Skip to content

monitor-load-balancer.md

Latest commit

 

History

History
292 lines (212 loc) · 14.3 KB

monitor-load-balancer.md

File metadata and controls

292 lines (212 loc) · 14.3 KB
title Monitor Azure Load Balancer
description Start here to learn how to monitor Azure Load Balancer by using Azure Monitor and Azure Monitor Insights.
ms.date 08/21/2024
ms.custom horz-monitor, template-how-to, subject-monitoring, engagement-fy23, devx-track-azurecli, devx-track-azurepowershell
ms.topic concept-article
author mbender-ms
ms.author mbender
ms.service azure-load-balancer

Monitor Azure Load Balancer

[!INCLUDE horz-monitor-intro]

Load Balancer provides other monitoring data through:

[!INCLUDE horz-monitor-insights]

Load Balancer insights provide:

  • Functional dependency view
  • Metrics dashboard
  • Overview tab
  • Frontend and Backend Availability tab
  • Data Throughput tab
  • Flow Distribution
  • Connection Monitors
  • Metric Definitions

For more information on Load Balancer insights, see Using Insights to monitor and configure your Azure Load Balancer.

[!INCLUDE horz-monitor-resource-types]

For more information about the resource types for Load Balancer, see Azure Load Balancer monitoring data reference.

[!INCLUDE horz-monitor-data-storage]

[!INCLUDE horz-monitor-platform-metrics]

You can analyze metrics for Load Balancer with metrics from other Azure services using metrics explorer by opening Metrics from the Azure Monitor menu. See Analyze metrics with Azure Monitor metrics explorer for details on using this tool.

For a list of available metrics for Load Balancer, see Azure Load Balancer monitoring data reference.

[!INCLUDE horz-monitor-resource-logs]

For the available resource log categories, their associated Log Analytics tables, and the log schemas for Load Balancer, see Azure Load Balancer monitoring data reference.

Creating a diagnostic setting

Resource logs aren't collected and stored until you create a diagnostic setting and route them to one or more locations. You can create a diagnostic setting with the Azure portal, Azure PowerShell, or the Azure CLI.

To use the Azure portal and for general guidance, see Create diagnostic setting to collect platform logs and metrics in Azure. To use PowerShell or the Azure CLI, see the following sections.

When you create a diagnostic setting, you specify which categories of logs to collect. The category for Load Balancer is AllMetrics.

PowerShell

Sign in to Azure PowerShell:

Connect-AzAccount

Log analytics workspace

To send resource logs to a Log Analytics workspace, enter these commands. Replace the bracketed values with your values:

## Place the load balancer in a variable. ##
$lbpara = @{
    ResourceGroupName = <your-resource-group-name>
    Name = <your-load-balancer-name>
}
$lb = Get-AzLoadBalancer @lbpara
    
## Place the workspace in a variable. ##
$wspara = @{
    ResourceGroupName = <your-resource-group-name>
    Name = <your-log-analytics-workspace-name>
}
$ws = Get-AzOperationalInsightsWorkspace @wspara
    
## Enable the diagnostic setting. ##
Set-AzDiagnosticSetting `
    -ResourceId $lb.id `
    -Name <your-diagnostic-setting-name> `
    -Enabled $true `
    -MetricCategory 'AllMetrics' `
    -WorkspaceId $ws.ResourceId

Storage account

To send resource logs to a storage account, enter these commands. Replace the bracketed values with your values:

## Place the load balancer in a variable. ##
$lbpara = @{
    ResourceGroupName = <your-resource-group-name>
    Name = <your-load-balancer-name>
}
$lb = Get-AzLoadBalancer @lbpara
    
## Place the storage account in a variable. ##
$storpara = @{
    ResourceGroupName = <your-resource-group-name>
    Name = <your-storage-account-name>
}
$storage = Get-AzStorageAccount @storpara
    
## Enable the diagnostic setting. ##
Set-AzDiagnosticSetting `
    -ResourceId $lb.id `
    -Name <your-diagnostic-setting-name> `
    -StorageAccountId $storage.id `
    -Enabled $true `
    -MetricCategory 'AllMetrics'

Event hub

To send resource logs to an event hub namespace, enter these commands. Replace the bracketed values with your values:

## Place the load balancer in a variable. ##
$lbpara = @{
    ResourceGroupName = <your-resource-group-name>
    Name = <your-load-balancer-name>
}
$lb = Get-AzLoadBalancer @lbpara
    
## Place the event hub in a variable. ##
$hubpara = @{
    ResourceGroupName = <your-resource-group-name>
    Name = <your-event-hub-name>
}
$eventhub = Get-AzEventHubNamespace @hubpara

## Place the event hub authorization rule in a variable. ##    
$hubrule = @{
    ResourceGroupName = 'myResourceGroup'
    Namespace = 'myeventhub8675'
}
$eventhubrule = Get-AzEventHubAuthorizationRule @hubrule

## Enable the diagnostic setting. ##
Set-AzDiagnosticSetting `
    -ResourceId $lb.Id `
    -Name 'myDiagSetting-event'`
    -EventHubName $eventhub.Name `
    -EventHubAuthorizationRuleId $eventhubrule.Id `
    -Enabled $true `
    -MetricCategory 'AllMetrics'

Azure CLI

Sign in to Azure CLI:

az login

Log analytics workspace

To send resource logs to a Log Analytics workspace, enter these commands. Replace the bracketed values with your values:

lbid=$(az network lb show \
    --name <your-load-balancer-name> \
    --resource-group <your-resource-group> \
    --query id \
    --output tsv)

wsid=$(az monitor log-analytics workspace show \
    --resource-group <your-resource-group> \
    --workspace-name <your-log-analytics-workspace-name> \
    --query id \
    --output tsv)
    
az monitor diagnostic-settings create \
    --name <your-diagnostic-setting-name> \
    --resource $lbid \
    --metrics '[{"category": "AllMetrics","enabled": true}]' \
    --workspace $wsid

Storage account

To send resource logs to a storage account, enter these commands. Replace the bracketed values with your values:

lbid=$(az network lb show \
    --name <your-load-balancer-name> \
    --resource-group <your-resource-group> \
    --query id \
    --output tsv)

storid=$(az storage account show \
        --name <your-storage-account-name> \
        --resource-group <your-resource-group> \
        --query id \
        --output tsv)
    
az monitor diagnostic-settings create \
    --name <your-diagnostic-setting-name> \
    --resource $lbid \
    --metrics '[{"category": "AllMetrics","enabled": true}]' \
    --storage-account $storid

Event hub

To send resource logs to an event hub namespace, enter these commands. Replace the bracketed values with your values:

lbid=$(az network lb show \
    --name <your-load-balancer-name> \
    --resource-group <your-resource-group> \
    --query id \
    --output tsv)

az monitor diagnostic-settings create \
    --name myDiagSetting-event \
    --resource $lbid \
    --metrics '[{"category": "AllMetrics","enabled": true}]' \
    --event-hub-rule /subscriptions/<your-subscription-id>/resourceGroups/<your-resource-group>/providers/Microsoft.EventHub/namespaces/<your-event-hub-namespace>/authorizationrules/RootManageSharedAccessKey

[!INCLUDE horz-monitor-activity-log]

Note

Load balancer activity logs will not include updates to NIC-based backend pools. To monitor and alert on updates to the load balancer backend pool for NIC-based backend pools, we recommend collecting logs on the NIC resource-level or at a resource group level instead.

[!INCLUDE horz-monitor-analyze-data]

[!INCLUDE horz-monitor-external-tools]

Analyzing Load Balancer Traffic with VNet flow logs

Virtual network flow logs are a feature of Azure Network Watcher that logs information about IP traffic flowing through a virtual network. Flow data from virtual network flow logs is sent to Azure Storage. From there, you can access the data and export it to any visualization tool, security information and event management (SIEM) solution, or intrusion detection system (IDS).

For general guidance on creating and managing virtual network flow logs, see Manage virtual network flow logs. Once you have created your virtual network flow logs, you can access the data on Log Analytics workspaces where you can also query and filter the data to identify traffic flowing through your Load Balancer. See Traffic analytics schema and data aggregation for more details on the virtual network flow logs schema.

You can also enable Traffic Analytics when you are creating your virtual network flow logs which provides insights and visualizations on the flow log data such as traffic distribution, traffic pattern, application ports utilized, and top talkers in your virtual network.

Log Analytics query for VNet flow logs

To view logs for inbound flows connected to a specific Load Balancer:

NTANetAnalytics
| where DestLoadBalancer == '<Subscription ID>/<Resource Group name>/<Load Balancer name>'

  1. Use the query above in your Log Analytics workspace and update the string with the valid values for your Load Balancer. To learn more about using Log Analytics, see Log Analytics tutorial.

  2. To view the source IP of the connection, either the SrcIp or SrcPublicIps column will be populated. All traffic originating from public non-malicious or Azure service-owned IP addresses will appear in SrcPublicIps and all other source IPs will appear in SrcIP. If you want more details on the type of traffic, you can use the FlowType column to filter for different types of IP addresses involved in the flow. See Traffic analytics schema and data aggregation notes for FlowType field definitions.

  3. Identify the backend pool instances being used in the inbound connection through any of the following columns: DestIP, MacAddress, DestVM, TargetResourceID, DestNic.

  4. Through these logs, you can gather further information about the connections going through your Load Balancer such as port information, protocol, and traffic size through packet and byte count sent from destination and source.

[!INCLUDE horz-monitor-kusto-queries]

[!INCLUDE horz-monitor-alerts]

[!INCLUDE horz-monitor-insights-alerts]

Load Balancer alert rules

The following table lists some suggested alert rules for Load Balancer. These alerts are just examples. You can set alerts for any metric, log entry, or activity log entry listed in the Azure Load Balancer monitoring data reference.

Alert type Condition Description
Load balancing rule unavailable due to unavailable VMs If data path availability split by Frontend IP address and Frontend Port (all known and future values) is equal to zero, or in a second independent alert, if health probe status is equal to zero, then fire alerts These alerts help determine if the data path availability for any configured load balancing rules isn't servicing traffic due to all VMs in the associated backend pool being probed down by the configured health probe. Review load balancer troubleshooting guide to investigate the potential root cause.
VM availability significantly low If health probe status split by Backend IP and Backend Port is equal to user defined probed-up percentage of total pool size (that is, 25% are probed up), then fire alert This alert determines if there are less than needed VMs available to serve traffic
Outbound connections to internet endpoint failing If SNAT Connection Count filtered to Connection State = Failed is greater than zero, then fire alert This alert fires when SNAT ports are exhausted and VMs are failing to initiate outbound connections.
Approaching SNAT exhaustion If Used SNAT Ports is greater than user defined number, then fire alert This alert requires a static outbound configuration where the same number of ports are always allocated. It then fires when a percentage of the allocated ports is used.

[!INCLUDE horz-monitor-advisor-recommendations]

Related content