Monitoring Network Load Balancing

This document provides you with the information you need to understand Cloud Monitoring for Network Load Balancing.

Network Load Balancing exports key metrics to Monitoring.

Monitoring metrics can be used for the following purposes:

  • Evaluate a load balancer's configuration, usage, and performance
  • Troubleshoot problems
  • Improve resource utilization and user experience

In addition to the predefined dashboards in Monitoring, you can create custom dashboards, set up alerts, and query the metrics through the Monitoring API.

Viewing Monitoring dashboards

  1. In the Cloud Console, go to the Monitoring page.
    Go to Monitoring
  2. In the navigation pane, select Dashboards.
  3. Select the Google Cloud Load Balancers dashboard. All the load balancers are displayed.
  4. Select the load balancer name from the list of load balancers.

In the Network Load Balancing details pane, you can see various details for the selected load balancer. This pane displays your current configurations.

In the Key metrics pane, you can see charts for each key metrics. Click Breakdowns to see specific breakdowns. This pane presents data informed by historical configurations, while the Network Load Balancing details pane only displays the current configurations. For more information, see Dashboards and charts.

Configuring monitoring and alerting using Cloud console

Refer to Metrics and resource types for a list of collected metrics. Refer to Filters for lists of attributes you can use to filter your results.

Defining Monitoring custom dashboards

You can create custom Monitoring dashboards over Network Load Balancing metrics.

  1. Go to Monitoring in the Google Cloud console.
    Go to Monitoring
  2. Select Dashboards > Create Dashboard.
  3. Click Add chart.
  4. Give the chart a title.
  5. Select metrics and filters.
    You can find Network Load Balancing metrics by searching for the loadbalancing.googleapis.com/l3/external prefix.
  6. Click Save.

Defining Monitoring alerts

You can define Monitoring alerts over various Network Load Balancing metrics:

  1. Go to Monitoring in the Google Cloud console.
    Go to Monitoring
  2. Select Alerting > Create a Policy.
  3. Select resource types and metrics.
    You can find Network Load Balancing resource types and metrics by searching for loadbalancing.googleapis.com/l3/external in the Find resource type and metric field.
  4. Specify a Configuration to set when an alert should be triggered.
  5. Add Filters if desired.
  6. Click Save.

Metric reporting frequency and retention

Metrics for the network load balancers are exported to Monitoring in 1-minute granularity batches. Monitoring data is retained for six weeks. The dashboard provides data analysis in default intervals of 1h (one hour), 6h (six hours), 1d (one day), 1w (one week), and 6w (six weeks). You can manually request analysis in any interval from six weeks (6w) to sixty seconds (60s).

Metrics and resource types

Metrics

The following metrics for network load balancers are reported into Monitoring. You can use these metric names when making API requests.

metric_name Type Description
loadbalancing.googleapis.com/l3/external/ingress_bytes_count counter The number of bytes sent from a client to a network load balancer backend. For TCP flows, only bytes in the application stream are counted.
loadbalancing.googleapis.com/l3/external/ingress_packets_count counter The number of packets sent from a client to a network load balancer backend.
loadbalancing.googleapis.com/l3/external/egress_bytes_count counter The number of bytes sent from a network load balancer backend to a client. For TCP flows, only bytes in on the application stream are counted.
loadbalancing.googleapis.com/l3/external/egress_packets_count counter The number of packets sent from a network load balancer backend to a client.
loadbalancing.googleapis.com/l3/external/rtt_latencies distribution A distribution of RTT measured over TCP connections for network load balancer flows.
Available only for TCP traffic.

Resource types

Network Load Balancing uses the following resource types:

  • tcp_lb_rule
  • udp_lb_rule
  • loadbalancing.googleapis.com/ExternalNetworkLoadBalancerRule

A network load balancer can support TCP, UDP, ESP, ICMP, and ICMPv6 traffic.

You can specify the resource type as tcp_lb_rule or udp_lb_rule to limit monitored data to either TCP or UDP protocols. You can specify the resource type as loadbalancing.googleapis.com/ExternalNetworkLoadBalancerRule to monitor all the supported protocols including TCP, UDP, ESP, ICMP, and ICMPv6 protocols.

Filters

Resource labels for tcp_lb_rule or udp_lb_rule

You can restrict your query by resource. You can also group results by these values.

resource.label.<var>LABEL_KEY</var>:
label_key Type Description
project string The identifier of the Google Cloud project associated with this resource.
load_balancer_name string The name of the load balancer.
region string The region where the load balancer's backend instance is located. For example, us-central1, europe-west1, asia-east1.
network_name string The VPC network in which the load balancer backend resides.
backend_target_type string The type of the backend target that handled the connection. Valid values are BACKEND_SERVICE or TARGET_POOL.
backend_target_name string The name of the backend target configured for this network load balancer.
forwarding_rule_name string The name of the forwarding rule used by the client to connect to the network load balancer.
forwarding_rule_network_tier string The network tier of the forwarding rule.
backend_name string The name of the backend instance group that handled the connection.
For a backend service-based network load balancer, the value is the name of the backend instance group that handled the connection.
For a target pool-based network load balancer, the value is the name of the target pool.
backend_type string The type of the backend that handled the connection. The valid values are INSTANCE_GROUP or TARGET_POOL.
For a backend service-based network load balancer, the value is INSTANCE_GROUP.
For a target pool-based network load balancer, the value is TARGET_POOL.
backend_scope string The scope of the backend group (name of the zone or region) that handled the connection. For a target pool-based network load balancer, the value is UNSUPPORTED_FOR_TARGET_POOL.
backend_scope_type string The type of the scope of the backend group that handled the connection. Valid values are ZONE or REGION. For a target pool-based network load balancer, the value is UNSUPPORTED_FOR_TARGET_POOL.
backend_failover_configuration string Failover configuration of the backend group that handled the connection. Valid values are PRIMARY, BACKUP, or UNKNOWN.
backend_subnetwork_name string The name of the subnetwork of the instance that received the connection.
endpoint_zone string Zone of the endpoint (Compute Engine or GKE instance) that handled the connection.

Resource labels for loadbalancing.googleapis.com/ExternalNetworkLoadBalancerRule

You can restrict your query by resource. You can also group results by these values.

resource.label.<var>LABEL_KEY</var>:
label_key Type Description
project string The identifier of the Google Cloud project associated with this resource.
region string The region where the load balancer's backend instance is located. For example, us-central1, europe-west1, asia-east1.
backend_network_name string The VPC network in which the load balancer backend resides.
backend_target_type string The type of the backend target that handled the connection. Valid values are BACKEND_SERVICE or TARGET_POOL.
backend_service_name string The name of the backend service that handled the connection. If the backend_target_type is TARGET_POOL, then the value is UNSUPPORTED_FOR_TARGET_POOL.
primary_target_pool string The name of the primary target pool. If the backend_target_type is BACKEND_SERVICE, then the value is UNSUPPORTED_FOR_BACKEND_SERVICE.
target_pool string The name of the target pool. If the backend_target_type is BACKEND_SERVICE, then the value is UNSUPPORTED_FOR_BACKEND_SERVICE.
forwarding_rule_name string The name of the forwarding rule used by the client to connect to the network load balancer.
backend_group_name string The name of the backend group that handled the connection. If the backend_target_type is TARGET_POOL, then the value is UNSUPPORTED_FOR_TARGET_POOL.
backend_group_type string The type of the backend group that handled the connection.
If the backend_target_type is BACKEND_SERVICE, then the value is INSTANCE_GROUP.
If the backend_target_type is TARGET_POOL, then the value is UNSUPPORTED_FOR_TARGET_POOL.
backend_group_scope string The scope of the backend group (name of the zone or region) that handled the connection. If the backend_target_type is TARGET_POOL, then the value is UNSUPPORTED_FOR_TARGET_POOL.
backend_subnetwork_name string The name of the subnetwork of the instance that received the connection.
backend_zone string Zone of the endpoint (Compute Engine or GKE instance) that handled the connection.

Metric labels

You can restrict your query by metric label. You can also group results by these values.

metric.label.<var>LABEL_KEY</var>:
label_key Type Description
client_country string Country of the client which initiated the connection to the network load balancer.
client_continent string Continent of the client which initiated the connection to the network load balancer.
protocol string The protocol in the connection for the loadbalancing.googleapis.com/ExternalNetworkLoadBalancerRule resource. This field is left blank for the tcp_lb_rule and udp_lb_rule resources.

Monitoring API requests

You can construct arbitrary queries over the Network Load Balancing metrics using the Monitoring API`s v3 projects.timeSeries.list request. The exact structure of the various definitions follow the generic monitoring filters semantics.

Example API requests

  1. Get all bytes sent from all network load balancers in the project, during 12:00-12:02 PM UTC on 2019-07-01, with 1m aggregates:

timeSeries.list parameters:

  • name: projects/PROJECT_ID
  • Filter: resource.type = "tcp_lb_rule" AND metric.name="loadbalancing.googleapis.com/l3/external/egress_bytes_count"
  • interval.start_time: 2019-07-01T12:00:00Z
  • interval.end_time: 2019-07-01T12:02:00Z
  • aggregation.alignmentPeriod: 60s
  • aggregation.crossSeriesReducer: REDUCE_SUM
  • aggregation.perSeriesAligner: ALIGN_SUM
  1. Get median RTT measurement over all network load balancers in the project, during 12:19-12:20 PM UTC on 2019-07-01, broken down by client country:

timeSeries.list parameters:

  • name: projects/PROJECT_ID
  • Filter: resource.type = "tcp_lb_rule" AND metric.name="loadbalancing.googleapis.com/l3/external/rtt_latencies"
  • interval.start_time: 2019-07-01T12:19:00Z
  • interval.end_time: 2019-07-01T12:20:00Z
  • aggregation.alignmentPeriod: 60s
  • aggregation.crossSeriesReducer: REDUCE_SUM
  • aggregation.groupByFields: metric.label.client_country
  • aggregation.perSeriesAligner: ALIGN_PERCENTILE_50
  1. Get total bytes from a specific network load balancer during 12:19-12:20 PM UTC on 2019-07-01, broken down by endpoint_zone and client continent:

timeSeries.list parameters:

  • name: projects/PROJECT_ID
  • Filter: resource.type = "tcp_lb_rule" AND resource.label.load_balancer_name = "netlb-bs-1" AND metric.name="loadbalancing.googleapis.com/l3/external/ingress_bytes_count"
  • interval.start_time: 2019-07-01T12:19:00Z
  • interval.end_time: 2017-07-01T12:20:00Z
  • aggregation.alignmentPeriod: 60s
  • aggregation.crossSeriesReducer: REDUCE_SUM
  • aggregation.groupByFields: metric.label.client_continent
  • aggregation.groupByFields: resource.label.endpoint_zone
  • aggregation.perSeriesAligner: ALIGN_SUM

What's next