This document provides you with the information you need to understand logging and monitoring metrics for Internal HTTP(S) Load Balancing.
You can enable logging on a per-backend service basis. A single internal HTTP(S) load balancer's URL map can reference more than one backend service, so you might need to enable logging for more than one backend service, depending on your configuration.
Enabling logging on an existing backend service
- Go to the Load balancing page in the Google Cloud Console.
Go to the Load balancing page
- Click the name of your load balancer.
- Click Edit .
- Click Backend Configuration.
- Click Edit next to your backend service.
- Click Advanced configurations (Session affinity, connection draining timeout).
- Click Enable logging.
- Set a Sample rate fraction. You can set it to anything from
- Click Update.
Update the backend service to enable logging with the gcloud compute instance-templates create command.
gcloud compute backend-services update BACKEND_SERVICE \ --enable-logging \ --logging-sample-rate=RATE \ --region=REGION
--enable-loggingenables logging for that backend service.
--logging-sample-ratelets you specify a value from
0.0means no requests are logged and
1.0means 100% of requests are logged. Only meaningful with the
--enable-loggingparameter. Enabling logging but setting the sampling rate to
0.0is equivalent to disabling logging.
After you enable logging on the backend service, each HTTP(S) request is logged by using Cloud Logging.
How to view logs
To view logs, go to the Logs Viewer.
internal HTTP(S) load balancer logs are indexed first by network and then by region.
- To see logs for all internal HTTP(S) load balancers, in the first pull-down menu select Internal HTTP/S Load Balancing Rule.
- To see logs for only one network, select Internal HTTP/S Load Balancing Rule, and then select the name of a network.
- To see logs for just one region of the network, select
Internal HTTP/S Load Balancing Rule >
Log fields of type boolean typically only appear if they have a value of
If a boolean field has a value of
false, that field is omitted from the log.
UTF-8 encoding is enforced for log fields. Characters that are not UTF-8 characters are replaced with question marks.
You can configure export of
logs-based metrics for
resource logs (
resource.type="internal_http_lb_rule). The metrics
created are based on the "Internal HTTP/S Load Balancing Rule
(Logs-based Metrics)" resource, which is available under
Cloud Monitoring dashboards:
What is logged
Internal HTTP(S) Load Balancing log entries contain information useful for monitoring and debugging your HTTP(S) traffic. Log entries contain the following types of information:
- General information shown in most Google Cloud logs, such as severity, project ID, project number, timestamp, and so on.
- HttpRequest log fields.
Logs are LogEntry protos.
The resource name of the log to which this log entry belongs.
In the form
||Timestamp format||The time at which the request began.|
The severity of the log entry. Defaults to
||HttpRequest object||An HttpRequest proto that describes the HTTP(S) request being logged.|
The resource name of the trace associated with the log entry, if
any. If it contains a relative resource name, the name is assumed to
be relative to
The span ID within the trace associated with the log entry. For
Trace spans, this is the same format that the Trace API v2 uses: a
16-character hexadecimal encoding of an 8-byte array, such as
||MonitoredResource object||The monitored resource associated with this log entry. See MonitoredResource object.|
||string||The name of the load balancer's VPC network.|
||string||The name of the load balancer's Google Cloud region.|
||string||The name of the URL map object configured to select a backend service.|
||string||The name of the forwarding rule object.|
||string||The name of the target proxy object referenced by the forwarding rule.|
The URL map path rule or route rule configured as part of the URL
map key. Can be
||string||The name of the backend selected to handle the request, based on the URL map path rule or route rule that matches the request.|
The type of the backend target (
||string||The name of the backend instance group or NEG.|
||string||The type of the backend, either instance group or NEG.|
The scope of the backend (either zone name or region name). Might be
The scope of the backend
Internal HTTP(S) Load Balancing exports monitoring data to Monitoring.
Monitoring metrics can be used for the following purposes:
- Evaluating a load balancer's configuration, usage, and performance
- Troubleshooting problems
- Improving resource utilization and user experience
In addition to the predefined dashboards in Monitoring, you can create custom dashboards, set up alerts, and query the metrics through the Monitoring API.
Viewing Cloud Monitoring metrics
To view the metrics for a monitored resource using Metrics Explorer, do the following:
- In the Google Cloud Console navigation pane, select Monitoring:
Go to Google Cloud Console
If this is the first access of Cloud Monitoring for this Google Cloud project, then Cloud Monitoring creates a Workspace. Typically, this process is automatic and completes within a few minutes. If prompted to either select a Workspace or to create a Workspace, select create.
- In the Monitoring navigation pane, click Metrics Explorer.
- Ensure Metric is the selected tab.
- Click in the box labeled
Find resource type and metric, and then select from the menu or
enter the name for the resource and metric. Use the following information to complete the
fields for this text box:
- For the Resource, select or enter internal_http_lb_rule. This selects the resource Internal HTTP/S Load Balancing Rule.
- For the Metric, select from the menu.
- Use the Filter, Group By, and Aggregator menus to modify how the data is displayed. For example, you can group by resource or metric labels. For more information, see Selecting metrics.
Defining alerting policies
You can create alerting policies to monitor the values of metrics and to notify you when those metrics violate a condition. The general steps for creating an alerting policy that monitors one or more Internal HTTP/S Load Balancer resources are as follows:
- In the Google Cloud Console, go to Monitoring.
- In the Monitoring navigation pane, select notificationsAlerting, and then select Create policy.
- Click Add condition:
- The settings in the Target pane specify the resource and metric to be monitored. Click the text box to enable a menu, and then select the resource Internal HTTP/S Load Balancer. Next, select a metric from the metrics list.
- The settings in the Configuration pane of the alerting policy determine when the alert is triggered. Most fields in this pane are populated with default values. For more information about the fields in the pane, see Configuration in the Alerting policies documentation.
- Click Add.
- To advance to the notifications section, click Next.
- Optional: To add notifications to your alerting policy, click
Notification channels. In the dialog, select one or more notification
channels from the menu, and then click OK.
If a notification channel that you want to add isn't listed, then click Manage notification channels. You are taken to the Notification channels page in a new browser tab. From this page, you can update the configured notification channels. After you have completed your updates, return to the original tab, click Refresh autorenew, and then select the notification channels to add to the alerting policy.
- To advance to the documentation section, click Next.
- Click Name and enter a name for the alerting policy.
- Optional: Click Documentation, and then add any information that you want included in a notification message.
- Click Save.
Defining Monitoring custom dashboards
You can create custom Monitoring dashboards over Internal HTTP(S) Load Balancing metrics:
- Go to Monitoring in the Google Cloud Console.
Go to Monitoring
- Select Dashboards > Create Dashboard.
- Click on Add Chart.
- Give the chart a title.
- Select metrics and filters. For metrics, the resource type is Internal HTTP/S Load Balancer.
- Click Save.
Metric reporting frequency and retention
Metrics for the load balancers are exported to Monitoring in 1-minute granularity batches. Monitoring data is retained for six (6) weeks. The dashboard provides data analysis in default intervals of 1H (one hour), 6H (six hours), 1D (one day), 1W (one week), and 6W (six weeks). You can manually request analysis in any interval from 6W to 1 minute.
Monitoring metrics for internal HTTP(S) load balancers
The following metrics for internal HTTP(S) load balancers are reported into Monitoring:
||The number of requests served by the internal HTTP(S) load balancer.|
|Request bytes count||
||The number of bytes sent as requests from clients to the internal HTTP(S) load balancer.|
|Response bytes count||
||The number of bytes sent as responses from the internal HTTP(S) load balancer to the client.|
||A distribution of the latency, in milliseconds, measured from the time the first byte of the request is received by the proxy to the time when the last byte of the response is sent by the proxy.|
||A distribution of the latency, in milliseconds, measured from when the proxy sends the first byte of the request to the backend to when the proxy receives the last byte of the response from the backend.|
Filtering dimensions for internal HTTP(S) load balancer metrics
Metrics are aggregated for each internal HTTP(S) load balancer. You can filter aggregated metrics by the following dimensions.
|BACKEND_SCOPE||The Google Cloud zone
or region of the backend group that served the client request, or
a special string for cases in which the backend group wasn't assigned.
|PROXY_REGION||Region of the internal HTTP(S) load balancer, client, and backend. Examples:
|BACKEND||The name of the backend instance group or NEG that served the client request.|
|BACKEND_TARGET||The name of the backend service that served the client request.|
|MATCHED_URL_RULE||The URL map path rule or route rule that matched the prefix of the client HTTP(S) request (up to 50 characters).|
Response code class fraction metric is supported for the entire
load balancer. No further granularity is supported.