Logs and metrics for caching

Each Cloud CDN request is logged in Cloud Logging. For information about enabling and disabling logging, see the External Application Load Balancer and Cloud CDN logging and monitoring overview.

Logs for Cloud CDN are associated with the external Application Load Balancer that your Cloud CDN backends are attached to. Cloud CDN logs are indexed first by forwarding rule and then by URL map.

To view Cloud CDN logs, follow these steps.

Console

1. In the Google Cloud console, go to the Logs Explorer page.

   Go to Logs Explorer

2. In the Resource menu, select Cloud HTTP Load Balancer.
3. View logs as follows:
   • View all logs: select the Resource menu, and then select All forwarding rules.
   • View the logs for a forwarding rule: select the forwarding rule name from the list of forwarding rules.
   • View the logs for a URL map used by a forwarding rule: select a forwarding rule, and then select a URL map.

Request served from the backend

To confirm that a request is served from a Cloud CDN-enabled backend, there are three main fields to look for, as follows:

• httpRequest: When a request is served from a backend, you can see that the cache is filled and you can confirm the request URL.
  • cacheFillBytes: NUMBER_OF_BYTES
  • cacheLookup: True
  • requestURL: URL
• jsonPayload: From the statusDetails field, you can confirm that the response was served by the backend.
  • statusDetails: "response_sent_by_backend"

Request served from cache

The following log entry shows a cache hit.

 {
    insertId: "1oek5rg3l3fxj7"
    jsonPayload: {
        @type: "type.googleapis.com/google.cloud.loadbalancing.type.LoadBalancerLogEntry"
        cacheId: "SFO-fbae48ad"
        statusDetails: "response_from_cache"
    }
    httpRequest: {
        requestMethod: "GET"
        requestUrl: "http://LOAD_BALANCER_IP_ADDRESS/static/us/three-cats.jpg"
        requestSize: "577"
        status: 304
        responseSize: "157"
        userAgent: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36"
        remoteIp: "CLIENT_IP_ADDRESS"
        cacheHit: true
        cacheLookup: true
    }
    resource: {
        type: "http_load_balancer"
        labels: {
            zone: "global"
            url_map_name: "URL_MAP_NAME"
            forwarding_rule_name: "FORWARDING_RULE_NAME"
            target_proxy_name: "TARGET_PROXY_NAME"
            backend_service_name: ""
            project_id: "PROJECT_ID"
        }
    }
    timestamp: "2020-06-08T23:41:30.078651Z"
    severity: "INFO"
    logName: "projects/PROJECT_ID/logs/requests"
    trace: "projects/PROJECT_ID/traces/241d69833e64b3bf83fabac8c873d992"
    receiveTimestamp: "2020-06-08T23:41:30.588272510Z"
    spanId: "7b6537d3672e08e1"
}

What is logged

In addition to general information contained in most logs, such as severity, project ID, project number, and timestamp, the external Application Load Balancer and Cloud CDN logs contain the following:

• The HttpRequest log field, which captures the HTTP status code, bytes returned, and whether a cache lookup or cache fill was performed.

• The jsonPayload.cacheId field, which indicates the location and cache instance that the cache response was served from. For example, a cache response served from a cache in Amsterdam would have a cacheId value of AMS-85e2bd4b, where AMS is the IATA code, and 85e2bd4b is an opaque identifier of the cache instance (because some Cloud CDN locations have multiple discrete caches).

• The statusDetails and cacheDetail fields of the jsonPayload.

You can filter on the following fields to determine the cache hit, miss, or revalidation status of a request served by Cloud CDN:

• Cache Hit

  jsonPayload.statusDetails=("response_from_cache" OR "byte_range_caching")

  or

  httpRequest.cacheHit=true
httpRequest.cacheValidatedWithOriginServer!=true

• Cache Hit Validated With Origin Server

  jsonPayload.statusDetails="response_from_cache_validated"

  or

  httpRequest.cacheHit=true
httpRequest.cacheValidatedWithOriginServer=true

• Cache Miss

  jsonPayload.statusDetails="response_sent_by_backend"

  or

  httpRequest.cacheHit!=true
httpRequest.cacheLookup=true

Log fields of type boolean typically only appear if they have a value of true. If a boolean field has a value of false, that field is omitted from the log.

UTF-8 encoding is enforced for these fields. Characters that are not UTF-8 characters are replaced with question marks.

When Cloud CDN serves a client request by initiating validation requests or byte range requests, it omits the serverIp field from the Cloud Logging log entry for the client request. This is because Cloud CDN can send requests to multiple server IP addresses in reaction to a single client request.

Each request initiated by Cloud CDN creates a Cloud Logging log entry. The resulting log entry contains a parentInsertId field inside the jsonPayload. You can use this field to identify the insertId of the log entry for the single client request that prompted Cloud CDN to initiate the validation request or byte range request. Furthermore, the log entry identifies Cloud CDN as the user agent.

Monitoring for Cloud CDN

Cloud CDN exports monitoring data to Cloud Monitoring. Monitoring is used to monitor the health of a Cloud CDN deployment.

Cloud Monitoring provides a set of dashboard definitions available on GitHub in the monitoring-dashboard-samples repository as JSON files. In the networking file, there is a Cloud CDN-specific dashboard called cloud-cdn-monitoring.json. Upload this custom dashboard to Monitoring by following the instructions in Installing sample dashboards.

Note: The dashboard does not filter on any specific backend; all load-balancing traffic for all backends is included. To filter on a specific backend, you need to add filter (resource.backend_target_name = 'my-backend') to each chart's Monitoring Query Language (MQL) query. Additionally, to filter on only backends that have Cloud CDN enabled, include filter (metric.cache_result!= 'DISABLED').

Monitoring Cloud CDN example queries

Monitoring lets you create custom dashboards. Dashboards can use any of the monitoring metrics for external Application Load Balancers. Following are some example MQL snippets that you can paste into custom Monitoring dashboards.

Request bytes count broken out by cache result

This query focuses on backends that have Cloud CDN enabled, which is done by including filter (metric.cache_result != 'DISABLED').

fetch https_lb_rule
| metric 'loadbalancing.googleapis.com/https/response_bytes_count'
| filter (metric.cache_result != 'DISABLED')
| align rate(1m)
| every 1m
| group_by [metric.cache_result],
    [value_response_bytes_count_aggregate:
       aggregate(value.response_bytes_count)]

Client roundtrip TCP latency at 95% for a specific backend target

This query includes filter (resource.backend_target_name = 'example-backend'), which narrows down traffic to the backend example-backend. A backend can be a Cloud Storage bucket, a Compute Engine VM group, or an external backend.

fetch https_lb_rule
| metric 'loadbalancing.googleapis.com/https/frontend_tcp_rtt'
| filter (resource.backend_target_name = 'example-backend')
| group_by 1m,
    [value_frontend_tcp_rtt_aggregate: aggregate(value.frontend_tcp_rtt)]
| every 1m
| group_by [metric.proxy_continent],
    [value_frontend_tcp_rtt_aggregate_percentile:
       percentile(value_frontend_tcp_rtt_aggregate, 95)]

Request count broken out by response code class for Cloud CDN-enabled backends

This query breaks out traffic by response code class (2xx, 3xx, 4xx, 5xx) to help separate client successes, client errors, and server errors.

fetch https_lb_rule
| metric 'loadbalancing.googleapis.com/https/request_count'
| filter (metric.cache_result != 'DISABLED')
| group_by 1h, [row_count: row_count()]
| every 1h
| group_by [metric.response_code_class],
    [row_count_aggregate: aggregate(row_count)]

Request count broken out by country of origin

This query shows traffic broken out by country of origin, which is determined by using client IP addresses.

fetch https_lb_rule
| metric 'loadbalancing.googleapis.com/https/request_count'
| align rate(1m)
| every 1m
| group_by [metric.client_country], [value_request_count_aggregate: aggregate(value.request_count)]

What's next

• To learn more about logging, including how to export logs to BigQuery, Pub/Sub, or Cloud Storage, and how to configure logs-based metrics for monitoring and alerting, see the Cloud Logging documentation.

• To learn about the fields that are included in the httpRequest log entry, see HttpRequest.