This page shows you how to configure request logging for Media CDN.
Enable logs
Logging is disabled by default to minimize the data stored. Logs are routed, filtered, and stored in Cloud Logging.
Enable logging for a specific Media CDN service:
gcloud edge-cache services update YOUR_SERVICE \ --enable-logging \ --logging-sample-rate=1.0
Disable logging:
gcloud edge-cache services update YOUR_SERVICE \ --no-enable-logging
Get the current state of logging on a service:
gcloud edge-cache services describe YOUR_SERVICE
The output is similar to the following:
... logConfig: enable: true sampleRate: 1.0 ...
Set the log sampling rate
For large volumes of requests, rather than capture a log for every request, you might prefer to sample logs and rely on metrics for proactive monitoring and investigation.
To only ingest and store logs for 10% of your requests, set
logConfig.sampleRate
to 0.1
:
gcloud edge-cache services update YOUR_SERVICE \ --enable-logging \ --logging-sample-rate=0.1
Logging must be enabled to change the sampling rate.
The sampling rate, unless otherwise specified, defaults to 0 (0%), which logs every request for the configured service.
Query logs
To query logs, you can use the Logs Explorer in the Google Cloud console, or you can use the Google Cloud CLI.
To view logs in the Google Cloud console, filter for the Media CDN resource type and (optionally) by project and service name.
The following logging query shows the logs for all of your Media CDN services:
resource.type="edgecache.googleapis.com/EdgeCacheRouteRule"
To filter for logs associated with a specific EdgeCacheService
resource and
project, extend this query:
resource.type="edgecache.googleapis.com/EdgeCacheRouteRule" resource.labels.resource_container="projects/12345678" resource.labels.service_name="MY_PROJECT"
To match on partial names or use regular expressions, see the Logging query language comparisons.
Example: Identify cache misses
A request for an object might be a cache miss but might cause a cache fill for the following reasons:
- A time to live (TTL) that is too short
- An overly specific cache key
- Too few requests to stay cached
To identify cache misses, you can filter for logs in the Logs Explorer.
The following log query shows requests that are cacheable (looked up), but that require a fill from the origin:
resource.type="edgecache.googleapis.com/EdgeCacheRouteRule" resource.labels.resource_container="projects/12345678" resource.labels.service_name="MY_PROJECT" jsonPayload.cacheStatus="miss"
If you need to filter by a specific URL, filter on the
httpRequest.requestUrl
field:
httpRequest.requestUrl = "URL"
To filter on part of the URL, such as the path component, use the has
operator:
# `:` is the `has` comparison operator httpRequest.requestUrl: "/videos/1381381_1080.mp4" # You can use `OR` or `AND` to filter on multiple values httpRequest.requestUrl: ("https://media-test.example.com/" OR "https://canary.example.net")
To learn more about Logging's comprehensive syntax for matching and filtering, see Write advanced queries using the Logging query language.
Filter stored logs
To filter logs before storing them—for example, only capturing pertinent fields to reduce the total log volume that you need to store and query—you can configure exclusion filters, which allow you to define a query (filter) that includes or excludes fields prior to storage.
You can also set up multiple filters—for example, capturing all cache miss requests or all requests for a specific hostname, and only taking a sample of all logs.
Route logs
Media CDN integrates directly with Cloud Logging, which helps you aggregate and route your logs in the following ways:
To storage destinations, such as Cloud Storage and BigQuery
To Pub/Sub topics, to which third-party destinations can subscribe to access the logs
For an overview of how logs are routed from Logging, see Routing and storage overview.
What's next
- To understand more about how to set up logs-based metrics to define your own metrics values based on logged requests, see the Log-based metrics overview.
- To learn more about the pricing for Logging, see Google Cloud Observability pricing.
- To read about how audit logging works and how to enable and configure audit logs for admin activity, see the Cloud Audit Logs documentation.