Deploy, secure, and monitor streaming service with Media CDN
Gaurav Madan
Customer Engineer, Networking
Prakash Daga
Product Manager
Media CDN is Google Cloud's media delivery CDN platform designed for streaming video and large object downloads. Media CDN complements Cloud CDN, Google Cloud’s web acceleration CDN platform. With Google Cloud’s Media CDN, customers can now leverage the same infrastructure that YouTube uses to serve its global users. Media CDN gives its customers a platform providing massive coverage and presence globally in over 206+ countries/territories and 1300+ cities. In this blog post, we will walk you through the steps of turning up Google Cloud Media CDN. We will cover the following topics:
- Setting up the Origin
- Setting up the CDN service
- Setting up required security and Media CDN integration with Cloud Armor
- Monitoring CDN service and Media CDN integration with Cloud Monitoring
While security requirements may differ from customer to customer and steps for setting security are optional, the steps for setting up of origin and CDN service will be mandatory in all scenarios. It is important to define a few terms, which are used in this document frequently :
Edge Cache Origin : In Media CDN, an Edge Cache Origin is a reference to a publicly-reachable HTTP(S) endpoint hosting the content you intend to serve via Media CDN. Therefore, from a configuration point of view, origin related configuration is defined under ‘Edge Cache origin’
Edge Cache Service : An Edge Cache Service is a public endpoint that makes thousands of global edge locations available for delivering media to your users. Therefore, from Media CDN configuration point of view, caching and routing related configurations are defined under ‘ Edge Cache service’
Setting up the Origin : Edge Cache Origin Configuration
Media CDN lets customers easily fetch content from origin infrastructure, located in multiple sources. The configurations related to Origin of content are defined under ‘Origins’. The configuration can be done from Google Cloud console (UI), or by creating a YAML file and importing the same using gcloud command. A sample origin configuration is shown below:
While defining the origins, some of important points to be considered are as follows:
a) A single Edge Cache service can map to multiple origins that contain manifests, video segments, and other static content. For example, there could be 2 Origins pointing to 2 different Google cloud storage buckets and both origins being successfully used in one given edge cache service.
b) Parameters like retries and failover behaviors are configured per origin. For example, for a given origin, a customer may configure the retry condition as ‘Connection-failure’, ‘gateway-error’, ‘HTTP-5xx’, ‘Forbidden’ etc. Once retry-conditions are identified and chosen, customers define the maximum-retry attempts and appropriate timeout values. Timeout values will signify how long Media CDN will wait to connect to your origin for it to respond to a request.
c) Failover Origins : Content availability is a top priority for Media CDN customers, and failover mechanisms are a common way of addressing availability for customer origins. In the event of Media CDN not able to reach primary origin, it can failover to alternate origins (aka failover origins)
d) A Customer needs to configure protocol over which communication to origin will function. Origins can be reached over HTTP/2, HTTPS (HTTP/1.1 over TLS), and unencrypted HTTP/1.1.
Setting up the CDN service: edge cache service configuration
Once an origin is configured, the next step is to get the edge cache service configured. Some of points to consider are as follows
a) Break the edge cache service configuration into ‘routes’. These routes match requests based on (at least) a host. For each route, we can define appropriate origin and define the properties of cache-behavior. For example we can define the first route to match a manifest file, point it to appropriate origin and define its CDN policy, CORS policy etc. Additionally, we will define second route to match the segments like *.ts, point them to the same or alternate origin and define appropriate CDN policy, CORS policy etc.
b) For each route, the customer has to next decide which cache-mode is appropriate for a given route rule. Cache modes allow customer to decide if Media CDN should unconditionally cache successful responses, overriding any cache directives set by the origin (FORCE_CACHE_ALL) or media CDN should cache static object and ensure that Valid caching directives from the origin are prioritized (CACHE_ALL_STATIC) or the matched object bypasses the cache (BYPASS_CACHE)
c) Define appropriate Time to Live(TTL) for CDN policies : 3 timers need attention here, how long edge cache can hold an object (Max TTL), how long client browser can cache an object (Client TTL), and a timer used when origin doesn’t have a existing valid TTL (default TTL). Customers should be aware that these TTL are optimistic timers. Content that is rarely accessed might be evicted from the cache prior to the TTL being reached.
Optionally, if Cross-Origin Resource Sharing (CORS) related configuration is needed, the same is configured under CORS policy. Details are documented here.
Setting up the security: edge cache security configuration
Security requirements vary differently depending on the deployment and Media CDN offers a comprehensive security integration. The most common asks on security are discussed below.
Securing the delivery of content : There can be multiple strategies to secure the content and one or multiple of these can be deployed by customer as per requirements:
- Media CDN integrates with Cloud Armor. The edge security policies of Cloud Armor can help allow/deny GEOs to access served content. Also, customers may choose to configure allow/deny source IP address blocks which can access served content (IP whitelist / blacklist strategy). The below example shows a sample rule of Cloud Armor policy which will allow users from India as or from a white-listed IP range to access content
Media CDN supports serving TLS-encrypted (HTTPS) traffic from your own domain name. Both self managed as well as Google managed certificates are supported with Media CDN
Securing the access to content
Media CDN offers the following signed request options for client authentication:
Signatures: Media CDN uses a single signature to help protect content.Signatures let you do full URL signing, including the host and protocol.
Tokens: Token authentication is a signed request signing format where all of the information used to sign the request is in a specially-constructed token. The token is appended to a query parameter of the request URL to make a signed request. In addition to verifying tokens, you can configure Media CDN to generate new tokens for user agents to use in future requests within a playback session. This behavior, called Dual-Token Authentication, increases protection of your master manifest assets from link sharing, and requires your application server to only sign the initial master manifest request, rather than each request in a playback session. Media CDN uses tokens to help protect content. Your application server will generate a short-duration token for the initial master manifest request, and Media CDN will return a long-duration token to authenticate the session’s remaining asset requests.
A more comprehensive defense-in-depth approach requires origins to be able to authenticate requests from Media CDN itself.
For such requirements in case origin is Google Cloud Storage(GCS) buckets, Media CDN makes authenticated requests to private GCS buckets using service accounts. Meaning thereby, if the GCS buckets are private, we need to grant the ‘objectViewer’ permission to Media CDN's service account in order to let Media CDN read from the bucket on cache fill.
For origin buckets not on GCS, Media CDN also supports AWS signature version 4, which can be useful in cases like origin on external storage buckets like AWS S3 buckets etc.
Setting up the monitoring: edge cache monitoring
It's important to monitor your media service closely to ensure that it's performing as expected. Media CDN allows all its customers to use no cost out-of-box monitoring, configure custom monitoring dashboards using Media CDN metrics if predefined dashboards don't serve the purpose or use rich logging data to have a deeper look into various data points related to edge cache. The out of box monitoring includes (not limited to)
Reporting on cache-summary results : HITs, MISS, UNCACHEABLE percentages and offload ratio.
Top geographies of audiences
Total number of requests, cache egress, cache fill, error percentage statistics
Origin time to first byte, latencies etc
All of the above monitoring statistics are available for each Media CDN edge service configured by the customer for a historical period of up to the last 6 weeks. Below snip shows a few of widgets from out of box monitoring dashboard
In addition to the above, Media CDN integrates with Cloud Monitoring. This gives users of Media CDN options to select metrics when using Metrics Explorer or create alerting policies based on these metrics. This also gives a uniform experience to use the same operations suite and start monitoring Media CDN using the same tools and procedures as for other Google Cloud products. Under metrics explorer, Media CDN offers a rich set of metrics, which can be used by a customer to create custom dashboards. Examples of some useful metrics are request count, request count by client, response byte count, response byte count by client, origin request count, origin response byte count, origin time to first byte, origin time to last byte etc. Following snip shows a sample of metrics available for monitoring :
Lastly, Media CDN offers near real-time logging, with export options to a variety of tools. Each HTTP request between the client and the edge is logged by Media CDN to Cloud Logging. HTTP log fields, client ASN, cache status, cache mode, details of Armor Edge security policy are some important fields shown in Media CDN logs. A sample Media CDN log entry is shown here.
