Media CDN lets you fetch content from your origin infrastructure, whether content is hosted within Google Cloud, in another cloud, or on-premises.
Each configuration can have one or more origins associated with it. Origin configurations tell Media CDN how to connect to your infrastructure, when and how to fail over, retry, and time out, and what protocol to use when connecting.
The following are origin features:
- Origins can be defined per host and per route, which allows a single
EdgeCacheService
resource to map to multiple origins that contain (for example) manifests, video segments, other static content, and so on. - Origins can be reached over HTTP/2, HTTPS, and unencrypted HTTP/1.1.
- Retries and failover behaviors are configured per origin, and can allow
the service to fail over on hard errors (such as connectivity failure) or retry
based on HTTP 404 (
Not Found
) or HTTP 429 (Too Many Requests
) responses. - Private resources inside Google Cloud or on-premises can be reached by configuring an external Application Load Balancer as an origin behind Media CDN.
- Redirect-following behavior is configured per origin. You can enable Media CDN to follow redirects to other origin servers.
Origin requirements
To allow Media CDN to cache origin responses larger than
1 MiB, an origin must include the following in the response headers for
both HEAD
and GET
requests, unless specified otherwise:
- A
Last-Modified
orETag
HTTP response header (a validator). - A valid HTTP
Date
header. - A valid
Content-Length
header. - The
Content-Range
response header, in response to aRange GET
request. TheContent-Range
header must have a valid value in the form ofbytes x-y/z
(wherez
is the object size).
The default origin protocol is HTTP/2. If your origins only support HTTP/1.1, you can set the protocol field explicitly for each origin.
Best practices for origin failover
When configuring multiple origins for failover or load balancing, ensure that
your media content and Vary
, ETag
, and Last-Modified
header behaviors are
consistent between your origins.
As a best practice, configure origin redirection only for origins that you trust
and control. Ensure that you trust every origin in a redirect chain because
each origin produces content that is served by your EdgeCacheService
.
Origin shielding
Media CDN provides a deeply tiered edge infrastructure that is designed to actively minimize cache fill wherever possible.
There are three primary layers of caching infrastructure:
- Deep edge caches, which serve the majority of traffic and off-load within a service providers network.
- Google's peering edge, which is connected to thousands of ISPs and acts as the mid-tier cache for edge caches, and for cases where those are not present within a given ISP, the user-facing cache.
- Long-tail caches within Google's network that other downstream caches fill from prior to the origin. These caches support significant fan-in and have substantial cache storage capacity, and act as an origin "shield".
An overview of this topology is shown here:
All layers of caching support request collapsing ("coalescing") to further reduce origin load. Based on observed, real-world workloads at scale:
- > 95% of cache fill uses a dedicated long-tail cache node within the region, in order to reduce cache fill costs and latency.
- Cache fill between the origin and Google's own edge infrastructure is entirely over Google's global private backbone network, which reduces cache fill latency and improves reliability—both are active benefits for live streaming workloads.
- Caches cross-fill from each other where advantageous to do so, further driving down cache fill rates.
- Media CDN has significant storage capacity across caches, which minimize the eviction rates even for long-tail, less popular content.
Customers might see different offload rates depending on their cache configuration, user load, workloads (such as live vs. on-demand), user distribution, and how much long-tail content (total corpus size) they serve to users across regions.
Request collapsing
Request collapsing (or "coalescing") actively collapses multiple user-driven cache fill requests for the same cache key into a single origin request, per edge node.
Combined with Origin Shielding, this further reduces origin load and egress bandwidth needs, and is the default behavior for Media CDN.
Collapsed requests have both the client-facing request logged and the (collapsed) cache fill request logged. The "leader" of the collapsed session is used to make the origin fill request, which means that the origin sees the headers (including the User-Agent) of that client only.
Requests that do not share the same cache key cannot be collapsed.
Origin connectivity
The following sections describe how Media CDN connects to origins, how HTTP requests are made, and how you can authenticate requests.
Supported origins and protocols
Media CDN directly supports any publicly reachable HTTP endpoint as an origin, including the following:
- Cloud Storage buckets, including private buckets through Identity and Access Management service accounts
- External Application Load Balancers
- Amazon S3-compatible buckets, including private buckets that use AWS Signature Version 4
- Other publicly available object storage, such as Azure Blob Storage
- Publicly available web servers, such as public VMs or on-premises hosts
Connectivity to origins is over secure tunnels and Google's global backbone network.
The following table details the supported origin protocols.
Protocol | Supported | SSL (TLS) required |
---|---|---|
HTTP/2 | Yes (default) | Yes |
HTTPS (HTTP/1.1 over TLS) | Yes | Yes |
HTTP/1.1 | Yes | No |
Media CDN uses HTTP/2 (h2) to connect to an origin by default. HTTP/2 and HTTPS both require a valid, publicly trusted TLS (SSL) certificate. A valid certificate is one that is unexpired, signed by a public Certificate Authority, and has a Subject Alternative Name matching the hostname sent to the origin.
Notes:
- If your origin does not support HTTP/2, you can explicitly set the protocol
(on a per-origin basis) to
HTTP
(HTTP/1.1) orHTTPS
(HTTP/1.1 over TLS). - When configuring HTTPS or HTTP/1.1 as the origin protocol, Media CDN doesn't negotiate an alternative protocol (such as HTTP/2). Similarly, when configuring HTTP/2, the connection doesn't fall back to HTTP/1.1, in order to be explicit about origin connectivity behavior.
- Media CDN automatically uses the correct port based on the protocol: port 80 for HTTP or port 443 for HTTPS and HTTP/2.
Origin request headers
When connecting to an origin, Media CDN uses the Host
header
from the client request by default.
The following table documents what the origin sees in the incoming request under different configuration scenarios:
Client Request | EdgeCacheService.hostRewrite |
EdgeCacheOrigin.hostRewrite |
originAddress | Host header / TLS SNI at origin |
---|---|---|---|---|
media.example.com | None | None | backend.example.com | media.example.com |
media.example.com | service.example.com | None | backend.example.com | service.example.com |
media.example.com | None | origin.example.com | backend.example.com | origin.example.com |
media.example.com | service.example.com | origin.example.com | backend.example.com | origin.example.com |
media.example.com | service.example.com | origin.example.com | gs://vod-content-bucket | set automatically based on the bucket name |
The primary origin and any failover origins see the same host header if they
share the same routeRule
or hostRewrite
configuration.
Any hostRewrite
settings are ignored when using a Cloud Storage
bucket as the origin because alternative host header values are not supported
by Cloud Storage buckets. The host header is automatically set based on
the bucket name.
The TLS SNI (Server Name Indication) value in the request (for HTTP/3, HTTP/2, and HTTPS requests) is set to the same value as the host header sent to the origin.
For information about rewriting host headers for per-route configuration, see Advanced routing. For information about setting per-origin override actions, see Example: Failover without redirect following.
Using private Cloud Storage buckets
Media CDN can pull content from any internet-reachable HTTP(S) endpoint. In some cases, you might wish to require authentication, in order to only allow Media CDN to pull content, and prevent unauthorized access. Cloud Storage supports this through IAM permissions.
For Cloud Storage origins, you can:
- Grant the Media CDN service account the
objectViewer
IAM permission on the Cloud Storage bucket(s) you are using as your origin(s). - Remove the
allUsers
permission. - (Optional) Remove the
allAuthenticatedUsers
permission.
To change permissions of a Cloud Storage bucket, you need the Storage Admin IAM role.
The Media CDN service account is owned by the Media CDN project, and it won't appear in your project's list of service accounts.
The service account has the following format, and only grants access to Media CDN resources in the projects that you explicitly allow.
service-PROJECT_NUM@gcp-sa-mediaedgefill.iam.gserviceaccount.com
To grant Media CDN access to a bucket, you can use the
Google Cloud console
or the gsutil
tool to give the service account the objectViewer
role:
gsutil iam ch \ serviceAccount:service-PROJECT_NUM@gcp-sa-mediaedgefill.iam.gserviceaccount.com:objectViewer gs://BUCKET
To remove all permissions from the allUsers
role for the given bucket, run the
following:
gsutil iam ch -d allUsers gs://BUCKET
You can validate that public access has been removed by opening an incognito
browser window and attempting to access a bucket object by using
https://storage.googleapis.com/BUCKET/object.ext
.
To allow EdgeCacheService
resources within one project access
to a Cloud Storage bucket in another project, you can grant the
Media CDN service account in that project access to the storage
bucket.
To do so, ensure that the PROJECT_NUM
in
service-PROJECT_NUM@gcp-sa-mediaedgefill.iam.gserviceaccount.com
is the
project number of the project with the EdgeCacheService
resources that need access. You
can repeat this for multiple projects, especially if you have multiple projects
housing different Media CDN environments (dev, staging,
production) and a separate project containing your video or media assets.
You can protect access to your Cloud Storage origin without enabling signed requests for that route.
Configuring private Cloud Storage does not prevent your cached content from being accessed from Media CDN directly. For information about how you can issue signed requests to individual users, see signed requests.
Authenticate requests
If you need to confirm that a request is coming from Media CDN, there are two supported approaches:
- Validate the connecting IP address is from Media CDN's cache
fill ranges. These ranges are shared across all customers, but are always
used by
EdgeCacheService
resources when connecting to an origin. - Add a custom request header with a token value you validate on the origin (for example, a random 16-byte value). Your origin can then reject requests that do not include this value.
For guidance about how to set request headers per route, see the documentation for custom headers.
Configure following origin redirects
Media CDN supports following redirects returned by your origin internally during cache fill, rather than returning redirect responses directly to the client. When Media CDN is configured to follow origin redirects, Media CDN retrieves content from the redirect location before caching and returning the redirected response to the client. Media CDN follows redirects across domains.
As a best practice, configure origin redirection only for origins that you trust
and control. Ensure that you trust every origin in a redirect chain because
each origin produces content that is served by your EdgeCacheService
.
To enable following origin redirects, add the following configuration to your
EdgeCacheOrigin
:
name: MY_ORIGIN
originAddress: "DOMAIN_NAME"
maxAttempts: 2
originRedirect:
redirectConditions: [FOUND, TEMPORARY_REDIRECT]
Media CDN only uses the protocol specified to reach all servers
specified in the redirects. Ensure that all the servers that
Media CDN might be redirected to support your desired protocols.
In particular, if the protocol is set to HTTPS, HTTP/2, or HTTP/3,
Media CDN doesn't fall back to HTTP/1.1 connections to follow
insecure redirects. The host header sent to the redirected origin matches the
redirected URL. Media CDN follows a single redirect per
EdgeCacheOrigin
attempt before returning the final response or evaluating
retry or failover conditions.
The redirectConditions
setting specifies which HTTP response codes cause
Media CDN to follow a redirect for each origin.
Condition | Description |
---|---|
MOVED_PERMANENTLY | Follow redirect for response code HTTP 301 |
FOUND | Follow redirect for response code HTTP 302 | SEE_OTHER | Follow redirect for response code HTTP 303 |
TEMPORARY_REDIRECT | Follow redirect for response code HTTP 307 |
PERMANENT_REDIRECT | Follow redirect for response code HTTP 308 |
Failover and timeouts
The following sections describe how to configure:
- Timeouts: Determine how long Media CDN waits to connect to your origin or for it to respond to a request.
- Retries: Determine whether Media CDN retries an origin HTTP request to your origin, and under what conditions.
- Failover: Determine whether Media CDN attempts to connect to a failover origin if the first is unavailable or returns a specific status code.
Tune origin timeouts
Timeouts allow you to configure when origin retry and failover behaviors are triggered, as well as when subsequent client-side failover may be triggered.
The following describes configurable timeouts that Media CDN supports:
connectTimeout
andmaxAttemptsTimeout
limit how long Media CDN takes to find a usable response.Both timeouts include the time that the origin takes to return headers and to determine whether to use a failover or redirect.
connectTimeout
applies independently for each origin attempt, whilemaxAttemptsTimeout
includes the time required to connect across all origin attempts, including failovers and redirects. Following a redirect counts as an additional attempt to connect to the origin, and counts toward themaxAttempts
set for the configured origin.When Media CDN encounters a non-redirect response, such as from a redirect or failover origin, the
readTimeout
andresponseTimeout
values apply. Redirected origins use theconnectTimeout
,readTimeout
, andresponseTimeout
values configured for theEdgeCacheOrigin
that encountered the redirect.responseTimeout
andreadTimeout
control how long a streamed response can take. After Media CDN determines that it's going to use an upstream response, neitherconnectTimeout
normaxAttemptsTimeout
matter. At this point,readTimeout
andresponseTimeout
come into effect.
Media CDN makes at most four origin attempts across all origins,
regardless of the maxAttempts
set by each EdgeCacheOrigin
.
Media CDN uses the maxAttemptsTimeout
value from the primary
EdgeCacheOrigin
. The per-attempt timeout values (connectTimeout
,
readTimeout
, and responseTimeout
) are configured for the EdgeCacheOrigin
of each attempt.
The following table describes the timeout fields:
Field | Default | Description |
---|---|---|
connectTimeout | 5 seconds | The maximum amount of time Media CDN can take from
starting the request to the origin until Media CDN determines
whether the response is usable. Practically, The timeout must be a value between 1 second and 15 seconds. |
maxAttemptsTimeout | 15 seconds | The maximum time across all connection attempts to the origin, including failover origins, before returning an error to the client. An HTTP 504 is returned if the timeout is reached before a response is returned. The timeout must be a value between 1 second and 30 seconds. This setting defines the total duration for all origin
connection attempts, including failover origins, in order to cap the
total time clients have to wait for content to start streaming. Only the first
|
readTimeout | 15 seconds | The maximum duration to wait between reads of a single HTTP response.
The |
responseTimeout | 30 seconds | The maximum duration to allow for a response to complete. The timeout must be a value between 1 second and 120 seconds. The duration is measured from the time that the first body bytes are received. If this timeout is reached before the response is complete, the response is truncated and logged. |
Consider the following examples:
Origin A
matches requests to "/segments/" and has amaxAttemptsTimeout
value of5s
, amaxAttempts
value of1
, and afailover_origin
value ofOrigin B
. TheconnectTimeout
value is at its default of5s
. If you attempt to connect toOrigin A
, and it fails within 1 second due to an invalid TLS certificate, you have ~4 seconds left to make a successful connection toOrigin B
.Origin C
matches requests to "/manifests/*, has amaxAttemptsTimeout
value of10s
, amaxAttempts
value of3
, andfailover_origin
not configured. TheconnectTimeout
value is at its default of5s
. Media CDN attempts to connect toOrigin C
up to three times, allowing up to 10 seconds (themaxAttemptsTimeout
limit) to make a successful connection.
Retry origin requests
Media CDN supports origin retries, allowing unsuccessful requests to the origin to be retried. You can specify how many times the current origin may be retried against before a failover origin (if configured) is attempted.
- Media CDN attempts to reach the primary origin up to the configured
maxAttempts
value, which defaults to1
. - Media CDN retries origin connections up to a maximum of three
times before failing and returning an HTTP 502 (
Bad Gateway
) error to the user. This includes any failover origin connections, which count towards the limit of three. - When configuring an origin resource, you can configure a primary origin by
using the
originAddress
field, and then an optionalfailoverOrigin
. ThefailoverOrigin
points at another origin resource.
The retryConditions
for each origin specifies what types of failures
triggers a retry:
Condition | Default | Description |
---|---|---|
CONNECT_FAILURE | ✔️ | Retry on failures include routing, DNS and TLS handshake errors, as well as TCP/UDP timeouts. |
HTTP_5XX | Retry on any HTTP 5xx response code. | |
GATEWAY_ERROR | Similar to 5xx, but only applies to response codes 502, 503 or 504. | |
RETRIABLE_4XX | Retry for retriable 4xx response codes, including 409 and 429. | |
NOT_FOUND | Retry on an HTTP 404 response code. | |
FORBIDDEN | Retry on an HTTP 403 response code. |
If you require active health-checking, round-robin, or load-aware steering across origins, you can configure an external Application Load Balancer as the primary origin.
Failover behavior
The following table describes how failover operates, and what response a client would observe:
Scenario | Failover configured | User-facing status |
---|---|---|
Media CDN attempts to connect to your origin, and receives no HTTP response after two attempts (default). | No | HTTP 502 (Bad Gateway) |
Media CDN attempts to connect to your primary origin, and fails to connect (TLS handshake error). An attempt is made against your configured failover origin, which returns an HTTP 404 error. | Yes | HTTP 404 (Not Found) |
Media CDN attempts to connect to both your primary and failover origin, but receives no HTTP status code. | Yes | HTTP 502 (Bad Gateway) |
If Media CDN receives a status code matching any configured retryConditions
, such as an HTTP 404 (Not
Found
) or HTTP 429 (Too Many Requests
) error, and subsequent retry and failover origin requests continue to fail, an
HTTP 502 (Bad Gateway
) is returned to the client after origin attempts are exhausted.
Example: Failover without redirect following
The following is a basic failover EdgeCacheOrigin
configuration:
name: FAILOVER_ORIGIN
originAddress: FAILOVER_DOMAIN_NAME
Media CDN retries the route's primary origin a maximum of three times
before attempting a failover origin. In this configuration, after trying the
primary origin three times, Media CDN attempts a single request
against the FAILOVER_ORIGIN
. If the failover origin
also fails to respond successfully, then Media CDN returns either
the entire origin response or, if no status code is received, an HTTP 502 (Bad
Gateway
) response.
Cache fill latency increases with the number of retries and failover events.
Increasing the origin timeout values (such as connectTimeout
) further
impacts cache fill latency because it increases the time spent waiting for an
overloaded or busy origin server to respond.
The following example demonstrates a configuration that sends fill requests to
MY_ORIGIN
. The configuration causes
Media CDN to retry on connection errors (such as DNS, TCP, or
TLS errors), HTTP 5xx responses from the origin, or an HTTP
404 (Not Found
). After two attempts, it fails over to
FAILOVER_ORIGIN
.
A total maximum of four attempts are made across your configured origins: the
original attempt plus up to three retry attempts. You can configure a maxAttempts
value
per origin to determine how many retries are made before attempting to fail
over.
name: MY_ORIGIN
originAddress: DOMAIN_NAME
maxAttempts: 2 # the number of attempts to make before trying the failoverOrigin
failoverOrigin: FAILOVER_ORIGIN
# what conditions trigger a retry or failover
retryConditions:
- CONNECT_FAILURE
- HTTP_5xx # any HTTP 5xx response
- NOT_FOUND # retry on a HTTP 404
timeout:
maxAttemptsTimeout: 10s # set a deadline for all retries and failover
If your origin requires an origin-specific host rewrite or other
origin-specific header modification, use the following originOverrideAction
configuration
example to set them:
name: FAILOVER_ORIGIN
originAddress: "FAILOVER_ORIGIN_HOST"
originOverrideAction:
urlRewrite:
hostRewrite: "FAILOVER_ORIGIN_HOST"
headerAction:
requestHeadersToAdd:
- headerName: "Authorization"
headerValue: "AUTH-KEY"
replace: true
The following is a completed configuration:
name: MY_ORIGIN
originAddress: DOMAIN_NAME
maxAttempts: 2 # the number of attempts to make before trying the failoverOrigin
failoverOrigin: FAILOVER_ORIGIN
# what conditions trigger a retry or failover
retryConditions:
- CONNECT_FAILURE
- HTTP_5xx # any HTTP 5xx response
- NOT_FOUND # retry on a HTTP 404
timeout:
maxAttemptsTimeout: 10s # set a deadline for all retries and failover
name: FAILOVER_ORIGIN
originAddress: "FAILOVER_ORIGIN_HOST"
originOverrideAction:
urlRewrite:
hostRewrite: "FAILOVER_ORIGIN_HOST"
headerAction:
requestHeadersToAdd:
- headerName: "Authorization"
headerValue: "AUTH-KEY"
replace: true
In the previous example, the originOverrideAction.hostRewrite
setting takes
precedence over any existing header
rewrites configured on routes that point to
this origin.
You can use requestHeadersToAdd
unique, per-origin headers requested by that
particular origin. A common use case adds static Authorization
headers.
Because these header manipulations are run during the origin request, headers
added per origin either replace or append to existing headers of the same field
name. By default, Media CDN appends to existing headers. To
replace existing headers, set headerAction.replace
to true
.
Example: Failover with redirect following
For example, suppose that you have configured the following EdgeCacheOrigin
resources
and that your EdgeCacheService
resource's routes are configured to use the
PrimaryOrigin
for cache fill:
name: PrimaryOrigin
originAddress: "primary.example.com"
maxAttempts: 2
failoverOrigin: "SecondaryOrigin"
retryConditions: [CONNECT_FAILURE]
originRedirect:
redirectConditions: [FOUND, TEMPORARY_REDIRECT]
name: SecondaryOrigin
originAddress: "secondary.example.com"
maxAttempts: 3
originRedirect:
redirectConditions: [FOUND, TEMPORARY_REDIRECT]
In this example, when Media CDN performs a cache fill,
Media CDN reads the configuration of the PrimaryOrigin
and
responds accordingly.
Suppose that Media CDN connects to primary.example.com
as
attempt #1 to contact the origin. If primary.example.com
returns a successful
response, Media CDN uses that response for the cache fill.
Suppose now that primary.example.com
returns an HTTP 302 Found Redirect
to
Location: b.example.com
. Then as attempt #2 to contact the origin,
Media CDN follows the redirect to b.example.com
. In this case,
Media CDN does the following:
- If
b.example.com
returns a successful response, Media CDN uses that response for the cache fill. - If
b.example.com
returns a redirect or a failure response, Media CDN fails over to the configuredSecondaryOrigin
. This is because in this example thePrimaryOrigin
is configured for twomaxAttempts
.
If Media CDN fails over to SecondaryOrigin
,
Media CDN uses the SecondaryOrigin
configuration and attempts
to connect to secondary.example.com
. This is attempt #1 to contact the origin,
and attempt #3 overall.
In this case, Media CDN does the following:
- If
secondary.example.com
returns a successful response, Media CDN uses that response for the cache fill. - If
secondary.example.com
returns an HTTP 302Found Redirect
toLocation: c.example.com
, Media CDN attempts to contactc.example.com
. In this example, this is attempt #2 forSecondaryOrigin
and attempt #4 overall.
If the attempt to contact c.example.com
returns a successful response,
Media CDN uses that response for the cache fill. If the attempt
returns a redirect that Media CDN is configured to follow,
Media CDN returns an HTTP 502 (Bad Gateway
) error because it has exhausted
the maximum attempts to contact an origin. Media CDN makes at
most four attempts across all origins, regardless of EdgeCacheOrigin
configurations. Lastly, if Media CDN fails to contact
c.example.com
, Media CDN returns either a 504 Gateway Timeout
or 502 Bad Gateway
response.
If you require health checking, round-robin, or load-aware steering across origins, you can configure an external Application Load Balancer as a primary origin.
Configure origins
The following sections provide examples on how to configure common origin types, including Cloud Storage buckets and external Application Load Balancer origins.
Cloud Storage integration
Media CDN supports Cloud Storage buckets as backends for content. Each service can reference multiple buckets by configuring routes for host, paths and other request attributes.
Cloud Storage buckets are configured by using the bucket URL, such as
gs://my-bucket
, as the origin address when creating an origin resource.
Console
In the Google Cloud console, go to the Media CDN Origins page.
Click Create origin.
Enter a name for the origin. For example:
cloud-storage-origin
.Enter an optional description for the origin.
For the Origin address, choose Select a Google Cloud Storage bucket.
Browse to your Cloud Storage bucket.
For Cloud Storage, retain the default protocol and port settings.
(Optional) Select a failover origin to try in case this origin becomes unreachable. You can update this field later.
Select retry conditions.
For Max attempts, select a maximum number of attempts to fill cache from this origin.
(Optional) Tune the timeouts:
- For Connect timeout, select the maximum duration to wait for the origin connection to be established.
- For Response timeout, select the maximum duration to allow for a response to complete.
- For Read timeout, select the maximum duration to wait between reads of a single HTTP connection or stream.
(Optional) Enter one or more labels for this origin.
Click Create origin.
gcloud
Use the gcloud edge-cache origins create
command:
gcloud edge-cache origins create cloud-storage-origin --origin-address="gs://my-bucket"
Create request issued for: [cloud-storage-origin]
originAddress: "gs://my-bucket"
This is the same whether the bucket is multi-regional, dual-region, or regional.
When configuring a service, you can route your video-on-demand content to one
bucket, and live streaming content to a second bucket: useful if you have
different teams managing each workflow. Similarly, you could route
eu-media.example.com
to a multi-region Cloud Storage bucket located
in the EU to reduce cache fill latency, and route us-media.example.com
(or
match on path, header or query parameter) to a US-based storage bucket.
For cases where write latency is critical, such as low-latency live streaming, you can configure a regional Cloud Storage endpoint as close to your users as possible.
Use an external Application Load Balancer as the origin
If you require active health-checking, round-robin, or load-aware steering across Compute Engine, GKE, or on-premises origins, you can configure an external Application Load Balancer as an origin.
This allows you to configure (for example) your live streaming packagers behind Media CDN or a group of Envoy proxies managed by Traffic Director for connecting back to on-premises infrastructure.
Load balancers allow you to configure backends for:
- Compute Engine VM instance groups.
- Network endpoint groups (including Compute Engine VMs and Google Kubernetes Engine clusters).
- Serverless network endpoint groups (Cloud Run, App Engine and Cloud Functions).
An architecture that combines an external Application Load Balancer origin for serving video manifests and a Cloud Storage origin for segment storage would resemble the following, with two origins mapped to different routes.
To configure an external Application Load Balancer as an origin, you need to create an origin resource with the IP address or public hostname pointing to your load balancer's forwarding rule(s). A public hostname (domain name) is preferred, because this is required for an SSL (TLS) certificate and for modern HTTP versions (HTTP/2 and HTTP/3).
You must also ensure that:
- Your load balancer has a route matching the hostname used for your
EdgeCacheService
resource. - (or) that you have configured a
urlRewrite.hostRewrite
for routes where your load balancer is configured as the origin. - That your load balancer has a publicly trusted SSL (TLS) certificate configured for these hostnames.
For example, if the public domain name pointed at your load balancer's
forwarding rule is origin-packager.example.com
, then you would create an
origin with the originAddress
set to this name:
Console
- Go to the Media CDN page in the Google Cloud console.
Go to Media CDN - Click the Origins tab.
- Click Create origin.
- Enter a name for the origin. For example:
load-balancer-origin
. - Enter an optional description for the origin.
- For the Origin address, choose Specify a FQDN or IP address.
- Enter the FQDN or IP address for your Google Cloud load balancer.
- Select a protocol and a port for Media CDN to connect to your origin.
- (Optional) Select a failover origin to try in case this origin becomes unreachable. You can update this field later.
- Select retry conditions.
- For Max attempts, select a maximum number of attempts to fill cache from this origin.
- (Optional) Tune the timeouts:
- For Connect timeout, select the maximum duration to wait for the origin connection to be established.
- For Response timeout, select the maximum duration to allow for a response to complete.
- For Read timeout, select the maximum duration to wait between reads of a single HTTP connection or stream.
- (Optional) Enter one or more labels for this origin.
- Click Create origin.
gcloud
Use the gcloud edge-cache origins
command:
gcloud edge-cache origins create lb-origin --origin-address="origin-packager.example.com"
Create request issued for: [lb-origin]
originAddress: "origin-packager.example.com"
If you are using the IP address of your forwarding rule as the origin address,
or do not have an SSL certificate attached to your load balancer, you can
set the protocol to HTTP
to fall back to unencrypted connections. We recommend
you only do this for development or testing purposes.
Configure the origin protocol
For origins that only support HTTPS (HTTP/1.1 over TLS) or HTTP/1.1 (without
TLS), set the protocol
field explicitly by doing the following:
Console
- Go to the Media CDN page in the Google Cloud console.
Go to Media CDN - Click the Origins tab.
- Select your origin and click Edit.
- For the protocol, select HTTPS or HTTP.
- If you selected HTTP, select port 80.
- Click Update origin.
gcloud
Use the gcloud edge-cache origins
command:
gcloud edge-cache origins update LEGACY_ORIGIN \ --protocol=HTTPS
The command updates the origin and returns the following:
Update request issued for: [LEGACY_ORIGIN] protocol: HTTPS
If your origin supports HTTP/2, you do not need to explicitly set the protocol.
Troubleshooting origins
If an origin isn't behaving as expected, check how to troubleshoot origins.