A cacheable response is an HTTP response that Cloud CDN can store and quickly retrieve, thus allowing for faster load times. Not all HTTP responses are cacheable.
Cache modes
With cache modes, you can control the factors that determine whether Cloud CDN caches your content.
Cloud CDN offers three cache modes, which define how responses are cached, whether Cloud CDN respects cache directives sent by the origin, and how cache TTLs are applied.
The available cache modes are shown in the following table:
Cache mode | Behavior |
---|---|
USE_ORIGIN_HEADERS |
Requires origin responses to set valid cache directives and valid
caching headers. Responses without these directives
are forwarded from the origin. This is the default behavior for Cloud CDN-enabled
backends created by using the |
CACHE_ALL_STATIC |
Automatically caches static content that
doesn't have the no-store , private , or
no-cache directive. Origin responses that set valid caching
directives are also cached. |
FORCE_CACHE_ALL |
Unconditionally caches responses, overring any cache directives set by the origin. Make sure not to cache private, per-user content (such as dynamic HTML or API responses) if using a shared backend with this mode configured. |
For setup instructions, see Setting the cache mode.
Static content
Static content is content that is always the same, even when accessed by different users. The CSS that you use to style your site, JavaScript to provide interactivity, video, and image content typically don't change for each user for a given URL (cache key), and thus benefit from being cached across Cloud CDN's global edge network.
When you set the cache mode to cache all static content, Cloud CDN automatically caches responses for the following:
- Web Assets, including CSS (
text/css
), JavaScript (application/javascript
) and all web fonts, including WOFF2 (font/woff2
) - Images, including JPEG (
image/jpg
) and PNG (image/png
) - Videos, including H.264, H.265, and MP4 (
video/mp4
) +. Audio files, including MP3 (image/mpeg
) and MP4 (audio/mp4
) - Formatted documents, including PDF (
application/pdf
)
The following table provides a summary.
Category | MIME types |
---|---|
Web assets | text/css text/ecmascript text/javascript application/javascript |
Fonts | Any Content-Type matching font/* |
Images | Any Content-Type matching image/* |
Videos | Any Content-Type matching video/* |
Audio | Any Content-Type matching audio/* |
Formatted document types | application/pdf and application/postscript |
Cloud CDN inspects the Content-Type
HTTP response header, which
reflects the MIME
type
of the content being served.
Note the following:
Your origin's web server software must set the
Content-Type
for each response. Many web servers automatically set theContent-Type
header, including NGINX, Varnish, and Apache.Cloud Storage sets the
Content-Type
header automatically on upload when you use the Cloud Console or thegsutil
tool to upload content.If a response is cacheable based on its MIME type but has a
Cache-Control
response header ofprivate
,no-store
, orno-cache
, or aSet-Cookie
header (see the full list of rules), it isn't cached.
Cloud CDN doesn't use file extensions in the URL path to determine whether a response is cacheable because many valid cacheable responses aren't reflected in URLs.
If you want to cache text/html
and application/json
content types, you must
set explicit Cache-Control
headers in the response, being
careful not to accidentally cache one user's data and serve it to all users.
Cacheable content
Cloud CDN caches only those responses that meet all the requirements in this section. Some of these requirements are specified by RFC 7234, and others are specific to Cloud CDN.
A response can be stored in Cloud CDN caches only if all of the following are true.
Attribute | Requirement |
---|---|
Served by | Backend service, backend bucket, or a custom origin with Cloud CDN enabled |
In response to | GET request |
Status code | 200 , 203 , 204 , 206 ,
300 , 301 , 302 , 307 ,
308 , 404 , 405 , 410 ,
421 , 451 , or 501 |
Cache directive | With the With the With the |
Freshness | With the With the With the |
Content | Contains a valid For example, a |
Size | Less than or equal to the maximum size |
For Cloud Storage backend buckets, following are several ways to satisfy these requirements:
Make your bucket publicly readable. This is the approach that we recommend for public content. With this setting, anyone on the internet can view and list your objects and their metadata, excluding ACLs. The recommended practice is to dedicate specific buckets for public objects. For more information, see Recommended bucket architecture.
Make the individual objects publicly readable. We don't recommend this approach.
By default, when the entire bucket is public, or the individual objects are
public and the individual objects don't specify Cache-Control
metadata,
Cloud Storage assigns a Cache-Control: public, max-age=3600
header to
the object. You can set different values by using
Cache-Control
metadata.
For an example that shows how to configure an external HTTP(S) load balancer with a backend bucket, see Setting up Cloud CDN with a backend bucket.
Maximum size
Cloud CDN enforces a maximum size for each response. Any response with a body larger than the maximum size is not cached but is still delivered to the client.
The maximum size varies depending on whether the origin server supports byte range requests.
Origin server supports byte range requests | Origin server does not support byte range requests |
---|---|
5 TB (5,497,558,138,880 bytes) | 10 MB (10,485,760 bytes) |
Nearly all modern web servers (including NGINX, Apache, and Varnish) support byte range requests.
Non-cacheable content based on origin headers
There are checks that block caching of responses.
A response isn't cached if it does not meet the requirements for Cacheable content, or if any of the following is true.
Attribute | Requirement |
---|---|
Served by | Backend service, backend bucket, or a custom origin that doesn't have Cloud CDN enabled |
Cookie | Has a Set-Cookie header |
Vary header |
Has a value other than Accept ,
Accept-Encoding , or Origin
|
Response directive | Response has a Cache-Control header with
no-store , no-cache , or private
directive (unless using the FORCE_CACHE_ALL cache mode, in
which case the Cache-Control header is ignored) |
Request directive | Request has a Cache-Control: no-store header |
Size | Larger than the maximum size |
These requirements might be relaxed in the future, allowing Cloud CDN to cache additional responses.
If Cache-Control: no-store
, no-cache
, or private
is present, but the
content is still being cached, this is due to one of the following:
- URL signing is configured.
- The Cloud CDN cache mode is set to force caching of all responses.
Preventing caching
To prevent private information from being cached in Cloud CDN caches, do the following:
- Make sure that Cloud CDN cache mode isn't set to the
FORCE_CACHE_ALL
mode, which unconditionally caches all responses. - Include a
Cache-Control: private
header in responses that should not be stored in Cloud CDN caches, or aCache-Control: no-store
header in responses that should not be stored in any cache, even a web browser's cache. - Do not sign URLs that provide access to private
information. When content is accessed by using a signed URL, it is potentially
eligible for caching regardless of any
Cache-Control
directives in the response.
Adding custom response headers
With custom response headers, you can specify headers that the external HTTP(S) load balancer adds to proxied responses. Custom response headers let you reflect the cache status to your clients, client geographic data, and your own static response headers.
For the list of headers, see Variables that can appear in the header value.
For instructions, see Working with custom response headers.
Cache keys
Each cache entry in a Cloud CDN cache is identified by a cache key. When a request comes into the cache, the cache converts the URI of the request into a cache key, and then compares it with keys of cached entries. If it finds a match, the cache returns the object associated with that key.
For backend services, Cloud CDN
defaults to using the complete request URI as the cache key. For example,
https://example.com/images/cat.jpg
is the complete URI for a particular
request for the cat.jpg
object. This string is used as the default cache key.
Only requests with this exact string match. Requests for
http://example.com/images/cat.jpg
or
https://example.com/images/cat.jpg?user=user1
do not match.
You can change which parts of the URI are used in the cache key. While the filename and path must always be part of the key, you can include or omit any combination of protocol, host, or query string when customizing your cache key. Using cache keys describes how to customize your cache keys.
URI part | Customization | Example URLs that have the same cache key |
---|---|---|
Protocol | Omit the protocol from the cache key. |
|
Host | Omit the host from the cache key. |
|
Query string | Omit the query string from the cache key. Selectively omit or include portions of the query string. |
|
In addition to including or omitting the entire query string, you can use portions of the query string by using include lists and exclude lists.
For backend buckets, the cache key consists of the URI without the protocol, host, or query string.
Thus, for a given backend bucket, the following URIs resolve to the same cached object:
http://example.com/images/cat.jpg
https://example.com/images/cat.jpg
https://example.com/images/cat.jpg?user=user1
http://example.com/images/cat.jpg?user=user1
https://example.com/images/cat.jpg?user=user2
https://media.example.com/images/cat.jpg
https://www.example.com/images/cat.jpg
Query string include list
You can selectively control which query string parameters Cloud CDN
incorporates into cache keys. For example, if you create an include list of
user
, thenhttps://example.com/images/cat.jpg?user=user1&color=blue
creates a cache key of https://example.com/images/cat.jpg?user=user1
that
also matches https://example.com/images/cat.jpg?user=user1&color=red
.
To use this option, you must include the query string, specify a non-empty include list, and not specify an exclude list.
Query string exclude list
You can selectively control which query string parameters Cloud CDN
ignores by using an exclude list. For example, if you create an exclude list of user
,
all query string parameters except user
are used in the cache key.
With the exclude list configured and an input of
https://example.com/images/cat.jpg?user=user1&color=blue
, Cloud CDN
creates a cache key of https://example.com/images/cat.jpg?color=blue
that also
matches https://example.com/images/cat.jpg?user=user2&color=blue
but not
https://example.com/images/cat.jpg?user=user1&color=red
.
To use this option, you must include the query string, specify a non-empty exclude list, and not specify an include list.
Vary
headers
The Vary
header indicates that the response varies depending on the client's
request headers. In addition to the request URI, Cloud CDN respects
Vary
headers that origin servers include in responses. For example, if a
response specifies Vary: Accept
, Cloud CDN uses one cache entry for
requests that specify Accept: image/webp,image/*,*/*;q=0.8
and another for
requests that specify Accept: */*
.
The table in the
Non-cacheable content section lists
the Vary
headers that allow content to be cached. Other Vary
header values
prevent content from being cached.
The FORCE_CACHE_ALL
cache mode does not override this behavior. The Vary
headers are important to avoid cache poisoning between multiple possible origin
server responses. It would be dangerous for FORCE_CACHE_ALL
to cause those
responses to be cached.
Vary
headers are sometimes used when serving compressed content.
Cloud CDN does not compress or decompress responses itself, but it can
serve responses that the origin server compressed. If your origin server chooses
whether to serve compressed or uncompressed content based on the value of the
Accept-Encoding
request header, make sure that the response specifies
Vary: Accept-Encoding
.
Expiration times and validation requests
With the USE_ORIGIN_HEADERS
cache mode, each cache entry in a
Cloud CDN cache has an expiration time defined by the Cache-Control:
s-maxage
, Cache-Control: max-age
, and/or Expires
headers in accordance with RFC 7234.
If more than one is present, Cache-Control: s-maxage
takes precedence over
Cache-Control: max-age
, and Cache-Control: max-age
takes precedence over
Expires
.
With the CACHE_ALL_STATIC
cache mode, any backend response carrying
static content is considered fresh, or for non-static content, freshness is
based on origin headers.
With the FORCE_CACHE_ALL
cache mode, all backend responses are
considered fresh.
When Cloud CDN receives a request, it looks up the corresponding cache entry and checks its age. If the cache entry exists and is fresh enough, the response can be served from the cache. However, if the expiration time has passed, Cloud CDN cannot serve a response without first contacting one of your backends.
Cloud CDN revalidates cached objects that are older than 30 days.
This allows for automatic invalidation and eviction of stale, user-generated
cached content. When a max-age
or s-maxage
value exceeds 30 days (2,592,000
seconds), Cloud CDN treats the value as if it were 2,592,000
seconds. Downstream clients still see the accurate values of max-age
and s-
maxage
, even if they exceed 30 days.
Cloud CDN can attempt to use the information in the cached response headers to validate the cache entry with the backend. This happens when both of the following are true:
- The previously cached response has a
Last-Modified
orETag
header. - The client request is a
GET
request that does not containIf-Modified-Since
orIf-None-Match
headers.
Cloud CDN performs this validation slightly differently depending on whether the response was cached by using byte range requests:
- If the response was cached by using byte range requests, Cloud CDN
initiates a separate validation request that includes
If-Modified-Since
and/orIf-None-Match
headers. - Otherwise, Cloud CDN adds
If-Modified-Since
and/orIf-None-Match
headers to the client request and forwards the modified request to the backend.
If the cached copy is still up to date, the backend can validate the existing
cache entry by sending a 304 Not Modified
response. In this case, the backend
sends only the response headers, not the response body. Cloud CDN
inserts the new response headers into the cache, updates the expiration time,
and serves the new response headers and cached response body to the client.
If the previously cached response does not have a Last-Modified
or an ETag
header, Cloud CDN ignores the expired cache entry and forwards
the client request to the backend unmodified.
A cache entry's expiration time is an upper bound on how long the cache entry remains valid. There is no guarantee that a cache entry remains in the cache until it expires. Cache entries for unpopular content can be evicted to make room for new content. Regardless of the specified expiration time, cache entries that aren't accessed for 30 days are automatically removed.
For more information, see Eviction and expiration.
TTL settings and overrides
You can fine-tune Cloud CDN behavior with regard to how long Cloud CDN waits before revalidating content.
For more information, see Using TTL settings and overrides.
Support for byte range requests
A response that satisfies the following criteria indicates that the origin server supports byte range requests:
- Status code:
200 OK
or206 Partial Content
- Header:
Accept-Ranges: bytes
- Header:
Content-Length
and/orContent-Range
- Header:
ETag
with a strong validator - Header:
Last-Modified
Cloud Storage supports byte range requests for most objects. However,
Cloud Storage does not support byte range requests for objects with
Content-Encoding: gzip
metadata unless the client request includes an Accept-
Encoding: gzip
header. If you have Cloud Storage objects larger than
10 MB, make sure that they do not have Content-Encoding: gzip
metadata. For information about how to edit object metadata, see Viewing and
editing object metadata.
Popular web server software also supports byte range requests. Consult your web server software's documentation for details about how to enable support. For more information about byte range requests, see the HTTP specification.
When an origin server supports byte range requests, a Cloud CDN cache declines to store an otherwise cacheable response the first time it is requested if either of the following is true:
- The response body is incomplete because the client requested only part of the content.
- The response body is larger than 1 MB (1,048,576 bytes).
When this happens and the response would otherwise satisfy the normal cacheability requirements, the cache records that the origin server supports byte range requests for that cache key and forwards the origin server's response to the client.
On a cache miss, the cache checks whether the origin server is known to support
byte range requests. If byte range requests are known to be supported for the
cache key, the cache does not forward the client request to the external HTTP(S) load balancer.
Instead, the cache initiates its own byte range cache fill
requests for the missing parts of the content. If your origin server returns
the requested byte range in a 206 Partial Content
response, the cache can
store that range for future requests.
A cache stores a 206 Partial Content
response only when it is received in
response to a byte range request that the cache initiated. Because a cache
doesn't initiate a byte range request unless it had previously recorded that the
origin server supports byte range requests for that cache key, a given cache
doesn't store content that's larger than 1 MB until the second time that
content is accessed.
Requests initiated by Cloud CDN
When your origin server supports byte range requests, Cloud CDN can send multiple requests to your origin server in reaction to a single client request. Cloud CDN can initiate two types of requests: validation requests and byte range requests.
If the response that indicated that your origin server supported byte range requests
for a particular cache key has expired, Cloud CDN initiates
a validation request to confirm that the content hasn't changed and that your
origin server still supports range requests for the content. If your origin
server responds with a 304 Not Modified
response, Cloud CDN
proceeds to serve the content by using byte ranges. Otherwise,
Cloud CDN forwards your origin server's response to the
client. You control expiration times by using the Cache-Control
and Expires
response headers.
On a cache miss, Cloud CDN initiates cache fill requests for a set of byte ranges that overlap the client request. If some ranges of the content requested by the client are present in the cache, Cloud CDN serves whatever it can from the cache and sends byte range requests for only the missing ranges to your origin server.
Each byte range request initiated by Cloud CDN specifies a range that begins at an offset that's a multiple of 2,097,136 bytes. With the possible exception of the final range, each range is also 2,097,136 bytes. If the content isn't a multiple of that size, the final range is smaller. The size and offsets used in byte range requests might change in the future.
As an example, consider a client request for bytes 1,000,000 through 3,999,999 of content that is not present in the cache. In this example, Cloud CDN could initiate two GET requests, one for the first 2,097,136 bytes of the content and another for the second 2,097,136 bytes. This results in 4,194,272 bytes of cache fill even though the client requested only 3,000,000 bytes.
When you use a Cloud Storage bucket as your origin, each GET request is billed as a separate Class B operation. You are charged for all GET requests processed by Cloud Storage, including any requests initiated by Cloud CDN. When a response is served entirely from a Cloud CDN cache, no GET requests are sent to Cloud Storage, and you are not charged for any Cloud Storage operations.
When Cloud CDN initiates a validation request or byte range
request, it does not include client-specific headers such as Cookie
or
User-Agent
.
What's next
- To understand how cache modes make it easier to cache content, see Using cache modes.
- To enable Cloud CDN for your HTTP(S) load balanced instances and storage buckets, see Using Cloud CDN.
- To learn about invalidating caches, see Cache invalidation overview.
- To find GFE points of presence, see Cache locations.