Setting up Cloud CDN with a backend bucket

Cloud CDN leverages Google Cloud global external HTTP(S) load balancers to provide routing, health checking, and Anycast IP support. Because global external HTTP(S) load balancers can have multiple backend instance types—Compute Engine VM instances, Google Kubernetes Engine Pods, Cloud Storage buckets, or external origins outside of Google Cloud—you can choose which backends (origins) to enable Cloud CDN for.

This setup guide shows you how to create a simple external HTTP(S) load balancer with Cloud CDN enabled. The example uses the following resources:

  • The default Virtual Private Cloud (VPC) network
  • A default URL map
  • A reserved external IP address
  • A Cloud Storage bucket as the backend
  • A single load balancer backend bucket that acts as a wrapper around the Cloud Storage bucket

A backend bucket supports the following:

  • Cloud Storage buckets of any storage class, including multi-region buckets
  • Cloud CDN policies for caching content at Google's global edge

To learn how Cloud CDN works, see the Cloud CDN overview.

Load balancer backends

An external HTTP(S) load balancer uses a URL map to direct traffic from specified URLs to specified services. The following table summarizes the types of backends where you can host content and services.

Load balancer backend configuration Typical content type Backend types
Backend service Dynamic (such as data)
  • Unmanaged instance groups
  • Managed instance groups
  • Network endpoint groups internal to Google Cloud
  • Network endpoint groups external to Google Cloud
Backend bucket Static (such as images)
  • Cloud Storage buckets (discussed on this page)

Before you begin

  • If you are using the gcloud or gsutil utilities, see Quickstart: Using the gsutil tool to install them.
  • Set a default project.

    Console

    1. In the Google Cloud Console, go to the Home page.

      Go to the Google Cloud home page

    2. To the right of Google Cloud, select a project from the pull-down menu.

    gcloud or gsutil

     gcloud config set project PROJECT_ID
    

    or

     gsutil config set project PROJECT_ID
    
    • Replace PROJECT_ID with the project you will use for this guide.

Creating a Cloud Storage bucket

If you have an existing Cloud Storage bucket that isn't already assigned to a load balancer, you can skip to the next step.

When you create a Cloud Storage bucket to use as the backend for an external HTTP(S) load balancer with Cloud CDN, we recommend that you choose a multi-region bucket, which automatically replicates objects across multiple Google Cloud regions. This can improve the availability of your content and improve failure tolerance across your application.

Console

  1. In the Google Cloud Console, open the Cloud Storage browser.

    Open the Storage browser

  2. Click Create bucket.
  3. Specify values for the fields in the following table, leaving all others at their defaults.

    Property Value (type the value or select an option as specified)
    Name For each bucket, enter a globally unique name. If the name you enter is not unique, you see a message to try another name.
    Location type Multi-region
    Location Select a region, such as us (multiple regions in United States).
    Default storage class Standard
    Access control Uniform
  4. Click Create.

  5. Note the name of the newly-created Cloud Storage bucket for the next step.

gsutil

gsutil mb -p PROJECT_ID -c standard -l us-east1 -b on gs://BUCKET_NAME

Copying a graphic file into your Cloud Storage bucket

To enable you to test the setup, copy a graphic file from a public Cloud Storage bucket to your own Cloud Storage bucket.

  1. Run the following command in Cloud Shell. Replace BUCKET_NAME with your unique Cloud Storage bucket name:

    gsutil cp gs://gcp-external-http-lb-with-bucket/three-cats.jpg gs://BUCKET_NAME/static/us/
    
  2. In the Cloud Console, click Refresh to verify that the graphic file is copied.

Making your Cloud Storage bucket public

This example makes your Cloud Storage bucket publicly readable. This is the recommended approach for public content. With this setting, anyone on the internet can view and list your objects and their metadata, excluding ACLs. The recommended practice is to dedicate specific Cloud Storage buckets for public objects. For more information, see Recommended bucket architecture.

The following are alternatives to making your Cloud Storage buckets public:

The following procedure grants all users access to view objects in your Cloud Storage bucket, making the bucket publicly readable.

Console

  1. In the Google Cloud Console, open the Cloud Storage browser.

    Open the Storage browser

  2. Navigate to the bucket and click the Permissions tab.
  3. Click Add members.
  4. In New members, enter allUsers.
  5. For the role, select Cloud Storage > Storage Object Viewer.
  6. Click Save.

gsutil

gsutil iam ch allUsers:objectViewer gs://BUCKET_NAME

Reserving an external IP address

Now that your Cloud Storage bucket is up and running, set up a global static external IP address that your customers use to reach your load balancer.

This step is optional, but recommended, because a static external IP address provides a single address to point your domain at.

Console

  1. In the Google Cloud Console, go to the External IP addresses page.

    Go to the External IP addresses page

  2. To reserve an IPv4 address, click Reserve static address.
  3. Assign a Name of example-ip.
  4. Set the Network Service Tier to Premium.
  5. Set the IP version to IPv4.
  6. Set the Type to Global.
  7. Click Reserve.

gcloud

gcloud compute addresses create example-ip \
    --network-tier=PREMIUM \
    --ip-version=IPV4 \
    --global

Note the IPv4 address that was reserved:

gcloud compute addresses describe example-ip \
    --format="get(address)" \
    --global

Creating the external HTTP(S) load balancer

If you want to create an HTTPS load balancer, you must have an SSL certificate resource that you can add to the load balancer's front end. For more information, see the SSL certificates overview.

Console

Start the external HTTP(S) load balancer configuration process

  1. In the Google Cloud Console, go to the Load balancing page.

    Go to the Load balancing page

  2. Under HTTP(S) load balancing, click Start configuration.
  3. Select From Internet to my VMs, and then click Continue.
  4. Set Name to http-lb, and then go to the next step.

Configure the backend and enable Cloud CDN

Create the load balancer's backend bucket, which serves as a wrapper for your Cloud Storage bucket. When creating or editing a backend bucket, you can enable Cloud CDN.

  1. Click Backend configuration.
  2. Under Backend services & backend buckets, click Create or select backend services & backend buckets, and then click Backend buckets > Create a backend bucket.
  3. Set the Name to backend-bucket1. This name doesn't need to be globally unique and can be different from the name of the actual Cloud Storage bucket.
  4. Under Cloud Storage bucket, click Browse.
  5. Select the Cloud Storage globally unique BUCKET_NAME that you created, and then click Select.
  6. Click Enable Cloud CDN.

  7. Click Create.

Configure host rules and path matchers

In Host and path rules, you can retain the default settings.

For a customized setup example, see Adding backend buckets to load balancers.

To learn more about host rules and path matchers, see URL maps overview.

Configure the frontend

  1. Click Frontend configuration.
  2. Verify that the options in the following table are configured with these values.

    Property Value (type a value or select an option as specified)
    Protocol HTTP
    Network Service Tier Premium
    IP version IPv4
    IP address example-ip
    Port 80

    If you want to create an HTTPS load balancer instead of an HTTP load balancer, you must have an SSL certificate (gcloud compute ssl-certificates list), and you must fill in the fields as follows.

    Property Value (type a value or select an option as specified)
    Protocol HTTPS
    Network Service Tier Premium
    IP version IPv4
    IP address example-ip
    Port 443
    Certificate Select a certificate or Create a new certificate
  3. Click Done.

Review the configuration

  1. Click Review and finalize.
  2. Review the Backend buckets, Host and path rules, and Frontend sections.
  3. Click Create.
  4. Wait for the load balancer to be created.
  5. Click the name of the load balancer (http-lb).
  6. Note the IP address of the load balancer for the next task. It's referred to as IP_ADDRESS.

Sending traffic to your backend bucket

After creating the global forwarding rule, it can take several minutes for your configuration to propagate worldwide. After several minutes have passed, you can start sending traffic to the load balancer's IP address.

Console

  1. In the Google Cloud Console, go to the Load balancing page.

    Go to the Load balancing page

  2. Click http-lb to expand the load balancer that you just created.

    In the Backend section, confirm that the backend bucket is healthy. There should be a green checkmark next to your backend bucket. If you see otherwise, first try reloading the page. It can take a few moments for the Cloud Console to indicate that backends are healthy.

  3. After the Cloud Console shows that the backend bucket is healthy, you can test your load balancer using a web browser by going to http://IP_ADDRESS/static/us/three-cats.jpg. Replace IP_ADDRESS with the load balancer's IP address. Your browser should render a page with content showing the graphic file.

gcloud

Use the curl command to test the response from the URL. Replace IP_ADDRESS with the load balancer's IPv4 address.

Note the IPv4 address that was reserved:

gcloud compute addresses describe example-ip \
    --format="get(address)" \
    --global

Send a curl request:

curl http://IP_ADDRESS/static/us/three-cats.jpg

Verifying that Cloud CDN is working

If you reload the http://ip-address/static/us/three-cats.jpg page several times in quick succession, there should be several cache hits.

The following log entry shows a cache hit. You can view cache hits by opening the Logs Viewer in the Google Cloud Console and filtering by the forwarding rule name.

Open the Logs Viewer

Logs Viewer

2020-06-08 16:41:30.078 PDT
GET
304
157 B
null
Chrome 83
http://LOAD_BALANCER_IP_ADDRESS/static/us/three-cats.jpg

CLIENT_IP_ADDRESS - "GET http://LOAD_BALANCER_IP_ADDRESS/static/us/three-cats.jpg" 304 157 "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36" Expand all | Collapse all{ httpRequest: { cacheHit: true cacheLookup: true remoteIp: "CLIENT_IP_ADDRESS" requestMethod: "GET" requestSize: "577" requestUrl: "http://LOAD_BALANCER_IP_ADDRESS/static/us/three-cats.jpg" responseSize: "157" status: 304 userAgent: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36" } insertId: "1oek5rg3l3fxj7" jsonPayload: { @type: "type.googleapis.com/google.cloud.loadbalancing.type.LoadBalancerLogEntry" cacheId: "SFO-fbae48ad" statusDetails: "response_from_cache" } logName: "projects/PROJECT_ID/logs/requests" receiveTimestamp: "2020-06-08T23:41:30.588272510Z" resource: { labels: { backend_service_name: "" forwarding_rule_name: "http-lb-forwarding-rule" project_id: "PROJECT_ID" target_proxy_name: "http-lb-target-proxy" url_map_name: "http-lb" zone: "global" } type: "http_load_balancer" } severity: "INFO" spanId: "7b6537d3672e08e1" timestamp: "2020-06-08T23:41:30.078651Z" trace: "projects/PROJECT_ID/traces/241d69833e64b3bf83fabac8c873d992" }

Disabling Cloud CDN

Console

Disable Cloud CDN for a single backend bucket

  1. In the Google Cloud Console, go to the Cloud CDN page.

    Go to the Cloud CDN page

  2. On the right side of the origin row, click Menu and then select Edit.
  3. Clear the checkboxes of any backend buckets that you want to stop from using Cloud CDN.
  4. Click Update.

Remove Cloud CDN for all backend buckets for an origin

  1. In the Cloud Console, go to the Cloud CDN page.

    Go to the Cloud CDN page

  2. On the right side of the origin row, click Menu and then select Remove.
  3. To confirm, click Remove.

gcloud

gcloud compute backend-buckets update BACKEND_BUCKET_NAME \
    --no-enable-cdn

Disabling Cloud CDN does not invalidate or purge caches. If you turn Cloud CDN off and back on again, most or all of your cached content might still be cached. To prevent content from being used by the caches, you must invalidate that content.

What's next