Setting up Cloud CDN with a managed instance group

Cloud CDN leverages Google Cloud global external HTTP(S) load balancers to provide routing, health checking, and Anycast IP support. Global external HTTP(S) load balancers can have multiple backend instance types, and you can choose which backends (or origins) to enable Cloud CDN for.

This setup guide shows you how to create an external HTTP(S) load balancer and a Compute Engine backend with Cloud CDN enabled. The load balancer has the following resources:

This setup guide shows you how to create an external HTTP(S) load balancer and a Compute Engine backend with the following resources:

If this example doesn't match your use case, see one of the following pages:

For general concepts, see the External HTTP(S) Load Balancing overview.

Load balancer topologies

For an HTTPS load balancer, you create the configuration that is illustrated in the following diagram.

HTTPS load balancer with a Compute Engine backend (click to enlarge)
HTTPS load balancer with a Compute Engine backend (click to enlarge)

For an HTTP load balancer, you create the configuration that is illustrated in the following diagram.

HTTP load balancer with a Compute Engine backend (click to enlarge)
HTTP load balancer with a Compute Engine backend (click to enlarge)

The sequence of events in the diagrams are as follows:

  1. A client sends a content request to the external IPv4 address defined in the forwarding rule.

  2. The load balancer checks whether the request can be served from cache. If so, the load balancer serves the requested content out of cache. If not, processing continues.

  3. For an HTTPS load balancer, the forwarding rule directs the request to the target HTTPS proxy.

    For an HTTP load balancer, the forwarding rule directs the request to the target HTTP proxy.

  4. The target proxy uses the rule in the URL map to determine that the single backend service receives all requests.

  5. The load balancer determines that the backend service has only one instance group and directs the request to a virtual machine (VM) instance in that group.

  6. The VM serves the content requested by the user.

HTTP(S) Load Balancing with a Compute Engine backend and Cloud CDN enabled (click to enlarge)
HTTP(S) Load Balancing with a Compute Engine backend and Cloud CDN enabled (click to enlarge)

Before you begin

Make sure that your setup meets the prerequisites.

Set up an SSL certificate resource

For an HTTPS load balancer, create an SSL certificate resource as described in the following:

We recommend using a Google-managed certificate.

This example assumes that you already have an SSL certificate resource named www-ssl-cert.

Set up permissions

To complete the steps in this guide, you must have permission to create Compute Engine instances, firewall rules, and reserved IP addresses in a project. You must have either a project owner or editor role, or you must have the following Compute Engine IAM roles.

Task Required role
Create instances Instance Admin
Add and remove firewall rules Security Admin
Create load balancer components Network Admin
Create a project (optional) Project Creator

For more information, see the following guides:

Creating a managed instance group

To set up a load balancer with a Compute Engine backend, your VMs need to be in an instance group. This guide describes how to create a managed instance group with Linux VMs that have Apache running, and then set up load balancing.

The managed instance group provides VMs running the backend servers of an external HTTP(S) load balancer. For demonstration purposes, backends serve their own hostnames.

Console

  1. In the Google Cloud Console, go to the Instance groups page.

    Go to the Instance groups page

  2. Click Create instance group.
  3. On the left, choose New managed instance group.
  4. For Name, enter lb-backend-example.
  5. Under Location, select Single zone.
  6. For Region, select your preferred region. This example uses us-east1.
  7. For Zone, select us-east1-b.
  8. Under Instance template, select Create a new instance template.
  9. For Name, enter lb-backend-template.
  10. Ensure that the Boot disk is set to a Debian image, such as Debian GNU/Linux 9 (stretch). These instructions use commands that are only available on Debian, such as apt-get.

  11. Do not configure the Firewall option.

    Because the external HTTP(S) load balancer is a proxy, you don't need to select Allow HTTPS traffic or Allow HTTP traffic under Firewall. In Configuring a firewall rule, you create the only necessary firewall rule for this load balancer.

  12. Under Management, security, disks, networking, sole tenancy, on the Management tab, insert the following script into the Startup script field.

    #! /bin/bash
apt-get update
apt-get install apache2 -y
a2ensite default-ssl
a2enmod ssl
vm_hostname="$(curl -H "Metadata-Flavor:Google" \
http://169.254.169.254/computeMetadata/v1/instance/name)"
echo "Page served from: $vm_hostname" | \
tee /var/www/html/index.html

  13. On the Networking tab, add the network tags: allow-health-check

  14. Click Save and continue.

  15. Under Autoscaling mode, select Don't autoscale.

  16. Under Number of instances, enter 2.

  17. To create the new instance group, click Create.

gcloud

  1. Create the template.

    gcloud compute instance-templates create lb-backend-template \
   --region=us-east1 \
   --network=default \
   --subnet=default \
   --tags=allow-health-check \
   --image-family=debian-9 \
   --image-project=debian-cloud \
   --metadata=startup-script='#! /bin/bash
     apt-get update
     apt-get install apache2 -y
     a2ensite default-ssl
     a2enmod ssl
     vm_hostname="$(curl -H "Metadata-Flavor:Google" \
     http://169.254.169.254/computeMetadata/v1/instance/name)"
     echo "Page served from: $vm_hostname" | \
     tee /var/www/html/index.html
     systemctl restart apache2'

  2. Create the managed instance group based on the template.

    gcloud compute instance-groups managed create lb-backend-example \
   --template=lb-backend-template --size=2 --zone=us-east1-b

Adding a named port to the instance group

For your instance group, define an HTTP service and map a port name to the relevant port. Once configured, the load balancing service forwards traffic to the named port.

Console

  1. In the Google Cloud Console, go to the Instance groups page.

    Go to the Instance groups page

  2. Click the name of your instance group (in this example lb-backend-example).
  3. On the instance group's Overview page, click Edit .
  4. Click Specify port name mapping.
  5. Click Add item.
  6. For the port name, enter http. For the port number, enter and 80.
  7. Click Save.

gcloud

Use the gcloud compute instance-groups set-named-ports command.

gcloud compute instance-groups set-named-ports lb-backend-example \
    --named-ports http:80 \
    --zone us-east1-b

Configuring a firewall rule

In this example, you create the fw-allow-health-check firewall rule. This is an ingress rule that allows traffic from the Google Cloud health checking systems (130.211.0.0/22 and 35.191.0.0/16). This example uses the target tag allow-health-check to identify the VMs.

Console

  1. In the Google Cloud Console, go to the Firewall page.

    Go to the Firewall page

  2. Click Create firewall rule to create the second firewall rule.
  3. For Name, enter fw-allow-health-check.
  4. Under Network, select Default.
  5. Under Targets, select Specified target tags.
  6. Populate the Target tags field with allow-health-check.
  7. Set Source filter to IP ranges.
  8. Set Source IP ranges to 130.211.0.0/22 and 35.191.0.0/16.
  9. Under Protocols and ports, select Specified protocols and ports.
  10. Select the tcp checkbox, and then type 80 for the port numbers.
  11. Click Create.

gcloud 

gcloud compute firewall-rules create fw-allow-health-check \
    --network=default \
    --action=allow \
    --direction=ingress \
    --source-ranges=130.211.0.0/22,35.191.0.0/16 \
    --target-tags=allow-health-check \
    --rules=tcp:80

Reserving an external IP address

Now that your instances are up and running, set up a global static external IP address that your customers use to reach your load balancer.

Console

  1. In the Google Cloud Console, go to the External IP addresses page.

    Go to the External IP addresses page

  2. To reserve an IPv4 address, click Reserve static address.
  3. For Name, enter lb-ipv4-1.
  4. Set Network Service Tier to Premium.
  5. Set IP version to IPv4.
  6. Set Type to Global.
  7. Click Reserve.

gcloud 

gcloud compute addresses create lb-ipv4-1 \
    --ip-version=IPV4 \
    --global

Note the IPv4 address that was reserved:

gcloud compute addresses describe lb-ipv4-1 \
    --format="get(address)" \
    --global

Setting up the load balancer

In this example, you are using HTTP or HTTPS between the client and the load balancer. For HTTPS, you need one or more SSL certificate resources to configure the proxy. We recommend using a Google-managed certificate.

Even if you're using HTTPS on the frontend, you can use HTTP on the backend. Google automatically encrypts traffic between Google Front Ends (GFEs) and your backends that reside within Google Cloud VPC networks.

Console

  1. In the Google Cloud Console, go to the Load balancing page.

    Go to the Load balancing page

  2. Click Create load balancer.
  3. Under HTTP(S) Load Balancing, click Start configuration.
  4. Select From Internet to my VMs, and then click Continue.
  5. For the load balancer Name, enter something like web-map-https or web-map-http.
  6. Click Backend configuration.
    1. Under Create or select backend services & backend buckets, select Backend services > Create a backend service.
    2. Add a name for your backend service, such as web-backend-service.
    3. Under Protocol, select HTTP.
    4. For the Named Port, enter http.
    5. In Backends > New backend > Instance group, select your instance group, lb-backend-example.
    6. For the Port numbers, enter 80.
    7. Retain the other default settings.
    8. Under Health check, select Create a health check, and then add a name for your health check, such as http-basic-check.
    9. Set the protocol to HTTP, and then click Save and continue.
    10. Select Enable Cloud CDN.
    11. Retain the other default settings.
    12. Click Create.
  7. In Host and path rules, retain the default settings.
  8. In Frontend configuration, use the following values:
    1. Set Protocol to HTTPS or HTTP.
    2. Set IP address to lb-ipv4-1, which you created earlier.
    3. Ensure that Port is set to 443 to allow HTTPS traffic or 80 for HTTP traffic.
    4. If you selected 443, click the Certificate drop-down list and select your primary SSL certificate.
    5. Click Done.
  9. Click Review and finalize.
  10. When you finish configuring the load balancer, click Create.
  11. Wait for the load balancer to be created.
  12. Click the name of the load balancer.
  13. On the Load balancer details screen, note the IP:Port for your load balancer.

gcloud

  1. Create a health check. 
    gcloud compute health-checks create http http-basic-check \
    --port 80
  2. Create a backend service. 
    gcloud compute backend-services create web-backend-service \
    --protocol=HTTP \
    --port-name=http \
    --health-checks=http-basic-check \
    --global
  3. Add your instance group as the backend to the backend service. 
    gcloud compute backend-services add-backend web-backend-service \
    --instance-group=lb-backend-example \
    --instance-group-zone=us-east1-b \
    --global

Setting up an HTTPS frontend

Skip this section for HTTP load balancers.

  1. For HTTPS, create a URL map to route the incoming requests to the default backend service. 
    gcloud compute url-maps create web-map-https \
    --default-service web-backend-service
  2. For HTTPS, if you haven't already done so, create the global SSL certificate resource, as shown in:

    The following example assumes that you already have a certificate file called certificate-file and a private key file called private-key-file. The example creates an SSL certificate resource called www-ssl-cert.

    gcloud compute ssl-certificates create www-ssl-cert \
    --certificate=certificate-file \
    --private-key=private-key-file \
    --global
  3. For HTTPS, create a target HTTPS proxy to route requests to your URL map. The proxy is the portion of the load balancer that holds the SSL certificate for an HTTPS load balancer, so you also load your certificate in this step. 
    gcloud compute target-https-proxies create https-lb-proxy \
    --url-map=web-map-https \
    --ssl-certificates=www-ssl-cert
  4. For HTTPS, create a global forwarding rule to route incoming requests to the proxy. 
    gcloud compute forwarding-rules create https-content-rule \
    --address=lb-ipv4-1 \
    --global \
    --target-https-proxy=https-lb-proxy \
    --ports=443

Setting up an HTTP frontend

Skip this section for HTTPS load balancers.

  1. For HTTP, create a URL map to route the incoming requests to the default backend service. 
    gcloud compute url-maps create web-map-http \
    --default-service web-backend-service
  2. For HTTP, create a target HTTP proxy to route requests to your URL map. 
    gcloud compute target-http-proxies create http-lb-proxy \
    --url-map=web-map-http
  3. For HTTP, create a global forwarding rule to route incoming requests to the proxy. 
    gcloud compute forwarding-rules create http-content-rule \
    --address=lb-ipv4-1 \
    --global \
    --target-http-proxy=http-lb-proxy \
    --ports=80

Enabling Cloud CDN

If you didn't already enable Cloud CDN when you created your backend service, you can do this now by updating the backend service:

gcloud compute backend-services update web-backend-service \
    --enable-cdn \
    --cache-mode=CACHE_MODE

Replace CACHE_MODE with one of the following:

  • CACHE_All_STATIC: Automatically caches static content. Responses that are marked as uncacheable (private, no-store, or no-cache directives in Cache-Control response headers) aren't cached. To cache dynamic content, the content must have valid caching headers. This is the default behavior for all new Cloud CDN-enabled backends.

  • USE_ORIGIN_HEADERS (default): Requires the origin to set valid caching headers to cache content. Responses without these headers aren't cached at Google's edge and require a full trip to the origin on every request, potentially impacting performance and increasing load on the origin server. This is the default behavior for all existing Cloud CDN-enabled backends.

  • FORCE_CACHE_ALL: Caches all content, ignoring any private, no-store, or no-cache directives in Cache-Control response headers. This might result in caching of private, per-user (user identifiable) content. You should only enable this on backends that are not serving private or dynamic content, such as Cloud Storage buckets.

To automatically cache static responses from your origin, you can use the CACHE_ALL_STATIC cache mode setting.

To control cacheability for each response by using HTTP cache directives, set the cache mode to use origin headers (USE_ORIGIN_HEADERS). For information about the cache directives that Cloud CDN understands and what's not cached by Cloud CDN, see Cacheable content and Non-cacheable content.

If your origin isn't serving any per-user dynamic content, you might want to cache all responses from the origin. To do this, use the FORCE_CACHE_ALL mode. This mode caches all responses, regardless of content type or cache directives.

If you don't explicitly select a cache mode when you enable Cloud CDN on a backend, the API and the gcloud command-line tool default to USE_ORIGIN_HEADERS, and the Cloud Console defaults to CACHE_ALL_STATIC.

Testing traffic sent to your instances

Now that the load balancing service is running, you can send traffic to the forwarding rule and watch the traffic be dispersed to different instances.

Console

  1. In the Google Cloud Console, go to the Load balancing page.

    Go to the Load balancing page

  2. Click the load balancer that you just created.
  3. In the Backend section, confirm that the VMs are healthy. The Healthy column should be populated, indicating that both VMs are healthy (2/2). If you see otherwise, first try reloading the page. It can take a few moments for the Cloud Console to indicate that the VMs are healthy. If the backends do not appear healthy after a few minutes, review the firewall configuration and the network tag assigned to your backend VMs.

  4. For HTTPS, if you are using a Google-managed certificate, confirm that your certificate resource's status is ACTIVE. For more information, see Google-managed SSL certificate resource status.
  5. After the Cloud Console shows that the backend instances are healthy, you can test your load balancer using a web browser by going to https://IP_ADDRESS (or http://IP_ADDRESS). Replace IP_ADDRESS with the load balancer's IP address.
  6. If you used a self-signed certificate for testing HTTPS, your browser displays a warning. You must explicitly instruct your browser to accept a self-signed certificate.

  7. Your browser should render a page with content showing the name of the instance that served the page, along with its zone (for example, Page served from: lb-backend-example-xxxx). If your browser doesn't render this page, review the configuration settings in this guide.

Disabling Cloud CDN

Console

Disable Cloud CDN for a single backend service

  1. In the Google Cloud Console, go to the Cloud CDN page.

    Go to the Cloud CDN page

  2. On the right side of the origin row, click Menu and then select Edit.
  3. Clear the checkboxes of any backend services that you want to stop from using Cloud CDN.
  4. Click Update.

Remove Cloud CDN for all backend services for an origin

  1. In the Cloud Console, go to the Cloud CDN page.

    Go to the Cloud CDN page

  2. On the right side of the origin row, click Menu and then select Remove.
  3. To confirm, click Remove.

gcloud 

gcloud compute backend-services update BACKEND_SERVICE_NAME \
    --no-enable-cdn

Disabling Cloud CDN does not invalidate or purge caches. If you disable and then re-enable Cloud CDN, most or all of your cached content might still be cached. To prevent content from being served from cache, you must invalidate that content.

What's next