Setting up an global external HTTP(S) load balancer (classic) with backend buckets

Backend buckets are Cloud Storage buckets backends for HTTP(S) Load Balancing.

Note: If your backends serve dynamic content over HTTP(S), consider using backend services instead of backend buckets.

Cloud Storage buckets as load balancer backends

An external HTTP(S) load balancer uses a URL map to direct traffic from specified URLs to your backends.

In the following diagram, the load balancer sends traffic with a path of /video/* to a Cloud Storage bucket called videos. All other requests go to a Cloud Storage bucket named website.

Distributing traffic to Cloud Storage (click to enlarge)

This document shows you how to create an external HTTP(S) load balancer to route requests for static content to Cloud Storage buckets. After you configure a load balancer with the backend buckets, requests to URL paths that begin with /love-to-fetch/ are sent to the BUCKET_2_NAME Cloud Storage bucket, and all other requests are sent to the BUCKET_1_NAME` Cloud Storage bucket.

In this example, BUCKET_1_NAME is named cats, and BUCKET_2_NAME is named dogs.

By default, Cloud Storage uses the same cache that Cloud CDN uses. If you enable Cloud CDN on the backend bucket, you can use Cloud CDN controls on your content. Cloud CDN controls include, for example, cache modes, signed URLs, and invalidation. Cloud CDN also lets you cache large content (> 10 MB). If you don't enable Cloud CDN on your backend bucket, you can only use origin Cache-Control headers to control caching for smaller content, as set by the Cloud Storage metadata.

Before you begin

Make sure that your setup meets the prerequisites.

Set a default project

1. If you are using the gcloud or gsutil utilities, you can install them both by using instructions in Quickstart: Using the gsutil Tool document.

2. Set a default project.

   Console

   a. Go to the Home page in the Google Cloud Console.
   Go to the Google Cloud home page
   
   b. To the right of Google Cloud, select a project from the pull-down menu.

   gcloud or gsutil

    gcloud config set project PROJECT_ID
   

   or

    gsutil config set project PROJECT_ID
   

   • <var>PROJECT_ID</var> - the project you will use for this guide.

Permissions

To follow this guide, you need to create Cloud Storage buckets and a load balancer in a project. You should be either a project owner or editor, or you should have the following Compute Engine IAM roles:

Task	Required Role
Create load balancer components	Network Admin
Create Cloud Storage buckets	Storage Object Admin

For more information, see the following guides:

• Access control
• IAM Conditions

Set up an SSL certificate resource

For an HTTPS load balancer, create an SSL certificate resource as described in the following:

• Using self-managed SSL certificates
• Using Google-managed SSL certificates

We recommend using a Google-managed certificate.

This example assumes that you already have an SSL certificate resource named www-ssl-cert.

Warning: Don't use a self-signed certificate for production purposes.

Preparing Cloud Storage buckets and content

The process for preparing the Cloud Storage buckets is as follows:

• Create the buckets.
• Copy content to the buckets.
• Provide public access to the buckets.

Creating Cloud Storage buckets

If you have an existing Cloud Storage bucket, you can skip to the next step.

When you create a bucket to use as the backend for a HTTP(S) Load Balancing, we recommend that you choose a multi-region bucket, which automatically replicates objects across multiple Google Cloud regions. This can improve the availability of your content and improve failure tolerance across your application.

Console

1. In the Cloud Console, open the Cloud Storage browser.
   Open the Cloud Storage browser
2. Click Create bucket.

3. Specify values for the following fields, leaving all others at their defaults:

   Property	Value (type the value or select an option as specified)
   Name	For each bucket, enter a globally unique name. If the name you enter is not unique, you'll see a message to try another name.
   Location type	For this example, select Region. For production deployments, we recommend multi-region buckets.
   Location	For one bucket, select us-east1 For the other bucket, select europe-north1
   Default storage class	Standard
   Access control	Uniform

4. Click Create.

5. Note the names of the newly-created Cloud Storage buckets for the next step, where they are referred to as BUCKET_1_NAME and BUCKET_2_NAME.

gsutil

gsutil mb -p PROJECT_ID -c standard -l us-east1 -b on gs://BUCKET_1_NAME

gsutil mb -p PROJECT_ID -c standard -l europe-north1 -b on gs://BUCKET_2_NAME

Copying graphic files into your Cloud Storage buckets

So that you can test the setup, copy graphic files from a public Cloud Storage bucket to your own Cloud Storage buckets.

1. Run the following command in Cloud Shell, replacing the bucket name variables with your Cloud Storage bucket names:

   gsutil cp gs://gcp-external-http-lb-with-bucket/three-cats.jpg gs://BUCKET_1_NAME/never-fetch/
   

   gsutil cp gs://gcp-external-http-lb-with-bucket/two-dogs.jpg gs://BUCKET_2_NAME/love-to-fetch/
   

2. In the Cloud Console, click Refresh Bucket to verify that the graphic file is copied.

Making the Cloud Storage buckets public

Make the Cloud Storage buckets publicly readable. With this setting, anyone on the internet can view your objects and their metadata, excluding ACLs. Anyone on the internet can also list the objects in your buckets. Don't include sensitive information in your public buckets.

To reduce the likelihood of accidental exposure of sensitive information, don't store public objects and sensitive data in the same bucket. For more information, see Recommended bucket architecture.

Console

Grant all users access to view objects in the bucket. Repeat this procedure for each bucket.

1. In the Cloud Console, open the Cloud Storage browser.
   Open the Cloud Storage browser
2. Navigate to the bucket and click the Permissions tab.
3. Click Add.
4. In New principals, enter allUsers.
5. For the role, select Cloud Storage > Storage Object Viewer.
6. Click ALLOW PUBLIC ACCESS.
7. Click Save.

gsutil

gsutil iam ch allUsers:objectViewer gs://BUCKET_1_NAME

gsutil iam ch allUsers:objectViewer gs://BUCKET_2_NAME

Reserving an external IP address

Now that your buckets are up and running, set up a global static external IP address that your customers use to reach your load balancer.

This step is optional, but recommended, because a static external IP address provides a single address to point your domain at.

Note: You can skip this step and have Google Cloud associate an ephemeral IP address with your load balancer's forwarding rule. An ephemeral IP address remains constant while the forwarding rule exists. If you need to delete the forwarding rule and re-add it, the forwarding rule might receive a new IP address. If needed, you can make an ephemeral IP address static.

Console

1. Go to the External IP addresses page in the Google Cloud Console.
   Go to the External IP addresses page
2. Click Reserve static address to reserve an IPv4 address.
3. Assign a Name of example-ip.
4. Set the Network tier to Premium.
5. Set the IP version to IPv4.
6. Set the Type to Global.
7. Click Reserve.
8. Ensure that the Type is set to Global.
9. Click Reserve.

gcloud

gcloud compute addresses create example-ip \
    --network-tier=PREMIUM \
    --ip-version=IPV4 \
    --global

Note the IPv4 address that was reserved:

gcloud compute addresses describe example-ip \
    --format="get(address)" \
    --global

Creating the HTTP load balancer with backend buckets

If you want to create an HTTPS load balancer, you must add an SSL certificate resource to the load balancer's frontend. For more information, see the SSL certificates overview.

Console

Start the configuration

1. In the Google Cloud Console, go to the Load balancing page.
   Go to Load balancing
2. Under HTTP(S) load balancing, click Start configuration.
3. Select From Internet to my VMs, and then click Continue.
4. Under Advanced traffic management, select Classic HTTP(S) Load Balancer.
5. Click Continue.
6. Set Name to `http-lb`.

Configuring the backend

1. Click Backend configuration.
2. For Backend services & backend buckets, click Create or select backend services & backend buckets, click Backend buckets > Create a backend bucket.
3. Set the Name to cats.
4. Click Browse under Cloud Storage bucket.
5. Select your BUCKET_1_NAME and click Select.
6. Click Create.

Repeat this step to create dogs, and browse to BUCKET_2_NAME.

Configuring host rules and path matchers

Host rules and path matchers are configuration components of an external HTTP(S) load balancer's URL map.

1. Click Host and path rules.
2. For cats, you can't change the default hosts and paths. This means that all unmatched requests go to cats.
3. For dogs, enter * in the Hosts field.
4. For dogs, enter /love-to-fetch/* in the Paths field.

Hosts	Paths	Backend
All unmatched (default)	All unmatched (default)	cats
*	/love-to-fetch/*	dogs

Configuring the frontend

1. Click Frontend configuration.
2. Verify that the following options are configured with these values:

   Property	Value (type a value or select an option as specified)
   Protocol	HTTP
   Network Service Tier	Premium
   IP version	IPv4
   IP address	example-ip
   Port	80

   If you want to create an HTTPS load balancer instead of an HTTP load balancer, you must have an SSL certificategcloud compute ssl-certificates list), and you must fill in the fields as follows:

   Property	Value (type a value or select an option as specified)
   Protocol	HTTP(S)
   Network Service Tier	Premium
   IP version	IPv4
   IP address	example-ip
   Port	443
   Select a certificate or create a new certificate
   (Optional) Enable HTTP to HTTPS Redirect	Use this checkbox to enable redirects from port 80 to port 443. Enabling this checkbox creates an additional partial HTTP load balancer that uses the same IP address as your HTTPS load balancer and redirects HTTP requests to your load balancer's HTTPS frontend. This checkbox can only be selected when the HTTPS protocol is selected and a reserved IP address is used.

3. Click Done.

Reviewing the configuration

1. Click Review and finalize.
2. Review the Backend Buckets, Host and Path rules, and Frontend.
3. Click Create.
4. Wait for the load balancer to be created.
5. Click the name of the load balancer (http-lb).
6. Note the IP address of the load balancer for the next task. It's referred to as IP_ADDRESS.

gcloud

Configuring the backend

    gcloud compute backend-buckets create cats \
        --load-balancing-scheme=EXTERNAL \
        --gcs-bucket-name=BUCKET_1_NAME
    

    gcloud compute backend-buckets create dogs \
        --load-balancing-scheme=EXTERNAL \
        --gcs-bucket-name=BUCKET_2_NAME
    

Configuring the URL map

    gcloud compute url-maps create http-lb \
        --default-backend-bucket=cats
    

    gcloud compute url-maps add-path-matcher http-lb \
        --path-matcher-name=path-matcher-2 \
        --new-hosts=* \
        --backend-bucket-path-rules="/love-to-fetch/*=dogs" \
        --default-backend-bucket=cats
    

Configuring the target proxy

    gcloud compute target-http-proxies create http-lb-proxy \
        --url-map=http-lb
    

Configuring the forwarding rule

    gcloud compute forwarding-rules create http-lb-forwarding-rule \
        --load-balancing-scheme=EXTERNAL \
        --network-tier=PREMIUM \
        --address=example-ip \
        --global \
        --target-http-proxy=http-lb-proxy \
        --ports=80
    

Sending traffic to your backend bucket

Several minutes after you have configured your load balancer, you can start sending traffic to the load balancer's IP address.

Console

1. Go to the Load balancing page in the Google Cloud Console.
   Go to the Load balancing page
2. Click http-lb to expand the load balancer that you just created.

3. You can test your load balancer using a web browser by going to the following URLs, where IP_ADDRESS is the load balancer's IP address. Your browser should render pages with content showing the graphic files:

   • http://IP_ADDRESS/love-to-fetch/two-dogs.jpg
   • http://IP_ADDRESS/never-fetch/three-cats.jpg

gcloud

Use the curl command to test the response from the URL. Replace IP_ADDRESS with the load balancer's IPv4 address:

curl http://IP_ADDRESS/love-to-fetch/two-dogs.jpg

curl http://IP_ADDRESS/never-fetch/three-cats.jpg

Limitations

• Backend buckets are only supported with global external HTTP(S) load balancers and global external HTTP(S) load balancer (classic). They aren't supported by regional external HTTP(S) load balancer or any other load balancer type.
• Backend buckets aren't supported with Identity-Aware Proxy.

What's next

• Setting up multi-region load balancing
• Setting up Cloud CDN with a backend bucket
• Using Terraform module examples for external HTTP(S) load balancers