Setting up a load balancer with backend buckets

Backend buckets allow you to use Google Cloud Storage buckets with HTTP(S) Load Balancing.

Using a Cloud Storage bucket as a load balancer backend

An external HTTP(S) load balancer uses a URL map to direct traffic from specified URLs to either a backend service or a backend bucket. One common use case is:

• Send requests for dynamic content, such as data, to a backend service.
• Send requests for static content, such as images, to a backend bucket.

In the following diagram, the load balancer sends traffic with a path of /love-to-fetch/ to a Cloud Storage bucket in the europe-north region. All other requests go to a Cloud Storage bucket in the us-east region.

Distributing traffic to Cloud Storage (click to enlarge)

This document shows you how to create an external HTTP(S) load balancer to route requests for static content to Cloud Storage buckets. After you configure a load balancer with the backend buckets, requests to URL paths that begin with /love-to-fetch are sent to the europe-north Cloud Storage bucket, and all other requests are sent to the us-east Cloud Storage bucket.

Note: The external HTTP(S) load balancer doesn't automatically balance traffic across these buckets based on the user's region. Requests to /never-fetch/object always go to your us bucket, and requests to /love-to-fetch/object always go to your bucket in europe-north.

Before you begin

Make sure that your setup meets the prerequisites.

Set a default project

1. If you are using the gcloud or gsutil utilities, you can install them both by using instructions in Quickstart: Using the gsutil Tool document.

2. Set a default project.

   Console

   a. Go to the Home page in the Google Cloud Console.
   Go to the Google Cloud home page
   
   b. To the right of Google Cloud, select a project from the pull-down menu.

   gcloud or gsutil

    gcloud config set project [PROJECT_ID]
   

   or

    gsutil config set project [PROJECT_ID]
   

   • [PROJECT_ID] - the project you will use for this guide.

Permissions

To follow this guide, you need to create Cloud Storage buckets and a load balancer in a project. You should be either a project owner or editor, or you should have the following Compute Engine IAM roles:

Task	Required Role
Create load balancer components	Network Admin
Create Cloud Storage buckets	Storage Object Admin

For more information, see the following guides:

• Access control
• IAM Conditions

Preparing Cloud Storage buckets and content

The process for preparing the Cloud Storage buckets is as follows:

• Create the buckets.
• Copy content to the buckets.
• Provide public access to the buckets.

Creating Cloud Storage buckets

If you have an existing Cloud Storage bucket, you can skip to the next step.

When you create a bucket to use as the backend for a HTTP(S) Load Balancing, we recommend that you choose a multi-region bucket, which automatically replicates objects across multiple Google Cloud regions. This can improve the availability of your content and improve failure tolerance across your application.

Console

1. In the Cloud Console, open the Cloud Storage browser.
   Open the Cloud Storage browser
2. Click Create bucket.

3. Specify values for the following fields, leaving all others at their defaults:

   Property	Value (type the value or select an option as specified)
   Name	For each bucket, enter a globally unique name. If the name you enter is not unique, you'll see a message to try another name.
   Location type	For this example, select Region. For production deployments, we recommend multi-region buckets.
   Location	For one bucket, select us-east1 For the other bucket, select europe-north1
   Default storage class	Standard
   Access control	Uniform

4. Click Create.

5. Note the names of the newly-created Cloud Storage buckets for the next step, where they are referred to as BUCKET_1_NAME and BUCKET_2_NAME.

gsutil

gsutil mb -p PROJECT_ID -c standard -l us-east1 -b on gs://BUCKET_1_NAME

gsutil mb -p PROJECT_ID -c standard -l europe-north1 -b on gs://BUCKET_2_NAME

Copying graphic files into your Cloud Storage buckets

So that you can test the setup, copy graphic files from a public Cloud Storage bucket to your own Cloud Storage buckets.

1. Run the following command in Cloud Shell, replacing the bucket name variables with your Cloud Storage bucket names:

   gsutil cp gs://gcp-external-http-lb-with-bucket/three-cats.jpg gs://BUCKET_1_NAME/never-fetch/
   

   gsutil cp gs://gcp-external-http-lb-with-bucket/two-dogs.jpg gs://BUCKET_2_NAME/love-to-fetch/
   

2. In the Cloud Console, click Refresh Bucket to verify that the graphic file is copied.

Making the Cloud Storage buckets public

Make the Cloud Storage buckets publicly readable. With this setting, anyone on the internet can view your objects and their metadata, excluding ACLs. Anyone on the internet can also list the objects in your buckets. Don't include sensitive information in your public buckets.

To reduce the likelihood of accidental exposure of sensitive information, don't store public objects and sensitive data in the same bucket. For more information, see Recommended bucket architecture.

Console

Grant all users access to view objects in the bucket. Repeat this procedure for each bucket.

1. In the Cloud Console, open the Cloud Storage browser.
   Open the Cloud Storage browser
2. Navigate to the bucket and click the Permissions tab.
3. Click and select Add members.
4. In New members enter allUsers.
5. For the role, select Storage > Storage object viewer.
6. Click Save.

gsutil

gsutil iam ch allUsers:objectViewer gs://BUCKET_1_NAME

gsutil iam ch allUsers:objectViewer gs://BUCKET_2_NAME

Reserving an external IP address

Now that your buckets are up and running, set up a global static external IP address that your customers use to reach your load balancer.

This step is optional, but recommended, because a static external IP address provides a single address to point your domain at.

Note: You can skip this step and have Google Cloud associate an ephemeral IP address with your load balancer's forwarding rule. An ephemeral IP address remains constant while the forwarding rule exists. If you need to delete the forwarding rule and re-add it, the forwarding rule might receive a new IP address.

Console

1. Go to the External IP addresses page in the Google Cloud Console.
   Go to the External IP addresses page
2. Click Reserve static address to reserve an IPv4 address.
3. Assign a Name of example-ip.
4. Set the Network tier to Premium.
5. Set the IP version to IPv4.
6. Set the Type to Global.
7. Click Reserve.
8. Ensure that the Type is set to Global.
9. Click Reserve.

gcloud

gcloud compute addresses create example-ip \
    --network-tier=PREMIUM \
    --ip-version=IPV4 \
    --global

Note the IPv4 address that was reserved:

gcloud compute addresses describe example-ip \
    --format="get(address)" \
    --global

Creating the HTTP load balancer with backend buckets

If you want to create an HTTPS load balancer, you must add an SSL certificate resource to the load balancer's frontend. For more information, see the SSL certificates overview.

Console

1. Go to the Load balancing page in the Google Cloud Console.
   Go to the Load balancing page
2. Under HTTP(S) load balancing, click Start configuration.
3. Select From Internet to my VMs.
4. Set Name to http-lb.

Configuring the backend

1. Click Backend configuration.
2. For Backend services & backend buckets, click Create or select backend services & backend buckets, click Backend buckets > Create a backend bucket.
3. Set the Name to cats.
4. Click Browse under Cloud Storage bucket.
5. Select your BUCKET_1_NAME and click Select.
6. Click Create.

Repeat this step to create dogs, and browse to BUCKET_2_NAME`.

Configuring host rules and path matchers

Host rules and path matchers are configuration components of an external HTTP(S) load balancer's URL map.

1. Click Host and path rules.
2. For cats, you can't change the default hosts and paths. This means that all unmatched requests go to cats.
3. For dogs, enter * in the Hosts field.

4. For dogs, enter /love-to-fetch/* in the Paths field.

   Hosts	Paths	Backend
   All unmatched (default)	All unmatched (default)	cats
   *	/love-to-fetch/*	dogs

Configuring the frontend

1. Click Frontend configuration.

2. Verify that the following options are configured with these values:

   Property	Value (type a value or select an option as specified)
   Protocol	HTTP
   Network Service Tier	Premium
   IP version	IPv4
   IP address	example-ip
   Port	80

   If you want to create an HTTPS load balancer instead of an HTTP load balancer, you must have an SSL certificate (gcloud compute ssl-certificates list), and you must fill in the fields as follows:

   Property	Value (type a value or select an option as specified)
   Protocol	HTTP(S)
   Network Service Tier	Premium
   IP version	IPv4
   IP address	example-ip
   Port	443
   Select a certificate or create a new certificate

3. Click Done.

Reviewing the configuration

1. Click Review and finalize.
2. Review the Backend Buckets, Host and Path rules, and Frontend.
3. Click Create.
4. Wait for the load balancer to be created.
5. Click the name of the load balancer (http-lb).
6. Note the IP address of the load balancer for the next task. It's referred to as IP_ADDRESS.

gcloud

Configuring the backend

gcloud compute backend-buckets create cats \
    --gcs-bucket-name=BUCKET_1_NAME

gcloud compute backend-buckets create dogs \
    --gcs-bucket-name=BUCKET_2_NAME

Configuring the URL map

gcloud compute url-maps create http-lb \
    --default-backend-bucket=cats

gcloud compute url-maps add-path-matcher http-lb \
    --path-matcher-name=path-matcher-2 \
    --new-hosts=* \
    --backend-bucket-path-rules="/love-to-fetch/*=dogs" \
    --default-backend-bucket=cats

Configuring the target proxy

gcloud compute target-http-proxies create http-lb-proxy \
    --url-map=http-lb

Configuring the forwarding rule

gcloud compute forwarding-rules create http-lb-forwarding-rule \
    --address=example-ip \
    --global \
    --target-http-proxy=http-lb-proxy \
    --ports=80

Sending traffic to your backend bucket

Several minutes after you have configured your load balancer, you can start sending traffic to the load balancer's IP address.

Console

1. Go to the Load balancing page in the Google Cloud Console.
   Go to the Load balancing page
2. Click http-lb to expand the load balancer that you just created.
3. In the Backend section, confirm that the backend buckets are healthy. There should be a green checkmark next to your backend buckets. If you see otherwise, first try reloading the page. It can take a few moments for the Cloud Console to indicate that backends are healthy.

4. After the Cloud Console shows that the backend buckets are healthy, you can test your load balancer using a web browser by going to the https://IP_ADDRESS/love-to-fetch/two-dogs.jpg and https://IP_ADDRESS/never-fetch/three-cats.jpg, where IP_ADDRESS is the load balancer's IP address. Your browser should render pages with content showing the graphic files:

   • http://IP_ADDRESS/love-to-fetch/two-dogs.jpg
   • http://IP_ADDRESS/never-fetch/three-cats.jpg

gcloud

Use the curl command to test the response from the URL. Replace IP_ADDRESS with the load balancer's IPv4 address:

curl http://IP_ADDRESS/love-to-fetch/two-dogs.jpg

curl http://IP_ADDRESS/never-fetch/three-cats.jpg

Limitations

• Backend buckets are only supported with external HTTP(S) load balancers
• Backend buckets aren't supported with Identity-Aware Proxy

What's next

• Setting up multi-region, content-based load balancing
• Setting up Cloud CDN with a backend bucket