Setting up a load balancer with backend buckets

Backend buckets allow you to use Google Cloud Storage buckets with HTTP(S) Load Balancing.

Using a Cloud Storage bucket as a load balancer backend

An external HTTP(S) load balancer uses a URL map to direct traffic from specified URLs to either a backend service or a backend bucket. One common use case is:

  • Send requests for dynamic content, such as data, to a backend service.
  • Send requests for static content, such as images, to a backend bucket.

In the following diagram, the load balancer sends traffic with a path of /static/eu/ to a Cloud Storage bucket in the europe-north region. All other requests go to a Cloud Storage bucket in the us-east region.

Distributing traffic to Cloud Storage (click to enlarge)
Distributing traffic to Cloud Storage (click to enlarge)

This document shows you how to create an external HTTP(S) load balancer to route requests for static content to Cloud Storage buckets. After you configure a load balancer with the backend buckets, requests to URL paths that begin with /static/eu are sent to the europe-north Cloud Storage bucket, and all other requests are sent to the us-east Cloud Storage bucket.

Before you begin

Make sure that your setup meets the prerequisites.

Set a default project

  1. If you are using the gcloud or gsutil utilities, you can install them both by using instructions in Quickstart: Using the gsutil Tool document.
  2. Set a default project.

    Console

    a. Go to the Home page in the Google Cloud Console.
    Go to the Google Cloud home page

    b. To the right of Google Cloud, select a project from the pull-down menu.

    gcloud or gsutil

     gcloud config set project [PROJECT_ID]
    

    or

     gsutil config set project [PROJECT_ID]
    
    • [PROJECT_ID] - the project you will use for this guide.

Permissions

To follow this guide, you need to create Cloud Storage buckets and a load balancer in a project. You should be either a project owner or editor, or you should have the following Compute Engine IAM roles:

Task Required Role
Create load balancer components Network Admin
Create Cloud Storage buckets Storage Object Admin

For more information, see the following guides:

Preparing Cloud Storage buckets and content

The process for preparing the Cloud Storage buckets is as follows:

  • Create the buckets.
  • Copy content to the buckets.
  • Provide public access to the buckets.

Creating Cloud Storage buckets

If you have an existing Cloud Storage bucket, you can skip to the next step.

When you create a bucket to use as the backend for a HTTP(S) Load Balancing, we recommend that you choose a multi-region bucket, which automatically replicates objects across multiple Google Cloud regions. This can improve the availability of your content and improve failure tolerance across your application.

Console

  1. In the Cloud Console, open the Cloud Storage browser.
    Open the Cloud Storage browser
  2. Click Create bucket.
  3. Specify values for the following fields, leaving all others at their defaults:

    Property Value (type the value or select an option as specified)
    Name For each bucket, enter a globally unique name. If the name you enter is not unique, you'll see a message to try another name.
    Location type For this example, select Region. For production deployments, we recommend multi-region buckets.
    Location For one bucket, select us-east1
    For the other bucket, select europe-north1
    Default storage class Standard
    Access control Uniform
  4. Click Create.

  5. Note the names of the newly-created Cloud Storage buckets for the next step, where they are referred to as BUCKET_1_NAME and BUCKET_2_NAME.

gsutil

gsutil mb -p PROJECT_ID -c standard -l us-east1 -b on gs://BUCKET_1_NAME
gsutil mb -p PROJECT_ID -c standard -l europe-north1 -b on gs://BUCKET_2_NAME

Copying graphic files into your Cloud Storage buckets

So that you can test the setup, copy graphic files from a public Cloud Storage bucket to your own Cloud Storage buckets.

  1. Run the following command in Cloud Shell, replacing the bucket name variables with your Cloud Storage bucket names:

    gsutil cp gs://gcp-external-http-lb-with-bucket/three-cats.jpg gs://BUCKET_1_NAME/static/us/
    
    gsutil cp gs://gcp-external-http-lb-with-bucket/two-dogs.jpg gs://BUCKET_2_NAME/static/eu/
    
  2. In the Cloud Console, click Refresh Bucket to verify that the graphic file is copied.

Making the Cloud Storage buckets public

Make the Cloud Storage buckets publicly readable. With this setting, anyone on the internet can view your objects and their metadata, excluding ACLs. Anyone on the internet can also list the objects in your buckets. Don't include sensitive information in your public buckets.

To reduce the likelihood of accidental exposure of sensitive information, don't store public objects and sensitive data in the same bucket. For more information, see Recommended bucket architecture.

Console

Grant all users access to view objects in the bucket. Repeat this procedure for each bucket.

  1. In the Cloud Console, open the Cloud Storage browser.
    Open the Cloud Storage browser
  2. Navigate to the bucket and click the Permissions tab.
  3. Click and select Add members.
  4. In New members enter allUsers.
  5. For the role, select Storage > Storage object viewer.
  6. Click Save.

gsutil

gsutil iam ch allUsers:objectViewer gs://BUCKET_1_NAME
gsutil iam ch allUsers:objectViewer gs://BUCKET_2_NAME

Reserving an external IP address

Now that your buckets are up and running, set up a global static external IP address that your customers use to reach your load balancer.

This step is optional, but recommended, because a static external IP address provides a single address to point your domain at.

Console

  1. Go to the External IP addresses page in the Google Cloud Console.
    Go to the External IP addresses page
  2. Click Reserve static address to reserve an IPv4 address.
  3. Assign a Name of example-ip.
  4. Set the Network tier to Premium.
  5. Set the IP version to IPv4.
  6. Set the Type to Global.
  7. Click Reserve.
  8. Ensure that the Type is set to Global.
  9. Click Reserve.

gcloud

gcloud compute addresses create example-ip \
    --network-tier=PREMIUM \
    --ip-version=IPV4 \
    --global

Note the IPv4 address that was reserved:

gcloud compute addresses describe example-ip \
    --format="get(address)" \
    --global

Creating the HTTP load balancer with backend buckets

If you want to create an HTTPS load balancer, you must add an SSL certificate resource to the load balancer's front end. For more information, see the SSL certificates overview.

  1. Go to the Load balancing page in the Google Cloud Console.
    Go to the Load balancing page
  2. Under HTTP(S) load balancing, click Start configuration.
  3. Select From Internet to my VMs.
  4. Set Name to http-lb.

Configuring the backend

  1. Click Backend configuration.
  2. For Backend services & backend buckets, click Create or select backend services & backend buckets, click Backend buckets > Create a backend bucket.
  3. Set the Name to backend-bucket1.
  4. Click Browse under Cloud Storage bucket.
  5. Select your BUCKET_1_NAME and click Select.
  6. Click Create.

Repeat this step to create backend-bucket2, and browse to BUCKET_2_NAME.

Configuring host rules and path matchers

Host rules and path matchers are configuration components of an external HTTP(S) load balancer's URL map.

  1. Click Host and path rules.
  2. For backend-bucket1, you can't change the default hosts and paths. This means that all unmatched requests go to backend-bucket1.
  3. For backend-bucket2, enter * in the Hosts field.
  4. For backend-bucket2, enter /static/eu/* in the Paths field.

    Hosts Paths Backend
    All unmatched (default) All unmatched (default) backend-bucket1
    * /static/eu/* backend-bucket2

Configuring the frontend

  1. Click Frontend configuration.
  2. Verify the following options are configured with these values:

    Property Value (type a value or select an option as specified)
    Protocol HTTP
    Network Service Tier Premium
    IP version IPv4
    IP address example-ip
    Port 80

    If you want to create an HTTPS load balancer instead of an HTTP load balancer, you must have an SSL certificate (gcloud compute ssl-certificates list), and you must fill in the fields as follows.

    Property Value (type a value or select an option as specified)
    Protocol HTTP(S)
    Network Service Tier Premium
    IP version IPv4
    IP address example-ip
    Port 443
    Select a certificate or create a new certificate
  3. Click Done.

Reviewing the configuration

  1. Click Review and finalize.
  2. Review the Backend Buckets, Host and Path rules, and Frontend.
  3. Click Create.
  4. Wait for the load balancer to be created.
  5. Click the name of the load balancer (http-lb).
  6. Note the IP address of the load balancer for the next task. It's referred to as IP_ADDRESS.

Sending traffic to your backend bucket

Now that you have configured your load balancer, you can start sending traffic to the load balancer's IP address.

Console

  1. Go to the Load balancing page in the Google Cloud Console.
    Go to the Load balancing page
  2. Click http-lb to expand the load balancer that you just created.
  3. In the Backend section, confirm that the backend buckets are healthy. There should be a green checkmark next to your backend buckets. If you see otherwise, first try reloading the page. It can take a few moments for the Cloud Console to indicate that backends are healthy.
  4. After the Cloud Console shows that the backend buckets are healthy, you can test your load balancer using a web browser by going to the https://IP_ADDRESS/static/eu/two-dogs.jpg and https://IP_ADDRESS/static/us/three-cats.jpg, where IP_ADDRESS is the load balancer's IP address. Your browser should render pages with content showing the graphic files:

    • http://IP_ADDRESS/static/eu/two-dogs.jpg
    • http://IP_ADDRESS/static/us/three-cats.jpg

gcloud

Use the curl command to test the response from the URL. Replace IP_ADDRESS with the load balancer's IPv4 address:

curl http://IP_ADDRESS/static/eu/two-dogs.jpg
curl http://IP_ADDRESS/static/us/three-cats.jpg

Limitations

  • Backend buckets are only supported with external HTTP(S) load balancers
  • Backend buckets aren't supported with Identity-Aware Proxy

What's next