Backend buckets allow you to use Google Cloud Storage buckets with HTTP(S) Load Balancing.
Using a Cloud Storage bucket as a load balancer backend
An external HTTP(S) load balancer uses a URL map to direct traffic from specified URLs to either a backend service or a backend bucket. One common use case is:
- Send requests for dynamic content, such as data, to a backend service.
- Send requests for static content, such as images, to a backend bucket.
In the following diagram, the load balancer sends traffic with a path of
/love-to-fetch/
to a Cloud Storage bucket in the europe-north
region.
All other requests go to a Cloud Storage bucket in the us-east
region.
This document shows you how to create an
external HTTP(S) load balancer
to route requests for static content to Cloud Storage
buckets. After you configure
a load balancer with the backend buckets, requests to URL paths
that begin with /love-to-fetch
are sent to the europe-north
Cloud Storage bucket, and all other requests are sent to the us-east
Cloud Storage bucket.
Before you begin
Make sure that your setup meets the prerequisites.
Set a default project
- If you are using the
gcloud
orgsutil
utilities, you can install them both by using instructions in Quickstart: Using the gsutil Tool document. -
Console
a. Go to the Home page in the Google Cloud Console.
Go to the Google Cloud home page
b. To the right of Google Cloud, select a project from the pull-down menu.gcloud or gsutil
gcloud config set project [PROJECT_ID]
or
gsutil config set project [PROJECT_ID]
[PROJECT_ID]
- the project you will use for this guide.
Permissions
To follow this guide, you need to create Cloud Storage buckets and a load balancer in a project. You should be either a project owner or editor, or you should have the following Compute Engine IAM roles:
Task | Required Role |
---|---|
Create load balancer components | Network Admin |
Create Cloud Storage buckets | Storage Object Admin |
For more information, see the following guides:
Preparing Cloud Storage buckets and content
The process for preparing the Cloud Storage buckets is as follows:
- Create the buckets.
- Copy content to the buckets.
- Provide public access to the buckets.
Creating Cloud Storage buckets
If you have an existing Cloud Storage bucket, you can skip to the next step.
When you create a bucket to use as the backend for a HTTP(S) Load Balancing, we recommend that you choose a multi-region bucket, which automatically replicates objects across multiple Google Cloud regions. This can improve the availability of your content and improve failure tolerance across your application.
Console
- In the Cloud Console, open the Cloud Storage browser.
Open the Cloud Storage browser - Click Create bucket.
Specify values for the following fields, leaving all others at their defaults:
Property Value (type the value or select an option as specified) Name For each bucket, enter a globally unique name. If the name you enter is not unique, you'll see a message to try another name. Location type For this example, select Region. For production deployments, we recommend multi-region buckets. Location For one bucket, select us-east1
For the other bucket, select europe-north1Default storage class Standard Access control Uniform Click Create.
Note the names of the newly-created Cloud Storage buckets for the next step, where they are referred to as
BUCKET_1_NAME
andBUCKET_2_NAME
.
gsutil
gsutil mb -p PROJECT_ID -c standard -l us-east1 -b on gs://BUCKET_1_NAME
gsutil mb -p PROJECT_ID -c standard -l europe-north1 -b on gs://BUCKET_2_NAME
Copying graphic files into your Cloud Storage buckets
So that you can test the setup, copy graphic files from a public Cloud Storage bucket to your own Cloud Storage buckets.
Run the following command in Cloud Shell, replacing the bucket name variables with your Cloud Storage bucket names:
gsutil cp gs://gcp-external-http-lb-with-bucket/three-cats.jpg gs://BUCKET_1_NAME/never-fetch/
gsutil cp gs://gcp-external-http-lb-with-bucket/two-dogs.jpg gs://BUCKET_2_NAME/love-to-fetch/
In the Cloud Console, click Refresh Bucket to verify that the graphic file is copied.
Making the Cloud Storage buckets public
Make the Cloud Storage buckets publicly readable. With this setting, anyone on the internet can view your objects and their metadata, excluding ACLs. Anyone on the internet can also list the objects in your buckets. Don't include sensitive information in your public buckets.
To reduce the likelihood of accidental exposure of sensitive information, don't store public objects and sensitive data in the same bucket. For more information, see Recommended bucket architecture.
Console
Grant all users access to view objects in the bucket. Repeat this procedure for each bucket.
- In the Cloud Console, open the Cloud Storage browser.
Open the Cloud Storage browser - Navigate to the bucket and click the Permissions tab.
- Click and select Add members.
- In New members enter
allUsers
. - For the role, select Storage > Storage object viewer.
- Click Save.
gsutil
gsutil iam ch allUsers:objectViewer gs://BUCKET_1_NAME
gsutil iam ch allUsers:objectViewer gs://BUCKET_2_NAME
Reserving an external IP address
Now that your buckets are up and running, set up a global static external IP address that your customers use to reach your load balancer.
This step is optional, but recommended, because a static external IP address provides a single address to point your domain at.
Console
- Go to the External IP addresses page in the Google Cloud Console.
Go to the External IP addresses page - Click Reserve static address to reserve an IPv4 address.
- Assign a Name of
example-ip
. - Set the Network tier to Premium.
- Set the IP version to IPv4.
- Set the Type to Global.
- Click Reserve.
- Ensure that the Type is set to Global.
- Click Reserve.
gcloud
gcloud compute addresses create example-ip \ --network-tier=PREMIUM \ --ip-version=IPV4 \ --global
Note the IPv4 address that was reserved:
gcloud compute addresses describe example-ip \ --format="get(address)" \ --global
Creating the HTTP load balancer with backend buckets
If you want to create an HTTPS load balancer, you must add an SSL certificate resource to the load balancer's frontend. For more information, see the SSL certificates overview.
Console
- Go to the Load balancing page in the Google Cloud Console.
Go to the Load balancing page - Under HTTP(S) load balancing, click Start configuration.
- Select From Internet to my VMs.
- Set Name to
http-lb
.
Configuring the backend
- Click Backend configuration.
- For Backend services & backend buckets, click Create or select backend services & backend buckets, click Backend buckets > Create a backend bucket.
- Set the Name to
cats
. - Click Browse under Cloud Storage bucket.
- Select your
BUCKET_1_NAME
and click Select. - Click Create.
Repeat this step to create dogs
, and browse
to BUCKET_2_NAME`.
Configuring host rules and path matchers
Host rules and path matchers are configuration components of an external HTTP(S) load balancer's URL map.
- Click Host and path rules.
- For
cats
, you can't change the default hosts and paths. This means that all unmatched requests go tocats
. - For
dogs
, enter*
in the Hosts field. For
dogs
, enter/love-to-fetch/*
in the Paths field.Hosts Paths Backend All unmatched (default) All unmatched (default) cats * /love-to-fetch/* dogs
Configuring the frontend
- Click Frontend configuration.
Verify that the following options are configured with these values:
Property Value (type a value or select an option as specified) Protocol HTTP Network Service Tier Premium IP version IPv4 IP address example-ip Port 80 If you want to create an HTTPS load balancer instead of an HTTP load balancer, you must have an SSL certificate (
gcloud compute ssl-certificates list
), and you must fill in the fields as follows:Property Value (type a value or select an option as specified) Protocol HTTP(S) Network Service Tier Premium IP version IPv4 IP address example-ip Port 443 Select a certificate or create a new certificate Click Done.
Reviewing the configuration
- Click Review and finalize.
- Review the Backend Buckets, Host and Path rules, and Frontend.
- Click Create.
- Wait for the load balancer to be created.
- Click the name of the load balancer (http-lb).
- Note the IP address of the load balancer for the next task. It's
referred to as
IP_ADDRESS
.
gcloud
Configuring the backend
gcloud compute backend-buckets create cats \ --gcs-bucket-name=BUCKET_1_NAME
gcloud compute backend-buckets create dogs \ --gcs-bucket-name=BUCKET_2_NAME
Configuring the URL map
gcloud compute url-maps create http-lb \ --default-backend-bucket=cats
gcloud compute url-maps add-path-matcher http-lb \ --path-matcher-name=path-matcher-2 \ --new-hosts=* \ --backend-bucket-path-rules="/love-to-fetch/*=dogs" \ --default-backend-bucket=cats
Configuring the target proxy
gcloud compute target-http-proxies create http-lb-proxy \ --url-map=http-lb
Configuring the forwarding rule
gcloud compute forwarding-rules create http-lb-forwarding-rule \ --address=example-ip \ --global \ --target-http-proxy=http-lb-proxy \ --ports=80
Sending traffic to your backend bucket
Now that you have configured your load balancer, you can start sending traffic to the load balancer's IP address.
Console
- Go to the Load balancing page in the Google Cloud Console.
Go to the Load balancing page - Click
http-lb
to expand the load balancer that you just created. - In the Backend section, confirm that the backend buckets are healthy. There should be a green checkmark next to your backend buckets. If you see otherwise, first try reloading the page. It can take a few moments for the Cloud Console to indicate that backends are healthy.
After the Cloud Console shows that the backend buckets are healthy, you can test your load balancer using a web browser by going to the https://
IP_ADDRESS
/love-to-fetch/two-dogs.jpg and https://IP_ADDRESS
/never-fetch/three-cats.jpg, whereIP_ADDRESS
is the load balancer's IP address. Your browser should render pages with content showing the graphic files:- http://
IP_ADDRESS
/love-to-fetch/two-dogs.jpg - http://
IP_ADDRESS
/never-fetch/three-cats.jpg
- http://
gcloud
Use the curl
command to test the response from the URL. Replace
IP_ADDRESS
with the load balancer's IPv4
address:
curl http://IP_ADDRESS/love-to-fetch/two-dogs.jpg
curl http://IP_ADDRESS/never-fetch/three-cats.jpg
Limitations
- Backend buckets are only supported with external HTTP(S) load balancers
- Backend buckets aren't supported with Identity-Aware Proxy