This document provides instructions for setting up an external proxy Network Load Balancer with a target SSL proxy and VM instance group backends. Before you start, read External proxy Network Load Balancer overview for information on how these load balancers work.
Setup overview
This example demonstrates how to set up an external proxy Network Load Balancer for a
service that exists in two regions: us-central1
and us-east1
. You will
configure the following:
- Four instances spread across two regions
- Instance groups for holding the instances
- Backend components, which include the following:
- Health check - used to monitor instance health
- Backend service - monitors instance groups and prevents them from exceeding configured usage
- Backends - hold the instance groups
- Frontend components, which include the following:
- An SSL certificate resource. You can use either a self-managed certificate, where you supply your own SSL certificate, or a Google-managed certificate, where Google issues a certificate that is valid for all of your domains. For more information, see Types of SSL certificates.
- The SSL proxy itself with its SSL certificate
- An external static IPv4 address and a forwarding rule that sends user traffic to the proxy
- An external static IPv6 address and a forwarding rule that sends user traffic to the proxy
- A firewall rule that allows traffic from the load balancer and health checker to the instances.
- Optionally, an SSL policy to control the features of SSL that your SSL proxy load balancer negotiates with clients.
After that, you'll test your configuration.
Permissions
To follow this guide, you must be able to create instances and modify a network in a project. You must be either a project owner or editor, or you must have all of the following Compute Engine IAM roles:
Task | Required Role |
---|---|
Create networks, subnets, and load balancer components | Network Admin |
Add and remove firewall rules | Security Admin |
Create instances | Compute Instance Admin |
For more information, see the following guides:
Configure instances and instance groups
This section shows how to create instances and instance groups, then add the instances to the instance groups. A production system would normally use managed instance groups based on instance templates, but this setup is quicker for initial testing.
Create instances
For testing purposes, install Apache on four instances, two in each of two instance groups. Typically, you would not use an external proxy Network Load Balancer for HTTP traffic, but Apache is commonly used and is easy to set up for testing.
Create these instances with the tag ssl-lb
, which the firewall rule will use
later.
Console
Create instances
In the Google Cloud console, go to the VM instances page.
Click Create instance.
Set Name to
ig-us-central1-1
.Set the Zone to us-central1-b.
Click Advanced options.
Click Networking and configure the following field:
- For Network tags, enter
ssl-lb
.
- For Network tags, enter
Click Management. Enter the following script into the Startup script field.
sudo apt-get update sudo apt-get install apache2 -y sudo a2ensite default-ssl sudo a2enmod ssl sudo service apache2 restart echo '<!doctype html><html><body><h1>ig-us-central1-1</h1></body></html>' | sudo tee /var/www/html/index.html
Leave the default values for rest of the fields.
Click Create.
Create
ig-us-central1-2
with the same settings, except with Startup script set to the following:sudo apt-get update sudo apt-get install apache2 -y sudo a2ensite default-ssl sudo a2enmod ssl sudo service apache2 restart echo '<!doctype html><html><body><h1>ig-us-central1-2</h1></body></html>' | sudo tee /var/www/html/index.html
Create
ig-us-east1-1
with the same settings, except with Zone set tous-east1-b
and Startup script set to the following:sudo apt-get update sudo apt-get install apache2 -y sudo a2ensite default-ssl sudo a2enmod ssl sudo service apache2 restart echo '<!doctype html><html><body><h1>ig-us-east1-1</h1></body></html>' | sudo tee /var/www/html/index.html
Create
ig-us-east1-2
with the same settings, except with Zone set tous-east1-b
and Startup script set to the following:sudo apt-get update sudo apt-get install apache2 -y sudo a2ensite default-ssl sudo a2enmod ssl sudo service apache2 restart echo '<!doctype html><html><body><h1>ig-us-east1-2</h1></body></html>' | sudo tee /var/www/html/index.html
gcloud
Create
ig-us-central1-1
in zoneus-central1-b
.gcloud compute instances create ig-us-central1-1 \ --image-family debian-10 \ --image-project debian-cloud \ --tags ssl-lb \ --zone us-central1-b \ --metadata startup-script="#! /bin/bash sudo apt-get update sudo apt-get install apache2 -y sudo a2ensite default-ssl sudo a2enmod ssl sudo service apache2 restart echo '<!doctype html><html><body><h1>ig-us-central1-1</h1></body></html>' | sudo tee /var/www/html/index.html EOF"
Create
ig-us-central1-2
in zoneus-central1-b
.gcloud compute instances create ig-us-central1-2 \ --image-family=debian-10 \ --image-project=debian-cloud \ --tags=ssl-lb \ --zone=us-central1-b \ --metadata=startup-script="#! /bin/bash sudo apt-get update sudo apt-get install apache2 -y sudo a2ensite default-ssl sudo a2enmod ssl sudo service apache2 restart echo '<!doctype html><html><body><h1>ig-us-central1-2</h1></body></html>' | sudo tee /var/www/html/index.html EOF"
Create
ig-us-east1-1
in zoneus-east1-b
.gcloud compute instances create ig-us-east1-1 \ --image-family=debian-10 \ --image-project=debian-cloud \ --tags=ssl-lb \ --zone=us-east1-b \ --metadata=startup-script="#! /bin/bash sudo apt-get update sudo apt-get install apache2 -y sudo a2ensite default-ssl sudo a2enmod ssl sudo service apache2 restart echo '<!doctype html><html><body><h1>ig-us-east1-1</h1></body></html>' | sudo tee /var/www/html/index.html EOF"
Create
ig-us-east1-2
in zoneus-east1-b
.gcloud compute instances create ig-us-east1-2 \ --image-family=debian-10 \ --image-project=debian-cloud \ --tags=ssl-lb \ --zone=us-east1-b \ --metadata=startup-script="#! /bin/bash sudo apt-get update sudo apt-get install apache2 -y sudo a2ensite default-ssl sudo a2enmod ssl sudo service apache2 restart echo '<!doctype html><html><body><h1>ig-us-east1-2</h1></body></html>' | sudo tee /var/www/html/index.html EOF"
Create an instance group for each zone and add instances
Console
In the Google Cloud console, go to the Instance groups page.
Click Create instance group.
Set the Name to
us-ig1
.Set the Zone to
us-central1-b
.Under Port mapping, click Add port. A load balancer sends traffic to an instance group through a named port. Create a named port to map the incoming traffic to a specific port number.
- Enter a Port name of
ssl-lb
and Port numbers of443
.
- Enter a Port name of
Under Instance definition, click Select existing instances.
From VM instances select
ig-us-central1-1
andig-us-central1-2
.Leave other settings as they are.
Click Create.
Repeat steps, but set the following:
- Name:
us-ig2
- Zone:
us-east1-b
- Port name of
ssl-lb
and Port numbers of443
- Instances: ig-us-east1-1 and ig-us-east1-2.
- Name:
Confirm that you now have two instance groups, each with two instances.
gcloud
Create the
us-ig1
instance group.gcloud compute instance-groups unmanaged create us-ig1 --zone us-central1-b
Set a named port for the instance group.
gcloud compute instance-groups set-named-ports us-ig1 \ --named-ports=ssl-lb:443 \ --zone=us-central1-b
Add
ig-us-central1-1
andig-us-central1-2
tous-ig1
gcloud compute instance-groups unmanaged add-instances us-ig1 \ --instances=ig-us-central1-1,ig-us-central1-2 \ --zone=us-central1-b
Create the
us-ig2
instance group.gcloud compute instance-groups unmanaged create us-ig2 --zone us-east1-b
Set a named port for the instance group.
gcloud compute instance-groups set-named-ports us-ig2 \ --named-ports=ssl-lb:443 \ --zone=us-east1-b
Add
ig-us-east1-1
andig-us-east1-2
tous-ig2
gcloud compute instance-groups unmanaged add-instances us-ig2 \ --instances=ig-us-east1-1,ig-us-east1-2 \ --zone=us-east1-b
You now have an instance group in each of two regions, each with two instances.
Create a firewall rule for the SSL load balancer
Configure the firewall to allow traffic from the load balancer and health checker to the instances.
Console
In the Google Cloud console, go to the Firewall policies page.
Click Create firewall rule.
In the Name field, enter
allow-ssl-lb-and-health
.Under Network, select default.
Under Targets, select Specified target tags.
Set Target tags to
ssl-lb
.Set Source filter to IPv4 ranges.
Set Source IPv4 ranges to
130.211.0.0/22
and35.191.0.0/16
.Under Protocols and ports, set Specified protocols and ports to
tcp:443
.Click Create.
gcloud
gcloud compute firewall-rules create allow-ssl-lb-and-health \ --source-ranges=130.211.0.0/22,35.191.0.0/16 \ --target-tags=ssl-lb \ --allow=tcp:443
If you are using a Google-managed certificate, confirm that your certificate resource's status is ACTIVE. For more information, see Google-managed SSL certificate resource status.
gcloud compute ssl-certificates list
Configure the load balancer
Console
In the Google Cloud console, go to the Load balancing page.
Click Create load balancer.
- On the Network Load Balancer (TCP/SSL) card, click Start configuration.
Under Internet facing or internal only select From Internet to my VMs.
Under Multiple regions or single region, select Multiple regions (or not sure yet).
Backend configuration
- Click Backend configuration.
- Set the Name of the load balancer to
my-ssl-lb
. - Set the Protocol to SSL.
- For Named port, enter
ssl-lb
. - Accept the default value for the Timeout.
- Leave the Backend type set to Instance groups.
- Under New backend, select instance group
us-ig1
. - Set Port numbers to
443
. - Leave other settings as they are.
- Under New backend, select instance group
us-ig2
. - Set Port numbers to
443
. - Leave other settings as they are.
- Click Done.
- Under Health check, select Create health check.
- Set the health check Name to
my-ssl-health-check
. - Set Protocol to
SSL
. - Leave the other settings the same.
- Click Save and continue.
- Set the health check Name to
-
Optional: Configure a default backend security policy. The default security policy throttles traffic over a user-configured threshold. For more information about default security policies, see the Rate limiting overview.
- To opt out of the Google Cloud Armor default security policy, select
None
in the backend security policy list menu. - In the Security section, select Default security policy.
- In the Policy name field, accept the automatically generated name or enter a name for your security policy.
- In the Request count field, accept the default request
count or enter an integer between
1
and10,000
. - In the Interval field, select an interval.
- In the Enforce on key field, choose one of the following values: All, IP address, or X-Forwarded-For IP address. For more information about these options, see Identifying clients for rate limiting.
- To opt out of the Google Cloud Armor default security policy, select
- Verify that there is a green check mark next to Backend configuration in the Google Cloud console. If not, double-check that you have completed all the steps above.
Frontend configuration
- Click Frontend configuration.
- Under New frontend IP and port, enter a Name of
my-ssl-lb-forwarding-rule
. - Set Protocol to SSL.
- Under IP address, select Create IP address.
- Enter a Name of
ssl-lb-static-ipv4
. - Click Reserve.
- Enter a Name of
- Under Certificate, select Create a new certificate.
- Enter a Name of
my-ssl-cert
. - If you choose Upload my certificate, complete these steps:
- Paste in your certificate or click Upload to navigate to your certificate file.
- Paste in your private key or click Upload to navigate to your private key file.
- If you choose Create Google managed certificate, enter a
Domain.
- To enter additional domains, click Add Domain.
- Click Create.
- To add certificate resources in addition to the primary SSL certificate resource, click Additional certificates. Then either select another certificate from the Certificates menu or click Create a new certificate and follow the instructions above.
- Enter a Name of
- (Optional) To create an SSL policy:
- Under SSL policy, select Create a policy.
- Enter a Name of
my-ssl-policy
. - For Minimum TLS Version, select TLS 1.0.
- For Profile, select Modern. The Enabled features and Disabled features are displayed.
- Click Create.
- (Optional) Turn on Proxy protocol if desired.
- Click Done.
- Click Add frontend IP and port.
- Enter a Name of
my-ssl-lb-ipv6-forwarding-rule
. - Set IP version to
IPv6
. - Under IP address, select Create IP address.
- Enter a Name of
ssl-lb-static-ipv6
. - Click Reserve.
- Enter a Name of
- Under Certificate, select
my-ssl-cert
.- To add certificate resources in addition to the primary SSL certificate resource, either select a certificate from the Certificates list or click Create a new certificate.
- (Optional) Use an SSL policy or turn on PROXY protocol if desired.
- Click Done.
- Verify that there is a green check mark next to Frontend configuration in the Google Cloud console. If not, double-check that you have completed all the steps above.
Review and finalize
- Click Review and finalize.
- Double-check your settings.
- Click Create.
gcloud
Create a health check.
gcloud compute health-checks create ssl my-ssl-health-check --port=443
Create a backend service.
gcloud compute backend-services create my-ssl-lb \ --global-health-checks \ --protocol=SSL \ --port-name=ssl-lb \ --health-checks=my-ssl-health-check \ --timeout=5m \ --global
Alternatively you could configure unencrypted communication between from the load balancer to the instances with
--protocol=TCP
.Add instance groups to your backend service.
gcloud compute backend-services add-backend my-ssl-lb \ --instance-group=us-ig1 \ --instance-group-zone=us-central1-b \ --balancing-mode=UTILIZATION \ --max-utilization=0.8 \ --global
gcloud compute backend-services add-backend my-ssl-lb \ --instance-group=us-ig2 \ --instance-group-zone=us-east1-b \ --balancing-mode=UTILIZATION \ --max-utilization=0.8 \ --global
Configure your SSL certificate resource.
If you are using self-managed certificates, you must already have at least one SSL certificate to upload. If you do not, see SSL certificates. When you use multiple SSL certificates, you must create them one at a time.
If you are using self-managed SSL certificates and you don't have a private key and signed certificate, you can create and use a self-signed certificate for testing purposes.
To create a self-managed SSL certificate resource:
gcloud compute ssl-certificates create my-ssl-cert \ --certificate=[CRT_FILE_PATH] \ --private-key=[KEY_FILE_PATH]
To create a Google-managed SSL certificate resource:
gcloud beta compute ssl-certificates create www-ssl-cert \ --domains=[DOMAIN_1],[DOMAIN_2]
Configure a target SSL proxy.
External proxy Network Load Balancers support creating a target SSL proxy that has from one to fifteen SSL certificates. Before you run this command, you must create an SSL certificate resource for each certificate.
If you want to turn on the proxy header, set it to
PROXY_V1
instead ofnone
. You can optionally attach an SSL policy to the target proxy. First, create the policy.gcloud compute ssl-policies create my-ssl-policy \ --profile=MODERN \ --min-tls-version=1.0
Then attached the policy to the target proxy.
gcloud compute target-ssl-proxies create my-ssl-lb-target-proxy \ --backend-service=my-ssl-lb \ --ssl-certificates=[SSL_CERT_1][,[SSL_CERT_2],...] \ --ssl-policy=my-ssl-policy \ --proxy-header=NONE
Reserve global static IP addresses.
Your customers use these IP addresses to access your load-balanced service.
gcloud compute addresses create ssl-lb-static-ipv4 \ --ip-version=IPV4 \ --global
gcloud compute addresses create ssl-lb-static-ipv6 \ --ip-version=IPV6 \ --global
Configure global forwarding rules.
Create global forwarding rules associated with the target proxy. Replace LB_STATIC_IP and LB_STATIC_IPV6 with the IP addresses you generated in Reserve global static IP addresses.
gcloud compute forwarding-rules create my-ssl-lb-forwarding-rule \ --global \ --target-ssl-proxy=my-ssl-lb-target-proxy \ --address=LB_STATIC_IP \ --ports=443
gcloud compute forwarding-rules create my-ssl-lb-ipv6-forwarding-rule \ --global \ --target-ssl-proxy=my-ssl-lb-target-proxy \ --address=LB_STATIC_IPV6 \ --ports=443
Connect your domain to your load balancer
After the load balancer is created, note the IP address that is associated with
the load balancer—for example, 30.90.80.100
. To point your domain to your
load balancer, create an A
record by using your domain registration service. If
you added multiple domains to your SSL certificate, you must add an A
record
for each one, all pointing to the load balancer's IP address. For example, to
create A
records for www.example.com
and example.com
, use the following:
NAME TYPE DATA www A 30.90.80.100 @ A 30.90.80.100
If you are using Google Domains, for more information, see Google Domains Help.
Test the load balancer
In your web browser, connect to your static IP address using HTTPS. In this
test setup, the instances are using self-signed certificates. Therefore, you
will see a warning in your browser the first time you access a page. Click
through the warning to see the actual page. Replace [IP_ADDRESS]
with either
the IPv4 or IPv6 address you created earlier.
https://[IP_ADDRESS]
You should see one of the hosts from the region closest to you. Reload the page until you see the other instance in that region. To see instances from the other region, stop the instances in the closest region.
Alternatively, you can use curl
from the your local machine's command line.
If you are using a self-signed certificate on the SSL proxy, you must also
specify -k
. The curl -k
option allows curl to work even if you have a
self-signed certificate or no certificate at all. If you have a normal
certificate, you can remove that parameter. You should only use the -k
parameter for testing your own site. Under normal circumstances, a valid
certificate is an important security measure and certificate warnings should
not be ignored.
Replace [IP_ADDRESS]
with either the IPv4 or IPv6 address you created earlier.
curl -k https://[IP_ADDRESS]
If you can't reach the load balancer, try the steps described under Troubleshooting your setup.
Additional configuration options
This section expands on the configuration example to provide alternative and additional configuration options. All of the tasks are optional. You can perform them in any order.
Set PROXY protocol for retaining client connection information
Google Cloud Load Balancing with SSL proxy terminates SSL connections from the client and creates new connections to the instances. Therefore, the original client IP and port information is not preserved by default.
To preserve and send the original connection information to your instances, enable PROXY protocol (version 1) on the load balancer's frontend. This protocol sends an additional header that contains the source IP address, destination IP address, and port numbers to the instance as a part of the request.
You can also set the PROXY protocol for TCP and SSL health checks.
Make sure that the load balancer's backend instances are running HTTP(S) servers that support PROXY protocol headers. If the HTTP(S) servers are not configured to support PROXY protocol headers, the backend instances return empty responses. For example, the PROXY protocol doesn't work with the Apache HTTP Server software. You can use different web server software, such as Nginx.
If you set the PROXY protocol for user traffic, you must also set it for
your health checks. If you are checking health and serving
content on the same port, set the health check's --proxy-header
to match your
load balancer setting.
The PROXY protocol header is typically a single line of user-readable text with the following format:
PROXY TCP4 <client IP> <load balancing IP> <source port> <dest port>\r\n
In this example, the client IP address is 192.0.2.1
, the load balancing IP
address is 198.51.100.1
, the client port is 15221
, and the destination port
is 443
:
PROXY TCP4 192.0.2.1 198.51.100.1 15221 443\r\n
When the client IP address is not known, the load balancer generates a PROXY protocol header in the following format:
PROXY UNKNOWN\r\n
Update PROXY protocol header for an existing target proxy
The example load balancer setup on this page shows you how to enable the PROXY protocol header while creating the load balancer. Use these steps to change the PROXY protocol header for an existing target SSL proxy.
Console
In the Google Cloud console, go to the Load balancing page.
Click Edit
for your load balancer.Click Frontend configuration.
Change the value of the Proxy protocol field.
Click Update to save your changes.
gcloud
gcloud compute target-ssl-proxies update my-ssl-lb-target-proxy \ --proxy-header=[NONE | PROXY_V1]
Configure session affinity
These procedures show you how to update a backend service for the example SSL proxy load balancer so that the backend service uses client IP affinity.
When client IP affinity is enabled, the load balancer directs a particular client's requests to the same backend VM based on a hash created from the client's IP address and the load balancer's IP address (the external IP address of an external forwarding rule).
Console
To enable client IP session affinity:
In the Google Cloud console, go to the Load balancing page.
Click Backends.
Click my-ssl-lb (the name of the backend service you created for this example) and click Edit.
On the Backend service details page, click Advanced configuration.
Under Session affinity, select Client IP from the menu.
Click Update.
gcloud
Use the following gcloud command to update the my-ssl-lb
backend
service, specifying client IP session affinity:
gcloud compute backend-services update my-ssl-lb \ --global \ --session-affinity=CLIENT_IP
API
To set client IP session affinity, make a PATCH
request to the
backendServices/patch
method.
PATCH https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/us-west1/backendServices/my-ssl-lb
{
"sessionAffinity": "CLIENT_IP"
}
Enable connection draining
You can enable connection draining on backend services to ensure minimal interruption to your users when an instance that is serving traffic is terminated, removed manually, or removed by an autoscaler. To learn more about connection draining, see the Enabling Connection Draining documentation.
Troubleshoot your setup
If pages fail to load when trying to access your load balancer's IP address, try the following steps to troubleshoot your setup.
Verify the health of instances
Verify that the instances are HEALTHY.
gcloud compute backend-services get-health my-ssl-lb
Verify that the firewall rule allows traffic
- The health checker and the load balancer both require these IP addresses
to allow traffic:
130.211.0.0/22
and35.191.0.0/16
. - If you are using SSL between the load balancer and the instances, use SSL
health checks. Ensure that the firewall rule allows
tcp:443
from the source IP address ranges130.211.0.0/22
and35.191.0.0/16
. - If you are using TCP between the load balancer and the instances, use TCP
health checks. Ensure that the firewall rule allows
tcp:80
from the source IP ranges130.211.0.0/22
and35.191.0.0/16
. - If you are using instance tags, make sure that the tag is listed under
TARGET_TAGS in the firewall rule, and make sure that all of your instances
have that tag. In this example, instances are tagged with
ssl-lb
.
gcloud compute firewall-rules list
NAME NETWORK SRC_RANGES RULES SRC_TAGS TARGET_TAGS allow-ssl-lb-and-health default 130.211.0.0/22,35.191.0.0/16 tcp:443 ssl-lb
Try to reach individual instances
Temporarily set a firewall rule that allows you to access your instances individually, and then try to load a page from a specific instance.
Open the firewall to allow traffic from any source to the tagged instances. Do this by creating a source filter that sets the source IP range to
0.0.0.0/0
for HTTPS traffic to instances with thessl-lb
tag.gcloud compute firewall-rules create allow-ssl-0-0-0-0 \ --source-ranges=0.0.0.0/0 \ --target-tags=ssl-lb \ --allow=tcp:443
Look up the
EXTERNAL_IP
address of one of the instances.gcloud compute instances list
NAME ZONE MACHINE_TYPE PREEMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS ig-us-central1-1 us-central1-b n1-standard-1 10.240.0.8 EXTERNAL_IP RUNNING ig-us-central1-2 us-central1-b n1-standard-1 10.240.0.11 EXTERNAL_IP RUNNING ig-us-east1-1 us-east1-b n1-standard-1 10.240.0.12 EXTERNAL_IP RUNNING ig-us-east1-2 us-east1-b n1-standard-1 10.240.0.13 EXTERNAL_IP RUNNING
Access one or more of your instances directly from your browser.
https://<var>EXTERNAL_IP</var>
If your instances are not accessible by this method, make sure that your software is running correctly. If it is, make sure that your load balancer firewall rule is correct.
gcloud compute firewall-rules describe allow-ssl-lb-and-health
allowed: IPProtocol: tcp ports: '443' creationTimestamp: '2016-02-20T22:27:15.094-08:00' description: '' id: '5304629236729177644' kind: compute#firewall name: allow-130-211-0-0-22-ssl network: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/networks/default selfLink: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/firewalls/allow-ssl-lb-and-health sourceRanges: 130.211.0.0/22,35.191.0.0/16 targetTags: ssl-lb
When you're sure the that instances are working, remove the "from anywhere" firewall rule.
gcloud compute firewall-rules delete allow-ssl-0-0-0-0
What's next
- See External proxy Network Load Balancer logging and monitoring for information about how monitoring works for external proxy Network Load Balancers.
- See SSL Policy overview for information about how SSL policies work.
- See Using SSL policies for information about configuring SSL policies.