Set up a regional external proxy Network Load Balancer with VM instance group backends

A regional external proxy Network Load Balancer is a proxy-based regional Layer 4 load balancer that enables you to run and scale your TCP service traffic in a single region behind an external regional IP address. These load balancers distribute external TCP traffic from the internet to backends in the same region.

This guide contains instructions to set up a regional external proxy Network Load Balancer with a managed instance group (MIG) backend.

Before you begin, read the External proxy Network Load Balancer overview.

In this example, we'll use the load balancer to distribute TCP traffic across backend VMs in two zonal managed instance groups in the us-west1 region. For purposes of the example, the service is a set of Apache servers configured to respond on port 110. Many browsers do not allow port 110, so the testing section uses curl.

In this example, you configure the deployment shown in the following diagram.

External proxy Network Load Balancer example configuration with instance group backends

A regional external proxy Network Load Balancer is a regional load balancer. All load balancer components (backend instance group, backend service, target proxy, and forwarding rule) must be in the same region.

Permissions

To follow this guide, you must be able to create instances and modify a network in a project. You must be either a project Owner or Editor, or you must have all of the following Compute Engine IAM roles.

Task	Required role
Create networks, subnets, and load balancer components	Compute Network Admin (roles/compute.networkAdmin)
Add and remove firewall rules	Compute Security Admin (roles/compute.securityAdmin)
Create instances	Compute Instance Admin (roles/compute.instanceAdmin)

For more information, see the following guides:

• Roles and permissions
• IAM Conditions

Configure the network and subnets

You need a VPC network with two subnets, one for the load balancer's backends and the other for the load balancer's proxies. This load balancer is regional. Traffic within the VPC network is routed to the load balancer if the traffic's source is in a subnet in the same region as the load balancer.

This example uses the following VPC network, region, and subnets:

• Network: a custom-mode VPC network named lb-network

• Subnet for backends: a subnet named backend-subnet in the us-west1 region that uses 10.1.2.0/24 for its primary IP address range

• Subnet for proxies: a subnet named proxy-only-subnet in the us-west1 region that uses 10.129.0.0/23 for its primary IP address range

Note: You can change the name of the network, the region, and the parameters for the subnets; however, subsequent steps in this guide use the network, region, and subnet parameters as named here.

Create the network and subnets

Console

1. In the Google Cloud console, go to the VPC networks page.

   Go to VPC networks

2. Click Create VPC network.

3. For Name, enter lb-network.

4. In the Subnets section, set the Subnet creation mode to Custom.

5. Create a subnet for the load balancer's backends. In the New subnet section, enter the following information:

   • Name: backend-subnet
   • Region: us-west1
   • IP address range: 10.1.2.0/24

6. Click Done.

7. Click Create.

gcloud

1. To create the custom VPC network, use the gcloud compute networks create command:

   gcloud compute networks create lb-network --subnet-mode=custom
   

2. To create a subnet in the lb-network network in the us-west1 region, use the gcloud compute networks subnets create command:

   gcloud compute networks subnets create backend-subnet \
       --network=lb-network \
       --range=10.1.2.0/24 \
       --region=us-west1
   

Create the proxy-only subnet

A proxy-only subnet provides a set of IP addresses that Google uses to run Envoy proxies on your behalf. The proxies terminate connections from the client and create new connections to the backends.

This proxy-only subnet is used by all Envoy-based load balancers in the us-west1 region of the lb-network VPC network.

Important: Don't try to assign addresses from the proxy-only subnet to your load balancer's forwarding rule or backends. You assign the forwarding rule's IP address and the backend instance IP addresses from a different subnet range (or ranges), not this one. Google Cloud reserves this subnet range for Google Cloud-managed proxies.

Console

If you're using the Google Cloud console, you can wait and create the proxy-only subnet later on the Load balancing page.

If you want to create the proxy-only subnet now, use the following steps:

1. In the Google Cloud console, go to the VPC networks page.

   Go to VPC networks

2. Click the name of the VPC network: lb-network.

3. Click Add subnet.

4. For Name, enter proxy-only-subnet.

5. For Region, select us-west1.

6. Set Purpose to Regional Managed Proxy.

7. For IP address range, enter 10.129.0.0/23.

8. Click Add.

gcloud

To create the proxy-only subnet, use the gcloud compute networks subnets create command:

gcloud compute networks subnets create proxy-only-subnet \
    --purpose=REGIONAL_MANAGED_PROXY \
    --role=ACTIVE \
    --region=us-west1 \
    --network=lb-network \
    --range=10.129.0.0/23

Create firewall rules

In this example, you create the following firewall rules:

• fw-allow-ssh. An ingress rule, applicable to the instances being load balanced, that allows incoming SSH connectivity on TCP port 22 from any address. You can choose a more restrictive source IP range for this rule; for example, you can specify only the IP ranges of the system from which you initiate SSH sessions. This example uses the target tag allow-ssh.

• fw-allow-health-check. An ingress rule, applicable to the instances being load balanced, that allows all TCP traffic from the Google Cloud health checking systems (in 130.211.0.0/22 and 35.191.0.0/16). This example uses the target tag allow-health-check.

• fw-allow-proxy-only-subnet. An ingress rule that allows connections from the proxy-only subnet to reach the backends.

Without these firewall rules, the default deny ingress rule blocks incoming traffic to the backend instances.

The target tags define the backend instances. Without the target tags, the firewall rules apply to all of your backend instances in the VPC network. When you create the backend VMs, make sure to include the specified target tags, as shown in Create a managed instance group.

Console

1. In the Google Cloud console, go to the Firewall policies page.

   Go to Firewall policies

2. Click Create firewall rule to create the rule to allow incoming SSH connections. Complete the following fields:

   • Name: fw-allow-ssh
   • Network: lb-network
   • Direction of traffic: Ingress
   • Action on match: Allow
   • Targets: Specified target tags
   • Target tags: allow-ssh
   • Source filter: IPv4 ranges
   • Source IPv4 ranges: 0.0.0.0/0
   • Protocols and ports:
     • Choose Specified protocols and ports.
     • Select the TCP checkbox, and then enter 22 for the port number.

3. Click Create.

4. Click Create firewall rule a second time to create the rule to allow Google Cloud health checks:

   • Name: fw-allow-health-check
   • Network: lb-network
   • Direction of traffic: Ingress
   • Action on match: Allow
   • Targets: Specified target tags
   • Target tags: allow-health-check
   • Source filter: IPv4 ranges
   • Source IPv4 ranges: 130.211.0.0/22 and 35.191.0.0/16

   • Protocols and ports:

     • Choose Specified protocols and ports.
     • Select the TCP checkbox, and then enter 80 for the port number.

     As a best practice, limit this rule to just the protocols and ports that match those used by your health check. If you use tcp:80 for the protocol and port, Google Cloud can use HTTP on port 80 to contact your VMs, but it cannot use HTTPS on port 443 to contact them.

5. Click Create.

6. Click Create firewall rule a third time to create the rule to allow the load balancer's proxy servers to connect to the backends:

   • Name: fw-allow-proxy-only-subnet
   • Network: lb-network
   • Direction of traffic: Ingress
   • Action on match: Allow
   • Targets: Specified target tags
   • Target tags: allow-proxy-only-subnet
   • Source filter: IPv4 ranges
   • Source IPv4 ranges: 10.129.0.0/23
   • Protocols and ports:
     • Choose Specified protocols and ports.
     • Select the TCP checkbox, and then enter 80 for the port number.

7. Click Create.

gcloud

1. Create the fw-allow-ssh firewall rule to allow SSH connectivity to VMs with the network tag allow-ssh. When you omit source-ranges, Google Cloud interprets the rule to mean any source.

   gcloud compute firewall-rules create fw-allow-ssh \
       --network=lb-network \
       --action=allow \
       --direction=ingress \
       --target-tags=allow-ssh \
       --rules=tcp:22
   

2. Create the fw-allow-health-check rule to allow Google Cloud health checks. This example allows all TCP traffic from health check probers; however, you can configure a narrower set of ports to meet your needs.

   gcloud compute firewall-rules create fw-allow-health-check \
       --network=lb-network \
       --action=allow \
       --direction=ingress \
       --source-ranges=130.211.0.0/22,35.191.0.0/16 \
       --target-tags=allow-health-check \
       --rules=tcp:80
   

3. Create the fw-allow-proxy-only-subnet rule to allow the region's Envoy proxies to connect to your backends. Set --source-ranges to the allocated ranges of your proxy-only subnet—in this example, 10.129.0.0/23.

   gcloud compute firewall-rules create fw-allow-proxy-only-subnet \
       --network=lb-network \
       --action=allow \
       --direction=ingress \
       --source-ranges=10.129.0.0/23 \
       --target-tags=allow-proxy-only-subnet \
       --rules=tcp:80
   

Reserve the load balancer's IP address

Reserve an external IP address for the load balancer. This procedure creates the load balancer's IP address in Standard Tier. Regional external proxy Network Load Balancers support both the Premium and Standard Network Service Tiers. However, creating this load balancer in the Premium Tier is not supported in the Google Cloud console. Use either gcloud or the REST API instead.

Console

1. In the Google Cloud console, go to the Reserve a static address page.

   Go to Reserve a static address

2. Choose a name for the new address.

3. For Network Service Tier, select Standard.

4. For IP version, select IPv4. IPv6 addresses are not supported.

5. For Type, select Regional.

6. For Region, select us-west1.

7. Leave the Attached to option set to None. After you create the load balancer, this IP address is attached to the load balancer's forwarding rule.

8. Click Reserve to reserve the IP address.

gcloud

1. To reserve a static external IP address, use the gcloud compute addresses create command:

   gcloud compute addresses create ADDRESS_NAME  \
       --region=us-west1 \
       --network-tier=STANDARD
   

   Replace ADDRESS_NAME with the name that you want to call this address.

2. To view the result, use the gcloud compute addresses describe command:

   gcloud compute addresses describe ADDRESS_NAME
   

Create a managed instance group

This section shows you how to create two managed instance group (MIG) backends in the us-west1 region for the load balancer. The MIG provides VM instances running the backend Apache servers for this example. Typically, a regional external proxy Network Load Balancer isn't used for HTTP traffic, but Apache is commonly-used software and is easy to set up for testing.

Console

Create an instance template

1. In the Google Cloud console, go to the Instance templates page.

   Go to Instance templates

2. Click Create instance template.

3. For Name, enter ext-reg-tcp-proxy-backend-template.

4. Ensure that the Boot disk is set to a Debian image, such as Debian GNU/Linux 10 (stretch). These instructions use commands that are only available on Debian, such as apt-get.

5. Click Advanced options.

6. Click Networking and configure the following fields:

   1. For Network tags, enter allow-ssh, allow-health-check, and allow-proxy-only-subnet.
   2. For Network interfaces, select the following:
      • Network: lb-network
      • Subnet: backend-subnet

7. Click Management. Enter the following script into the Startup script field:

    #! /bin/bash
    apt-get update
    apt-get install apache2 -y
    a2ensite default-ssl
    a2enmod ssl
    vm_hostname="$(curl -H "Metadata-Flavor:Google" \
    http://metadata.google.internal/computeMetadata/v1/instance/name)"
    echo "Page served from: $vm_hostname" | \
    tee /var/www/html/index.html
    systemctl restart apache2
    

8. Click Create.

Create a managed instance group

1. In the Google Cloud console, go to the Instance groups page.

   Go to Instance groups

2. Click Create instance group.

3. Select New managed instance group (stateless). For more information, see Create a MIG with stateful disks.

4. For Name, enter mig-a.

5. For Location, select Single zone.

6. For Region, select us-west1.

7. For Zone, select us-west1-a.

8. For Instance template, select ext-reg-tcp-proxy-backend-template.

9. Specify the number of instances that you want to create in the group.

   For this example, specify the following options for Autoscaling:

   • For Autoscaling mode, select Off:do not autoscale.
   • For Maximum number of instances, enter 2.

10. For Port mapping, click Add port.

    • For Port name, enter tcp80.
    • For Port number, enter 80.

11. Click Create.

12. To create a second managed instance group, repeat the Create a managed instance group steps and use the following settings:

    • Name: mig-c
    • Zone: us-west1-c

    Keep all the other settings the same.

gcloud

The Google Cloud CLI instructions in this guide assume that you are using Cloud Shell or another environment with bash installed.

1. To create a VM instance template with HTTP server, use the gcloud compute instance-templates create command:

   gcloud compute instance-templates create ext-reg-tcp-proxy-backend-template \
       --region=us-west1 \
       --network=lb-network \
       --subnet=backend-subnet \
       --tags=allow-ssh,allow-health-check,allow-proxy-only-subnet \
       --image-family=debian-10 \
       --image-project=debian-cloud \
       --metadata=startup-script='#! /bin/bash
          apt-get update
          apt-get install apache2 -y
          a2ensite default-ssl
          a2enmod ssl
          vm_hostname="$(curl -H "Metadata-Flavor:Google" \
          http://metadata.google.internal/computeMetadata/v1/instance/name)"
          echo "Page served from: $vm_hostname" | \
          tee /var/www/html/index.html
          systemctl restart apache2'
   

2. Create a managed instance group in the us-west1-a zone:

   gcloud compute instance-groups managed create mig-a \
       --zone=us-west1-a \
       --size=2 \
       --template=ext-reg-tcp-proxy-backend-template
   

3. Create a managed instance group in the us-west1-c zone:

   gcloud compute instance-groups managed create mig-c \
       --zone=us-west1-c \
       --size=2 \
       --template=ext-reg-tcp-proxy-backend-template
   

Configure the load balancer

Console

Start your configuration

1. In the Google Cloud console, go to the Load balancing page.

   Go to Load balancing

2. Click Create load balancer.
3. For Type of load balancer, select Network Load Balancer (TCP/UDP/SSL) and click Next.
4. For Proxy or passthrough, select Proxy load balancer and click Next.
5. For Public facing or internal, select Public facing (external) and click Next.
6. For Global or single region deployment, select Best for regional workloads and click Next.
7. Click Configure.

Basic configuration

1. For Name, enter my-ext-tcp-lb.
2. For Region, select us-west1.
3. For Network, select lb-network.

Reserve a proxy-only subnet

Note: If you already created the proxy-only subnet, the Reserve subnet button isn't displayed. You can skip this section and continue with the steps in the Configure the backends section.

1. Click Reserve subnet.
2. For Name, enter proxy-only-subnet.
3. For IP address range, enter 10.129.0.0/23.
4. Click Add.

Configure the backends

1. Click Backend configuration.
2. For Backend type, select Instance group.
3. For Protocol, select TCP.
4. For Named port, enter tcp80.
5. Configure the first backend:
   1. For New backend, select instance group mig-a.
   2. For Port numbers, enter 80.
   3. Retain the remaining default values, and then click Done.
6. Configure the second backend:
   1. Click Add backend.
   2. For New backend, select instance group mig-c.
   3. For Port numbers, enter 80.
   4. Retain the remaining default values, and then click Done.
7. Configure the health check:
   1. For Health check, select Create a health check.
   2. Set the health check name to tcp-health-check.
   3. For Protocol, select TCP.
   4. Set Port to 80.
8. Retain the remaining default values, and then click Save.
9. In the Google Cloud console, verify that there is a check mark next to Backend configuration. If not, double-check that you have completed all of the steps.

Configure the frontend

1. Click Frontend configuration.
2. For Name, enter ext-reg-tcp-forwarding-rule.
3. For Subnetwork, select backend-subnet.
4. For IP address, select the IP address reserved previously: LB_IP_ADDRESS
5. For Port number, enter 110. The forwarding rule only forwards packets with a matching destination port.
6. For Proxy protocol, select Off because the PROXY protocol doesn't work with the Apache HTTP Server software. For more information, see PROXY protocol.
7. Click Done.
8. In the Google Cloud console, verify that there is a check mark next to Frontend configuration. If not, double-check that you have completed all the previous steps.

Review and finalize

1. Click Review and finalize.
2. Review your load balancer configuration settings.
3. Optional: Click Equivalent code to view the REST API request that is used to create the load balancer.
4. Click Create.

gcloud

1. Create a regional health check:

   gcloud compute health-checks create tcp tcp-health-check \
       --region=us-west1 \
       --use-serving-port
   

2. Create a backend service:

   gcloud compute backend-services create ext-reg-tcp-proxy-bs \
       --load-balancing-scheme=EXTERNAL_MANAGED \
       --protocol=TCP \
       --port-name=tcp80 \
       --region=us-west1 \
       --health-checks=tcp-health-check \
       --health-checks-region=us-west1
   

3. Add instance groups to your backend service:

   gcloud compute backend-services add-backend ext-reg-tcp-proxy-bs \
       --region=us-west1 \
       --instance-group=mig-a \
       --instance-group-zone=us-west1-a \
       --balancing-mode=UTILIZATION \
       --max-utilization=0.8
    

   gcloud compute backend-services add-backend ext-reg-tcp-proxy-bs \
       --region=us-west1 \
       --instance-group=mig-c \
       --instance-group-zone=us-west1-c \
       --balancing-mode=UTILIZATION \
       --max-utilization=0.8
   

4. Create a target TCP proxy:

   gcloud compute target-tcp-proxies create ext-reg-tcp-target-proxy \
       --backend-service=ext-reg-tcp-proxy-bs \
       --proxy-header=NONE \
       --region=us-west1
   

   If you want to turn on the proxy header, set it to PROXY_V1 instead of NONE. In this example, don't enable the PROXY protocol because it doesn't work with the Apache HTTP Server software. For more information, see PROXY protocol.

5. Create the forwarding rule. For --ports, specify a single port number from 1-65535. This example uses port 110. The forwarding rule only forwards packets with a matching destination port.

   gcloud compute forwarding-rules create ext-reg-tcp-forwarding-rule \
       --load-balancing-scheme=EXTERNAL_MANAGED \
       --network=lb-network \
       --subnet=backend-subnet \
       --region=us-west1 \
       --target-tcp-proxy=ext-reg-tcp-target-proxy \
       --target-tcp-proxy-region=us-west1 \
       --address=ext-reg-tcp-ip-address \
       --ports=110
   

Test the load balancer

Note: It might take a few minutes for the load balancer configuration to propagate after you first deploy it.

Now that you have configured your load balancer, you can test sending traffic to the load balancer's IP address.

1. Get the load balancer's IP address.

   To get the IPv4 address, run the following command:

   gcloud compute addresses describe ADDRESS_NAME
   

2. Send traffic to your load balancer by running the following command. Replace LB_IP_ADDRESS with your load balancer's IPv4 address.

   curl -m1 LB_IP_ADDRESS:9090
   

Additional configuration options

This section expands on the configuration example to provide alternative and additional configuration options. All of the tasks are optional. You can perform them in any order.

Enable session affinity

The example configuration creates a backend service without session affinity.

These procedures show you how to update a backend service for the example load balancer created previously so that the backend service uses client IP affinity or generated cookie affinity.

When client IP affinity is enabled, the load balancer directs a particular client's requests to the same backend VM based on a hash created from the client's IP address and the load balancer's IP address (the internal IP address of an internal forwarding rule).

To enable client IP session affinity, complete the following steps.

Console

1. In the Google Cloud console, go to the Load balancing page.

   Go to Load balancing

2. Click Backends.

3. Click ext-reg-tcp-proxy-bs (the name of the backend service that you created for this example), and then click Edit.

4. On the Backend service details page, click Advanced configuration.

5. For Session affinity, select Client IP.

6. Click Update.

gcloud

To update the ext-reg-tcp-proxy-bs backend service and specify client IP session affinity, use the gcloud compute backend-services update ext-reg-tcp-proxy-bs command:

gcloud compute backend-services update ext-reg-tcp-proxy-bs \
    --region=us-west1 \
    --session-affinity=CLIENT_IP

What's next

• To learn how external proxy Network Load Balancers work, see the External proxy Network Load Balancer overview.
• To set up logging and monitoring for your load balancer, see Proxy Network Load Balancer logging and monitoring.
• To delete the resources that you created so you are not billed for them in the future, see Clean up the load balancer setup.