Set up a regional internal proxy Network Load Balancer with an external backend

Before following this guide, familiarize yourself with the Internet NEG overview documentation, including the limitations.

The following architecture diagram shows a regional internal proxy Network Load Balancer frontend with an external backend.

Figure 1. A regional internal proxy Network Load Balancer with an external backend (click to enlarge).

Permissions

To follow this guide, you need to create an internet NEG and create or modify a proxy Network Load Balancer in a project. You should be either a project Owner or Editor (roles/owner or roles/editor), or you should have both of the following Compute Engine IAM roles.

Task	Required role
Create and modify load balancer components	Compute Network Admin (roles/compute.networkAdmin)
Create and modify NEGs	Compute Instance Admin (roles/compute.instanceAdmin)

Set up your external backend environment outside Google Cloud

To set up your external backend environment, see the following sections.

Configure network endpoints

Configure a network endpoint to expose your external backend to Google Cloud. Make sure that the endpoint—either an IP:Port combination or a fully-qualified domain name (FQDN) and port—is reachable over the internet. This endpoint is later referenced from the internet NEG.

For detailed configuration requirements for internet NEG endpoints, see the Internet NEGs overview.

Allow the external backend to receive traffic from Google Cloud

This step can be completed after you've created the proxy-only subnet and set up the Cloud NAT gateway.

To allow requests from Google Cloud to reach your external backend, you'll need to perform the following steps:

1. Configure a Cloud NAT gateway with IP addresses that are used for egress traffic from Google Cloud. The gateway maps the proxy-only subnet range to the external IP addresses. For the steps, see Set up a Cloud NAT gateway.
2. Make sure that your external backend environment is configured to allow traffic from Google Cloud to reach the external backend. For example, if you used pre-reserved IP addresses for the NAT gateway, you'll allowlist those IP addresses on your external environment. You'll likely need to work with the network or security admin of your external environment to set this up.

Set up your Google Cloud environment

You'll need a VPC network with two subnets: one for the load balancer components and the other for the region's proxy-only subnet. Then you'll create the load balancer with an internet NEG backend.

Create the VPC network and subnet

This subnet is used to create the load balancer's components.

Cloud console

1. In the Google Cloud console, go to the VPC networks page.
   Go to VPC networks
2. Click Create VPC network.
3. Enter a Name: LB_NETWORK.
4. In the Subnets section:
   • Set the Subnet creation mode to Custom.
   • In the New subnet section, enter the following information:
     • Name: LB_SUBNET_NAME
     • Region: REGION
     • IP address range: LB_SUBNET_RANGE
   • Click Done.
5. Click Create.

gcloud

1. Create the custom VPC network by using the gcloud compute networks create command:

   gcloud compute networks create LB_NETWORK \
     --subnet-mode=custom
   

2. Create a subnet in the LB_NETWORK network.

   gcloud compute networks subnets create LB_SUBNET_NAME \
     --network=LB_NETWORK \
     --range=LB_SUBNET_RANGE \
     --region=REGION
   

Configure the proxy-only subnet

This proxy-only subnet is used by all regional Envoy-based load balancers in the REGION region.

Console

1. In the Google Cloud console, go to the VPC networks page.
   Go to VPC networks
2. Select a Network from the list.
3. Click Add subnet.
4. Enter a Name: PROXY_ONLY_SUBNET_NAME.
5. Select a Region: REGION.
6. Set Purpose to Regional Managed Proxy.
7. Enter an IP address range: PROXY_ONLY_SUBNET_RANGE.
8. Click Add.

gcloud

Create the proxy-only subnet with the gcloud compute networks subnets create command.

gcloud compute networks subnets create PROXY_ONLY_SUBNET_NAME \
  --purpose=REGIONAL_MANAGED_PROXY \
  --role=ACTIVE \
  --region=REGION \
  --network=LB_NETWORK \
  --range=PROXY_ONLY_SUBNET_RANGE

Set up a Cloud NAT gateway

Before you configure the Cloud NAT gateway, make sure you've reviewed the associated limitations and pricing considerations. For details, see Regional NEGs: Use a Cloud NAT gateway.

The following commands describe how to set up a Cloud NAT gateway. The Cloud NAT gateway can be configured to use either automatic NAT external IP addresses, in which allocation is based on demand, or to use a manually pre-reserved set of external IP addresses. The gateway maps the proxy-only subnet range to the external IP addresses.

Set up automatic NAT allocated IP addresses

When you create a Cloud NAT gateway with automatic NAT IP address allocation, you can specify the Network Service Tiers (Premium Tier or Standard Tier) from which the Cloud NAT gateway allocates the IP addresses.

Console

1. In the Google Cloud console, go to the Cloud NAT page.

   Go to Cloud NAT

2. Click Get started or Create Cloud NAT gateway.

   Note: If this is the first Cloud NAT gateway that you're creating, click Get started. If you already have existing gateways, then instead of Get started, Google Cloud displays the Create Cloud NAT gateway button. To create another gateway, click Create Cloud NAT gateway.

3. Enter a gateway name LB_NAT_CONFIG.

4. For NAT type, select Public.

5. In the Network list, select LB_NETWORK.

6. In the Region list, select REGION.

7. Create a Cloud Router in the region.

8. For Source endpoint type, select Managed proxy load balancers.

9. In the Source list, select Custom.

   • For Subnets, select PROXY_ONLY_SUBNET_NAME.

10. In the Cloud NAT IP addresses list, select Automatic (recommended).

11. For Network service tier, choose either Premium or Standard.

12. Click Create.

gcloud

Use dynamically allocated IP addresses if your external backend environment doesn't require you to allowlist specific Google Cloud IP addresses that can send traffic to the external backend.

1. Create a Cloud Router:

   gcloud beta compute routers create ROUTER_NAME \
     --network=LB_NETWORK \
     --region=REGION

2. Set up the Cloud NAT gateway.

   gcloud beta compute routers nats create LB_NAT_CONFIG \
     --router=ROUTER_NAME \
     --endpoint-types=ENDPOINT_TYPE_MANAGED_PROXY_LB \
     --nat-custom-subnet-ip-ranges=PROXY_ONLY_SUBNET_NAME \
     --auto-allocate-nat-external-ips \
     --region=REGION
   

Replace the following:

• LB_NAT_CONFIG: the name of your NAT configuration.

• ROUTER_NAME: the name of your Cloud Router.

• REGION: the region of the NAT to create. If not specified, you might be prompted to select a region (interactive mode only).

• PROXY_ONLY_SUBNET_NAME: the name of your proxy only subnet.

Set up manually allocated IP addresses

Use manually allocated IP addresses only if your external backend environment requires you to use an allowlist for specific Google Cloud IP addresses. If the external backend environment doesn't need an allowlist, use dynamic allocation instead as shown previously.

When creating a Cloud NAT gateway, you can choose to manually assign NAT IP addresses from either Premium Tier or Standard Tier or both, subject to certain conditions.

Warning: If you provision fewer NAT IP addresses than the number of assigned Envoy proxies, requests sent to the internet NEG might result in HTTP 5xx errors. To ensure that you are informed when such an event occurs, set up an alert for the nat_allocation_failed metric. Contact support if you need help calculating the number of IP addresses that must be allocated for your load balancer in a specific region.

Console

1. In the Google Cloud console, go to the Cloud NAT page.

   Go to Cloud NAT

2. Click Get started or Create Cloud NAT gateway.

   Note: If this is the first Cloud NAT gateway that you're creating, click Get started. If you already have existing gateways, then instead of Get started, Google Cloud displays the Create Cloud NAT gateway button. To create another gateway, click Create Cloud NAT gateway.

3. Enter a gateway name LB_NAT_CONFIG.

4. In the Network list, select LB_NETWORK.

5. In the Region list, select REGION.

6. Select or create a Cloud Router in the region.

7. For Source endpoint type, select Managed proxy load balancers.

8. In the Source list, select Custom.

   • In the Subnets, select PROXY_ONLY_SUBNET_NAME.

9. In the Cloud NAT IP addresses list, select Manual.

10. For Network service tier, choose either Premium or Standard.

11. Select or create a static reserved external IP address to use for NAT.

    Note: You can only select or create IP addresses based on the tier that you have selected in the previous step.

12. If you want to specify additional IP addresses, click Add IP address, and then select or create an additional static reserved external IP address.

13. Click Create.

gcloud

1. Create the IP addresses. Because the gateway performs one-to-one NAT translation, you must make sure that the pool of reserved IP addresses is big enough to handle the amount of traffic you're expecting. Insufficiently allocated NAT IP addresses could result in traffic loss.

   gcloud compute addresses create IP_ADDRESS_NAME_1 IP_ADDRESS_NAME_2 [IP_ADDRESS_NAME_3 ...] \
     --region=REGION
   

2. Create a Cloud Router:

   gcloud compute routers create ROUTER_NAME \
     --network=LB_NETWORK \
     --region=REGION

3. Set up the Cloud NAT gateway.

   gcloud beta compute routers nats create LB_NAT_CONFIG \
     --router=ROUTER_NAME \
     --endpoint-types=ENDPOINT_TYPE_MANAGED_PROXY_LB \
     --nat-custom-subnet-ip-ranges=PROXY_ONLY_SUBNET_NAME \
     --nat-external-ip-pool=IP_ADDRESS_NAME_1,IP_ADDRESS_NAME_2,[IP_ADDRESS_NAME_3 ...] \
     --region=REGION
   

   Replace the following:

• LB_NAT_CONFIG: the name of your NAT configuration.

• ROUTER_NAME: the name of your Cloud Router.

• PROXY_ONLY_SUBNET_NAME: the name of your proxy only subnet.

• REGION: the region of the NAT to create. If not specified, you might be prompted to select a region (interactive mode only).

For more information, see Specify subnet ranges for NAT in the Cloud NAT documentation.

Make sure that you use an allowlist for the NAT IP address ranges on your external backend environment, so that your external backend can receive traffic from Google Cloud.

Reserve the load balancer's IP address

Reserve an internal IP address for the load balancer.

Console

You can reserve a standalone internal IP address using the Google Cloud console.

1. Go to the VPC networks page.

   Go to VPC networks

2. Click the network that was used to configure hybrid connectivity between the environments.
3. Click Static internal IP addresses and then click Reserve static address.
4. Enter a Name: LB_IP_ADDRESS.
5. For the Subnet, select LB_SUBNET_NAME.
6. If you want to specify which IP address to reserve, under Static IP address, select Let me choose, and then fill in a Custom IP address. Otherwise, the system automatically assigns an IP address in the subnet for you.
7. If you want to use this IP address with multiple forwarding rules, under Purpose, choose Shared.
8. Click Reserve to finish the process.

gcloud

1. Using the gcloud CLI, run the compute addresses create command:

   gcloud compute addresses create LB_IP_ADDRESS \
     --region=REGION \
     --subnet=LB_SUBNET_NAME \
   

2. Use the compute addresses describe command to view the allocated IP address:

   gcloud compute addresses describe LB_IP_ADDRESS \
     --region=REGION
   

Set up the internet NEG

You can create an internet NEG using either INTERNET_FQDN_PORT endpoints or INTERNET_IP_PORT endpoints.

Console

Create a NEG with INTERNET_FQDN_PORT endpoints

1. In the Google Cloud console, go to the Network endpoint group page.

   Go to Network endpoint group

2. Click Create network endpoint group.

3. Specify an INTERNET_NEG_NAME for your Internet NEG. For more information, see Resource naming convention.

4. In the Network endpoint group type list, select Network endpoint group (Internet) and then do the following:

   • In the Scope list, select Regional.
   • Optional: In the Region list, change the REGION for this NEG.
   • In the Network list, select LB_NETWORK.
   • In the Default port box, enter DEFAULT_PORT_NUMBER.
   • In the Add endpoints through list, select Fully qualified domain name and port.
5. Select Create.

Add INTERNET_FQDN_PORT endpoints to the NEG

1. In the Google Cloud console, go to the Network endpoint group page.

   Go to Network endpoint group

2. Click INTERNET_NEG_NAME.

3. Enter the Fully qualified domain name such as myorg.example.com. You must specify the FQDN objects in standard FQDN syntax.

4. Optional: For Port type, select Custom. If the Port type is Default, the default port of the NEG is used.

5. In the Port number box, enter PORT_NUMBER_1.
6. Select Create.

Create a NEG with INTERNET_IP_PORT endpoints

1. In the Google Cloud console, go to the Network endpoint group page.

   Go to Network endpoint group

2. Click Create network endpoint group.

3. Specify a name INTERNET_NEG_NAME for your Internet NEG. For more information, see Resource naming convention.

4. In the Network endpoint group type list, select Network endpoint group (Internet) and then do the following:

   • In the Scope list, select Regional.
   • Optional: In the Region list, change the REGION for this NEG.
   • In the Network list, select LB_NETWORK.
   • In the Default port box, enter DEFAULT_PORT_NUMBER.
   • In the Add endpoints through list, select IP and port.
5. Select Create.

Add INTERNET_IP_PORT endpoints to the NEG

1. In the Google Cloud console, go to the Network endpoint group page.

   Go to Network endpoint group

2. Click INTERNET_NEG_NAME.
3. In the IP address field, enter IP_ADDRESS_1.

4. Optional: In the Port type list, select Custom. If the Port type is Default, the default port of the NEG is used.

5. In the Port number field, enter a PORT_NUMBER_1.
6. Select Create.

gcloud

To create a NEG with INTERNET_FQDN_PORT endpoints:

1. Create the NEG resource.

   gcloud beta compute network-endpoint-groups create INTERNET_NEG_NAME \
       --network-endpoint-type=INTERNET_FQDN_PORT \
       --default-port=DEFAULT_PORT_NUMBER \
       --network=LB_NETWORK \
       --region=REGION
   

2. Add endpoints to the NEG. If a port isn't specified, the default port of the NEG is used.

   gcloud beta compute network-endpoint-groups update INTERNET_NEG_NAME \
       --add-endpoint="fqdn=FULLY_QUALIFIED_DOMAIN_NAME_1,port=PORT_NUMBER_1" \
       [--add-endpoint="fqdn=FULLY_QUALIFIED_DOMAIN_NAME_2,port=PORT_NUMBER_2" \]
       --region=REGION
   

   Replace the following:

   • FULLY_QUALIFIED_DOMAIN_NAME: the fully qualified domain name for the endpoint
   • PORT_NUMBER: the port number for the endpoint

   You can add up to 256 endpoints per NEG.

If your domain is resolvable over the internet, no other configuration is needed to set up DNS. However, if you're using private FQDNs, you'll need to configure Cloud DNS to facilitate DNS resolution. The name must be hosted on Cloud DNS or be resolvable through DNS forwarding from Cloud DNS to an on-premises DNS.

Start by creating a Cloud DNS zone to host the DNS records in your project. Then add the DNS records to it. Refer the Cloud DNS documentation for specific configuration steps.

To create a NEG with INTERNET_IP_PORT endpoints:

1. Create the NEG resource.

   gcloud beta compute network-endpoint-groups create INTERNET_NEG_NAME \
       --network-endpoint-type=INTERNET_IP_PORT \
       --default-port=DEFAULT_PORT_NUMBER \
       --network=LB_NETWORK \
       --region=REGION
   

2. Add endpoints to the NEG. If a port isn't specified, the default port of the NEG is used.

   gcloud beta compute network-endpoint-groups update INTERNET_NEG_NAME \
       --add-endpoint="ip=IP_ADDRESS_1,port=PORT_NUMBER_1" \
       [--add-endpoint="ip=IP_ADDRESS_2,port=PORT_NUMBER_2" \]
       --region=REGION
   

   Replace the following:

   • IP_ADDRESS: the IP address for the endpoint
   • PORT_NUMBER: the port number for the endpoint

   You can repeat this step to add up to 256 endpoints per NEG.

Create the load balancer

Console

Start your configuration

1. In the Google Cloud console, go to the Load balancing page.

   Go to Load balancing

2. Click Create load balancer.
3. For Type of load balancer, select Network Load Balancer (TCP/UDP/SSL) and click Next.
4. For Proxy or passthrough, select Proxy load balancer and click Next.
5. For Public facing or internal, select Internal and click Next.
6. For Cross-region or single region deployment, select Best for regional workloads and click Next.
7. Click Configure.

Basic configuration

1. Enter a Load balancer name.
2. For Region, select REGION.
3. For Network, select LB_NETWORK.

Reserve a proxy-only subnet

Note: If you already created the proxy-only subnet, the Reserve subnet button isn't displayed. You can skip this section and continue with the steps in the Frontend configuration section.

To reserve a proxy-only subnet:

1. Click Reserve subnet.
2. For Name, enter PROXY_ONLY_SUBNET_NAME.
3. For IP address range, enter PROXY_ONLY_SUBNET_RANGE.
4. Click Add.

Frontend configuration

1. Click Frontend configuration.
2. Enter a Name.
3. For Subnetwork, select LB_SUBNET_NAME.
4. For IP address, select the IP address reserved previously: LB_IP_ADDRESS.
5. For Port number, specify a single port number between 1-65535. The forwarding rule only forwards packets with a matching destination port.
6. Click Done.

Backend configuration

1. Click Backend configuration.
2. Click Backend services and backend buckets.
3. Click Create a backend service.
4. Enter a name.
5. For Backend type, select Internet network endpoint group.
6. For Protocol, select TCP.
7. For Backends, in the New backend window, select the Regional internet network endpoint group created in the previous step.
8. Click Done.
9. Configure the health check:
   1. For Health check, select Create a health check.
   2. Set the health check name to TCP_HEALTH_CHECK_NAME.
   3. For Protocol, select TCP.
   4. Set Port to 80.
   5. Retain the remaining default values, and then click Save.
10. Click Create.

Review and finalize

1. Click Review and finalize.
2. If everything looks correct, click Create.

gcloud

1. Optional: Create a health check. Health check probes for external backends use the distributed Envoy health checks and are later NAT-translated.

     gcloud compute health-checks create tcp TCP_HEALTH_CHECK_NAME \
         --region=REGION \
         --use-serving-port
     

2. Create a backend service:

     gcloud compute backend-services create BACKEND_SERVICE \
         --load-balancing-scheme=INTERNAL_MANAGED \
         --health-checks=TCP_HEALTH_CHECK_NAME \
         --health-checks-region=REGION \
         --region=REGION
     

3. Add the internet NEG to the backend service:

     gcloud compute backend-services add-backend BACKEND_SERVICE \
         --network-endpoint-group=INTERNET_NEG_NAME \
         --network-endpoint-group-region=REGION \
         --region=REGION
      

     gcloud compute backend-services add-backend BACKEND_SERVICE \
         --network-endpoint-group=INTERNET_NEG_NAME \
         --network-endpoint-group-region=REGION \
         --region=REGION
      

4. Create a target TCP proxy to route requests to your backend service.

     gcloud compute target-tcp-proxies create TARGET_TCP_PROXY_NAME \
         --backend-service=BACKEND_SERVICE \
         --region=REGION
     

5. Create a forwarding rule to route incoming requests to the proxy. For `--ports`, specify a single port number from 1-65535. The forwarding rule only forwards packets with a matching destination port.

     gcloud compute forwarding-rules create FORWARDING_RULE_NAME \
         --load-balancing-scheme=INTERNAL_MANAGED \
         --network-tier=PREMIUM \
         --network=LB_NETWORK \
         --subnet=LB_SUBNET_NAME \
         --address=LB_IP_ADDRESS \
         --target-tcp-proxy=TARGET_TCP_PROXY_NAME \
         --target-tcp-proxy-region=REGION \
         --region=REGION \
         --ports=PORT_NUMBER
     

Test the load balancer

Now that you have configured your load balancer, you can start sending traffic to the load balancer's IP address.

Create a client VM

This example creates a client VM (vm-client) in the same region as the load balancer. The client is used to validate the load balancer's configuration and demonstrate expected behavior.

gcloud

The client VM can be in any zone in the same REGION as the load balancer, and it can use any subnet in the same VPC network.

gcloud compute instances create vm-client \
    --image-family=debian-10 \
    --image-project=debian-cloud \
    --tags=allow-ssh \
    --network=LB_NETWORK \
    --subnet=LB_SUBNET_NAME \
    --zone=ZONE

Send traffic to the load balancer

It might take a few minutes for the load balancer configuration to propagate after you first deploy it.

1. Connect via SSH to the client instance.

   gcloud compute ssh vm-client \
     --zone=ZONE
   

2. Verify that the load balancer is serving backend hostnames as expected.

   1. Use the compute addresses describe command to view the load balancer's IP address:

      gcloud compute addresses describe LB_IP_ADDRESS \
        --region=REGION
      

      Make a note of the IP address.

   2. You can use curl to send traffic to the load balancer on the IP address and port specified when creating the load balancer forwarding rule. Testing whether the internet NEG backends are responding to requests depends on the service running on the external endpoints.

Additional configuration options

To enable even more capabilities for your regional internal proxy Network Load Balancer, such as global access (if your clients are in a different region), see Set up an internal Application Load Balancer with VM instance group backends: Additional configuration.

What's next

• To set up monitoring for your regional internal proxy Network Load Balancer, see Using monitoring.
• Clean up the load balancer setup.