Internal TCP/UDP load balancers as next hops

Internal TCP/UDP Load Balancing is a regional load balancer that enables you to run and scale your services behind an internal load balancing IP address. You can use an internal TCP/UDP load balancer as the next gateway to which packets are forwarded along the path to their final destination. To do this, you set the load balancer as the next hop in a custom static route.

This configuration is useful in the following cases:

• When you need to load balance traffic across multiple VMs that are functioning as virtual network address translation (NAT) appliances.

• When you need a load balancer to serve as a next hop for a default route. When virtual machine (VM) instances in your Virtual Private Cloud (VPC) network send traffic to the internet, the traffic is routed through load-balanced gateway virtual appliances.

• When you need to send traffic through multiple load balancers in two or more directions by using the same set of multi-NIC VMs as backends. To do this, you create a load balancer and use it as the next hop for a custom static route in each VPC network. Each internal TCP/UDP load balancer operates within a single VPC network, distributing traffic to the network interfaces of backend VMs in that network.

Load balancing to multiple NICs (click to enlarge)

In the preceding diagram, a VM instance group serves as the backend for two different load balancers. This use case is called internal TCP/UDP load balancers as next hops with common backends because multiple NICs (nic0 and nic1, in this case) on the backend VMs are being load balanced. This deployment is allowed because Internal TCP/UDP Load Balancing supports load balancing to any interface on backend VM instances (not just the primary interface nic0).

In contrast, two sets of VM instances can serve as the backends for two different load balancers. You might use two sets of backends when you have different traffic profiles for incoming and outgoing traffic. This use case is called internal TCP/UDP load balancers as next hops with different backends because only the primary interfaces (nic0) on the backend VMs are load balanced. This setup is shown in the following diagram:

Load balancing to single NICs (click to enlarge)

You can create a custom static route to pass TCP and UDP traffic to an internal TCP/UDP load balancer where the load balancer is the next hop for the static route. The route can be the default route (0.0.0.0/0), an external (publicly-routable) CIDR prefix, or an internal CIDR prefix, if the prefix doesn't conflict with a subnet route. For example, you can replace your default route (0.0.0.0/0) with a route that directs traffic to third-party backend VMs for packet processing.

To use the custom static route, the VM instances in each VPC network must be in the same region as their associated load balancer.

If a Cloud Router is in the same region as a load balancer, you can advertise this route to other connected networks by using Cloud Router custom route advertisements. For more information, see Internal TCP/UDP Load Balancing and connected networks.

Caution: When sending traffic to Google APIs and services, don't route the traffic through a next-hop VM or through a next-hop internal TCP/UDP load balancer. Instead, route such traffic through a next-hop default internet gateway. This includes traffic to Google APIs and services sent by means of Private Google Access.

For more information, see:

• Load balancing to multiple backend NICs
• Load balancing to a single backend NIC

Benefits of using your internal TCP/UDP load balancer as a next hop

When the load balancer is a next hop for a static route, you don't need to explicitly configure your clients to send traffic to the load balancer or to each backend VM. You can integrate your backend VMs in a bump-in-the-wire fashion.

Using an internal TCP/UDP load balancer as a next hop for a static route provides the same benefits as Internal TCP/UDP Load Balancing. The load balancer's health check ensures that new connections are routed to healthy backend VMs. By using a managed instance group as a backend, you can configure autoscaling to grow or shrink the set of VMs based on service demand.

Specifications

Following are specifications for using internal TCP/UDP load balancers as next hops.

Client IP session affinity

Client IP session affinity is one available session affinity option. It is a two-tuple affinity that uses the source IP address and the destination IP address as inputs for a hash function.

When using an internal TCP/UDP load balancer by itself, the destination IP address is the IP address of the load balancer's forwarding rule. Client IP session affinity in this context means that connections from a client with a constant source IP address are delivered to the same backend VM if the backend VM is healthy.

In contrast, when using an internal TCP/UDP load balancer as a next hop for a static route, the destination IP address varies because the load balancer's backend VMs process and route packets to different destinations. Using Client IP session affinity in this context doesn't cause packets to be processed by the same backend VM, even if the client has a constant source IP address.

Destination range

The destination of a custom static route cannot be equal to or more specific than a subnet route. Note that more specific means that the subnet mask is longer. This rule applies to all custom static routes, including when the next hop is an internal TCP/UDP load balancer. For example, suppose that your subnet route is 10.140.0.0/20. The destination of a custom static route can't be the same (10.140.0.0/20), and it can't be more specific, as in 10.140.0.0/22.

Same VPC network and region

Custom static routes that use internal TCP/UDP load balancers as next hops are limited to the following:

• A single VPC network. The load balancer and the custom static route must be in the same VPC network.

• A single region or all regions. Unless you configure global access, the custom static route is only available to resources in the same region as the load balancer. This regional restriction is enforced even though the route itself is part of the routing table for the entire VPC network. If you enable global access, the custom static route is available to resources in any region.

Order of operations

You must create an internal TCP/UDP load balancer before you can create a custom static route that uses it as a next hop. The load balancer must exist before you can create the route. If you try to create a route that refers to a nonexistent load balancer, Google Cloud returns an error.

You specify an internal TCP/UDP load balancer next hop by using the forwarding rule's name and the load balancer's region, not by using the internal IP address associated with the forwarding rule.

After you've created a route with a next hop that refers to an internal TCP/UDP load balancer, you cannot delete the load balancer unless you first delete the route. Specifically, you can't delete an internal forwarding rule until no custom static route uses that forwarding rule as a next hop.

Backend requirements

• You must configure all the internal TCP/UDP load balancer's backend VMs to allow IP forwarding (--can-ip-forward = True). For more information, see Considerations for instance-based or load balancer-based routing.

• You cannot use an internal TCP/UDP load balancer whose backends are Google Kubernetes Engine (GKE) nodes as a next hop for a custom static route. Software on the nodes can only route traffic to Pods if the destination matches an IP address managed by the cluster, not an arbitrary destination.

Processing of TCP, UDP, and other protocol traffic

When an internal TCP/UDP load balancer is deployed as a next hop, Google Cloud forwards all TCP and UDP traffic on all ports to the backend VMs, regardless of the following:

• The forwarding rule's protocol and port configuration
• The backend service's protocol configuration

The load balancer processes only TCP and UDP traffic and ignores all other traffic, such as ICMP. Traffic that does not use the TCP or UDP protocol is handled by the next most specific route in your VPC network. The routes for non-TCP and non-UDP traffic have the following characteristics:

• They do not have an internal TCP/UDP load balancer as the next hop.
• They are chosen according to the routing order.

For example, suppose you have the following routes in your VPC network:

• Destination: 1.1.1.1/32, Next hop: internal TCP/UDP load balancer
• Destination: 1.0.0.0/8, Next hop: default internet gateway

For clients that are located in the same region as the load balancer (or in any region when global access is enabled), TCP and UDP traffic destined to 1.1.1.1/32 uses the route with the internal TCP/UDP load balancer next hop.

For non-TCP and non-UDP traffic (such as ICMP pings), the 1.0.0.0/8 route is used instead.

Additional specifications

• A custom static route cannot use network tags if the route has an internal TCP/UDP load balancer as the route's next hop.

• You cannot create multiple custom static routes with the same priority, the same destination, and the same internal TCP/UDP load balancer next hop. Consequently, each custom static route with the same next hop load balancer must have at least a unique destination or a unique priority.

• If you have different internal TCP/UDP load balancers as next hops for multiple routes that have the same destination and priority, Google Cloud doesn't distribute the traffic among the load balancers. Instead, Google Cloud chooses only one of the load balancers as the next hop for all traffic that matches the destination and ignores the other load balancers.

• When you're using an internal TCP/UDP load balancer as the next hop for a custom static route, you can't use the --purpose=SHARED_LOADBALANCER_VIP flag with the internal forwarding rule's IP address. This is because the shared internal IP address can indirectly reference more than one backend service.

For more information, see the Routes overview.

Use cases

You can use an internal TCP/UDP load balancer as a next hop in multiple deployments and topologies.

For each example, note the following guidelines:

• Each VM interface must be in a separate VPC network.

• You cannot use backend VMs or load balancers to route traffic between subnets in the same VPC network because subnet routes cannot be overridden.

• The internal TCP/UDP load balancer is a software-defined pass-through load balancer. NAT and proxying are only performed by the backend VMs—the firewall virtual appliances.

Using an internal TCP/UDP load balancer as the next hop to a NAT gateway

This use case load balances traffic from internal VMs to multiple NAT gateway instances that route traffic to the internet.

NAT use case (click to enlarge)

Hub and spoke: Exchanging next-hop routes by using VPC Network Peering

In addition to exchanging subnet routes, you can configure VPC Network Peering to export and import custom static and dynamic routes. Custom static routes that use network tags or that have a next hop of the default internet gateway are excluded.

You can configure a hub-and-spoke topology with your next-hop firewall virtual appliances located in the hub VPC network by doing the following:

• In the hub VPC network, create an internal TCP/UDP load balancer with firewall virtual appliances as the backends.
• In the hub VPC network, create a custom static route, and set the next hop to be the internal TCP/UDP load balancer.
• Connect the hub VPC network to each of the spoke VPC networks by using VPC Network Peering.
• For each peering, configure the hub network to export its custom routes, and configure the corresponding spoke network to import custom routes. The route with the load balancer next hop is one of the routes that the hub network exports.

The next hop firewall appliance load balancer in the hub VPC network is usable in each spoke network, according to the routing order.

Hub and spoke (click to enlarge)

Load balancing to multiple NICs

In the following use case, the backend VMs are virtual appliance instances (for example, firewall instances or NAT gateways) with NICs in multiple VPC networks. Firewall instances and NAT gateways can be provided by a third party as virtual appliances. The virtual appliances are Compute Engine VMs with multiple NICs.

This example shows a single set of backend virtual appliances in a managed VM instance group.

In the VPC network called testing, the internal TCP/UDP load balancer has a forwarding rule called fr-ilb1. In the example, this load balancer distributes traffic to the nic0 interface on each virtual appliance, but it could be any NIC.

In the VPC network called production, a different internal TCP/UDP load balancer has a forwarding rule called fr-ilb2. This load balancer distributes traffic to a different interface, nic1 in this example.

Traffic with multi-NIC load balancing (click to enlarge)

Traffic flows when load balancing to multiple NICs (click arrow to expand)

When sending a packet, the request and response paths are as follows.

Request path Traffic Flow 1

1. The client VM in the VPC network named testing sends a packet with source IP address 10.30.1.100 and destination IP address 10.50.1.100.
1. In the testing VPC network, a custom static route for the 10.50.1.0/24 destination sends the packet to the load balancer that uses the fr-ilb1 forwarding rule.
2. The load balancer delivers the packet to the nic0 interface of a healthy backend VM. In this example, the backend VM that is selected is firewall-instance-a, a firewall virtual appliance.
2. In addition to performing its firewall filtering function, the backend instance uses routing configured within its operating system to modify the packet, performing Source Network Address Translation (SNAT). SNAT changes the packet's source IP address to the IP address of the firewall instance's secondary network interface, nic1, which is located in the VPC network named production. In this example, firewall-instance-a changes the source IP address of the packet to the IP address of its nic1 interface, 10.50.1.2.
1. The firewall instance consults its internal routing table, and sends the packet out to nic1.
2. A subnet route in the production VPC network delivers the packet to the VM instance at 10.50.1.100. The received packet has a source IP address belonging to nic1 of the firewall instance that processed the packet (firewall-instance-a, 10.50.1.2 for this example), and the destination IP address 10.50.1.100.

Response path Traffic Flow 1

When the VM instance at 10.50.1.100 sends an established response packet, that packet is routed as follows, so that it is delivered to the client at 10.30.1.100:

3. The response packet has a source IP address of 10.50.1.100 and a destination IP address belonging to the firewall instance that sent the corresponding incoming packet. For this example, the IP address is 10.50.1.2, on nic1 of firewall-instance-a. The response packet must be sent to the same firewall instance that delivered the corresponding incoming packet. A subnet route in the production VPC network delivers the packet to that firewall instance.
1. Routing configured within the operating system of the firewall instance modifies the packet, performing Destination Network Address Translation (DNAT), changing the destination to 10.30.1.100.
2. The firewall instance consults its internal routing table, and delivers the packet to the instance's network interface (nic0), which is located in the testing VPC network.
4. A subnet route in the testing VPC network delivers the packet to the client instance at 10.30.1.100. The received packet has a source IP address of 10.50.1.100 and a destination IP address of 10.30.1.100.

Request path Traffic Flow 2

5. The client VM in the VPC network named production sends a packet with source IP address 10.50.1.100 and destination IP address 10.30.1.100.
1. In the production VPC network, a custom static route for the 10.30.1.0/24 destination sends the packet to the load balancer that uses the fr-ilb2 forwarding rule.
2. The load balancer delivers the packet to the nic1 interface of a healthy backend VM. In this example, the backend VM selected is firewall-instance-a, a firewall virtual appliance.
6. Routing configured within the operating system of the firewall instance modifies the packet, performing SNAT, changing the packet's source IP address to the IP address of the firewall instance's primary network interface, nic0, which is located in the VPC network named testing. In this example, firewall-instance-a changes the source IP address of the packet to the IP address of its nic0 interface, 10.30.1.2.
1. The firewall instance consults its internal routing table, and sends the packet out to nic0.
2. A subnet route in the testing VPC network delivers the packet to the VM instance at 10.30.1.100. The received packet has a source IP address belonging to nic0 of the firewall instance that processed the packet (firewall-instance-a, 10.30.1.2 for this example), and the destination IP address 10.30.1.100.

Response path Traffic Flow 2

When the VM instance at 10.30.1.100 sends an established response packet, that packet is routed as follows, so that it is delivered to the client at 10.50.1.100:

7. The response packet has a source IP address of 10.30.1.100 and a destination IP address belonging to the firewall instance that sent the corresponding incoming packet. For this example, the IP address is 10.30.1.2, on nic0 of firewall-instance-a. The response packet must be sent to the same firewall instance that delivered the corresponding incoming packet. A subnet route in the testing VPC network delivers the packet to that firewall instance.
1. Routing configured within the operating system of the firewall instance modifies the packet, performing DNAT, changing the destination to 10.50.1.100.
2. The firewall instance consults its internal routing table, and delivers the packet to the instance's network interface (nic1), which is located in the production VPC network.
8. A subnet route in the production VPC network delivers the packet to the client instance at 10.50.1.100. The received packet has a source IP address of 10.30.1.100 and a destination IP address of 10.50.1.100.

For a detailed configuration setup, see Load balancing to multiple backend NICs.

Using an internal TCP/UDP load balancer as the next hop with a single NIC

Like the previous use case, the backend VMs are virtual appliance instances (for example, firewall instances or NAT gateways) with NICs in multiple VPC networks.

The following examples show two sets of backend virtual appliances in two managed VM instance groups.

The backends are summarized as follows.

Load-balanced instances for testing-to-production traffic	Load-balanced instances for production-to-testing traffic
fw-instance-a fw-instance-b fw-instance-c	fw-instance-d fw-instance-e fw-instance-f

In the testing-to-production direction, traffic originates in the VPC network called testing. In the testing network, the internal TCP/UDP load balancer has a forwarding rule called fr-ilb1. This example uses internal TCP/UDP load balancers with backend services that do not have a specified network parameter. This means that each load balancer only distributes traffic to the primary network interface of each backend VM.

Traffic with single-NIC load balancing (testing-to-production) (click to enlarge)

Traffic flows when load balancing to a single NIC testing-to-production (click arrow to expand)

When sending a packet, the request and response paths are as follows.

Request path traffic flow testing-to-production

1. The client VM in the VPC network named testing sends a packet with source IP address 10.30.1.100 and destination IP address 10.50.1.100.
1. In the testing VPC network, a custom static route for the 10.50.1.0/24 destination sends the packet to the load balancer that uses the fr-ilb1 forwarding rule.
2. The load balancer delivers the packet to the nic0 interface of a healthy backend VM. In this example, the backend VM selected is firewall-instance-a, a firewall virtual appliance.
2. In addition to performing its filtering function, the firewall instance uses routing configured within its operating system to modify the packet, performing Source Network Address Translation (SNAT). SNAT changes the packet's source IP address to the IP address of the firewall instance's secondary network interface, nic1, which is located in the VPC network named production. In this example, firewall-instance-a changes the source IP address of the packet to the IP address of its nic1 interface, 10.50.1.2.
1. The firewall instance consults its internal routing table, and sends the packet out to nic1.
2. A subnet route in the production VPC network delivers the packet to the VM instance at 10.50.1.100. The received packet has a source IP address belonging to nic1 of the firewall instance that processed the packet (firewall-instance-a, 10.50.1.2 for this example), and the destination IP address 10.50.1.100.

Response path traffic flow production-to-testing

When the VM instance at 10.50.1.100 sends an established response packet, that packet is routed as follows, so that it is delivered to the client at 10.30.1.100:

3. The response packet has a source IP address of 10.50.1.100 and a destination IP address belonging to the firewall instance that sent the corresponding incoming packet. For this example, the IP address is 10.50.1.2, on nic1 of firewall-instance-a. The response packet must be sent to the same firewall instance that delivered the corresponding incoming packet. A subnet route in the production VPC network delivers the packet to that firewall instance.
1. Routing configured within the operating system of the firewall instance modifies the packet, performing Destination Network Address Translation (DNAT), changing the destination to 10.30.1.100.
2. The firewall instance consults its internal routing table, and delivers the packet to the instance's network interface (nic0), which is located in the testing VPC network.
4. A subnet route in the testing VPC network delivers the packet to the client instance at 10.30.1.100. The received packet has a source IP address of 10.50.1.100 and a destination IP address of 10.30.1.100.

In the production-to-testing direction, traffic originates in the VPC network called production. In the production network, a different internal TCP/UDP load balancer has a forwarding rule called fr-ilb2. Again, this example uses internal TCP/UDP load balancers with backend services that do not have a specified network parameter. This means that each load balancer only distributes traffic to the primary network interface of each backend VM. Because each virtual appliance can have only one nic0, this deployment requires a second set of virtual appliances, the first set for testing-to-production load balancing and the second set for production-to-testing load balancing.

Traffic with single-NIC load balancing (production-to-testing) (click to enlarge)

Traffic flows when load balancing to a single NIC production-to-testing (click arrow to expand)

When sending a packet, the request and response paths are as follows.

Request path traffic flow production-to-testing

5. The client VM in the VPC network named production sends a packet with source IP address 10.50.1.100 and destination IP address 10.30.1.100.
1. In the production VPC network, a custom static route for the 10.30.1.0/24 destination sends the packet to the load balancer that uses the fr-ilb2 forwarding rule.
2. The load balancer delivers the packet to the nic0 interface of a healthy backend VM. In this example, the backend VM selected is firewall-instance-d, a firewall virtual appliance.
6. Routing configured within the operating system of the firewall instance modifies the packet, performing SNAT, changing the packet's source IP address to the IP address of the firewall instance's secondary network interface, nic1, which is located in the VPC network named testing. In this example, firewall-instance-d changes the source IP address of the packet to the IP address of its nic1 interface, 10.30.1.5.
1. The firewall instance consults its internal routing table, and sends the packet out to nic1.
2. A subnet route in the testing VPC network delivers the packet to the VM instance at 10.30.1.100. The received packet has a source IP address belonging to nic1 of the firewall instance that processed the packet (firewall-instance-d, 10.30.1.5 for this example), and the destination IP address 10.30.1.100.

Response path traffic flow testing-to-production

When the VM instance at 10.30.1.100 sends an established response packet, that packet is routed as follows, so that it is delivered to the client at 10.50.1.100:

7. The response packet has a source IP address of 10.30.1.100 and a destination IP address belonging to the firewall instance that sent the corresponding incoming packet. For this example, the IP address is 10.30.1.5, on nic1 of firewall-instance-d. The response packet must be sent to the same firewall instance that delivered the corresponding incoming packet. A subnet route in the testing VPC network delivers the packet to that firewall instance.
1. Routing configured within the operating system of the firewall instance modifies the packet, performing DNAT, changing the destination to 10.50.1.100.
2. The firewall instance consults its internal routing table, and delivers the packet to the instance's network interface (nic0), which is located in the production VPC network.
8. A subnet route in the production VPC network delivers the packet to the client instance at 10.50.1.100. The received packet has a source IP address of 10.30.1.100 and a destination IP address of 10.50.1.100.

What's next

• To configure an internal TCP/UDP load balancer to be a next hop, see Setting up Internal TCP/UDP Load Balancing for third-party appliances or Deploying centralized VM-based appliances using internal TCP/UDP load balancer as the next hop.
• To configure and test an internal TCP/UDP load balancer, see Setting up Internal TCP/UDP Load Balancing.
• To troubleshoot next hop issues with your internal TCP/UDP load balancer, see Troubleshooting Internal TCP/UDP Load Balancing.