Internal regional TCP proxy load balancer overview

The Google Cloud internal regional TCP proxy load balancer is a proxy-based regional load balancer powered by open source Envoy proxy software and the Andromeda network virtualization stack.

This load balancer provides a single regional internal IPv4 address to host your TCP services. An internal regional TCP proxy load balancer first terminates the TCP connection between the client and the load balancer at an Envoy proxy. The proxy opens a second TCP connection to backends hosted in Google Cloud, on-premises, or other cloud environments.

The internal regional TCP proxy load balancer uses the Premium Network Service Tier.

Benefits

The internal regional TCP proxy load balancer supports the following:

Load balancing to backends on Google Cloud, on-premises, or other cloud environments. This is especially useful when used with Private Service Connect to expose producer services from workloads on-premises to consumers in Google Cloud.
Intelligent routing. The load balancer routes requests to backends within a region based on configurable target capacities.
Port remapping. The port used by the load balancer's forwarding rule does not have to match the port used when making connections to its backends. For example, the forwarding rule could use TCP port 80, while the connection to the backends could use TCP port 8080.
Listening on any port. You can configure the load balancer's forwarding rule to listen on any port you choose.
Locality policies. Within a backend instance group or network endpoint group, you can configure how requests are distributed to member instances or endpoints.
Global access. When global access is enabled, clients from any region can access the load balancer and its backends.

Architecture

The following diagram shows the Google Cloud resources required for an internal regional TCP proxy load balancer.

Internal regional TCP proxy load balancer components (click to enlarge). — Internal regional TCP proxy load balancer components (click to enlarge)

Proxy-only subnet

In the diagram above, the proxy-only subnet provides a set of IP addresses that Google uses to run Envoy proxies on your behalf. You must create a proxy-only subnet in each region of a VPC network where you use an Envoy-based load balancer (such as the internal regional TCP proxy load balancer, internal HTTP(S) load balancer, etc.) All Envoy-based load balancers in a region and VPC network share the same proxy-only subnet. The purpose of the proxy-only subnet must be set to REGIONAL_MANAGED_PROXY. Further:

Proxy-only subnets are only used for Envoy proxies, not your backends.
Backend VMs or endpoints of all Envoy-based load balancers in a region and VPC network receive connections from the same proxy-only subnet.
The IP address of the load balancer is not located in the proxy-only subnet. The load balancer's IP address is defined by its internal managed forwarding rule, which is described below.

Forwarding rules and IP addresses

Regional forwarding rules route traffic by IP address, port, and protocol to a load balancing configuration consisting of a target proxy and a backend service.

A regional internal managed forwarding rule specifies an internal IP address, port, and regional target TCP proxy. Clients use the IP address and port to connect to the load balancer's Envoy proxies – the forwarding rule's IP address is the IP address of the load balancer (sometimes called a virtual IP address or VIP).

The internal IP address associated with the forwarding rule can come from any subnet in the same network and region. Note that:

The IP address can (but does not need to) come from the same subnet as the backend instance groups.
The IP address must not come from the reserved proxy-only subnet that has its --purpose flag set to REGIONAL_MANAGED_PROXY.
If you want to share an internal IP address with multiple forwarding rules, set the --purpose flag to SHARED_LOADBALANCER_VIP.

Internal managed forwarding rules support all possible ports but each forwarding rule can reference exactly one port.

Forwarding rules and global access

An internal regional TCP proxy load balancer's forwarding rules are regional, even when global access is enabled. After you enable global access, the regional internal forwarding rule's allowGlobalAccess flag is set to true.

Target proxies

The internal regional TCP proxy load balancer terminates TCP connections from the client and creates new connections to the backends. By default, the original client IP address and port information is not preserved. You can preserve this information by using the PROXY protocol. The target proxy routes incoming requests directly to the load balancer's backend service.

Backend service

A backend service directs incoming traffic to one or more attached backends. A backend is either an instance group or a network endpoint group. The backend contains balancing mode information to define fullness based on connections (or, for instance group backends only, utilization).

Each internal regional TCP proxy load balancer has a single backend service resource.

Supported backends

The backend service supports the following types of backends:

All instance group backends: instance groups can be zonal unmanaged, zonal managed, or regional managed
All network endpoint group backends: zonal NEGs with GCE_VM_IP_PORT endpoints and hybrid connectivity NEGs with NON_GCP_PRIVATE_IP_PORT endpoints are supported.

All of the backends must be of the same type (instance groups or NEGs). You can simultaneously use different types of instance group backends, or you can simultaneously use different types of NEG backends, but you cannot use instance group and NEG backends together on the same backend service.

You can mix zonal NEGs and hybrid NEGs within the same backend service.

To ensure minimal interruptions to your users, you can enable connection draining on backend services. Such interruptions might happen when a backend is terminated, removed manually, or removed by an autoscaler. To learn more about using connection draining to minimize service interruptions, see Enabling connection draining.

Protocol for communicating with the backends

When you configure a backend service for an internal regional TCP proxy load balancer, you set the protocol that the backend service uses to communicate with the backends. The load balancer uses only the protocol that you specify, and does not attempt to negotiate a connection with the other protocol. The internal regional TCP proxy load balancer only supports TCP for communicating with the backends.

Health check

Each backend service specifies a health check that periodically monitors the backends' readiness to receive a connection from the load balancer. This reduces the risk that requests might be sent to backends that can't service the request. Health checks do not check if the application itself is working.

Firewall rules

An internal regional TCP proxy load balancer requires the following firewall rules:

An ingress allow rule that permits traffic from the Google health check probes.
- 35.191.0.0/16
- 130.211.0.0/22
Currently, health check probes for hybrid NEGs originate from Google's centralized health checking mechanism. If you cannot allow traffic that originates from the Google health check ranges to reach your hybrid endpoints and would prefer to have the health check probes originate from private IP addresses instead, speak to your Google account representative to get your project allowlisted for distributed Envoy health checks.
An ingress allow rule that permits traffic from the proxy-only subnet.

The ports for these firewall rules must be configured as follows:

Allow traffic to the destination port for each backend service's health check.
For instance group backends: Determine the ports to be configured by the mapping between the backend service's named port and the port numbers associated with that named port on each instance group. The port numbers can vary among instance groups assigned to the same backend service.
For GCE_VM_IP_PORT NEG backends: Allow traffic to the port numbers of the endpoints.

Client access

By default, clients must be in the same region as the load balancer. Clients can be in the same network or in a VPC network connected by using VPC Network Peering. You can enable global access to allow clients from any region to access your load balancer.

Internal regional TCP proxy load balancer with global access (click to enlarge)

The following table summarizes client access.

Global access disabled	Global access enabled
Clients must be in the same region as the load balancer. They also must be in the same VPC network as the load balancer or in a VPC network that is connected to the load balancer's VPC network by using VPC Network Peering.	Clients can be in any region. They still must be in the same VPC network as the load balancer or in a VPC network that's connected to the load balancer's VPC network by using VPC Network Peering.
On-premises clients can access the load balancer through Cloud VPN tunnels or VLAN attachments. These tunnels or attachments must be in the same region as the load balancer.	On-premises clients can access the load balancer through Cloud VPN tunnels or VLAN attachments. These tunnels or attachments can be in any region.

Shared VPC architecture

The internal regional TCP proxy load balancer supports networks that use Shared VPC. Shared VPC lets you maintain a clear separation of responsibilities between network administrators and service developers. Your development teams can focus on building services in service projects, and the network infrastructure teams can provision and administer load balancing. If you're not already familiar with Shared VPC, read the Shared VPC overview documentation.

IP address	Forwarding rule	Target proxy	Backend components
An internal IP address must be defined in the same project as the backends. For the load balancer to be available in a Shared VPC network, the internal IP address must be defined in the same service project where the backend VMs are located, and it must reference a subnet in the desired Shared VPC network in the host project. The address itself comes from the primary IP range of the referenced subnet.	An internal forwarding rule must be defined in the same project as the backends. For the load balancer to be available in a Shared VPC network, the internal forwarding rule must be defined in the same service project where the backend VMs are located, and it must reference the same subnet (in the Shared VPC network) that the associated internal IP address references.	The target proxy must be defined in the same project as the backends.	In a Shared VPC scenario, the backend VMs are typically located in a service project. A regional internal backend service and health check must be defined in that service project.

IP address

Forwarding rule

Target proxy

Backend components

An internal IP address must be defined in the same project as the backends.

For the load balancer to be available in a Shared VPC network, the internal IP address must be defined in the same service project where the backend VMs are located, and it must reference a subnet in the desired Shared VPC network in the host project. The address itself comes from the primary IP range of the referenced subnet.

An internal forwarding rule must be defined in the same project as the backends.

For the load balancer to be available in a Shared VPC network, the internal forwarding rule must be defined in the same service project where the backend VMs are located, and it must reference the same subnet (in the Shared VPC network) that the associated internal IP address references.

The target proxy must be defined in the same project as the backends.

In a Shared VPC scenario, the backend VMs are typically located in a service project. A regional internal backend service and health check must be defined in that service project.

Traffic distribution

An internal regional TCP proxy load balancer distributes traffic to its backends as follows:

Connections originating from a single client are sent to the same zone as long as healthy backends (instance groups or NEGs) within that zone are available and have capacity, as determined by the balancing mode. For internal regional TCP proxy load balancers, the balancing mode can be CONNECTION (instance group or NEG backends) or UTILIZATION (instance group backends only).
Connections from a client are sent to the same backend if you have configured session affinity.
After a backend is selected, traffic is then distributed among instances (in an instance group) or endpoints (in a NEG) according to a load balancing policy. For the load balancing policy algorithms supported, see the localityLbPolicy setting in the regional backend service API documentation.

Session affinity

Session affinity allows you to configure the load balancer's backend service to send all requests from the same client to the same backend, as long as the backend is healthy and has capacity.

The internal regional TCP proxy load balancer offers client IP affinity, which forwards all requests from the same client IP address to the same backend, as long as that backend has capacity and remains healthy.

Failover

If a backend becomes unhealthy, traffic is automatically redirected to healthy backends within the same region. The load balancer implements a gentle failover algorithm per zone. Rather than waiting for all the backends in a zone to become unhealthy, the load balancer starts redirecting traffic to a different zone when the ratio of healthy to unhealthy backends in any zone falls below a certain percentage threshold (70%; this threshold is non-configurable). If all backends in all zones are unhealthy, the load balancer immediately terminates the client connection.

Load balancing for GKE applications

If you are building applications in Google Kubernetes Engine, you can use standalone zonal NEGs to load balance traffic directly to containers. With standalone NEGs you are responsible for creating the Service object that creates the zonal NEG, and then associating the NEG with the backend service so that the load balancer can connect to the Pods.

Quotas and limits

For information about quotas and limits, see Load balancing resource quotas.

Limitations

The internal regional TCP proxy load balancer does not support IPv6 traffic.
The internal regional TCP proxy load balancer does not support Shared VPC deployments where the load balancer's frontend is in one host or service project and the backend service and backends are in another service project (also known as cross-project service referencing).