Traffic distribution for external passthrough Network Load Balancers

This page describes the following concepts to help you further understand and customize how external passthrough Network Load Balancers distribute traffic: session affinity, connection tracking, weighted load balancing, connection draining, UDP fragmentation, and failover.

The way that external passthrough Network Load Balancers distribute new connections depends on whether you have configured failover:

• If you haven't configured failover, an external passthrough Network Load Balancer distributes new connections to its healthy backend VMs if at least one backend VM is healthy. When all backend VMs are unhealthy, the load balancer distributes new connections among all backends as a last resort. In this situation, the load balancer routes each new connection to an unhealthy backend VM.
• If you have configured failover, an external passthrough Network Load Balancer distributes new connections among healthy backend VMs in its active pool, according to a failover policy that you configure. When all backend VMs are unhealthy, you can choose from one of the following behaviors:
  • (Default) The load balancer distributes traffic to only the primary VMs. This is done as a last resort. The backup VMs are excluded from this last-resort distribution of connections.
  • The load balancer drops traffic.

For details about how connections are distributed, see the next section Backend selection and connection tracking.

For details about how failover works, see the Failover section.

Backend selection and connection tracking

External passthrough Network Load Balancers use configurable backend selection and connection tracking algorithms to determine how traffic is distributed to backend VMs.

External passthrough Network Load Balancers use the following algorithm to distribute packets among backend VMs (in its active pool, if you have configured failover):

1. If the load balancer has an entry in its connection tracking table matching the characteristics of an incoming packet, the packet is sent to the backend specified by the connection tracking table entry. The packet is considered to be part of a previously established connection, so the packet is sent to the backend VM that the load balancer previously determined and recorded in its connection tracking table.

2. If the load balancer receives a packet for which it has no connection tracking entry, the load balancer does the following:

   1. The load balancer selects a backend. The load balancer calculates a hash based on the configured session affinity. It uses this hash to select a backend from among the ones that are healthy (unless all backends are unhealthy, in which case all backends are considered as long as the failover policy hasn't been configured to drop traffic in this situation). The default session affinity, NONE, uses the following hash algorithms:

      • For TCP and unfragmented UDP packets, a 5-tuple hash of the packet's source IP address, source port, destination IP address, destination port, and the protocol.
      • For fragmented UDP packets and all other protocols, a 3-tuple hash of the packet's source IP address, destination IP address, and the protocol.

      Backend selection can be customized by using a hash algorithm that uses fewer pieces of information. For all the supported options, see session affinity options.

      In addition, note the following:

      If you enable weighted load balancing, the hash based backend selection becomes weighted based on the weights reported by backend instances. For example:

      • Consider a backend service configured with session affinity set to NONE and a forwarding rule with protocol UDP. If there are two healthy backend instances with weights 1 and 4, then the backends will get 20% and 80% of the UDP packets respectively.
      • Consider a backend service that is configured with 3-tuple session affinity and connection tracking. The session affinity is CLIENT_IP_PROTO and connection tracking mode is PER_SESSION. If there are three healthy backend instances with weights 0, 2 and 6, then the backends will get traffic for 0%, 25%, and 75% of the new source IP addresses (the source IP addresses for which there are no existing connection tracking table entries) respectively. Traffic for existing connections will go to the previously assigned backends.
        Caution: If the weights change frequently, weighted load balancing can be detrimental to session affinity. There are multiple load-balancers and connection tracking is not replicated across them. For sustained session affinity, it is recommended that you don't use weighted load balancing or use it infrequently.

   2. The load balancer adds an entry to its connection tracking table. This entry records the selected backend for the packet's connection so that all future packets from this connection are sent to the same backend. Whether connection tracking is used depends on the protocol:

      • TCP packets. Connection tracking is always enabled, and cannot be turned off. By default, connection tracking is 5-tuple, but it can be configured to be less than 5-tuple. When it is 5-tuple, TCP SYN packets are treated differently. Unlike non-SYN packets, they discard any matching connection tracking entry and always select a new backend.
        
        The default 5-tuple connection tracking is used when:

        • tracking mode is PER_CONNECTION (all session affinities), or,
        • tracking mode is PER_SESSION and the session affinity is NONE, or,
        • tracking mode is PER_SESSION and the session affinity is CLIENT_IP_PORT_PROTO.

      • UDP, ESP, and GRE packets. Connection tracking is enabled only if session affinity is set to something other than NONE.

      • ICMP and ICMPv6 packets. Connection tracking cannot be used.

      For additional details about when connection tracking is enabled, and what tracking method is used when connection tracking is enabled, see connection tracking mode.

      In addition, note the following:

      • An entry in the connection tracking table expires 60 seconds after the load balancer processes the last packet that matched the entry. For more information, see Idle timeout.
      • Depending on the protocol, the load balancer might remove connection tracking table entries when backends become unhealthy. For details and how to customize this behavior, see Connection persistence on unhealthy backends.

Session affinity options

Session affinity controls the distribution of new connections from clients to the load balancer's backend VMs. Session affinity is specified for the entire regional external backend service, not on a per backend basis.

External passthrough Network Load Balancers support the following session affinity options:

• None (NONE). 5-tuple hash of source IP address, source port, protocol, destination IP address, and destination port
• Client IP, Destination IP (CLIENT_IP). 2-tuple hash of source IP address and destination IP address
• Client IP, Destination IP, Protocol (CLIENT_IP_PROTO). 3-tuple hash of source IP address, destination IP address, and protocol
• Client IP, Client Port, Destination IP, Destination Port, Protocol (CLIENT_IP_PORT_PROTO). 5-tuple hash of source IP address, source port, protocol, destination IP address, and destination port

To learn how these session affinity options affect the backend selection and connection tracking methods, see this table.

Connection tracking policy

This section describes the settings that control the connection tracking behavior of external passthrough Network Load Balancers. A connection tracking policy includes the following settings:

• Tracking mode
• Connection persistence on unhealthy backends
• Idle timeout

Connection tracking mode

Whether connection tracking is enabled depends only on the protocol of the load-balanced traffic and the session affinity settings. Tracking mode specifies the connection tracking algorithm to be used when connection tracking is enabled. There are two tracking modes: PER_CONNECTION (default) and PER_SESSION.

• PER_CONNECTION (default). This is the default tracking mode. With this connection tracking mode, TCP traffic is always tracked per 5-tuple, regardless of the session affinity setting. For UDP, ESP, and GRE traffic, connection tracking is enabled when the selected session affinity is not NONE. UDP, ESP, and GRE packets are tracked using the tracking methods described in this table.

• PER_SESSION. If session affinity is CLIENT_IP or CLIENT_IP_PROTO, configuring this mode results in 2-tuple and 3-tuple connection tracking, respectively, for all protocols (except ICMP and ICMPv6, which are not connection-trackable). For other session affinity settings, PER_SESSION mode behaves identically to PER_CONNECTION mode.

To learn how these tracking modes work with different session affinity settings for each protocol, see the following table.

Backend selection		Connection tracking mode
Session affinity setting	Hash method for backend selection	PER_CONNECTION (default)	PER_SESSION
Default: No session affinity (NONE)	TCP and unfragmented UDP: 5-tuple hash Fragmented UDP and all other protocols: 3-tuple hash	TCP: 5-tuple connection tracking All other protocols: connection tracking off	TCP: 5-tuple connection tracking All other protocols: connection tracking off
Client IP, Destination IP (CLIENT_IP)	All protocols: 2-tuple hash	TCP and unfragmented UDP: 5-tuple connection tracking Fragmented UDP, ESP, and GRE: 3-tuple connection tracking All other protocols: connection tracking off	TCP, UDP, ESP, GRE: 2-tuple connection tracking All other protocols: connection tracking off
Client IP, Destination IP, Protocol (CLIENT_IP_PROTO)	All protocols: 3-tuple hash	TCP and unfragmented UDP: 5-tuple connection tracking Fragmented UDP, ESP, and GRE: 3-tuple connection tracking All other protocols: connection tracking off	TCP, UDP, ESP, GRE: 3-tuple connection tracking All other protocols: connection tracking off
Client IP, Client Port, Destination IP, Destination Port, Protocol (CLIENT_IP_PORT_PROTO)	TCP and unfragmented UDP: 5-tuple hash Fragmented UDP and all other protocols: 3-tuple hash	TCP and unfragmented UDP: 5-tuple connection tracking Fragmented UDP, ESP, and GRE: 3-tuple connection tracking All other protocols: connection tracking off	TCP and unfragmented UDP: 5-tuple connection tracking Fragmented UDP, ESP, and GRE: 3-tuple connection tracking All other protocols: connection tracking off

To learn how to change the connection tracking mode, see Configure a connection tracking policy.

Connection persistence on unhealthy backends

The connection persistence settings control whether an existing connection persists on a selected backend VM or endpoint after that backend becomes unhealthy, as long as the backend remains in the load balancer's configured backend group (in an instance group or a NEG).

The behavior described in this section does not apply to cases where you remove a backend instance from an instance group or remove a backend endpoint from its NEG, or remove the instance group or the NEG from the backend service. In such cases, established connections only persist as described in connection draining.

The following connection persistence options are available:

• DEFAULT_FOR_PROTOCOL (default)
• NEVER_PERSIST
• ALWAYS_PERSIST

The following table summarizes connection persistence options and how connections persist for different protocols, session affinity options, and tracking modes.

Connection persistence on unhealthy backends option	Connection tracking mode
	PER_CONNECTION	PER_SESSION
DEFAULT_FOR_PROTOCOL	TCP: connections persist on unhealthy backends (all session affinities) All other protocols: connections never persist on unhealthy backends	TCP: connections persist on unhealthy backends if session affinity is NONE or CLIENT_IP_PORT_PROTO All other protocols: connections never persist on unhealthy backends
NEVER_PERSIST	All protocols: connections never persist on unhealthy backends
ALWAYS_PERSIST	TCP: connections persist on unhealthy backends (all session affinities) ESP, GRE, UDP: connections persist on unhealthy backends if session affinity is not NONE ICMP, ICMPv6: not applicable because they are not connection-trackable This option should only be used for advanced use cases.	Configuration not possible

Note: When the load balancer moves a connection from an unhealthy backend, the load balancer creates a new tracking table entry. The new entry maps the connection to a different backend based on the configured session affinity. The new entry remains valid just as any other entry would. If the unhealthy backend returns to being healthy, that alone does not cause the connection tracking table entry to be removed (unless it triggers a failback and connection draining on failover and failback has been disabled).

TCP connection persistence behavior on unhealthy backends

Whenever a TCP connection with 5-tuple tracking persists on an unhealthy backend:

• If the unhealthy backend continues to respond to packets, the connection continues until it is reset or closed (by either the unhealthy backend or the client).
• If the unhealthy backend sends a TCP reset (RST) packet or does not respond to packets, then the client might retry with a new connection, letting the load balancer select a different, healthy backend. TCP SYN packets always select a new, healthy backend.

To learn how to change connection persistence behavior, see Configure a connection tracking policy.

Idle timeout

Entries in connection tracking tables expire 60 seconds after the load balancer processes the last packet that matched the entry. This idle timeout value can't be modified.

Weighted load balancing

Caution: If the weights change frequently, weighted load balancing can be detrimental to session affinity. There are multiple load-balancers and connection tracking is not replicated across them. For sustained session affinity, we recommend that you not use weighted load balancing or use it infrequently.

You can configure a network load balancer to distribute traffic across the load balancer's backend instances based on the weights reported by an HTTP health check using weighted load balancing.

Weighted load balancing requires that you configure both of the following:

• You must set the locality load balancer policy (localityLbPolicy) of the backend service to WEIGHTED_MAGLEV.
• You must configure the backend service with an HTTP health check. The HTTP health check responses must contain a custom HTTP response header field X-Load-Balancing-Endpoint-Weight to specify the weights with values from 0 to 1000 for each backend instance.

If you use the same backend (instance group or NEG) for multiple backend service-based external passthrough Network Load Balancers using weighted load balancing, we recommend using a unique request-path for each health check of the backend service. This allows the endpoint instance to assign different weights to different backend services. For more information, see Success criteria for HTTP, HTTPS, and HTTP/2 health checks.

For selecting a backend for a new connection, backends are assigned a strict priority order based on their weight and health status as shown in the following table:

Weight	Healthy	Backend selection priority
Weight greater than zero	Yes	4
Weight greater than zero	No	3
Weight equals zero	Yes	2
Weight equals zero	No	1

Only the highest priority backends are eligible for backend selection. If all eligible backends have zero weight, the load balancer distributes new connections among all eligible backends, treating them with equal weight. For examples of weighted load balancing, see Backend selection and connection tracking.

Weighted load balancing can be used in the following scenarios:

• If some connections process more data than others, or some connections live longer than others, the backend load distribution might get uneven. By signaling a lower per-instance weight, an instance with high load can reduce its share of new connections, while it keeps servicing existing connections.

• If a backend is overloaded and assigning more connections might break existing connections, such backends assign zero weight to itself. By signaling zero weight, a backend instance stops servicing new connections, but continues to service existing ones.

• If a backend is draining existing connections before maintenance, it assigns zero weight to itself. By signaling zero weight, the backend instance stops servicing new connections, but continues to service existing ones.

For more information, see Configure weighted load balancing.

Connection draining

Connection draining is a process applied to established connections in the following cases:

• A VM or endpoint is removed from a backend (instance group or NEG).
• A backend removes a VM or endpoint (by replacement, abandonment, when rolling upgrades, or scaling down).
• A backend is removed from a backend service.

By default, connection draining is disabled. When disabled, established connections are terminated as quickly as possible. When connection draining is enabled, established connections are allowed to persist for a configurable timeout, after which the backend VM instance is terminated.

For more details about how connection draining is triggered and how to enable connection draining, see Enabling connection draining.

UDP fragmentation

Note: The following recommendations for how to handle fragmented UDP packets with backend service-based external passthrough Network Load Balancers are different from the recommendations for legacy target pool-based external passthrough Network Load Balancers.

Backend service-based external passthrough Network Load Balancers can process both fragmented and unfragmented UDP packets. If your application uses fragmented UDP packets, keep the following in mind:

• UDP packets might become fragmented before reaching a Google Cloud VPC network.
• Google Cloud VPC networks forward UDP fragments as they arrive (without waiting for all fragments to arrive).
• Non-Google Cloud networks and on-premises network equipment might forward UDP fragments as they arrive, delay fragmented UDP packets until all fragments have arrived, or discard fragmented UDP packets. For details, see the documentation for the network provider or network equipment.

If you expect fragmented UDP packets and need to route them to the same backends, use the following forwarding rule and backend service configuration parameters:

• Forwarding rule configuration: Use only one UDP or L3_DEFAULT forwarding rule per load-balanced IP address, and configure the forwarding rule to accept traffic on all ports. This ensures that all fragments arrive at the same forwarding rule. Even though the fragmented packets (other than the first fragment) lack a destination port, configuring the forwarding rule to process traffic for all ports also configures it to receive UDP fragments that have no port information. To configure all ports, either use the Google Cloud CLI to set --ports=ALL or use the API to set allPorts to True.

• Backend service configuration: Set the backend service's session affinity to CLIENT_IP (2-tuple hash) or CLIENT_IP_PROTO (3-tuple hash) so that the same backend is selected for UDP packets that include port information and UDP fragments (other than the first fragment) that lack port information. Set the backend service's connection tracking mode to PER_SESSION so that the connection tracking table entries are built by using the same 2-tuple or 3-tuple hashes.

Failover

You can configure an external passthrough Network Load Balancer to distribute connections among VM instances or endpoints in primary backends (instance groups or NEGs), and then switch, if needed, to using failover backends. Failover provides yet another method of increasing availability, while also giving you greater control over how to manage your workload when your primary backends aren't healthy.

By default, when you add a backend to an external passthrough Network Load Balancer's backend service, that backend is a primary backend. You can designate a backend to be a failover backend when you add it to the load balancer's backend service, or by editing the backend service later.

For more details about how failover works, see Failover overview for external passthrough Network Load Balancers.

What's next

• To configure an external passthrough Network Load Balancer with a backend service for TCP or UDP traffic only (supporting IPv4 and IPv6 traffic), see Set up an external passthrough Network Load Balancer with a backend service.
• To configure an external passthrough Network Load Balancer for multiple IP protocols (supporting IPv4 and IPv6 traffic), see Set up an external passthrough Network Load Balancer for multiple IP protocols.
• To configure an external passthrough Network Load Balancer with a zonal NEG backend, see Set up an external passthrough Network Load Balancer with zonal NEGs
• To learn how to transition an external passthrough Network Load Balancer from a target pool backend to a regional backend service, see Transitioning an external passthrough Network Load Balancer from a target pool to a backend service.