Traffic distribution for internal passthrough Network Load Balancers

This page describes the following concepts to help you further understand and customize how internal passthrough Network Load Balancers distribute traffic: session affinity, connection tracking, UDP fragmentation, and failover.

The way that an internal passthrough Network Load Balancer distributes new connections depends on whether you have configured failover:

• If you haven't configured failover, an internal passthrough Network Load Balancer distributes new connections to its healthy backend VMs if at least one backend VM is healthy. When all backend VMs are unhealthy, the load balancer distributes new connections among all backends as a last resort. In this situation, the load balancer routes each new connection to an unhealthy backend VM.

• If you have configured failover, an internal passthrough Network Load Balancer distributes new connections among VMs in its active pool, according to a failover policy that you configure. When all backend VMs are unhealthy, you can choose from one of the following behaviors:

  • (Default) The load balancer distributes traffic to only the primary VMs. This is done as a last resort. The backup VMs are excluded from this last-resort distribution of connections.
  • The load balancer is configured to drop traffic.

The method for distributing new connections depends on the load balancer's session affinity setting.

The health check state controls the distribution of new connections. By default, TCP connections persist on unhealthy backends. For more details and how to change this behavior, see Connection persistence on unhealthy backends.

Backend selection and connection tracking

Internal passthrough Network Load Balancers use configurable backend selection and connection tracking algorithms to determine how traffic is distributed to backend VMs. Internal passthrough Network Load Balancers use the following algorithm to distribute packets among backend VMs (in its active pool, if you have configured failover):

1. If the load balancer has an entry in its connection tracking table matching the characteristics of an incoming packet, the packet is sent to the backend specified by the connection tracking table entry. The packet is considered to be part of a previously established connection, so the packet is sent to the backend VM that the load balancer previously determined and recorded in its connection tracking table.

2. If the load balancer receives a packet for which it has no connection tracking entry, the load balancer does the following:

   1. The load balancer selects a backend. The load balancer calculates a hash based on the configured session affinity. It uses this hash to select a backend:

      • If at least one backend is healthy, the hash selects one of the healthy backends.
      • If all backends are unhealthy, and there's no failover policy configured, the hash selects one of the backends.
      • If all backends are unhealthy, there is a failover policy configured, and the failover policy is not configured to drop traffic in this situation, the hash selects one of the primary VM backends.

      The default session affinity, NONE, uses a 5-tuple hash of the packet's source IP address, source port, destination IP address, destination port, and the protocol

      Backend selection can be customized by using a hash algorithm that uses fewer pieces of information. For all the supported options, see session affinity options.

   2. The load balancer adds an entry to its connection tracking table. This entry records the selected backend for the packet's connection so that all future packets from this connection are sent to the same backend. Whether connection tracking is used depends on the protocol:

      For TCP and UDP packets, connection tracking is always enabled, and cannot be turned off. By default, connection tracking is 5-tuple, but it can be configured to be less than 5-tuple.

      When the connection tracking hash is 5-tuple, TCP SYN packets always create a new entry in the connection tracking table.
      
      The default 5-tuple connection tracking is used when:

      • tracking mode is PER_CONNECTION (all session affinities), or,
      • tracking mode is PER_SESSION and the session affinity is NONE, or,
      • tracking mode is PER_SESSION and the session affinity is CLIENT_IP_PORT_PROTO.

      For additional details about when connection tracking is enabled, and what tracking method is used when connection tracking is enabled, see connection tracking mode.

      In addition, note the following:

      • By default, an entry in the connection tracking table expires 600 seconds after the load balancer processes the last packet that matched the entry. For details about how to customize the idle timeout, see Idle timeout.
      • Depending on the protocol, the load balancer might remove connection tracking table entries when backends become unhealthy. For details and how to customize this behavior, see Connection persistence on unhealthy backends.

Session affinity options

Session affinity controls the distribution of new connections from clients to the load balancer's backend VMs. You set session affinity when your backend VMs need to keep track of state information for their clients. This is a common requirement for web applications.

Session affinity works on a best-effort basis.

Internal passthrough Network Load Balancers support the following session affinity options, which you specify for the entire internal backend service, not per backend instance group.

• None (NONE). 5-tuple hash of source IP address, source port, protocol, destination IP address, and destination port
• Client IP, no destination (CLIENT_IP_NO_DESTINATION). 1-tuple hash created from just the source IP address.
• Client IP (CLIENT_IP). 2-tuple hash of source IP address and destination IP address. External passthrough Network Load Balancers call this session affinity option Client IP, Destination IP.
• Client IP, Destination IP, Protocol (CLIENT_IP_PROTO). 3-tuple hash of source IP address, destination IP address, and protocol
• Client IP, Client Port, Destination IP, Destination Port, Protocol (CLIENT_IP_PORT_PROTO). 5-tuple hash of source IP address, source port, protocol, destination IP address, and destination port

Unless you use the load balancer as a next hop for a custom static route, a packet's destination IP address must match the IP address of the load balancer's forwarding rule for the packet to be delivered to the load balancer. For considerations when using the load balancer as a next hop for a custom static route, Session affinity and next hop internal passthrough Network Load Balancer.

Note: For internal UDP load balancers, setting session affinity is supported in the gcloud CLI and the API. You cannot set session affinity for UDP traffic by using the Google Cloud console.

Session affinity and next hop internal passthrough Network Load Balancer

When you configure a static route to use a next hop internal passthrough Network Load Balancer, the load balancer uses the same backend selection and connection tracking method previously discussed. Backend selection is still accomplished by calculating a hash according to the configured session affinity. Except for CLIENT_IP_NO_DESTINATION session affinity, the backend selection hash depends, in part, on the packet destination IP address.

When a internal passthrough Network Load Balancer load balancer is a next hop of a static route, the destination IP address is not limited to the forwarding rule IP address of the load balancer. Instead, the destination IP address of the packet can be any IP address that fits within the destination range of the static route.

If the number of configured and healthy backend VMs is constant (when failover is either not configured, or, when failover is configured but no failover or failback events occur), the load balancer behaves as follows:

• If there's just one configured and healthy backend VM (in the active pool, if failover is configured), it doesn't matter what session affinity you use because all hashes are mapped to the one backend VM.

• If there are two or more configured and healthy backend VMs (in the active pool, if failover is configured), your choice of session affinity is important:

  • If you need the same backend VM to process all packets from a client, based solely on the source IP address of the packet, regardless of the packet destination IP addresses, use CLIENT_IP_NO_DESTINATION session affinity. Depending on traffic patterns, some backend VMs might receive more packets or more connections than other backend VMs.

  • If you use a session affinity option that's not CLIENT_IP_NO_DESTINATION, the load balancer selects a backend VM based on information that at least includes both the source IP address and the destination IP address of the packet. Packets sent from the same client, using the same source IP address, but different destination IP addresses, can be routed to different backend VMs.

Connection tracking policy

This section describes the settings that control the connection tracking behavior of internal passthrough Network Load Balancers. A connection tracking policy includes the following settings:

• Tracking mode
• Connection persistence on unhealthy backends
• Idle timeout

Connection tracking mode

Tracking mode specifies the connection tracking algorithm to be used. There are two tracking modes: PER_CONNECTION (default) and PER_SESSION.

• PER_CONNECTION (default). In this mode, TCP and UDP traffic are always tracked per 5-tuple, regardless of the session affinity setting. This implies that the key for connection tracking (5-tuple) can be different from the configured session affinity setting (for example, 3-tuple with the CLIENT_IP_PROTO setting). As a result, the session affinity may be split and new connections for a session may select a different backend if the set of backends or their health changes.

• PER_SESSION. In this mode, TCP and UDP traffic is tracked according to the configured session affinity. That is, if session affinity is CLIENT_IP or CLIENT_IP_PROTO, configuring this mode results in 2-tuple and 3-tuple connection tracking, respectively. This might be desirable in scenarios where breaking affinity is expensive and should be avoided even after more backends are added.

The connection tracking mode setting is redundant if session affinity is set to NONE or CLIENT_IP_PORT_PROTO. To learn how these tracking modes work with different session affinity settings for each protocol, see the following table.

Backend selection		Connection tracking mode
Session affinity setting	Hash method for backend selection	PER_CONNECTION (default)	PER_SESSION
Default: No session affinity NONE	5-tuple hash	5-tuple connection tracking	5-tuple connection tracking
Client IP, no destination CLIENT_IP_NO_DESTINATION	1-tuple hash	5-tuple connection tracking	1-tuple connection tracking
Client IP CLIENT_IP (same as Client IP, Destination IP for external passthrough Network Load Balancers)	2-tuple hash	5-tuple connection tracking	2-tuple connection tracking
Client IP, Destination IP, Protocol CLIENT_IP_PROTO	3-tuple hash	5-tuple connection tracking	3-tuple connection tracking
Client IP, Client Port, Destination IP, Destination Port, Protocol CLIENT_IP_PORT_PROTO	5-tuple hash	5-tuple connection tracking	5-tuple connection tracking

To learn how to change the connection tracking mode, see Configure a connection tracking policy.

Connection persistence on unhealthy backends

The connection persistence on unhealthy backends settings control whether an existing connection persists on a selected backend after that backend becomes unhealthy (as long as the backend remains in the load balancer's configured backend instance group).

The behavior described in this section does not apply to cases where you remove a backend VM from its instance group, or remove the instance group from the backend service. In such cases, established connections only persist as described in Enabling connection draining.

The following connection persistence options are available:

• DEFAULT_FOR_PROTOCOL (default)
• NEVER_PERSIST
• ALWAYS_PERSIST

The following table summarizes connection persistence options and how connections persist for different protocols, session affinity options, and tracking modes.

Connection persistence on unhealthy backends option	Connection tracking mode
	PER_CONNECTION	PER_SESSION
DEFAULT_FOR_PROTOCOL	TCP: connections persist on unhealthy backends (all session affinities) UDP: connections never persist on unhealthy backends	TCP: connections persist on unhealthy backends if session affinity is NONE or CLIENT_IP_PORT_PROTO UDP: connections never persist on unhealthy backends
NEVER_PERSIST	TCP, UDP: connections never persist on unhealthy backends
ALWAYS_PERSIST	TCP, UDP: connections persist on unhealthy backends (all session affinities) This option should only be used for advanced use cases.	Configuration not possible

To learn how to change connection persistence behavior, see Configure a connection tracking policy.

Note: When the load balancer moves a connection from an unhealthy backend, the load balancer creates a new tracking table entry. The new entry maps the connection to a different backend based on the configured session affinity. The new entry remains valid just as any other entry would. If the unhealthy backend returns to being healthy, that alone does not cause the connection tracking table entry to be removed (unless it triggers a failback and connection draining on failover and failback has been disabled).

Idle timeout

By default, an entry in the connection tracking table expires 600 seconds after the load balancer processes the last packet that matched the entry. This default idle timeout value can be modified only when the connection tracking is less than 5-tuple (that is, when session affinity is configured to be either CLIENT_IP or CLIENT_IP_PROTO, and the tracking mode is PER_SESSION).

The maximum configurable idle timeout value is 57,600 seconds (16 hours).

To learn how to change the idle timeout value, see Configure a connection tracking policy.

Connections for single-client deployments

When testing connections to the IP address of an internal passthrough Network Load Balancer that only has one client, you should keep the following in mind:

• If the client VM is not a VM being load balanced—that is, it is not also a backend VM, new connections are delivered to the load balancer's healthy backend VMs. However, because all session affinity options rely on at least the client system's IP address, connections from the same client might be distributed to the same backend VM more frequently than you might expect.

  Practically, this means that you cannot accurately monitor traffic distribution through an internal passthrough Network Load Balancer by connecting to it from a single client. The number of clients needed to monitor traffic distribution varies depending on the load balancer type, the type of traffic, and the number of healthy backends.

• If the client VM is also a backend VM of the load balancer, connections sent to the IP address of the load balancer's forwarding rule are always answered by the same backend VM (which is also the client VM). This happens regardless of whether the backend VM is healthy. It happens for all traffic sent to the load balancer's IP address, not just traffic on the protocol and ports specified in the load balancer's internal forwarding rule.

  For more information, see sending requests from load-balanced VMs.

Connection draining

Connection draining is a process applied to established connections in the following cases:

• A VM or endpoint is removed from a backend (instance group or NEG).
• A backend removes a VM or endpoint (by replacement, abandonment, when rolling upgrades, or scaling down).

By default, connection draining is disabled. When disabled, established connections are terminated as quickly as possible. When connection draining is enabled, established connections are allowed to persist for a configurable timeout, after which the backend VM instance is terminated.

For more details about how connection draining is triggered and how to enable connection draining, see Enabling connection draining.

UDP fragmentation

Internal passthrough Network Load Balancers can process both fragmented and unfragmented UDP packets. If your application uses fragmented UDP packets, keep the following in mind:

• UDP packets might become fragmented before reaching a Google Cloud VPC network.
• Google Cloud VPC networks forward UDP fragments as they arrive (without waiting for all fragments to arrive).
• Non-Google Cloud networks and on-premises network equipment might forward UDP fragments as they arrive, delay fragmented UDP packets until all fragments have arrived, or discard fragmented UDP packets. For details, see the documentation for the network provider or network equipment.

If you expect fragmented UDP packets and need to route them to the same backends, use the following forwarding rule and backend service configuration parameters:

• Forwarding rule configuration: Use only one UDP forwarding rule per load-balanced IP address, and configure the forwarding rule to accept traffic on all ports. This ensures that all fragments arrive at the same forwarding rule. Even though the fragmented packets (other than the first fragment) lack a destination port, configuring the forwarding rule to process traffic for all ports also configures it to receive UDP fragments that have no port information. To configure all ports, either use the Google Cloud CLI to set --ports=ALL or use the API to set allPorts to True.

• Backend service configuration: Set the backend service's session affinity to CLIENT_IP (2-tuple hash) or CLIENT_IP_PROTO (3-tuple hash) so that the same backend is selected for UDP packets that include port information and UDP fragments (other than the first fragment) that lack port information. Set the backend service's connection tracking mode to PER_SESSION so that the connection tracking table entries are built by using the same 2-tuple or 3-tuple hashes.

Failover

An internal passthrough Network Load Balancer lets you designate some backends as failover backends. These backends are only used when the number of healthy VMs in the primary backend instance groups has fallen below a configurable threshold. By default, if all primary and failover VMs are unhealthy, as a last resort Google Cloud distributes new connections only among all the primary VMs.

When you add a backend to an internal passthrough Network Load Balancer's backend service, by default that backend is a primary backend. You can designate a backend to be a failover backend when you add it to the load balancer's backend service, or by editing the backend service later.

For a detailed conceptual overview of failover in internal passthrough Network Load Balancers, see Failover for internal passthrough Network Load Balancers.

What's next

• To configure and test an internal passthrough Network Load Balancer that uses failover, see Configure failover for internal passthrough Network Load Balancers.
• To configure and test an internal passthrough Network Load Balancer, see Set up an internal passthrough Network Load Balancer.