Configure load balancing networking mode

Version 1.16 

• Latest
• Version 1.16
• Earlier versions (unsupported)

Preview

This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA products and features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

This document describes the two Dataplane V2 networking modes for load balancing and how to enable them for GKE on Bare Metal. Prior to GKE on Bare Metal release 1.16.0, the Dataplane V2 networking mode for load balancing isn't configurable. Dataplane V2 supported only source network address translation (SNAT) load balancing. With cluster versions 1.16.0 and higher, you can configure the load balancing network mode to either SNAT (default) or Direct Server Return (DSR) when you create a cluster.

SNAT load balancing

Source network address translation (SNAT) is the default network mode for Dataplane V2 load balancing. However, in SNAT mode, the client IP address isn't preserved for backend Pods. When the packet from the client arrives at the load balancer node, it's translated and forwarded to the destination worker node with the backend Pod. The backend Pod sees that the request is coming from the load balancer node, instead of the client location. As a result, the reply is returned to the load balancer node and reversed translated and sent back to the client.

Packet flow for SNAT

With SNAT networking mode for Dataplane V2 load balancing, here's the packet flow from an external client to a backend Pod and back:

• Service of type LoadBalancer is assigned to the load balancer node and its IP address 172.16.20.16 is advertised by the MetalLB speaker running on that node.

• Dataplane V2 translates the source IP address and port with SNAT to <LB_NODE_IP>:52000 and forwards the packet to the worker node.

• The response is sent back to the load balancer node and the destination address is reverse translated.

The following diagram shows the packet flow for SNAT mode:

DSR load balancing

Direct Server Return (DSR) overcomes issues with SNAT load balancing. In DSR mode, the load balancer node uses IP Options to save the client source address. The worker node can recover the original packet and forward it to the backend Pod. The backend Pod can see that the packet is coming from client IP address instead of the load balancer node IP address. As a result, the return packet directly returns to the client IP address instead of traveling back to the load balancer node.

This mode not only solves the client IP address visibility problem, it also saves the bandwidth for the load balancer node. Return traffic doesn't have to go through the load balancer node, and the load balancer node doesn't have to do connection tracking anymore. This approach saves memory and frees the forwarding port. For asymmetric workloads, where download traffic is much higher than request traffic, the DSR flow reduces bandwidth significantly.

Packet flow for DSR

With DSR networking mode for Dataplane V2 load balancing, here's the packet flow from and external client to a backend Pod and back:

• Service of type LoadBalancer is assigned to the load balancer node and its IP address 172.16.20.16 is advertised by the MetalLB speaker running on that node.

• Dataplane V2 forwards the packet to the worker node and uses IP Options to save the original client source IP address and port.

• Dataplane V2 on the worker node forwards the packet to the backend Pod.

• The backend Pod recovers the source IP address and port and replies to the client.

The following diagram shows the packet flow for DSR mode:

DSR limitation

DSR is supported for non-tunnel mode only, which means the flat-mode network model has to be enabled. For instructions on enabling the flat-mode network model, see Implement flat IPv4 mode network model. If the flat-mode network model isn't enabled, the preflight check should fail if you try to enable DSR.

Enable DSR mode

As noted earlier, SNAT mode is enabled by default. To enable DSR mode, add the following annotation to your cluster configuration file before you create your cluster. While in Preview, this feature uses an annotation preview.baremetal.cluster.gke.io/dpv2-lbmode-dsr: enable to enable DSR mode. To disable DSR mode, remove the annotation or set it to disable. Other values aren't accepted, and raise validation errors.

apiVersion: baremetal.cluster.gke.io/v1
kind: Cluster
metadata:
  name: bm
  namespace: cluster-bm
  annotations:
    preview.baremetal.cluster.gke.io/dpv2-lbmode-dsr: enable  # valid options: enable or disable
spec:
...

The value can't be modified after the cluster is created. Make sure you've configured the correct option for your needs before you create the cluster.