IP masquerade agent


This page explains how IP masquerading works in Google Kubernetes Engine (GKE) and provides configuration options for different scenarios.

Overview

IP masquerading is a form of source network address translation (SNAT) used to perform many-to-one IP address translations. GKE can use IP masquerading to change the source IP addresses of packets sent from Pods. When IP masquerading applies to a packet emitted by a Pod, GKE changes the packet's source address from the Pod IP to the underlying node's IP address. Masquerading a packet's source is useful when a recipient is configured to receive packets only from the cluster's node IP addresses.

On Linux nodes, GKE configures iptables rules. GKE uses the ip-masq-agent DaemonSet to configure the appropriate dataplane. IP masquerading is not supported with Windows Server node pools.

For a more general overview of IP masquerading in any Kubernetes implementation, see IP Masquerade Agent User Guide.

Masquerading in GKE

The following factors determine when and how GKE performs IP masquerading or SNAT:

The following table describes valid masquerading outcomes based on the cluster configuration:

Cluster configuration SNAT behavior

GKE preserves the source Pod IP addresses for packets sent to destinations specified in the nonMasqueradeCIDRs list.

GKE changes source Pod IP addresses to source node IP addresses for packets sent to destinations not specified in the nonMasqueradeCIDRs list.

GKE preserves the source Pod IP addresses for packets sent to a set of default non-masquerade destinations.

GKE changes source Pod IP addresses to source node IP addresses for packets sent to destinations outside of the default non-masquerade destinations.

GKE preserves the source Pod IP addresses for packets sent to all destinations.

Change this behavior by ensuring ip-masq-agent is installed and you have specified a nonMasqueradeCIDRs list in the ip-masq-agent ConfigMap.

Masquerading scenarios

The following examples summarize possible masquerading configurations and their corresponding masquerading results.

Scenario: Masquerading to destinations out of 100.64.0.0/24 and 10.0.0.0/8

This scenario is relevant if you want the following masquerading behavior:

  • The cluster preserves the source Pod IP addresses (non-masquerading) for packets sent to destinations 100.64.0.0/24 and 10.0.0.0/8.
  • The cluster changes the source Pod IP addresses (masquerading) to node IP addresses for packets sent to all other destinations.

To achieve this masquerading behavior, configure your cluster as follows:

  • Deploy the ip-masq-agent DaemonSet.
  • Set the nonMasqueradeCIDRs list in the ip-masq-agent ConfigMap to [100.64.0.0/24,10.0.0.0/8].

Scenario: Masquerade to destinations outside of the default non-masquerade destinations

This scenario is relevant if you want the following masquerading behavior:

  • The cluster preserves source Pod IP addresses (non-masquerading) for packets sent to the default non-masquerade destinations.

To achieve this masquerading behavior, configure your cluster as follows:

  • Create the cluster without the --disable-default-snat flag.
  • Ensure that either or both of the following are true:
    • The ip-masq-agent DaemonSet is not deployed, or
    • The nonMasqueradeCIDRs key is absent in the ip-masq-agent ConfigMap.

Scenario: Not masquerade to any destinations

This scenario is relevant if you want the following behavior:

  • The cluster preserves source Pod IP addresses (non-masquerading) for packets sent to all destinations.

To achieve this behavior, configure your cluster as one of the following options:

  • Option 1:

    • Create the cluster with the --disable-default-snat flag.
    • Ensure that either or both of these are true:
      • Ensure the ip-masq-agent DaemonSet is not deployed, or
      • Ensure the nonMasqueradeCIDRs key is absent in the ip-masq-agent ConfigMap.
  • Option 2:

    • Deploy the ip-masq-agent DaemonSet.
    • Set the nonMasqueradeCIDRs list in the ip-masq-agent ConfigMap to [0.0.0.0/0].

Diagnostic containers and Pods with hostNetwork: true

IP masquerading is not applicable to packets sent from the following containers. Unless you specify a custom source IP address for packets, these containers send packets with sources set to the node's primary internal IP address:

When is the ip-masq-agent installed

The ip-masq-agent DaemonSet is automatically installed on nodes of your cluster if you created the cluster without the --disable-default-snat flag, and one or more of the following conditions are true:

  • The cluster has network policy enabled.
  • The cluster's Pod IP address range does not match or is not within 10.0.0.0/8.
  • The cluster has Workload Identity enabled.

For the ip-masq-agent DaemonSet to be effective, you must also specify the nonMasqueradeCIDRs list in the ip-masq-agent ConfigMap. For more information, see how to configure an IP masquerade agent.

Default non-masquerade destinations

The following table summarizes GKE default non-masquerade destination ranges.

GKE versions Destination ranges
Versions earlier than 1.14 Varies by version and image type.
Versions equal to or later than:
  • 1.14.1-gke.14
  • 1.14.2-gke.1
  • 10.0.0.0/8
    172.16.0.0/12
    192.168.0.0/16
    100.64.0.0/10
    192.0.0.0/24
    192.0.2.0/24
    192.88.99.0/24
    198.18.0.0/15
    198.51.100.0/24
    203.0.113.0/24
    240.0.0.0/4

    The default non-masquerade destinations are only relevant when the cluster meets the following conditions:

    For a cluster meeting the preceding conditions, GKE takes the following actions:

    • The cluster preserves source Pod IP addresses if the packet's destination is in the list of default non-masquerade destinations.
    • The cluster changes the source IP addresses from the Pod IP address to the IP address of the node running the Pod, if the packet's destination is not in the list of default non-masquerade destinations.

    If you configure the cluster where both the ip-masq-agent DaemonSet is deployed and a nonMasqueradeCIDRs list is specified in the ip-masq-agent ConfigMap, then the default non-masquerade destinations no longer apply. This is because the non-masquerade destinations are specified in the nonMasqueradeCIDRs list instead.

    Effect of the --disable-default-snat flag

    The following describe the purpose of the --disable-default-snat flag:

    • When you create a cluster with the --disable-default-snat flag, GKE preserves source Pod IP addresses for packets sent to all destinations if one or both of the following are true:

      • The ip-masq-agent DaemonSet is not deployed.
      • The nonMasqueradeCIDRs list is not specified in the ip-masq-agent ConfigMap (including the case where the ip-masq-agent ConfigMap is not specified at all).

      If your cluster meets either or both of these conditions, then GKE preserves source Pod IP addresses for packets sent to any destination.

    • If your cluster has Pods which use privately used public IP addresses, you must create the cluster with the --disable-default-snat flag.

    • The --disable-default-snat flag does not override a valid ip-masq-agent DaemonSet and configuration. If you configure the cluster such that both the ip-masq-agent DaemonSet is deployed and a nonMasqueradeCIDRs list is specified in the ip-masq-agent ConfigMap, then the --disable-default-snat flag is no longer relevant. With this DaemonSet and its configuration, non-masquerade destinations are explicitly specified in the nonMasqueradeCIDRs list, so masquerading is implied for packets sent to destinations not included in the nonMasqueradeCIDRs list.

    Masquerading to link-local destinations

    The link-local IP address destinations fit in the 169.254.0.0/16 IP range. By default, GKE preserves source Pod IP addresses for packets sent to 169.254.0.0/16. This default configuration applies to all GKE versions regardless of whether the following conditions are met:

    • The ip-masq-agent DaemonSet is deployed.
    • The nonMasqueradeCIDRs list is specified in the ip-masq-agent ConfigMap.
    • The --disable-default-snat flag is specified when creating the cluster.

    You can configure GKE to change source Pod IP addresses to node IP addresses when sending packets to 169.254.0.0/16 if you deploy the ip-masq-agent DaemonSet and set masqLinkLocal to True in the ip-masq-agent ConfigMap. When masqLinkLocal is not set, GKE assumes masqLinkLocal is False.

    What's next