IP masquerade agent

IP masquerade agent

This page explains how IP masquerading works in GKE and the configuration options for IP masquerade agent.

Overview

IP masquerading is a form of network address translation (NAT) used to perform many-to-one IP address translations, which allows multiple clients to access a destination using a single IP address. A GKE cluster uses IP masquerading so that destinations outside of the cluster only receive packets from node IP addresses instead of Pod IP addresses. This is useful in environments that expect to only receive packets from node IP addresses.

The following sections describe how to configure IP masquerading in your GKE cluster.

Before you begin

To prepare for this task, perform the following steps:

Ensure that you have enabled the Google Kubernetes Engine API.

Enable Google Kubernetes Engine API

Ensure that you have installed the Cloud SDK.
Set your default project ID:
```
gcloud config set project [PROJECT_ID]
```
If you are working with zonal clusters, set your default compute zone:
```
gcloud config set compute/zone [COMPUTE_ZONE]
```
If you are working with regional clusters, set your default compute region:
```
gcloud config set compute/region [COMPUTE_REGION]
```
Update gcloud to the latest version:
```
gcloud components update
```

Note: You can override these default settings in gcloud commands using the --project, --zone, and --region operational flags.

Masquerading in GKE

GKE uses iptables rules, along with the ip-masq-agent DaemonSet, to change the source IP address of packets sent from Pods to certain destinations. When a Pod sends a packet to a destination IP address in a specified masquerade range, the node's IP address is used as the packet's source address (instead of the Pod's IP address).

GKE performs IP masquerading for all destinations except:

If ip-masq-agent is installed in your cluster and you defined your own nonMasqueradeCIDRs in the ip-masq-agent ConfigMap, then GKE preserves the source IP addresses as Pod IP addresses for packets sent to destinations specified in nonMasqueradeCIDRs.
If ip-masq-agent is not installed or if no CIDRs are specified in nonMasqueradeCIDRs, then GKE preserves the source IP addresses for packets sent to a set of default non-masquerade destinations. These defaults depend on GKE version and node image type.

Additional information is available in the IP Masquerade Agent User Guide in the Kubernetes documentation.

When `ip-masq-agent` is included

The ip-masq-agent DaemonSet is automatically installed with --nomasq-all-reserved-ranges argument in GKE cluster, if one or more of the following is true:

The cluster has a network policy
The Pod's CIDR range is not within 10.0.0.0/8

You can change the destination ranges by specifying the nonMasqueradeCIDRs in the ip-agent-masq ConfigMap. If your cluster doesn't include the ip-masq-agent, you can manually install ip-masq-agent.

Default non-masquerade destinations

The following table summarizes GKE default non-masquerade destination ranges when nonMasqueradeCIDRs is not specified in the ip-masq-agent ConfigMap.

GKE versions	Node image type	Destination ranges	Notes
All versions	All image types	169.254.0.0/16 (link-local range)	You can change this behavior by setting `masqLinkLocal` to `True` in the `ip-masq-agent` ConfigMap.
Versions before 1.14	Container-Optimized OS with containerd (cos_containerd) or `ip-masq-agent` started with the `--nomasq-all-reserved-ranges` argument	10.0.0.0/8 172.16.0.0/12 192.168.0.0/16	You can change this behavior by specifying a list of CIDRs in `nonMasqueradeCIDRs` in the `ip-masq-agent` ConfigMap.
Versions before 1.14	Other image types without `ip-masq-agent` enabled	10.0.0.0/8
Versions 1.14.1-gke.14, 1.14.2-gke.1 and later	All image types	10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 100.64.0.0/10 192.0.0.0/24 192.0.2.0/24 192.88.99.0/24 198.18.0.0/15 198.51.100.0/24 203.0.113.0/24 240.0.0.0/4

Example `iptables` output

The follow example demonstrates iptables rules in the NAT table, as maintained by ip-masq-agent:

iptables -t nat -L IP-MASQ

RETURN     all  --  anywhere             169.254.0.0/16       /* ip-masq-agent: cluster-local traffic should not be subject to MASQUERADE */ ADDRTYPE match dst-type !LOCAL
RETURN     all  --  anywhere             10.0.0.0/8           /* ip-masq-agent: cluster-local traffic should not be subject to MASQUERADE */ ADDRTYPE match dst-type !LOCAL
RETURN     all  --  anywhere             172.16.0.0/12        /* ip-masq-agent: cluster-local traffic should not be subject to MASQUERADE */ ADDRTYPE match dst-type !LOCAL
RETURN     all  --  anywhere             192.168.0.0/16       /* ip-masq-agent: cluster-local traffic should not be subject to MASQUERADE */ ADDRTYPE match dst-type !LOCAL
MASQUERADE  all  --  anywhere            anywhere             /* ip-masq-agent: outbound traffic should be subject to MASQUERADE (this match must come after cluster-local CIDR matches) */ ADDRTYPE match dst-type !LOCAL

`ip-masq-agent` ConfigMap parameters

The following keys in the ip-masq-agent ConfigMap allow you to specify non-masquerade destination ranges and change the behavior of ip-masq-agent. The ConfigMap file must be written in YAML or JSON and must be named config.

nonMasqueradeCIDRs: A list of strings in CIDR notation that specify the destination IP address ranges for preserving Pod IP addresses.
masqLinkLocal: A boolean value which indicates whether to masquerade traffic to the link-local prefix (169.254.0.0/16). Default is false.
resyncInterval: Represents the amount of time that passes before ip-masq-agent reloads its configuration (writing to /etc/config/ip-masq-agent from the ConfigMap). The format is Nx, where N is an integer and x is a time unit such as s or ms. If unspecified, the default is 60 seconds.

Specifying `nonMasqueradeCIDRs`

The following ConfigMap, config, preserves source address for packets sent to 10.0.0.0/8:

nonMasqueradeCIDRs:
  - 10.0.0.0/8
resyncInterval: 60s

Ignoring the link-local Range

To have ip-masq-agent enable IP masquerading for the link-local range, set the value of the masqLinkLocal key to true in the ConfigMap.

For example:

nonMasqueradeCIDRs:
  - 10.0.0.0/8
resyncInterval: 60s
masqLinkLocal: true

Adding a ConfigMap to your Cluster

To add a ConfigMap to your cluster, run the following command from your shell or terminal window:

kubectl create configmap [CONFIGMAP_NAME] \
--from-file config \
--namespace kube-system

where:

[CONFIGMAP_NAME] is you choose for the ConfigMap resource, such as ip-masq-agent.

For example:

kubectl create configmap ip-masq-agent --from-file config --namespace kube-system

After the sync is complete, you should see the changes in iptables:

iptables -t nat -L IP-MASQ
Chain IP-MASQ (1 references)
target     prot opt source               destination
RETURN     all  --  anywhere             169.254.0.0/16       /* ip-masq-agent: cluster-local traffic should not be subject to MASQUERADE */ ADDRTYPE match dst-type !LOCAL
RETURN     all  --  anywhere             10.0.0.0/8           /* ip-masq-agent: cluster-local */
MASQUERADE  all  --  anywhere            anywhere             /* ip-masq-agent: outbound traffic should be subject to MASQUERADE (this match must come after cluster-local CIDR matches) */ ADDRTYPE match dst-type !LOCAL

Manually Creating an `ip-masq-agent` Resource (optional)

You might need to manually create and configure ip-masq-agent. To do so, you need to deploy a ip-masq-agent resource in your cluster.

First, save the following into a file named ip-masq-agent.yaml locally.

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: ip-masq-agent
  namespace: kube-system
spec:
  template:
    metadata:
      labels:
        k8s-app: ip-masq-agent
    spec:
      hostNetwork: true
      containers:
      - name: ip-masq-agent
        image: gcr.io/google-containers/ip-masq-agent-amd64:v2.4.1
        args:
            - --masq-chain=IP-MASQ
            # To non-masquerade reserved IP ranges by default, uncomment the line below.
            # - --nomasq-all-reserved-ranges
        securityContext:
          privileged: true
        volumeMounts:
          - name: config
            mountPath: /etc/config
      volumes:
        - name: config
          configMap:
            # Note this ConfigMap must be created in the same namespace as the daemon pods - this spec uses kube-system
            name: ip-masq-agent
            optional: true
            items:
              # The daemon looks for its config in a YAML file at /etc/config/ip-masq-agent
              - key: config
                path: ip-masq-agent
      tolerations:
      - effect: NoSchedule
        operator: Exists
      - effect: NoExecute
        operator: Exists
      - key: "CriticalAddonsOnly"
        operator: "Exists"

Then, run the following command:

kubectl apply -f ip-masq-agent.yaml

This should create a daemonset named ip-masq-agent that runs on all nodes in your cluster.

To configure the ip-masq-agent tool, refer to Configuring the agent using a ConfigMap.

What's next

Was this page helpful? Let us know how we did:

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see our Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated December 5, 2019.

Overview

Before you begin

Masquerading in GKE

When ip-masq-agent is included

Default non-masquerade destinations

Example iptables output

ip-masq-agent ConfigMap parameters

Specifying nonMasqueradeCIDRs

Ignoring the link-local Range

Adding a ConfigMap to your Cluster

Manually Creating an ip-masq-agent Resource (optional)

What's next

Send feedback about...

When `ip-masq-agent` is included

Example `iptables` output

`ip-masq-agent` ConfigMap parameters

Specifying `nonMasqueradeCIDRs`

Manually Creating an `ip-masq-agent` Resource (optional)