IP Masquerade Agent

This page explains how Kubernetes Engine's IP masquerade agent works.

Overview

IP masquerading is a form of network address translation (NAT) used to perform many-to-one IP address translations. Masquerading masks multiple source IP addresses behind a single address.

Using IP masquerading in your clusters can increase their security by preventing individual Pod IP addresses from being exposed to traffic outside link-local range (169.254.0.0/16) and additional arbitrary IP ranges.

Additionally, IP masquerading allows you to configure communication between IP ranges without masquerade, such as Pods in the 192.168.0.0/16 range interacting with networking resources in the cluster CIDR range (10.0.0.0/8).

Before you begin

To prepare for this task, perform the following steps:

Ensure that you have enabled the Kubernetes Engine API.

Enable Kubernetes Engine API

Ensure that you have installed the Cloud SDK.
Set your default project ID:
```
gcloud config set project [PROJECT_ID]
```
If you are working with zonal clusters, set your default compute zone:
```
gcloud config set compute/zone [COMPUTE_ZONE]
```
If you are working with regional clusters, set your default compute region:
```
gcloud config set compute/region [COMPUTE_REGION]
```
Update gcloud to the latest version:
```
gcloud components update
```

Note: You can override these default settings in gcloud commands using the --project, --zone, and --region operational flags.

How `ip-masq-agent` works

IP masquerading in GKE is handled by the ip-masq-agent tool. ip-masq-agent is automatically enabled in clusters running Kubernetes version 1.7 or later, in clusters with a network policy, or if the cluster's CIDR is not within 10.0.0.0/8.

ip-masq-agent configures iptables rules to handle masquerading node/Pod IP addresses when sending traffic to destinations outside the node’s and cluster's IP ranges. Pod IP addresses are masked behind their node's address.

By default, the agent is configured to treat the three private IP ranges specified by RFC 1918 as non-masquerade CIDRs:

10.0.0.0/8, which is the cluster CIDR used for Pod IP addresses
172.16.0.0/12
192.168.0.0/16

By default, IP masquerading also treats the link-local range (169.254.0.0/16) as a non-masquerade CIDR.

By default, the agent reloads its configuration file from the /etc/config/ip-masq-agent file in its container every 60 seconds.

Below are the default set of iptables rules applied by ip-masq-agent:

iptables -t nat -L IP-MASQ-AGENT
RETURN     all  --  anywhere             169.254.0.0/16       /* ip-masq-agent: cluster-local traffic should not be subject to MASQUERADE */ ADDRTYPE match dst-type !LOCAL
RETURN     all  --  anywhere             10.0.0.0/8           /* ip-masq-agent: cluster-local traffic should not be subject to MASQUERADE */ ADDRTYPE match dst-type !LOCAL
RETURN     all  --  anywhere             172.16.0.0/12        /* ip-masq-agent: cluster-local traffic should not be subject to MASQUERADE */ ADDRTYPE match dst-type !LOCAL
RETURN     all  --  anywhere             192.168.0.0/16       /* ip-masq-agent: cluster-local traffic should not be subject to MASQUERADE */ ADDRTYPE match dst-type !LOCAL
MASQUERADE  all  --  anywhere            anywhere             /* ip-masq-agent: outbound traffic should be subject to MASQUERADE (this match must come after cluster-local CIDR matches) */ ADDRTYPE match dst-type !LOCAL

Configuring the agent using a ConfigMap

If the default set of rules shown above are not sufficient for your cluster, you can use a ConfigMap to customize the affected IP ranges.

The ConfigMap file must be written in YAML or JSON and must be named config.

The file can contain three keys, all of which are optional:

nonMasqueradeCIDRs: A list of strings in CIDR notation that specify the IP address ranges that do not use IP masquerading.
masqLinkLocal: A boolean value which indicates whether to masquerade traffic to the link-local prefix (169.254.0.0/16). Default is false.
resyncInterval: An integer value representing the interval at which the agent attempts to sync its ConfigMap file from the disk. The format is Nx, where N is an integer and x is a time unit such as s or ms.

For example, the following ConfigMap, config, allows 10.0.0.0/8 to be excluded by ip-masq-agent:

nonMasqueradeCIDRs:
  - 10.0.0.0/8
resyncInterval: 60s

Ignoring the link-local range

By default, the link-local range (169.254.0.0/16) is exempted from IP masquerading.

To have ip-masq-agent enable IP masquerading for the link-local range, set the value of the masqLinkLocal key to true in the ConfigMap.

For example:

nonMasqueradeCIDRs:
  - 10.0.0.0/8
resyncInterval: 60s
masqLinkLocal: true

Adding a ConfigMap to your cluster

To add a ConfigMap to your cluster, run the following command from your shell or terminal window:

kubectl create configmap [CONFIGMAP_NAME] \
--from-file config \
--namespace kube-system

where:

[CONFIGMAP_NAME] is you choose for the ConfigMap resource, such as ip-masq-agent.

For example:

kubectl create configmap ip-masq-agent --from-file config --namespace kube-system

After the sync is complete, you should see the changes in iptables:

iptables -t nat -L IP-MASQ-AGENT
Chain IP-MASQ-AGENT (1 references)
target     prot opt source               destination
RETURN     all  --  anywhere             169.254.0.0/16       /* ip-masq-agent: cluster-local traffic should not be subject to MASQUERADE */ ADDRTYPE match dst-type !LOCAL
RETURN     all  --  anywhere             10.0.0.0/8           /* ip-masq-agent: cluster-local */
MASQUERADE  all  --  anywhere            anywhere             /* ip-masq-agent: outbound traffic should be subject to MASQUERADE (this match must come after cluster-local CIDR matches) */ ADDRTYPE match dst-type !LOCAL

Manually creating an `ip-masq-agent` resource (optional)

You might need to manually create and configure ip-masq-agent. To do so, you need to deploy a ip-masq-agent resource in your cluster.

Run the following command:

kubectl apply -f \
https://raw.githubusercontent.com/kubernetes-incubator/ip-masq-agent/master/ip-masq-agent.yaml

This command applies a configuration file provided by Kubernetes.

Next, apply the masq-agent-ds-ready label to nodes and DaemonSets which use the ip-masq-agent resource you created. Labelling nodes and DaemonSets indicates that ip-masq-agent should operate on these resources:

kubectl label nodes [NAME] beta.kubernetes.io/masq-agent-ds-ready=true
kubectl label daemonsets [NAME] beta.kubernetes.io/masq-agent-ds-ready=true

where [NAME] is the name of the resource(s).

To configure the ip-masq-agent tool, refer to Configuring the agent using a ConfigMap.

What's next

Was this page helpful? Let us know how we did:

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 3.0 License, and code samples are licensed under the Apache 2.0 License. For details, see our Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated September 17, 2018.

IP Masquerade Agent

Overview

Before you begin

How ip-masq-agent works

Configuring the agent using a ConfigMap

Ignoring the link-local range

Adding a ConfigMap to your cluster

Manually creating an ip-masq-agent resource (optional)

What's next

Send feedback about...

How `ip-masq-agent` works

Manually creating an `ip-masq-agent` resource (optional)