This page explains how Kubernetes Engine's IP masquerade agent works.
Overview
IP masquerading is a form of network address translation (NAT) used to perform many-to-one IP address translations. Masquerading masks multiple source IP addresses behind a single address.
Using IP masquerading in your clusters can increase their security by preventing
individual Pod IP addresses from being exposed to traffic outside link-local
range (169.254.0.0/16) and additional arbitrary IP ranges.
Additionally, IP masquerading allows you to configure communication between IP
ranges without masquerade, such as Pods in the 192.168.0.0/16 range
interacting with networking resources in the cluster CIDR range (10.0.0.0/8).
Before you begin
To prepare for this task, perform the following steps:
- Ensure that you have installed the Cloud SDK.
- Set your default project ID:
gcloud config set project [PROJECT_ID]
- Set your default compute zone:
gcloud config set compute/zone [COMPUTE_ZONE]
- Update all
gcloudcommands to the latest version:gcloud components update
How ip-masq-agent works
IP masquerading in Kubernetes Engine is handled by the ip-masq-agent tool.
ip-masq-agent is automatically enabled in clusters running Kubernetes version
1.7 or later, in clusters with a network policy, or if the cluster's CIDR
is not within 10.0.0.0/8.
ip-masq-agent configures iptables rules to handle masquerading node/Pod
IP addresses when sending traffic to destinations outside the node’s and
cluster's IP ranges. Pod IP addresses are masked behind their node's address.
By default, the agent is configured to treat the three private IP ranges specified by RFC 1918 as non-masquerade CIDRs:
10.0.0.0/8, which is the cluster CIDR used for Pod IP addresses172.16.0.0/12192.168.0.0/16
By default, IP masquerading also treats the link-local range (169.254.0.0/16)
as a non-masquerade CIDR.
By default, the agent reloads its configuration file from the
/etc/config/ip-masq-agent file in its container every 60 seconds.
Below are the default set of iptables rules applied by ip-masq-agent:
iptables -t nat -L IP-MASQ-AGENT
RETURN all -- anywhere 169.254.0.0/16 /* ip-masq-agent: cluster-local traffic should not be subject to MASQUERADE */ ADDRTYPE match dst-type !LOCAL
RETURN all -- anywhere 10.0.0.0/8 /* ip-masq-agent: cluster-local traffic should not be subject to MASQUERADE */ ADDRTYPE match dst-type !LOCAL
RETURN all -- anywhere 172.16.0.0/12 /* ip-masq-agent: cluster-local traffic should not be subject to MASQUERADE */ ADDRTYPE match dst-type !LOCAL
RETURN all -- anywhere 192.168.0.0/16 /* ip-masq-agent: cluster-local traffic should not be subject to MASQUERADE */ ADDRTYPE match dst-type !LOCAL
MASQUERADE all -- anywhere anywhere /* ip-masq-agent: outbound traffic should be subject to MASQUERADE (this match must come after cluster-local CIDR matches) */ ADDRTYPE match dst-type !LOCAL
Configuring the agent using a ConfigMap
If the default set of rules shown above are not sufficient for your cluster, you can use a ConfigMap to customize the affected IP ranges.
The ConfigMap file must be written in YAML or JSON and must be named
config.
The file can contain three keys, all of which are optional:
nonMasqueradeCIDRs: A list of strings in CIDR notation that specify the IP address ranges that do not use IP masquerading.masqLinkLocal: A boolean value which indicates whether to masquerade traffic to the link-local prefix (169.254.0.0/16). Default isfalse.resyncInterval: An integer value representing the interval at which the agent attempts to sync its ConfigMap file from the disk. The format isNx, whereNis an integer andxis a time unit such assorms.
For example, the following ConfigMap, config, allows
10.0.0.0/8 to be excluded by ip-masq-agent:
nonMasqueradeCIDRs: - 10.0.0.0/8 resyncInterval: 60s
Ignoring the link-local range
By default, the link-local range (169.254.0.0/16) is exempted from IP
masquerading.
To have ip-masq-agent enable IP masquerading for the link-local range, set the
value of the masqLinkLocal key to
true in the ConfigMap.
For example:
nonMasqueradeCIDRs: - 10.0.0.0/8 resyncInterval: 60s masqLinkLocal: true
Adding a ConfigMap to your cluster
To add a ConfigMap to your cluster, run the following command from your shell or terminal window:
kubectl create configmap [CONFIGMAP_NAME] \ --from-file config \ --namespace kube-system
where:
[CONFIGMAP_NAME]is you choose for the ConfigMap resource, such asip-masq-agent.
For example:
kubectl create configmap ip-masq-agent --from-file config --namespace kube-system
After the sync is complete, you should see the changes in iptables:
iptables -t nat -L IP-MASQ-AGENT
Chain IP-MASQ-AGENT (1 references)
target prot opt source destination
RETURN all -- anywhere 169.254.0.0/16 /* ip-masq-agent: cluster-local traffic should not be subject to MASQUERADE */ ADDRTYPE match dst-type !LOCAL
RETURN all -- anywhere 10.0.0.0/8 /* ip-masq-agent: cluster-local */
MASQUERADE all -- anywhere anywhere /* ip-masq-agent: outbound traffic should be subject to MASQUERADE (this match must come after cluster-local CIDR matches) */ ADDRTYPE match dst-type !LOCAL
Manually creating an ip-masq-agent resource (optional)
You might need to manually create and configure ip-masq-agent. To do so, you
need to deploy a ip-masq-agent resource in your cluster.
Run the following command:
kubectl apply -f \
https://raw.githubusercontent.com/kubernetes-incubator/ip-masq-agent/master/ip-masq-agent.yaml
This command applies a configuration file provided by Kubernetes.
Next, apply the masq-agent-ds-ready label to nodes and DaemonSets which
use the ip-masq-agent resource you created. Labelling nodes and DaemonSets
indicates that ip-masq-agent should operate on these resources:
kubectl label nodes [NAME] beta.kubernetes.io/masq-agent-ds-ready=true kubectl label daemonsets [NAME] beta.kubernetes.io/masq-agent-ds-ready=true
where [NAME] is the name of the resource(s).
To configure the ip-masq-agent tool, refer to
Configuring the agent using a ConfigMap.
