IP masquerade agent

IP masquerade agent

This page explains how Google Kubernetes Engine's IP masquerade agent works.

Overview

IP masquerading is a form of network address translation (NAT) used to perform many-to-one IP address translations. Masquerading masks multiple source IP addresses behind a single address.

Using IP masquerading in your clusters can increase their security by preventing individual Pod IP addresses from being exposed to traffic outside link-local range (169.254.0.0/16) and additional arbitrary IP ranges.

Additionally, IP masquerading allows you to configure communication between IP ranges without masquerade, such as Pods in the 192.168.0.0/16 range interacting with networking resources in the cluster CIDR range (10.0.0.0/8).

Before you begin

To prepare for this task, perform the following steps:

Ensure that you have enabled the Google Kubernetes Engine API.

Enable Google Kubernetes Engine API

Ensure that you have installed the Cloud SDK.
Set your default project ID:
```
gcloud config set project [PROJECT_ID]
```
If you are working with zonal clusters, set your default compute zone:
```
gcloud config set compute/zone [COMPUTE_ZONE]
```
If you are working with regional clusters, set your default compute region:
```
gcloud config set compute/region [COMPUTE_REGION]
```
Update gcloud to the latest version:
```
gcloud components update
```

Note: You can override these default settings in gcloud commands using the --project, --zone, and --region operational flags.

How `ip-masq-agent` Works

IP masquerading in GKE is handled by the ip-masq-agent tool. ip-masq-agent is automatically enabled in GKE v1.7 and higher, when one or more of the following additional conditions is met:

The cluster has a network policy
The cluster's CIDR range is not within 10.0.0.0/8

If your cluster uses GKE v1.7 or higher but does not conform to the other conditions for ip-masq-agent to be enabled automatically, you can create theip-masq-agent agent manually.

ip-masq-agent configures iptables rules to handle masquerading node/Pod IP addresses when sending traffic to destinations outside the node’s and cluster's IP ranges. Pod IP addresses are masked behind their node's address.

By default, the agent is configured to treat the three private IP ranges specified by RFC 1918 as non-masquerade CIDRs:

10.0.0.0/8, which is the cluster CIDR used for Pod IP addresses
172.16.0.0/12
192.168.0.0/16

By default, IP masquerading also treats the link-local range (169.254.0.0/16) as a non-masquerade CIDR.

By default, the agent reloads its configuration file from the /etc/config/ip-masq-agent file in its container every 60 seconds.

Below are the default set of iptables rules applied by ip-masq-agent:

iptables -t nat -L IP-MASQ
RETURN     all  --  anywhere             169.254.0.0/16       /* ip-masq-agent: cluster-local traffic should not be subject to MASQUERADE */ ADDRTYPE match dst-type !LOCAL
RETURN     all  --  anywhere             10.0.0.0/8           /* ip-masq-agent: cluster-local traffic should not be subject to MASQUERADE */ ADDRTYPE match dst-type !LOCAL
RETURN     all  --  anywhere             172.16.0.0/12        /* ip-masq-agent: cluster-local traffic should not be subject to MASQUERADE */ ADDRTYPE match dst-type !LOCAL
RETURN     all  --  anywhere             192.168.0.0/16       /* ip-masq-agent: cluster-local traffic should not be subject to MASQUERADE */ ADDRTYPE match dst-type !LOCAL
MASQUERADE  all  --  anywhere            anywhere             /* ip-masq-agent: outbound traffic should be subject to MASQUERADE (this match must come after cluster-local CIDR matches) */ ADDRTYPE match dst-type !LOCAL

For GKE version 1.14.1-gke.14, 1.14.2-gke.1 and higher, ip-masq-agent is deployed by GKE with the --nomasq-all-reserved-ranges option enabled by default. This option adds the following IP ranges to the default non-masquerade rules: 100.64.0.0/10, 192.0.0.0/24, 192.0.2.0/24, 192.88.99.0/24, 198.18.0.0/15, 198.51.100.0/24, 203.0.113.0/24, 240.0.0.0/4. You can also add these IP ranges by adding the flag to your manifest file when deploying the agent manually.

Configuring the Agent using a ConfigMap

If the default set of rules shown above are not sufficient for your cluster, you can use a ConfigMap to customize the affected IP ranges.

The ConfigMap file must be written in YAML or JSON and must be named config.

The file can contain three keys, all of which are optional:

nonMasqueradeCIDRs: A list of strings in CIDR notation that specify the IP address ranges that do not use IP masquerading.
masqLinkLocal: A boolean value which indicates whether to masquerade traffic to the link-local prefix (169.254.0.0/16). Default is false.
resyncInterval: An integer value representing the interval at which the agent attempts to sync its ConfigMap file from the disk. The format is Nx, where N is an integer and x is a time unit such as s or ms.

For example, the following ConfigMap, config, allows 10.0.0.0/8 to be excluded by ip-masq-agent:

nonMasqueradeCIDRs:
  - 10.0.0.0/8
resyncInterval: 60s

Ignoring the link-local Range

By default, the link-local range (169.254.0.0/16) is exempted from IP masquerading.

To have ip-masq-agent enable IP masquerading for the link-local range, set the value of the masqLinkLocal key to true in the ConfigMap.

For example:

nonMasqueradeCIDRs:
  - 10.0.0.0/8
resyncInterval: 60s
masqLinkLocal: true

Adding a ConfigMap to your Cluster

To add a ConfigMap to your cluster, run the following command from your shell or terminal window:

kubectl create configmap [CONFIGMAP_NAME] \
--from-file config \
--namespace kube-system

where:

[CONFIGMAP_NAME] is you choose for the ConfigMap resource, such as ip-masq-agent.

For example:

kubectl create configmap ip-masq-agent --from-file config --namespace kube-system

After the sync is complete, you should see the changes in iptables:

iptables -t nat -L IP-MASQ
Chain IP-MASQ (1 references)
target     prot opt source               destination
RETURN     all  --  anywhere             169.254.0.0/16       /* ip-masq-agent: cluster-local traffic should not be subject to MASQUERADE */ ADDRTYPE match dst-type !LOCAL
RETURN     all  --  anywhere             10.0.0.0/8           /* ip-masq-agent: cluster-local */
MASQUERADE  all  --  anywhere            anywhere             /* ip-masq-agent: outbound traffic should be subject to MASQUERADE (this match must come after cluster-local CIDR matches) */ ADDRTYPE match dst-type !LOCAL

Manually Creating an `ip-masq-agent` Resource (optional)

You might need to manually create and configure ip-masq-agent. To do so, you need to deploy a ip-masq-agent resource in your cluster.

First, save the following into a file named ip-masq-agent.yaml locally.

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: ip-masq-agent
  namespace: kube-system
spec:
  template:
    metadata:
      labels:
        k8s-app: ip-masq-agent
    spec:
      hostNetwork: true
      containers:
      - name: ip-masq-agent
        image: gcr.io/google-containers/ip-masq-agent-amd64:v2.4.1
        args:
            - --masq-chain=IP-MASQ
            # To non-masquerade reserved IP ranges by default, uncomment the line below.
            # - --nomasq-all-reserved-ranges
        securityContext:
          privileged: true
        volumeMounts:
          - name: config
            mountPath: /etc/config
      volumes:
        - name: config
          configMap:
            # Note this ConfigMap must be created in the same namespace as the daemon pods - this spec uses kube-system
            name: ip-masq-agent
            optional: true
            items:
              # The daemon looks for its config in a YAML file at /etc/config/ip-masq-agent
              - key: config
                path: ip-masq-agent
      tolerations:
      - effect: NoSchedule
        operator: Exists
      - effect: NoExecute
        operator: Exists
      - key: "CriticalAddonsOnly"
        operator: "Exists"

Then, run the following command:

kubectl apply -f ip-masq-agent.yaml

This should create a daemonset named ip-masq-agent that runs on all nodes in your cluster.

To configure the ip-masq-agent tool, refer to Configuring the agent using a ConfigMap.

What's next

Was this page helpful? Let us know how we did:

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see our Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated May 29, 2019.

Overview

Before you begin

How ip-masq-agent Works

Configuring the Agent using a ConfigMap

Ignoring the link-local Range

Adding a ConfigMap to your Cluster

Manually Creating an ip-masq-agent Resource (optional)

What's next

Send feedback about...

How `ip-masq-agent` Works

Manually Creating an `ip-masq-agent` Resource (optional)