Example GKE setup

This page shows you how to configure a sample Cloud NAT setup with Google Kubernetes Engine (GKE). Before setting up Cloud NAT, read the Cloud NAT overview.

Prerequisites

You need to do the following before setting up Cloud NAT.

Get IAM permissions

The roles/compute.networkAdmin role gives you permissions to create a NAT gateway on Cloud Router, reserve and assign NAT IP addresses, and specify subnetworks (subnets) whose traffic should use network address translation by the NAT gateway.

Set up Google Cloud

Before you get started, set up the following items in Google Cloud.

  1. Sign in to your Google Account.

    If you don't already have one, sign up for a new account.

  2. In the Google Cloud Console, on the project selector page, select or create a Google Cloud project.

    Go to the project selector page

  3. Make sure that billing is enabled for your Cloud project. Learn how to confirm that billing is enabled for your project.

  4. Install and initialize the Cloud SDK.

Setting up the GKE example

Use this example if you want to see a simple Cloud NAT configuration working with GKE.

Step 1: Create a VPC network and subnet

If you already have a network and subnet, you can skip this step.

Console

  1. In the Google Cloud Console, go to the VPC networks page.

    Go to the VPC networks page

  2. Click Create VPC network.

  3. Enter a Name of custom-network1.

  4. Under Subnets, set Subnet creation mode to Custom.

  5. Under New subnet, enter a Name of subnet-us-east-192.

  6. In Region, select us-east4.

  7. Enter an IP address range of 192.168.1.0/24.

  8. Click Done, and then click Create.

gcloud

  1. Create a new custom mode VPC network in your project:

    gcloud compute networks create custom-network1 \
        --subnet-mode custom

    Output:

    NAME             MODE     IPV4_RANGE   GATEWAY_IPV4
    custom-network1  custom

  2. Specify the subnet prefix for your first region. In this example, we assign 192.168.1.0/24 to region us-east4.

    gcloud compute networks subnets create subnet-us-east-192 \
       --network custom-network1 \
       --region us-east4 \
       --range 192.168.1.0/24

    Output:

    NAME                REGION    NETWORK          RANGE
    subnet-us-east-192  us-east4  custom-network1  192.168.1.0/24

Step 2: Create a private cluster

Console

  1. In the Cloud Console, go to the Kubernetes clusters page.

    Go to the Kubernetes clusters page

  2. Click Create cluster.

  3. For Name, enter nat-test-cluster.

  4. Set the Location type to Zonal.

  5. Set the Zone to us-east4-c.

  6. In the navigation pane, click Networking.

  7. Select Private cluster.

  8. Clear the Access master using its external IP address checkbox.

  9. Enter a Master IP range of 172.16.0.0/28.

  10. Set Network to custom-network1.

  11. To create and start the cluster, click Create.

gcloud

gcloud container clusters create "nat-test-cluster" \
    --zone "us-east4-c" \
    --username "admin" \
    --cluster-version "latest" \
    --machine-type "e2-medium" \
    --image-type "COS" \
    --disk-type "pd-standard" \
    --disk-size "100" \
    --scopes "https://www.googleapis.com/auth/compute","https://www.googleapis.com/auth/devstorage.read_only","https://www.googleapis.com/auth/logging.write","https://www.googleapis.com/auth/monitoring","https://www.googleapis.com/auth/servicecontrol","https://www.googleapis.com/auth/service.management.readonly","https://www.googleapis.com/auth/trace.append" \
    --num-nodes "3" \
    --enable-cloud-logging \
    --enable-cloud-monitoring \
    --enable-private-nodes \
    --enable-private-endpoint \
    --master-ipv4-cidr "172.16.0.0/28" \
    --enable-ip-alias \
    --network "projects/PROJECT_ID/global/networks/custom-network1" \
    --subnetwork "projects/PROJECT_ID/regions/us-east4/subnetworks/subnet-us-east-192" \
    --max-nodes-per-pool "110" \
    --enable-master-authorized-networks \
    --addons HorizontalPodAutoscaling,HttpLoadBalancing,KubernetesDashboard \
    --enable-autoupgrade \
    --enable-autorepair

Step 3: Create a firewall rule that allows SSH connections

Console

  1. In the Cloud Console, go to the Firewall page.

    Go to the Firewall page

  2. Click Create firewall rule.

  3. Enter a Name of allow-ssh.

  4. Specify a Network of custom-network1.

  5. Set Direction of traffic to Ingress.

  6. Set Action on match to Allow.

  7. Set Targets to All instances in the network.

  8. Set Source filter to IP ranges.

  9. Set Source IP ranges to 35.235.240.0/20.

  10. Set Protocols and ports to Specified protocols and ports.

  11. Select the tcp checkbox and enter port 22.

  12. Click Create.

gcloud

gcloud compute firewall-rules create allow-ssh \
    --network custom-network1 \
    --source-ranges 35.235.240.0/20 \
    --allow tcp:22

Step 4: Create IAP SSH permissions for one of your nodes

In a later step, use IAP to connect to your node.

Console

  1. In the Cloud Console, go to the Identity-Aware Proxy page.

    Go to the Identity-Aware Proxy page

  2. Select the SSH and TCP resources tab.

  3. Select the checkbox next to the first node in the list under All Tunnel Resources > us-east4-c. Its name will be similar to gke-nat-test-cluster-default-pool-b50db58d-075t.

  4. Write down the name of the node; later you'll use it to test connectivity.

  5. In the right pane, click Add member.

  6. To grant users, groups, or service accounts access to the resources, in the New members field, specify their email addresses.

    If you are just testing this feature, you can enter your own email address.

  7. To grant the members access to the resources through Cloud IAP's TCP forwarding feature, in the Role drop-down list, select Cloud IAP > IAP-secured Tunnel User.

  8. Click Save.

gcloud

For this step, use the Console instructions.

Step 5: Log in to the node and confirm that it cannot reach the internet

Console

  1. In the Cloud Console, go to the VM instances page.

    Go to the VM instances page

  2. Find the node that you created IAP SSH permissions for. In the Connect column, click the SSH drop-down arrow, and then select Open in browser window.

    If this is the first time that you are connecting to the instance, Google Cloud generates the SSH keys for you.

  3. From the node prompt, find the process ID of the kube-dns container:

    ps aux | grep -i "\s/kube-dns"

    The output row looks something like the following. The process ID is in the second column. In this example, the process ID is 2387.

    root      2387  0.0  0.6  38812 24792 ?        Ssl  01:27   0:03 /kube-dns --domain=cluster.local. --dns-port=10053 --config-dir=/kube-dns-config --v=2
  4. Access the container:

    sudo nsenter --target PROCESS_ID --net /bin/bash
  5. From kube-dns, attempt to connect to the internet:

    curl example.com

    You should get no result. If you do, you might not have created your cluster as a private cluster, or there might be some other problem. To troubleshoot, see VMs can reach the internet unexpectedly without Cloud NAT.

    To end the command, you might have to enter Ctrl+C.

gcloud

  1. Find the name of one of your cluster nodes:

    gcloud compute instances list

    A node name looks something like gke-nat-test-cluster-default-pool-1a4cbd06-3m8v. Make a note of the node name and use that name wherever you see NODE_NAME in the following commands.

  2. Add a Compute Engine SSH key to your local host:

    ssh-add ~/.ssh/google_compute_engine
    
  3. Connect to the node:

    gcloud compute ssh NODE_NAME \
        --zone us-east4-c \
        --tunnel-through-iap
  4. From the node prompt, find the process ID of the kube-dns container:

    ps aux | grep -i "\s/kube-dns"

    The output row looks something like the following. The process ID is in the second column. In this example, the process ID is 2387. In the following commands, replace PROCESS_ID with this number.

    root      2387  0.0  0.6  38812 24792 ?        Ssl  01:27   0:03 /kube-dns --domain=cluster.local. --dns-port=10053 --config-dir=/kube-dns-config --v=2
  5. Access the container:

    sudo nsenter --target PROCESS_ID --net /bin/bash
  6. From kube-dns, attempt to connect to the internet:

    curl example.com

    You should get no result.To end the command, you might have to enter Ctrl+C.

Step 6: Create a NAT configuration using Cloud Router

You must create the Cloud Router in the same region as the instances that use Cloud NAT. Cloud NAT is only used to place NAT information onto the VMs. It is not used as part of the actual NAT gateway.

This configuration allows all instances in the region to use Cloud NAT for all primary and alias IP ranges. It also automatically allocates the external IP addresses for the NAT gateway. For more options, see the gcloud command-line interface documentation.

Console

  1. In the Cloud Console, go to the Cloud NAT page.

    Go to the Cloud NAT page

  2. Click Get started or Create NAT gateway.

  3. Enter a Gateway name of nat-config.

  4. Set the VPC network to custom-network1.

  5. Set the Region to us-east4.

  6. Under Cloud Router, select Create new router.

    1. Enter a Name of nat-router.
    2. Click Create.
  7. Click Create.

gcloud

  1. Create a Cloud Router:

    gcloud compute routers create nat-router \
        --network custom-network1 \
        --region us-east4
  2. Add a configuration to the router:

    gcloud compute routers nats create nat-config \
        --router-region us-east4 \
        --router nat-router \
        --nat-all-subnet-ip-ranges \
        --auto-allocate-nat-external-ips

Step 7: Attempt to connect to the internet again

It might take up to three minutes for the NAT configuration to propagate, so wait at least a minute before trying to access the internet again.

If you are not still logged in to kube-dns, reconnect by using the procedure in Step 5. After you are logged in, rerun the curl command:

curl example.com

You should see output that contains the following content:


<html>
<head>
<title>Example Domain</title>
...
...
...
</head>

<body>
<div>
    <h1>Example Domain</h1>
    <p>This domain is established to be used for illustrative examples in documents. You can use this
    domain in examples without prior coordination or asking for permission.</p>
    <p><a href="http://www.iana.org/domains/example">More information...</a></p>
</div>
</body>
</html>

What's next