Example GKE Setup

Introduction

This page shows you how to configure a sample Cloud NAT setup with Google Kubernetes Engine. Before setting up Cloud NAT, read the Cloud NAT Overview.

Prerequisites

IAM permissions

  • The roles/compute.networkAdmin role can create a NAT gateway on Cloud Router, reserve/assign NAT IPs, and specify subnets whose traffic should use NAT translation by the NAT gateway.

Set up Google Cloud

Before you get started, set up the following items in Google Cloud.

  1. Sign in to your Google Account.

    If you don't already have one, sign up for a new account.

  2. In the Cloud Console, on the project selector page, select or create a Google Cloud project.

    Go to the project selector page

  3. Make sure that billing is enabled for your Google Cloud project. Learn how to confirm billing is enabled for your project.

  4. Install and initialize the Cloud SDK.
  gcloud config set project PROJECT_ID

You can also view a project ID that is already set:

  gcloud config list --format='text(core.project)'

Example GKE setup

Step 1: Create a VPC network and subnet

If you already have a network and subnet, you can skip this step.

Console

  1. Go to the VPC networks page in the Google Cloud Console.
    Go to the VPC networks page
  2. Click Create VPC network.
  3. Enter a Name of custom-network1.
  4. Under Subnets, set Subnet creation mode to Custom.
  5. Enter a Name of subnet-us-east-192.
  6. Select a Region of us-east4.
  7. Enter an IP address range of 192.168.1.0/24.
  8. Click Done.
  9. Click Create.

gcloud

  1. Create a new custom mode VPC network in your project.

    gcloud compute networks create custom-network1 \
        --subnet-mode custom
    NAME            MODE   IPV4_RANGE GATEWAY_IPV4
    custom-network1 custom
  2. Specify the subnet prefix for your first region. In this example, we're assigning 192.168.1.0/24 to region us-east4.

    gcloud compute networks subnets create subnet-us-east-192 \
       --network custom-network1 \
       --region us-east4 \
       --range 192.168.1.0/24
    NAME                  REGION      NETWORK         RANGE
    subnet-us-east-192    us-east4 custom-network1 192.168.1.0/24

Step 2: Create a private cluster

Console

  1. Go to the Kubernetes engine page.

    Go to the Kubernetes engine page

  2. Click the Create cluster button.
  3. Select Standard cluster.
  4. Specify a Name of nat-test-cluster for your cluster.
  5. Set the Location type to Zonal.
  6. Set the Zone to us-east4-c.
  7. Open Advanced options.
  8. Under Networking, check the Enable VPC-native (using alias IP) checkbox.
  9. Set Network to custom-network1.
  10. Under Network security, check the Private cluster checkbox.
  11. Uncheck the Access master using its external IP address checkbox.
  12. Enter a Master IP range of 172.16.0.0/28.
  13. Click the Create button to create and start the cluster.

gcloud

gcloud container clusters create "nat-test-cluster" \
    --zone "us-east4-c" \
    --username "admin" \
    --cluster-version "latest" \
    --machine-type "n1-standard-1" \
    --image-type "COS" \
    --disk-type "pd-standard" \
    --disk-size "100" \
    --scopes "https://www.googleapis.com/auth/compute","https://www.googleapis.com/auth/devstorage.read_only","https://www.googleapis.com/auth/logging.write","https://www.googleapis.com/auth/monitoring","https://www.googleapis.com/auth/servicecontrol","https://www.googleapis.com/auth/service.management.readonly","https://www.googleapis.com/auth/trace.append" \
    --num-nodes "3" \
    --enable-cloud-logging \
    --enable-cloud-monitoring \
    --enable-private-nodes \
    --enable-private-endpoint \
    --master-ipv4-cidr "172.16.0.0/28" \
    --enable-ip-alias \
    --network "projects/PROJECT_ID/global/networks/custom-network1" \
    --subnetwork "projects/PROJECT_ID/regions/us-east4/subnetworks/subnet-us-east-192" \
    --max-nodes-per-pool "110" \
    --enable-master-authorized-networks \
    --addons HorizontalPodAutoscaling,HttpLoadBalancing,KubernetesDashboard \
    --enable-autoupgrade \
    --enable-autorepair

Step 3: Create a firewall rule that allows SSH connections

Console

  1. Go to the Firewall rules page in the Google Cloud Console.
    Go to the Firewall rules page
  2. Click Create firewall rule.
  3. Enter a Name of allow-ssh.
  4. Specify a Network of custom-network1.
  5. Set Direction of traffic to ingress.
  6. Set Action on match to allow.
  7. Set Targets to All instances in the network.
  8. Set Source filter to IP ranges.
  9. Set Source IP ranges to 35.235.240.0/20.
  10. Set Protocols and ports to Specified protocols and ports.
  11. Select tcp and specify port 22.
  12. Click Create.

gcloud

gcloud compute firewall-rules create allow-ssh \
    --network custom-network1 \
    --source-ranges 35.235.240.0/20 \
    --allow tcp:22

Step 4: Create IAP SSH permissions one of your nodes

In a later step, use IAP to connect to your node.

Console

  1. Go to the Identity-Aware Proxy page.

    Go to the Identity-Aware Proxy page

  2. Select the SSH and TCP Resources tab.
  3. Select the checkbox next to the first node in the list under All Tunnel Resources > us-east4-c. It will have a name like gke-nat-test-cluster-default-pool-b50db58d-075t.
  4. Write down of the name of the node. You will use it to test connectivity later.
  5. Click Add member in the right-side pane.
  6. Specify which users, groups, or service accounts for which you want to grant access to the resources by specifying their email addresses in the New members field.
    If you are just testing this feature, you can enter your own email address.
  7. Grant the members access to the resources through Cloud IAP's TCP forwarding feature by opening the Select a role drop-down list and selecting Cloud IAP > IAP-secured tunnel user.
  8. Click Save.

gcloud

Use the Console instructions for this step.

Step 5: Log into node and confirm that it cannot reach the Internet

Console

  1. Go to the VM instances page.

    Go to the VM instances page

  2. Find the node you created IAP SSH permissions for. Under the Connect column, click the SSH button and select Open in browser window.

    If this is the first time you are connecting to the instance, GCP generates the SSH keys for you.

  3. From the node prompt, find the process ID of the kube-dns container:

    ps aux | grep -i "\s/kube-dns"

    The output row will look something like the following. The process ID is the second column. In this example, the process ID is 2387.

    root      2387  0.0  0.6  38812 24792 ?        Ssl  01:27   0:03 /kube-dns --domain=cluster.local. --dns-port=10053 --config-dir=/kube-dns-config --v=2
  4. Access the container:

    sudo nsenter --target PROCESS_ID --net /bin/bash
  5. From kube-dns, attempt to connect to the Internet:

    curl example.com

    You should get no result. If you do, you may not have created your cluster as a private cluster, or there may be some other problem. See Instances can reach the Internet without Cloud NAT for troubleshooting.

    You might have to enter Ctrl-C to end the command.

gcloud

  1. Find the name of one of your cluster nodes:

    gcloud compute instances list

    A node name will be something like gke-nat-test-cluster-default-pool-1a4cbd06-3m8v. Make a note of the node name and use that name wherever you see NODE_NAME in the commands below.

  2. Add a Compute Engine SSH key to your local host.

    ssh-add ~/.ssh/google_compute_engine
    

  3. Connect to the node:

    gcloud compute ssh NODE_NAME \
        --zone us-east4-c \
        --tunnel-through-iap
  4. From the node prompt, find the process ID of the kube-dns container:

    ps aux | grep -i "\s/kube-dns"

    The output row will look something like the following. The process ID is the second column. In this example, the process ID is 2387. Replace PROCESS_ID with this number in the commands below.

    root      2387  0.0  0.6  38812 24792 ?        Ssl  01:27   0:03 /kube-dns --domain=cluster.local. --dns-port=10053 --config-dir=/kube-dns-config --v=2
  5. Access the container:

    sudo nsenter --target PROCESS_ID --net /bin/bash
  6. From kube-dns, attempt to connect to the Internet:

    curl example.com

    You should get no result. You might have to enter Ctrl-C to end the command.

Step 6: Create a NAT configuration using Cloud Router

You must create the Cloud Router in the same region as the instances that use Cloud NAT. Cloud NAT is only used to place NAT information onto the VMs. It is not used as part of the actual NAT gateway.

This configuration allows all instances in the region to use Cloud NAT for all primary and alias IP ranges. It also automatically allocates the external IP addresses for the NAT gateway. See the gcloud command-line interface documentation for more options.

Console

  1. Go to the Cloud NAT page in the Google Cloud Console.
    Go to the Cloud NAT page
  2. Click Get started or Create NAT gateway.
  3. Enter a Gateway name of nat-config.
  4. Set the VPC network to custom-network1.
  5. Set the Region to us-east4.
  6. Under Cloud Router, select Create new router.
    1. Enter a Name of nat-router.
    2. Click Create.
  7. Click Create.

gcloud

Create a Cloud Router

gcloud compute routers create nat-router \
    --network custom-network1 \
    --region us-east4

Add a configuration to the router

gcloud compute routers nats create nat-config \
    --router-region us-east4 \
    --router nat-router \
    --nat-all-subnet-ip-ranges \
    --auto-allocate-nat-external-ips

Step 7: Attempt to connect to the Internet again

It may take up to 3 minutes for the NAT configuration to propagate, so wait at least a minute before trying to access the Internet again.

If you are not still logged into kube-dns, reconnect using the procedure in Step 5 above. Once you are logged in, re-run the curl command:

curl example.com

You should see output that contains the following content:


<html>
<head>
<title>Example Domain</title>
...
...
...
</head>

<body>
<div>
    <h1>Example Domain</h1>
    <p>This domain is established to be used for illustrative examples in documents. You may use this
    domain in examples without prior coordination or asking for permission.</p>
    <p><a href="http://www.iana.org/domains/example">More information...</a></p>
</div>
</body>
</html>

What's next

Kunde den här sidan hjälpa dig? Berätta:

Skicka feedback om ...