Example GKE Setup

Introduction

This page shows you how to configure a sample Cloud NAT setup with Google Kubernetes Engine. Before setting up Cloud NAT, read the Cloud NAT Overview.

Prerequisites

IAM permissions

  • The roles/compute.networkAdmin role can create a NAT gateway on Cloud Router, reserve/assign NAT IPs, and specify subnets whose traffic should use NAT translation by the NAT gateway.

Set up Google Cloud Platform

Before you get started, set up the following items in GCP.

  1. Sign in to your Google Account.

    If you don't already have one, sign up for a new account.

  2. Select or create a Google Cloud Platform project.

    Go to the Manage resources page

  3. Make sure that billing is enabled for your Google Cloud Platform project.

    Learn how to enable billing

  4. Install and initialize the Cloud SDK.
  gcloud config set project [PROJECTID]

You can also view a project ID that is already set:

  gcloud config list --format='text(core.project)'

Example GKE setup

Step 1: Create a VPC network and subnet

If you already have a network and subnet, you can skip this step.

Console

  1. Go to the VPC networks page in the Google Cloud Platform Console.
    Go to the VPC networks page
  2. Click Create VPC network.
  3. Enter a Name of custom-network1.
  4. Under Subnets, set Subnet creation mode to Custom.
  5. Enter a Name of subnet-us-central-192.
  6. Select a Region of us-central1.
  7. Enter an IP address range of 192.168.1.0/24.
  8. Click Done.
  9. Click Create.

gcloud

  1. Create a new custom mode VPC network in your project.

    gcloud compute networks create custom-network1 \
        --subnet-mode custom
    NAME            MODE   IPV4_RANGE GATEWAY_IPV4
    custom-network1 custom
  2. Specify the subnet prefix for your first region. In this example, we're assigning 192.168.1.0/24 to region us-central1.

    gcloud compute networks subnets create subnet-us-central-192 \
       --network custom-network1 \
       --region us-central1 \
       --range 192.168.1.0/24
    NAME                  REGION      NETWORK         RANGE
    subnet-us-central-192 us-central1 custom-network1 192.168.1.0/24

Step 2: Create a bastion host for testing

To test Cloud NAT, you must use a test VM instance that has no external IP address. But, you cannot directly connect via SSH to an instance that doesn't have an external IP address. To connect to the instance that doesn't have an external IP address, you must first connect to an instance that does have an external IP address, then connect to the other instance via internal IP addresses.

In this step, create a bastion host VM.

In a later step, use this VM to connect to your test instance.

Console

  1. Go to the VM instances page.

    Go to the VM instances page

  2. Click the Create or Create instance button.
  3. Specify a Name of bastion-1 for your instance.
  4. Set the Region to us-central1.
  5. Set the Zone to us-central1-c.
  6. Click the Management, security, disks, networking, sole tenancy link.
  7. Click the Networking tab.
  8. Under Network interfaces, click the pencil icon for the VM's default interface.
    1. Set the Network to custom-network1.
    2. Set the Subnetwork to subnet-us-central-192.
    3. Click Done.
  9. Click the Create button to create and start the instance.

gcloud

gcloud compute instances create bastion-1 \
    --image-family debian-9 \
    --image-project debian-cloud \
    --network custom-network1 \
    --subnet subnet-us-central-192 \
    --zone us-central1-c

Step 3: Create a private cluster

Console

  1. Go to the Kubernetes engine page.

    Go to the Kubernetes engine page

  2. Click the Create cluster button.
  3. Select Standard cluster.
  4. Specify a Name of nat-test-cluster for your cluster.
  5. Set the Location type to Zonal.
  6. Set the Zone to us-central1-c.
  7. Open Advanced options.
  8. Under Networking, check the Enable VPC-native (using alias IP) checkbox.
  9. Set Network to custom-network1.
  10. Under Network security, check the Private cluster checkbox.
  11. Uncheck the Access master using its external IP address checkbox.
  12. Enter a Master IP range of 172.16.0.0/28.
  13. Click the Create button to create and start the cluster.

gcloud

gcloud container --project "[PROJECT_ID] " clusters create "nat-test-cluster" \
    --zone "us-central1-c" \
    --username "admin" \
    --cluster-version "latest" \
    --machine-type "n1-standard-1" \
    --image-type "COS" \
    --disk-type "pd-standard" \
    --disk-size "100" \
    --scopes "https://www.googleapis.com/auth/compute","https://www.googleapis.com/auth/devstorage.read_only","https://www.googleapis.com/auth/logging.write","https://www.googleapis.com/auth/monitoring","https://www.googleapis.com/auth/servicecontrol","https://www.googleapis.com/auth/service.management.readonly","https://www.googleapis.com/auth/trace.append" \
    --num-nodes "3" \
    --enable-cloud-logging \
    --enable-cloud-monitoring \
    --enable-private-nodes \
    --enable-private-endpoint \
    --master-ipv4-cidr "172.16.0.0/28" \
    --enable-ip-alias \
    --network "projects/[PROJECT_ID]/global/networks/custom-network1" \
    --subnetwork "projects/[PROJECT_ID]/regions/us-central1/subnetworks/subnet-us-central-192" \
    --max-nodes-per-pool "110" \
    --enable-master-authorized-networks \
    --addons HorizontalPodAutoscaling,HttpLoadBalancing,KubernetesDashboard \
    --enable-autoupgrade \
    --enable-autorepair

Step 4: Create a firewall rule that allows SSH connections

Console

  1. Go to the Firewall rules page in the Google Cloud Platform Console.
    Go to the Firewall rules page
  2. Click Create firewall rule.
  3. Enter a Name of allow-ssh.
  4. Specify a Network of custom-network1.
  5. Set Direction of traffic to ingress.
  6. Set Action on match to allow.
  7. Set Targets to All instances in the network.
  8. Set Source filter to IP ranges.
  9. Set Source IP ranges to 0.0.0.0/0.
  10. Set Protocols and ports to Specified protocols and ports.
  11. Select tcp and specify port 22.
  12. Click Create.

gcloud

gcloud compute firewall-rules create allow-ssh \
    --network custom-network1 \
    --allow tcp:22

Step 5: Log into node and confirm that it cannot reach the Internet

Console

  1. Go to the Kubernetes clusters page.

    Go to the Kubernetes clusters page

  2. Click nat-test-cluster.
  3. Click the Nodes tab.
  4. In another browser window, go to the VM instances page.

    Go to the VM instances page

  5. In the Connect column of bastion-1, select Open in browser window.

    If this is the first time you are connecting to the instance, GCP generates the SSH keys for you.

  6. From bastion-1, connect to one of your GKE nodes. You can copy and paste the node name from the list in your first window.

    ssh [NODE_NAME] -A

    If this is the first time you are connecting to the instance, GCP generates the SSH keys for you.

  7. From the node prompt, find the process ID of the kube-dns container:

    ps aux | grep -i "\s/kube-dns"

    The output row will look something like the following. The process ID is the second column. In this example, the process ID is 2387.

    root      2387  0.0  0.6  38812 24792 ?        Ssl  01:27   0:03 /kube-dns --domain=cluster.local. --dns-port=10053 --config-dir=/kube-dns-config --v=2
  8. Access the container:

    sudo nsenter --target [PROCESS_ID] --net /bin/bash
  9. From kube-dns, attempt to connect to the Internet:

    curl example.com

    You should get no result. If you do, you may not have created your cluster as a private cluster, or there may be some other problem. See Instances can reach the Internet without Cloud NAT for troubleshooting.

gcloud

  1. Find the name of one of your cluster nodes:

    gcloud compute instances list

    A node name will be something like gke-nat-test-cluster-default-pool-1a4cbd06-3m8v. Make a note of the node name.

  2. Add a Compute Engine SSH key to your local host.

    ssh-add ~/.ssh/google_compute_engine
    

  3. Connect to bastion-1:

    gcloud compute ssh bastion-1 --zone us-central1-c -- -A

    If this is the first time you are connecting to the instance, GCP generates the SSH keys for you.

  4. From bastion-1, connect to one of your GKE nodes. You can copy and paste the node name from the list in your first window.

    ssh [NODE_NAME] -A

    If this is the first time you are connecting to the instance, GCP generates the SSH keys for you.

  5. From the node prompt, find the process ID of the kube-dns container:

    ps aux | grep -i "\s/kube-dns"

    The output row will look something like the following. The process ID is the second column. In this example, the process ID is 2387.

    root      2387  0.0  0.6  38812 24792 ?        Ssl  01:27   0:03 /kube-dns --domain=cluster.local. --dns-port=10053 --config-dir=/kube-dns-config --v=2
  6. Access the container:

    sudo nsenter --target [PROCESS_ID] --net /bin/bash
  7. From kube-dns, attempt to connect to the Internet:

    curl example.com

    You should get no result.

Step 6: Create a NAT configuration using Cloud Router

You must create the Cloud Router in the same region as the instances that use Cloud NAT. Cloud NAT is only used to place NAT information onto the VMs. It is not used as part of the actual NAT gateway.

This configuration allows all instances in the region to use Cloud NAT for all primary and alias IP ranges. It also automatically allocates the external IP addresses for the NAT gateway. See the gcloud command-line interface documentation for more options.

Console

  1. Go to the Cloud NAT page in the Google Cloud Platform Console.
    Go to the Cloud NAT page
  2. Click Get started or Create NAT gateway.
  3. Enter a Gateway name of nat-config.
  4. Set the VPC network to custom-network1.
  5. Set the Region to us-central1.
  6. Under Cloud Router, select Create new router.
    1. Enter a Name of nat-router.
    2. Click Create.
  7. Click Create.

gcloud

Create a Cloud Router

gcloud compute routers create nat-router \
    --network custom-network1 \
    --region us-central1

Add a configuration to the router

gcloud compute routers nats create nat-config \
    --router-region us-central1 \
    --router nat-router \
    --nat-all-subnet-ip-ranges \
    --auto-allocate-nat-external-ips

Step 7: Attempt to connect to the Internet again

It may take up to 3 minutes for the NAT configuration to propagate, so wait at least a minute before trying to access the Internet again.

If you are not still logged into kube-dns, reconnect using the procedure in Step 5 above. Once you are logged in, re-run the curl command:

curl example.com

You should see output that contains the following content:


<html>
<head>
<title>Example Domain</title>
...
...
...
</head>

<body>
<div>
    <h1>Example Domain</h1>
    <p>This domain is established to be used for illustrative examples in documents. You may use this
    domain in examples without prior coordination or asking for permission.</p>
    <p><a href="http://www.iana.org/domains/example">More information...</a></p>
</div>
</body>
</html>

What's next

Kunde den här sidan hjälpa dig? Berätta:

Skicka feedback om ...