Creating a Routes-based Cluster

This page shows how to create a routes-based cluster in Google Kubernetes Engine.

Overview

In Google Kubernetes Engine, clusters can be distinguished according to the way they route traffic from one Pod to another Pod. A cluster that uses Google Cloud Routes is called a routes-based cluster. A cluster that uses Alias IPs is called a VPC-native cluster.

VPC-native is the recommended type and is the default for new clusters. To create a routes-based cluster, you must explicitly turn off the VPC-native option.

Before you begin

To prepare for this task, perform the following steps:

Ensure that you have enabled the Google Kubernetes Engine API.

Enable Google Kubernetes Engine API

Ensure that you have installed the Cloud SDK.
Set your default project ID:
```
gcloud config set project [PROJECT_ID]
```
If you are working with zonal clusters, set your default compute zone:
```
gcloud config set compute/zone [COMPUTE_ZONE]
```
If you are working with regional clusters, set your default compute region:
```
gcloud config set compute/region [COMPUTE_REGION]
```
Update gcloud to the latest version:
```
gcloud components update
```

Note: You can override these default settings in gcloud commands using the --project, --zone, and --region operational flags.

Creating a routes-based cluster

gcloud

To create a routes-based cluster, include the --no-enable-ip-alias flag in the cluster creation command:

gcloud container clusters create [CLUSTER_NAME] --no-enable-ip-alias

where [CLUSTER_NAME] is a name that you choose for your cluster.

Console

Visit the Google Kubernetes Engine menu in Cloud Console.

Visit the Google Kubernetes Engine menu
Click Create cluster
Enter a name for your cluster.
Click Advanced options.
Under VPC-native, deselect Enable VPC-native (using alias IP).
Click Create

Verifying that your cluster uses routes

gcloud

List your cluster nodes:

kubectl get nodes

The output shows the names of your nodes:

NAME                                 STATUS   ...     AGE    VERSION
gke-xxx-default-pool-83e239a7-kcg8   Ready    ...     42m    v1.9.7-gke.6
gke-xxx-default-pool-83e239a7-qm6b   Ready    ...     42m    v1.9.7-gke.6
gke-xxx-default-pool-83e239a7-wnrq   Ready    ...     42m    1.9.7-gke.6

List your routes:

gcloud compute routes list

In the output, in the NEXT_HOP column, look for the name of one of your cluster nodes:

NAME                 NETWORK        DEST_RANGE         NEXT_HOP
...
[ROUTE_NAME]         default        10.24.0.0/24       [YOUR_NODE_NAME]
...

In the output, you can see that the route provides a next hop for any packet that is destined for a particular Pod address range.

Console

Visit the Google Kubernetes Engine menu in Cloud Console.

Visit the Google Kubernetes Engine menu
Click your cluster name to open a page of cluster details.
Under Node Pools, locate Instance groups, and click the name of your instance group. The Instance groups page opens and displays a list of your cluster nodes. Make a note of your node names.
Visit the Routes page in Cloud Console.

Visit the Routes page
In the list of routes, in the Next hop column, look for the name of one of your cluster nodes. Click the route name in that row.
The Route details page opens, and you can see that the route provides a next hop for any packet that is destined for a particular Pod address range.

Pods per node

In a routes-based cluster, each node is allocated a /24 range of IP addresses for Pods. With a /24 range, there are 256 addresses, but the maximum number of Pods per node is 110. By having approximately twice as many available IP addresses as possible Pods, Kubernetes is able to mitigate IP address reuse as Pods are added to and removed from a node.

Pod address range

A routes-based cluster has a range of IP addresses that are used for Pods and Services. Even though the range is used for both Pods and Services, it is called the Pod address range. The last /20 of the Pod address range is used for Services. A /20 range has 2¹² = 4096 addresses. So 4096 addresses are used for Services, and the rest of the range is used for Pods.

In command output, the Pod address range is called clusterIpv4Cidr, and the range of addresses used for Services is called servicesIpv4Cidr. For example, the output of gcloud container clusters describe includes output similar to this:

clusterIpv4Cidr: 10.96.0.0/16
...
servicesIpv4Cidr: 10.96.240.0/20

For GKE version 1.7 and later, the Pod address range can be from any RFC 1918 block: 10.0.0.0/8, 172.16.0.0/12 or 192.168.0.0/16. For earlier versions, the Pod address range must be from 10.0.0.0/8.

You can customize Pod address range by specifying a CIDR range. For example, you could specify the range 10.96.0.0/16.

gcloud

gcloud container clusters create [CLUSTER_NAME] --no-enable-ip-alias \
    --cluster-ipv4-cidr 10.96.0.0/16

where [CLUSTER_NAME] is a name that you choose for your cluster.

Console

Visit the Google Kubernetes Engine menu in Cloud Console.

Visit the Google Kubernetes Engine menu
Click Create cluster
Enter a name for your cluster.
Click Advanced options.
For Pod address range, enter 10.96.0.0/16.
Click Create

Considerations for Cluster Sizing

The maximum number of nodes, Pods, and Services for a given GKE cluster is determined by the size of the cluster subnet and the size of the Pod address range. You cannot change the Pod address range size after you create a cluster. When you create a cluster, ensure that you choose a Pod address range large enough to accommodate the cluster's anticipated growth.

The following table explains how to choose address ranges that are sufficient for a cluster that has 900 nodes:

Range	Guidance
Nodes	Node IP addresses are taken from the primary range of the cluster subnet. Your cluster subnet must be large enough to hold the total number of nodes in your cluster. For example, if you plan to create a 900-node cluster, the cluster subnet must be at least a /22 in size. A /22 range has 2¹⁰ = 1024 addresses. Subtract the 4 reserved IP addresses, and you get 1020, which is sufficient for the 900 nodes.
Pod address range	Each node has a /24 range of IP addresses for Pods. A /24 range has 2⁸ = 256 addresses. Recall that 4096 addresses in the Pod address range are used for Services. The remaining portion of the Pod address range is used for Pods, and must be big enough to hold the number of nodes x 256 addresses. Suppose you plan to create a 900-node cluster. Then you need 900 x 256 = 230,400 addresses for Pods. Now suppose you have a /14 Pod address range. A /14 range has 2¹⁸ = 262,144 addresses. Subtract the 4096 addresses used for Services, and you get 258,048, which is sufficient for 900 nodes.

Range

Guidance

Nodes

Node IP addresses are taken from the primary range of the cluster subnet. Your cluster subnet must be large enough to hold the total number of nodes in your cluster.

For example, if you plan to create a 900-node cluster, the cluster subnet must be at least a /22 in size. A /22 range has 2¹⁰ = 1024 addresses. Subtract the 4 reserved IP addresses, and you get 1020, which is sufficient for the 900 nodes.

Pod address range

Each node has a /24 range of IP addresses for Pods. A /24 range has 2⁸ = 256 addresses. Recall that 4096 addresses in the Pod address range are used for Services. The remaining portion of the Pod address range is used for Pods, and must be big enough to hold the number of nodes x 256 addresses.

Suppose you plan to create a 900-node cluster. Then you need 900 x 256 = 230,400 addresses for Pods. Now suppose you have a /14 Pod address range. A /14 range has 2¹⁸ = 262,144 addresses. Subtract the 4096 addresses used for Services, and you get 258,048, which is sufficient for 900 nodes.

Defaults and Limits for Range Sizes

The following table gives the minimum, maximum, and default sizes for the cluster subnet and the Pod address range.

Range	Default size	Minimum size	Maximum size
Nodes	/20, which has 2¹² = 4,096 addresses. Subtract 4 reserved addresses, and you get 4092 addresses for nodes.	/29, which has 2³ = 8 addresses. Subtract 4 reserved addresses, and you get 4 addresses for nodes.	/7, which has 2²⁵ addresses. This is approximately 33 million addresses for nodes.
Pod address range	/14, which has 2¹⁸ = 262,144 addresses.	/19, which has 2¹³ = 8192 addresses.	/9, which has 2²³ = 8,388,608 addresses.

Restrictions

You cannot migrate a VPC-native cluster to a routes-based cluster.
You cannot migrate an routes-based cluster to a VPC-native cluster.

What's next

Var denne siden nyttig? Si fra hva du synes:

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see our Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Sist oppdatert desember 4, 2019.

Creating a Routes-based Cluster

Overview

Before you begin

Creating a routes-based cluster

gcloud

Console

Verifying that your cluster uses routes

gcloud

Console

Pods per node

Pod address range

gcloud

Console

Considerations for Cluster Sizing

Defaults and Limits for Range Sizes

Restrictions

What's next

Send tilbakemelding om ...