Add authorized networks for control plane access


This page shows you how to use authorized networks to restrict the origins from where you can access the control planes of your Google Kubernetes Engine (GKE) clusters.

How authorized networks work

Authorized networks provide an IP-based firewall that controls access to the GKE control plane. The access to the control plane depends on the type of GKE cluster and the source IP addresses. With authorized networks, you configure the IP addresses you want to allow access to the GKE cluster's control plane endpoint as a CIDR block list.

The following terms are used to differentiate which IP-based firewall rules are in place, depending on your type of cluster:

IP addresses distinctions:

  • Public IP addresses of all Compute Engine VMs in Google Cloud: Public IP addresses assigned to any VM used by any customer hosted on Google Cloud. Google Cloud owns these IP addresses. To learn more, see Where can I find Compute Engine IP ranges?
  • Google Cloud platform IP addresses: IP addresses used by Google Cloud products such as Cloud Run or Cloud Functions. Any client hosted on Google Cloud can instantiate these IP addresses. Google Cloud owns these IP addresses.
  • Google-reserved IP addresses: Public IP addresses for GKE cluster management purposes. These IP addresses include GKE managed processes and other production Google services. Google owns these IP addresses.
  • Internet public IP addresses: Non-RFC 1918 and non-Google public IP addresses. Neither Google nor Google Cloud products own these public IP addresses.
  • GKE cluster IP address ranges: IP addresses assigned to the cluster that GKE uses for the cluster's nodes, Pods, and Services.
  • Privately used IP addresses: IP addresses from your cluster's VPC network. These IP addresses can include your cluster IP address, on-premises networks, the RFC 1918 ranges, or the privately used public IP (PUPI) addresses that include non-RFC 1918 ranges.

Cluster types:

  • Legacy public clusters: Public clusters running on legacy networks or VPC networks with public IP addresses assigned to nodes, and that have not been migrated to Private Service Connect architecture.
  • Legacy private clusters: Clusters based on VPC Network Peering for control plane connectivity from nodes with private IP addresses only.
  • PSC-based cluster created as public or private: Clusters that use Private Service Connect architecture for the communication between the GKE cluster control plane and nodes. To determine if your cluster uses Private Service Connect, see the GKE control plane.

Access to control plane endpoints

The following table shows the default authorized network status at cluster creation. Based on the type of GKE cluster and control plane endpoint, you can determine the following:

  • The preset IP addresses that can always access the GKE control plane.
  • The IP addresses that you can configure to access the cluster's control plane with authorized networks enabled.

The configurable IP addresses can access the cluster's control plane when you allowlist them and enable authorized networks.

Type of GKE cluster and control plane endpoint Default control plane authorized network status Preset IP addresses that can always access the GKE control plane1 Configurable IP address that can access the GKE control plane with authorized networks enabled2
Legacy public clusters or private clusters based on VPC Network Peering
Legacy public clusters with a public endpoint Disabled
  • Public IP addresses of all Compute Engine VMs in Google Cloud
  • Google Cloud platform IP addresses
  • Google-reserved IP addresses
  • GKE cluster IP address ranges
  • Allowlisted internet public IP addresses to access the cluster's public IP endpoint.
Legacy private clusters with public and private endpoints Enabled
  • Google-reserved IP addresses
  • GKE cluster IP address ranges (nodes, Pods, and services have private IP addresses only).
  • Allowlisted privately used IP addresses from the cluster network to access the cluster's private endpoint.
  • Allowlisted internet public IP addresses to access the cluster's public endpoint if the --enable-private-endpoint flag is disabled.
Clusters that use Private Service Connect
Clusters created as public Disabled
  • All private IP addresses from the cluster's network (excluding learned VPC Network Peering routes).

When the –enable-google-cloud flag is set (default), the following IP addresses can access the GKE control plane:

  • Public IP addresses of all Compute Engine VMs in Google Cloud
  • Google Cloud platform IP addresses
  • Google-reserved IP addresses
  • Public IP addresses of your GKE cluster nodes.

When the –no-enable-google-cloud flag is set, the following IP addresses can access the GKE control plane:

  • Google-reserved IP addresses
  • GKE cluster IP address ranges (nodes, Pods, and services have private IP addresses only).

To learn more, see changing cluster isolation.

  • Allowlisted internet public IP addresses to access the cluster's public endpoint
Clusters created as private Enabled

GKE cluster IP address ranges such as nodes, Pods, and Services have private IP addresses only.

By default, when the cluster is created, the –enable-google-cloud is disabled. However, when the –enable-google-cloud flag is set, the following IP addresses can access the GKE control plane:

  • Public IP addresses of all Compute Engine VMs in Google Cloud
  • Google Cloud platform IP addresses
  • Google-reserved IP addresses
  • Public IP addresses of your GKE cluster nodes.

When the –no-enable-google-cloud flag is set, the following IP addresses can access the GKE control plane:

  • Google-reserved IP addresses
  • GKE cluster IP address ranges (nodes, Pods, and services have private IP addresses only).

To learn more, see changing cluster isolation.

  • Allowlisted internet public IP addresses to access the cluster's public endpoint if the --enable-private-endpoint flag is disabled.
  • Allowlisted privately used IP addresses from the cluster network to access the cluster's private endpoint
  1. The IP addresses that can always access the GKE control plane, regardless of authorized networks being enabled or disabled

  2. You must allowlist these configurable IP addresses to access your cluster control plane.

Limitations

  • If you expand a subnet that is used by a cluster with authorized networks, you must update the authorized network configuration to include the expanded IP address range.
  • The number of authorized IP address ranges you can specify for public and private IP addresses depends on the type of cluster:

    • Public cluster: 50 IP address ranges
    • Private cluster: 100 IP address ranges
    • PSC-based clusters: 100 IP address ranges

Before you begin

Before you start, make sure you have performed the following tasks:

  • Enable the Google Kubernetes Engine API.
  • Enable Google Kubernetes Engine API
  • If you want to use the Google Cloud CLI for this task, install and then initialize the gcloud CLI. If you previously installed the gcloud CLI, get the latest version by running gcloud components update.

Create a cluster with authorized networks

You can create a cluster with one or more authorized networks by using the Google Cloud CLI, the Google Cloud console, or the GKE API.

gcloud

Run the following command:

gcloud container clusters create-auto CLUSTER_NAME \
    --enable-master-authorized-networks \
    --master-authorized-networks CIDR1,CIDR2,...

Replace the following:

  • CLUSTER_NAME: the name of your cluster.
  • CIDR1,CIDR2,...: A comma-delimited list of the CIDR values for the authorized networks. For example, 8.8.8.8/32,8.8.8.0/24.

Console

  1. Go to the Google Kubernetes Engine page in the Google Cloud console.

    Go to Google Kubernetes Engine

  2. Click Create.

  3. In the Autopilot or the Standard section, click Configure.

  4. Configure your cluster as needed.

  5. In the navigation menu, click Networking.

  6. Under Advanced networking options, select the Enable control plane authorized networks checkbox.

  7. Click Add authorized network.

  8. Enter a Name for the network.

  9. For Network, enter a CIDR range that you want to grant access to your cluster control plane.

  10. Click Done. Add additional authorized networks as needed.

  11. Click Create.

API

Specify the masterAuthorizedNetworksConfig object in your cluster create request:

"masterAuthorizedNetworksConfig": {
  "enabled": true,
  "cidrBlocks": [
    {
      "displayName": string,
      "cidrBlock": string
    }
  ]
}

For more information, refer to MasterAuthorizedNetworksConfig.

You can configure a private cluster with one or more authorized networks. For more information, see Private clusters.

Create a cluster with limited control plane access

GKE assigns a public IP address (external endpoint) to the control plane in public clusters. You can set further cluster isolation on public clusters that use Private Service Connect to privately connect nodes and control plane. To check if your cluster uses Private Service Connect, see Public clusters with Private Service Connect.

You can create a cluster and instruct GKE to block access from the following origins to the control plane:

You can create a cluster and define control plane access by using the Google Cloud CLI or the Google Cloud console.

gcloud

Run the following command:

gcloud container clusters create-auto CLUSTER_NAME
    --no-enable-google-cloud-access

Replace CLUSTER_NAME with the name of the GKE cluster.

With this command, the no-enable-google-cloud-access flag prevents the control plane from being accessible from IP addresses owned by Google Cloud.

Console

  1. Go to the Google Kubernetes Engine page in the Google Cloud console.

    Go to Google Kubernetes Engine

  2. Click Create.

  3. In the Autopilot or the Standard section, click Configure.

  4. Configure your cluster as needed.

  5. In the navigation menu, click Networking.

  6. Under Advanced networking options, select the Enable control plane authorized networks checkbox.

  7. Clear the Allow access through Google Cloud public IP addresses check box to prevent the control plane from being accessible from IP addresses owned by Google Cloud.

  8. Click Add authorized network.

  9. Enter a Name for the network.

  10. For Network, enter a CIDR range that you want to grant access to your cluster control plane.

  11. Click Done. Add additional authorized networks as needed.

  12. Click Create.

Add an authorized network to an existing cluster

You can add an authorized network to an existing cluster using the gcloud CLI or the Google Cloud console.

gcloud

Run the following command:

gcloud container clusters update CLUSTER_NAME \
    --enable-master-authorized-networks \
    --master-authorized-networks CIDR1,CIDR2,...

Replace the following:

  • CLUSTER_NAME: the name of your existing cluster.
  • CIDR1,CIDR2,...: A comma-delimited list of the CIDR values for the authorized networks. For example, 8.8.8.8/32,8.8.8.0/24.

Console

  1. Go to the Google Kubernetes Engine page in the Google Cloud console.

    Go to Google Kubernetes Engine

  2. Click the name of the cluster you want to modify.

  3. Under Networking, in the Control plane authorized networks field, click Edit control plane authorized networks.

  4. Select the Enable control plane authorized networks checkbox.

  5. Click Add authorized network.

  6. Enter a Name for the network.

  7. For Network, enter a CIDR range that you want to grant access to your cluster control plane.

  8. Click Done. Add additional authorized networks as needed.

  9. Click Save Changes.

API

Specify the desiredMasterAuthorizedNetworksConfig field in your cluster update request. In the field, specify a MasterAuthorizedNetworksConfig object:

"desiredMasterAuthorizedNetworksConfig": {
    object(MasterAuthorizedNetworksConfig)
  }

Verify an authorized network

You can verify an authorized network in an existing cluster using the gcloud CLI or the Google Cloud console.

gcloud

Run the following command:

gcloud container clusters describe CLUSTER_NAME

The output is similar to the following:

...
masterAuthorizedNetworksConfig:
  cidrBlocks:
  - cidrBlock: 8.8.8.8/32
  - cidrBlock: 8.8.4.4/32
  enabled: true
...

Console

  1. Go to the Google Kubernetes Engine page in the Google Cloud console.

    Go to Google Kubernetes Engine

  2. Click the name of the cluster you want to modify.

  3. Under Networking, the Control plane authorized networks field displays the allowed CIDRs.

API

Send a get request. Look for the CIDR blocks under the masterAuthorizedNetworksConfig field. For example:

"masterAuthorizedNetworksConfig": {
"enabled": true,
 "cidrBlocks": [
  {
    "displayName": "Office",
    "cidrBlock": "192.0.2.0/24"
  }
]
}

Disable authorized networks

You can disable authorized networks for an existing cluster using the gcloud CLI or the Google Cloud console.

gcloud

Run the following command:

gcloud container clusters update CLUSTER_NAME \
  --no-enable-master-authorized-networks

Console

  1. Go to the Google Kubernetes Engine page in the Google Cloud console.

    Go to Google Kubernetes Engine

  2. Click the name of the cluster you want to modify.

  3. Under Networking, in the Control plane authorized networks field, click Edit control plane authorized networks.

  4. Clear the Enable control plane authorized networks checkbox.

  5. Click Save Changes.

Remove authorized networks

gcloud

Run the following command:

gcloud container clusters update CLUSTER_NAME \
    --enable-master-authorized-networks

Console

  1. Go to the Google Kubernetes Engine page in the Google Cloud console.

    Go to Google Kubernetes Engine

  2. Click the name of the cluster you want to modify.

  3. Under Networking, in the Control plane authorized networks field, click Edit control plane authorized networks.

  4. Click Delete.

  5. Click Save Changes.

Troubleshooting

The following sections explain how to resolve common issues with authorized networks.

Too many CIDR blocks

gcloud returns the following error when attempting to create or update a cluster with more than 50 CIDR blocks:

ERROR: (gcloud.container.clusters.update) argument --master-authorized-networks: too many args

To resolve this issue, if your cluster is public, ensure that you specify no more than 50 CIDR blocks. If your cluster is private, specify no more than 100 CIDR blocks.

Unable to connect to the server

kubectl commands time out due to incorrectly configured CIDR blocks:

Unable to connect to the server: dial tcp MASTER_IP: getsockopt: connection timed out

When you create or update a cluster, ensure that you specify the correct CIDR blocks.

What's next