This page explains changes in access scopes for clusters running Kubernetes version 1.10.
What are access scopes?
When you create a cluster running Kubernetes version 1.9 or earlier, the cluster's default service account is granted a set of default access scopes. Access scopes are the legacy method of specifying permissions for your nodes, and for workloads running on your nodes if the workloads are using application default credentials (ADC).
In Google Kubernetes Engine, for version 1.9 and earlier, clusters are granted
storage-ro access scopes by
gcloud and Google Cloud Platform Console:
compute-rwgrants full access to all Google Compute Engine API methods
storage-rogrants read-only access to all Google Cloud Storage resources, including private images stored in Google Container Registry
Changes in access scopes
Beginning with Kubernetes version 1.10,
gcloud and GCP Console no
longer grants the
compute-rw access scope on new clusters and new node pools
by default. Furthermore, if
--scopes is specified in
gcloud container create,
gcloud no longer silently adds
These changes ensure that your default service account only has the permissions necessary to run your cluster, which improves the security of your project.
Are my workloads affected?
If your workloads do not require calling the APIs for which access is granted by these scopes, no action is required.
However, if your workloads require permission to interact with Compute Engine, your workloads might be affected. We recommend creating a custom service account with the equivalent Cloud Identity & Access Management roles.
If you override the scopes and your workloads require permission to interact with Cloud Storage (including pulling private container images from Container Registry), the storage-ro scope must also be included.
Alternatively, to replicate the behavior of pre-1.10 clusters, see Reverting to legacy access scopes.
Compute Engine and Cloud Storage return 403 errors to applications running in Kubernetes 1.10 clusters that lack the necessary scopes.
Additionally, the response to a request with insufficient scopes includes an HTTP header detailing the necessary scopes. For example:
WWW-Authenticate: Bearer realm="https://accounts.google.com/", error=insufficient_scope, scope="https://www.googleapis.com/auth/compute.readonly"
Configuring a custom service account for workloads
Cloud Identity & Access Management (Cloud IAM) is the access control system for granting authorized roles to users and service accounts within your GCP project. A service account is a special Google account which performs tasks, such as deploying applications, on your behalf. You use Cloud IAM to create a service account, then use Cloud IAM policy bindings to secure the account.
If your workloads require access to Compute Engine, grant the service account the Compute Engine Admin role. If your workloads need to pull private images from Container Registry, grant the Storage ObjectViewer role.
Create a service account
To create a custom service account called
kubernetes-engine-node-sa, run the
export NODE_SA_NAME=kubernetes-engine-node-sa gcloud iam service-accounts create $NODE_SA_NAME \ --display-name "GKE Node Service Account" export NODE_SA_EMAIL=`gcloud iam service-accounts list --format='value(email)' \ --filter='displayName:Node Service Account'`
Grant minimal roles
To configure the service account with the minimal necessary roles and
permissions for GKE, run the following commands.
$PROJECT is your
export PROJECT=`gcloud config get-value project` gcloud projects add-iam-policy-binding $PROJECT \ --member serviceAccount:$NODE_SA_EMAIL \ --role roles/monitoring.metricWriter gcloud projects add-iam-policy-binding $PROJECT \ --member serviceAccount:$NODE_SA_EMAIL \ --role roles/monitoring.viewer gcloud projects add-iam-policy-binding $PROJECT \ --member serviceAccount:$NODE_SA_EMAIL \ --role roles/logging.logWriter
Grant additional roles
To grant the service account the Compute Engine Admin role, run the following command:
gcloud projects add-iam-policy-binding $PROJECT \ --member serviceAccount:$NODE_SA_EMAIL \ --role roles/compute.admin
To grant the Storage ObjectViewer role:
gcloud projects add-iam-policy-binding $PROJECT \ --member serviceAccount:$NODE_SA_EMAIL \ --role roles/storage.objectviewer
To learn how to grant service accounts access to private images stored in Container Registry, see Granting users and other projects access to a registry.
Creating a cluster or node pool with the custom service account
To create a cluster that uses the custom service account, run the following command:
gcloud container clusters create --service-account=$NODE_SA_EMAIL
To create a node pool in an existing cluster:
gcloud container node-pools create --service-account=$NODE_SA_EMAIL
Reverting to legacy access scopes
If you want to continue using the legacy access scopes in clusters running Kubernetes version 1.10, you must add the scopes manually.
To create a cluster using GCP Console, perform the following steps:
Visit the Google Kubernetes Engine menu in GCP Console.
Click Create cluster.
- From Master Version select the latest Kubernetes 1.10 version.
- Configure the cluster as desired. Don't click Create yet.
- Click Advanced edit for the default node pool.
- In the Access scopes section, select Set access for each API.
- Select Read Write for Compute Engine. Storage should be Read Only by default.
- Click Save to exit the Advanced edit overlay.
- Click Create.
To create a cluster, run the following command.
gcloud container clusters create example-cluster --scopes compute-rw,gke-default
To create a node pool in an existing cluster:
gcloud container node-pools create example-pool --scopes compute-rw,gke-default