This page explains changes in access scopes for clusters running Kubernetes version 1.10 and higher. These changes only affect your clusters' interactions with Cloud Storage and Compute Engine.
What are access scopes?
Access scopes are the legacy method of specifying permissions for your nodes,
and for workloads running on your nodes if the workloads are using application
default credentials (ADC). In clusters running
Kubernetes versions prior to 1.10, the cluster's default service
account was granted a set of default access
scopes. These clusters were granted the following access scopes by gcloud
and
Google Cloud Console:
compute-rw
grants full access to all Compute Engine API methods.storage-ro
grants read-only access to all Cloud Storage resources, including private images stored in Container Registry.
Changes in access scopes
Beginning with Kubernetes version 1.10, gcloud
and Cloud Console no
longer grants the compute-rw
access scope on new clusters and new node pools
by default. Furthermore, if --scopes
is specified in
gcloud container create
, gcloud
no longer silently adds compute-rw
or
storage-ro
.
These changes ensure that your default service account only has the permissions necessary to run your cluster, which improves the security of your project.
Are my workloads affected?
If your workloads do not require calling the APIs for which access is granted by these scopes, no action is required.
However, if your workloads require permission to interact with Compute Engine or Cloud Storage, your workloads might be affected. We recommend creating a custom service account with the equivalent Identity and Access Management (IAM) roles.
If you override the scopes and your workloads require permission to interact with Cloud Storage (including pulling private container images from Container Registry), the storage-ro scope must also be included.
Alternatively, to replicate the behavior of pre-1.10 clusters, see Reverting to legacy access scopes.
Impact
Compute Engine and Cloud Storage return 403 errors to applications running in Kubernetes 1.10 clusters that lack the necessary scopes.
Additionally, the response to a request with insufficient scopes includes an HTTP header detailing the necessary scopes. For example:
WWW-Authenticate: Bearer realm="https://accounts.google.com/", error=insufficient_scope, scope="https://www.googleapis.com/auth/compute.readonly"
Configuring a custom service account for workloads
IAM is the access control system for granting authorized roles to users and service accounts within your GCP project. A service account is a special Google account that performs tasks, such as deploying applications, on your behalf. You use IAM to create a service account, then use IAM policy bindings to secure the account.
If your workloads require access to Compute Engine, grant the service account the Compute Engine Admin role. If your workloads need to pull private images from Container Registry, grant the Storage Object Viewer role.
Create a service account
To create a custom service account named kubernetes-engine-node-sa
, run the
following commands:
export NODE_SA_NAME=kubernetes-engine-node-sa
gcloud iam service-accounts create $NODE_SA_NAME \
--display-name "GKE Node Service Account"
export NODE_SA_EMAIL=`gcloud iam service-accounts list --format='value(email)' \
--filter='displayName:GKE Node Service Account'`
Grant minimal roles
To configure the service account with the minimal necessary roles and
permissions for your GKE node to function, run the following
commands, where $PROJECT
is your
project ID:
export PROJECT=`gcloud config get-value project`
gcloud projects add-iam-policy-binding $PROJECT \
--member serviceAccount:$NODE_SA_EMAIL \
--role roles/monitoring.metricWriter
gcloud projects add-iam-policy-binding $PROJECT \
--member serviceAccount:$NODE_SA_EMAIL \
--role roles/monitoring.viewer
gcloud projects add-iam-policy-binding $PROJECT \
--member serviceAccount:$NODE_SA_EMAIL \
--role roles/logging.logWriter
Grant additional roles
To grant the service account the Compute Engine Admin role, run the following command:
gcloud projects add-iam-policy-binding $PROJECT \
--member serviceAccount:$NODE_SA_EMAIL \
--role roles/compute.admin
To grant the Storage Object Viewer role, run the following command:
gcloud projects add-iam-policy-binding $PROJECT \
--member serviceAccount:$NODE_SA_EMAIL \
--role roles/storage.objectViewer
To learn how to grant service accounts access to private images stored in Container Registry, see Granting IAM permissions.
Creating a cluster or node pool with the custom service account
To create a cluster that uses the custom service account, run the following command:
gcloud container clusters create --service-account=$NODE_SA_EMAIL
To create a node pool in an existing cluster:
gcloud container node-pools create --service-account=$NODE_SA_EMAIL
Reverting to legacy access scopes
If you want to continue using the legacy access scopes in clusters running Kubernetes version 1.10 and higher, you must add the scopes manually.
Console
To create a cluster using Cloud Console:
Visit the Google Kubernetes Engine menu in Cloud Console.
Click add_box Create.
Under Master Version, select a Kubernetes version.
Configure the cluster as desired. Don't click Create yet.
From the navigation pane, under default-pool, click Security.
In the Access scopes section, select Set access for each API.
For Compute Engine, select Read Write. For Storage, it should be Read Only by default.
Click Create.
gcloud
To create a cluster, run the following command:
gcloud container clusters create example-cluster --scopes compute-rw,gke-default
To create a node pool in an existing cluster:
gcloud container node-pools create example-pool --scopes compute-rw,gke-default