Migrating from Legacy Access Scopes

This page explains changes in access scopes for clusters running Kubernetes version 1.10.

What are access scopes?

When you create a cluster running Kubernetes version 1.9 or earlier, the cluster's default service account is granted a set of default access scopes. Access scopes are the legacy method of specifying permissions for your nodes, and for workloads running on your nodes if the workloads are using application default credentials (ADC).

In Google Kubernetes Engine, for version 1.9 and earlier, clusters are granted compute-rw and storage-ro access scopes by gcloud and Google Cloud Platform Console:

• compute-rw grants full access to all Google Compute Engine API methods
• storage-ro grants read-only access to all Google Cloud Storage resources, including private images stored in Google Container Registry

Changes in access scopes

Beginning with Kubernetes version 1.10, gcloud and GCP Console no longer grants the compute-rw access scope on new clusters and new node pools by default. Furthermore, if --scopes is specified in gcloud container create, gcloud no longer silently adds compute-rw or storage-ro.

These changes ensure that your default service account only has the permissions necessary to run your cluster, which improves the security of your project.

Note: If your gcloud configuration includes container/new_scopes_behavior true, this behavior is already enabled for all Kubernetes versions.

Are my workloads affected?

If your workloads do not require calling the APIs for which access is granted by these scopes, no action is required.

However, if your workloads require permission to interact with Compute Engine, your workloads might be affected. We recommend creating a custom service account with the equivalent Cloud Identity & Access Management roles.

If you override the scopes and your workloads require permission to interact with Cloud Storage (including pulling private container images from Container Registry), the storage-ro scope must also be included.

Alternatively, to replicate the behavior of pre-1.10 clusters, see Reverting to legacy access scopes.

Impact

Compute Engine and Cloud Storage return 403 errors to applications running in Kubernetes 1.10 clusters that lack the necessary scopes.

Additionally, the response to a request with insufficient scopes includes an HTTP header detailing the necessary scopes. For example:

WWW-Authenticate: Bearer realm="https://accounts.google.com/", error=insufficient_scope, scope="https://www.googleapis.com/auth/compute.readonly"

Configuring a custom service account for workloads

Cloud Identity & Access Management (Cloud IAM) is the access control system for granting authorized roles to users and service accounts within your GCP project. A service account is a special Google account which performs tasks, such as deploying applications, on your behalf. You use Cloud IAM to create a service account, then use Cloud IAM policy bindings to secure the account.

If your workloads require access to Compute Engine, grant the service account the Compute Engine Admin role. If your workloads need to pull private images from Container Registry, grant the Storage ObjectViewer role.

Create a service account

To create a custom service account called kubernetes-engine-node-sa, run the following commands:

export NODE_SA_NAME=kubernetes-engine-node-sa
gcloud iam service-accounts create $NODE_SA_NAME \
  --display-name "GKE Node Service Account"
export NODE_SA_EMAIL=`gcloud iam service-accounts list --format='value(email)' \
  --filter='displayName:Node Service Account'`

Note: These commands export two variables, $NODE_SA_NAME and $NODE_SA_EMAIL, that are used in commands documented later on this page.

Grant minimal roles

To configure the service account with the minimal necessary roles and permissions for GKE, run the following commands. $PROJECT is your project ID:

export PROJECT=`gcloud config get-value project`
gcloud projects add-iam-policy-binding $PROJECT \
  --member serviceAccount:$NODE_SA_EMAIL \
  --role roles/monitoring.metricWriter
gcloud projects add-iam-policy-binding $PROJECT \
  --member serviceAccount:$NODE_SA_EMAIL \
  --role roles/monitoring.viewer
gcloud projects add-iam-policy-binding $PROJECT \
  --member serviceAccount:$NODE_SA_EMAIL \
  --role roles/logging.logWriter

Grant additional roles

Caution: The compute.admin role should not be granted to this service account unless necessary. Granting this role allows workloads to create, delete, read metadata on, and ssh into nodes, among other operations.

To grant the service account the Compute Engine Admin role, run the following command:

gcloud projects add-iam-policy-binding $PROJECT \
  --member serviceAccount:$NODE_SA_EMAIL \
  --role roles/compute.admin

Caution: The storage.objectviewer role grants the service account project-wide access to all Cloud Storage and Container Registry resources.

To grant the Storage ObjectViewer role:

gcloud projects add-iam-policy-binding $PROJECT \
  --member serviceAccount:$NODE_SA_EMAIL \
  --role roles/storage.objectviewer

To learn how to grant service accounts access to private images stored in Container Registry, see Granting users and other projects access to a registry.

Creating a cluster or node pool with the custom service account

To create a cluster that uses the custom service account, run the following command:

gcloud container clusters create --service-account=$NODE_SA_EMAIL

To create a node pool in an existing cluster:

gcloud container node-pools create --service-account=$NODE_SA_EMAIL

Note: You can also find the service account's email address by running the following command:

gcloud iam service-accounts list --format='value(email)' \
      --filter='displayName:Node Service Account'

Reverting to legacy access scopes

If you want to continue using the legacy access scopes in clusters running Kubernetes version 1.10, you must add the scopes manually.

Console

To create a cluster using GCP Console, perform the following steps:

1. Visit the Google Kubernetes Engine menu in GCP Console.

   Visit the Google Kubernetes Engine menu

2. Click Create cluster.

3. From Master Version select the latest Kubernetes 1.10 version.
4. Configure the cluster as desired. Don't click Create yet.
5. Click Advanced edit for the default node pool.
6. In the Access scopes section, select Set access for each API.
7. Select Read Write for Compute Engine. Storage should be Read Only by default.
8. Click Save to exit the Advanced edit overlay.
9. Click Create.

gcloud

To create a cluster, run the following command.

gcloud container clusters create example-cluster --scopes compute-rw,gke-default

To create a node pool in an existing cluster:

gcloud container node-pools create example-pool --scopes compute-rw,gke-default