Configuring cluster access for kubectl

This page explains how to configure cluster access for the kubectl command-line tool in Google Kubernetes Engine.

Overview

If you run multiple clusters within your Google Cloud project, you need to choose which cluster kubectl talks to. You can set a default cluster for kubectl by setting the current context in Kubernetes' kubeconfig file. Additionally, you can run kubectl commands against a specific cluster using the --cluster flag.

The following sections explain how kubeconfig works, how to set a default cluster for kubectl, and how to run individual kubectl commands against a specific cluster.

Before you begin

Before you start, make sure you have performed the following tasks:

Set up default gcloud settings using one of the following methods:

  • Using gcloud init, if you want to be walked through setting defaults.
  • Using gcloud config, to individually set your project ID, zone, and region.

Using gcloud init

If you receive the error One of [--zone, --region] must be supplied: Please specify location, complete this section.

  1. Run gcloud init and follow the directions:

    gcloud init

    If you are using SSH on a remote server, use the --console-only flag to prevent the command from launching a browser:

    gcloud init --console-only
  2. Follow the instructions to authorize gcloud to use your Google Cloud account.
  3. Create a new configuration or select an existing one.
  4. Choose a Google Cloud project.
  5. Choose a default Compute Engine zone.

Using gcloud config

  • Set your default project ID: 
    gcloud config set project PROJECT_ID
  • If you are working with zonal clusters, set your default compute zone: 
    gcloud config set compute/zone COMPUTE_ZONE
  • If you are working with regional clusters, set your default compute region: 
    gcloud config set compute/region COMPUTE_REGION
  • Update gcloud to the latest version: 
    gcloud components update

Kubernetes configuration file

Kubernetes uses a YAML file called kubeconfig to store cluster authentication information for kubectl. kubeconfig contains a list of contexts to which kubectl refers when running commands. By default, the file is saved at $HOME/.kube/config.

A context is a group of access parameters. Each context contains a Kubernetes cluster, a user, and a namespace. The current context is the cluster that is currently the default for kubectl: all kubectl commands run against that cluster.

When you create a cluster using gcloud container clusters create, an entry is automatically added to the kubeconfig in your environment, and the current context changes to that cluster:

gcloud container clusters create my-cluster
Creating my-cluster...done
Fetching cluster endpoint and auth data.
kubeconfig entry generated for my-cluster

When you create a cluster using Google Cloud Console or using gcloud from a different computer, your environment's kubeconfig is not updated. Additionally, if a project team member uses gcloud to create a cluster from their computer, their kubeconfig is updated but yours is not. Follow the instructions below to add these clusters to your local kubeconfig.

About the cluster endpoint

All clusters have a canonical endpoint. The endpoint exposes the Kubernetes API server that kubectl and other services use to communicate with your cluster control plane (master). The endpoint IP is displayed in Cloud Console under the Endpoints field of the cluster's Details tab, and in the output of gcloud container clusters describe in the endpoint field.

When you run gcloud container clusters get-credentials, you see that the command gets the cluster endpoint as part of updating kubeconfig.

Private clusters have two separate endpoint IP addresses: privateEndpoint, which is an internal IP address, and publicEndpoint, which is an external one. The endpoint field refers to the external IP, unless public access to the endpoint is disabled, in which case the private IP will be used.

Running get-credentials uses the IP specified in the endpoint field by default. For private clusters if you prefer to use the internal IP as the endpoint, see Generating a kubeconfig entry using a private cluster's internal IP address.

About authentication for kubectl

All GKE clusters are configured to accept Google Cloud user and service account identities, by validating the credentials presented by kubectl and retrieving the email address associated with the user or service account identity. As a result, the credentials for those accounts must include the userinfo.email OAuth scope in order to successfully authenticate.

When you use gcloud to set up your environment's kubeconfig for a new or existing cluster, gcloud gives kubectl the same credentials used by gcloud itself. For example, if you use gcloud auth login, your personal credentials are provided to kubectl, including the userinfo.email scope. This allows the GKE cluster to authenticate the kubectl client.

Alternatively, you may choose to configure kubectl to use the credentials of a Google Cloud service account, while running on a Compute Engine instance. However, by default, the userinfo.email scope is not included in credentials created by Compute Engine instances. Therefore, you must add this scope explicitly, such as by using the --scopes flag when the Compute Engine instance is created.

Once users or Google Cloud service accounts are authenticated, they must also be authorized to perform any action on a GKE cluster. See role-based access control for more information on how to configure authorization.

For information on supported authentication methods when connecting to the Kubernetes API server, see Authenticating to the Kubernetes API server.

Viewing the current context for kubectl

To view the current context for kubectl, run the following command:

kubectl config current-context

Viewing kubeconfig

To view your environment's kubeconfig, run the following command:

kubectl config view

The command returns a list of all clusters for which kubeconfig entries have been generated. If a GKE cluster is listed, you can run kubectl commands against it in your current environment. Otherwise, you need to generate a kubeconfig entry for the cluster.

Generating a kubeconfig entry

To run kubectl commands against a cluster created in Cloud Console, from another computer, or by another member of the project, you need to generate a kubeconfig entry in your environment.

Generate a kubeconfig entry by running the following command:

gcloud container clusters get-credentials cluster-name

where cluster-name is the name of the cluster.

This command requires you to have the container.clusters.get permission. The least-privileged IAM role that provides this permission is container.clusterViewer.

The kubeconfig entry contains either:

Generating a kubeconfig entry using a private cluster's internal IP address

When you run get-credentials, you can specify the --internal-ip to write a private cluster's internal IP address to kubeconfig:

gcloud container clusters get-credentials --internal-ip cluster-name

Setting a default cluster for kubectl commands

If you have previously generated a kubeconfig entry for clusters, you can switch the current context for kubectl to that cluster by running:

gcloud container clusters get-credentials

For example, consider a project with two clusters, my-cluster and my-new-cluster. The current context is my-new-cluster, but you want to run all kubectl commands against my-cluster. To switch the current context from my-new-cluster to my-cluster, you'd run the following command:

gcloud container clusters get-credentials my-cluster

Running individual kubectl commands against a specific cluster

You can run individual kubectl commands against a specific cluster by passing in the name of the cluster as it appears in kubeconfig as the argument for the --cluster flag.

For example, consider an environment with two clusters, my-cluster and my-new-cluster, in which the current context is my-cluster. You want to deploy an application to my-new-cluster, but you don't want to change the current context. To deploy the application to my-new-cluster without changing the current context, you'd run the following command:

kubectl run my-app --image gcr.io/my-bucket/my-app:1.0 --cluster my-new-cluster

Troubleshooting

Insufficient authentication scopes

When you run gcloud container clusters get-credentials you receive the following error:

ERROR: (gcloud.container.clusters.get-credentials) ResponseError: code=403, message=Request had insufficient authentication scopes.

This error occurs because you are attempting to access the Kubernetes Engine API from a Compute Engine VM that does not have the cloud-platform scope. For instructions on changing the scopes on your Compute Engine VM instance, see Creating and enabling service accounts for instances.

What's next