Credential Rotation

This page explains how to perform a credential rotation in Kubernetes Engine.

Overview

You can perform a Credential rotation to revoke and issue new credentials for your cluster.

Google recommends that you use credential rotation regularly to reduce credential lifetime and further secure your Kubernetes Engine cluster.

In addition to rotating credentials, Credential rotation also performs an IP rotation.

Before you begin

To prepare for this task, perform the following steps:

  • Ensure that you have enabled the Kubernetes Engine API.
  • Enable Kubernetes Engine API
  • Ensure that you have installed the Cloud SDK.
  • Set your default project ID:
    gcloud config set project [PROJECT_ID]
  • Set your default compute zone:
    gcloud config set compute/zone [COMPUTE_ZONE]
  • Update all gcloud commands to the latest version:
    gcloud components update

How credential rotation works

Credential rotation is a multi-step process that includes migrating to a new IP address:

  • When you initiate a credential rotation, your cluster master begins serving on the new IP address in addition to the original IP address. New credentials are issued to workloads and the control plane.
  • After you initiate a rotation, you must update your cluster's API clients (such as development machines using the kubectl command-line interface) to begin communicating with the master over the new IP address.
  • When you complete the rotation, the master ceases serving traffic over the previous IP address, and old credentials are revoked.

Performing a credential rotation

The following sections explain how to perform a credential rotation.

Initiating the rotation

To initiate a credential rotation, run the following command where:

  1. Set CLUSTER_NAME to the name of the cluster.

    CLUSTER_NAME=[CLUSTER_NAME]

  2. Run the following command, which creates new credentials and issues them to the control plane. It also configures the cluster master to serve on two IP Address, the original IP and a new IP. This causes brief downtime for the cluster API.

    gcloud container clusters update ${CLUSTER_NAME} --start-credential-rotation
    

Once the master has been reconfigured, Kubernetes Engine automatically updates your cluster's nodes to use the new IP and credentials. Each node pool is marked as "requires recreation." Kubernetes Engine does not finish the credential rotation until the automatic recreation is complete.

Inspecting the rotation

To monitor the rotation operation, run the following command:

gcloud container operations list | grep "AUTO_UPGRADE_NODES.*RUNNING"

This command returns the operation ID for the update operation.

To poll the operation, pass the operation ID to the following command:

gcloud container operations wait [OPERATION_ID]

Node pools are recreated one-by-one, and each has its own operation. If you have multiple node pools, you can use the above instructions to poll each operation.

Updating API clients

Once credential rotation has been initiated, you must update all API clients outside of the cluster (such as kubectl on developer machines) to use the new credential.

To update your API clients, run the following command for each client:

gcloud container clusters get-credentials [CLUSTER_NAME]

Completing the rotation

To complete the rotation, run the following commands:

  1. Set CLUSTER_NAME to the name of the cluster.

    CLUSTER_NAME=[CLUSTER_NAME]

  2. Run the following command which configures the cluster master to serve only with the new credential. This causes brief downtime for the cluster API.

    gcloud container clusters update ${CLUSTER_NAME} --complete-credential-rotation
    

What's next

Was this page helpful? Let us know how we did:

Send feedback about...

Kubernetes Engine