Upgrading a Cluster

This page explains how to upgrade or downgrade the version of Kubernetes that your Kubernetes Engine cluster runs.

See Versioning and Upgrades to learn about how Kubernetes versions are made available for your cluster master and nodes. To learn more about clusters, refer to Cluster Architecture.

Before you begin

To prepare for this task, perform the following steps:

  • Ensure that you have enabled the Kubernetes Engine API.
  • Enable Kubernetes Engine API
  • Ensure that you have installed the Cloud SDK.
  • Set your default project ID:
    gcloud config set project [PROJECT_ID]
  • Set your default compute zone:
    gcloud config set compute/zone [COMPUTE_ZONE]
  • Update all gcloud commands to the latest version:
    gcloud components update

About upgrading clusters

When you run an upgrade, Kubernetes Engine stops scheduling, drains, and deletes all of the cluster's nodes and their Pods one at a time. Replacement nodes are recreated with the same name as their predecessors. Each node must be recreated successfully for the upgrade to complete. When the new nodes register with the master, Kubernetes Engine marks the nodes as schedulable.

New node instances run the desired Kubernetes version as well as:

Supported versions

To find the supported Kubernetes master and node versions for upgrades and downgrades, run the following command:

gcloud container get-server-config

Nodes can be downgraded to a version older than the master version. Downgrading the cluster master version is not supported.

Supported versions are also documented in the Supported Versions page.

Saving your data to persistent disks

Data stored in host hostPath and emptyDir volumes is deleted during upgrades.

Before running an upgrade, you must ensure that any data you wish to keep is stored in a Pod with a Google Compute Engine persistent disk volume. Persistent disks are unmounted, rather than erased, during upgrades, and their data is "handed off" between Pods.

The following restrictions pertain to persistent disks:

  • The nodes on which Pods are running must be Compute Engine VMs
  • Those VMs need to be in the same Compute Engine project and zone as the persistent disk

To learn how to add a persistent disk to an existing node instance, see Adding or Resizing Persistent Disks.

Manually upgrading your nodes

You can manually upgrade your nodes using the Google Cloud Platform Console or the gcloud command-line tool.

gcloud

The following command upgrades your nodes to the version that your master is running:

gcloud container clusters upgrade [CLUSTER_NAME]

where [CLUSTER_NAME] is the name of the cluster to be upgraded.

To install a specific release of Kubernetes, use the optional --cluster-version flag:

gcloud container clusters upgrade [CLUSTER_NAME] --cluster-version [CLUSTER_VERSION]

where [CLUSTER_VERSION] is the Kubernetes version to which the nodes are upgraded. For example, --cluster-version=1.7.2 or cluster-version=latest.

For more information about specifying versions, refer to the Versioning and Upgrades documentation.

For more information, refer to the gcloud container clusters upgrade documentation.

Console

To upgrade a cluster using GCP Console, perform the following steps:

  1. Visit the Kubernetes Engine menu in GCP Console.

    Visit the Kubernetes Engine menu

  2. Select the desired cluster.

  3. Click Edit.
  4. From the Node Pools menu, select the desired node pool.
  5. From Node version, click Change.
  6. Select the desired version, then click Change.
  7. Click Save.

Automatically upgrading your nodes

Node Auto-Upgrades instructs Kubernetes Engine to upgrade your nodes' version of Kubernetes whenever a new version becomes available.

To enable automatic node upgrades, refer to the Node Auto-Upgrades documentation.

Checking upgrade status

You can check the status of an upgrade using gcloud beta container operations.

To see a list of every running and completed operation in the cluster, run the following command:

gcloud beta container operations list

Each operation is assigned an operation ID and an operation type as well as start and end times, target cluster, and status. The list appears similar to the following example:

NAME                              TYPE                ZONE           TARGET              STATUS_MESSAGE  STATUS  START_TIME                      END_TIME
operation-1505407677851-8039e369  CREATE_CLUSTER      us-west1-a     my-cluster                          DONE    20xx-xx-xxT16:47:57.851933021Z  20xx-xx-xxT16:50:52.898305883Z
operation-1505500805136-e7c64af4  UPGRADE_CLUSTER     us-west1-a     my-cluster                          DONE    20xx-xx-xxT18:40:05.136739989Z  20xx-xx-xxT18:41:09.321483832Z
operation-1505500913918-5802c989  DELETE_CLUSTER      us-west1-a     my-cluster                          DONE    20xx-xx-xxT18:41:53.918825764Z  20xx-xx-xxT18:43:48.639506814Z

To get more information about a specific operation, specify the operation ID in the following command:

gcloud beta container operations describe [OPERATION_ID]

For example:

gcloud beta container operations describe operation-1507325726639-981f0ed6
endTime: '20xx-xx-xxT21:40:05.324124385Z'
name: operation-1507325726639-981f0ed6
operationType: UPGRADE_CLUSTER
selfLink: https://container.googleapis.com/v1/projects/.../kubernetes-engine/docs/zones/us-central1-a/operations/operation-1507325726639-981f0ed6
startTime: '20xx-xx-xxT21:35:26.639453776Z'
status: DONE
targetLink: https://container.googleapis.com/v1/projects/.../kubernetes-engine/docs/zones/us-central1-a/clusters/...
zone: us-central1-a

Canceling a node upgrade

You can cancel an upgrade at any time. When you cancel an upgrade:

  • Nodes that have started the upgrade complete it
  • Nodes that have not started the upgrade do not upgrade
  • Nodes that have already successfully completed the upgrade are unaffected and are not rolled back

First, get the upgrade's operation ID using the following command:

gcloud container operations list

Then, to cancel the upgrade, run the following command:

gcloud beta container operations cancel [OPERATION_ID]

Refer to the gcloud beta container operations cancel documentation.

Rolling back a node upgrade

You can roll back node pools that failed to upgrade, or whose upgrades were canceled, to their previous version of Kubernetes. You cannot roll back node pools once they have been successfully upgraded. Nodes that have not started an upgrade are unaffected.

To roll back an upgrade, run the following command:

gcloud container node-pools rollback [POOL_NAME] --cluster [CLUSTER_NAME] \
    [--zone [COMPUTE_ZONE]]

where:

  • [POOL_NAME] is the name of the node pool to roll back
  • [CLUSTER_NAME] is the name of the cluster from which to roll back the node pool
  • [COMPUTE_ZONE] is your project's compute zone

Refer to the gcloud container node-pools rollback documentation.

Manually initiate a master upgrade

You can manually upgrade your masters using the GCP Console or the gcloud command-line tool. Once you've upgraded your cluster's master, the nodes can be upgraded to the same version.

gcloud

To upgrade your cluster master's version, first run the following command to see the available versions:

gcloud container get-server-config

To upgrade to the latest version, run the following command:

gcloud container clusters upgrade [CLUSTER_NAME] --master

To upgrade to a specific version that is not the default, run the following command:

gcloud container clusters upgrade [CLUSTER_NAME] --master --cluster-version [CLUSTER_VERSION]

Refer to the gcloud container clusters upgrade documentation.

Console

To manually update your cluster master, perform the following steps:

  1. Visit the Kubernetes Engine menu in Google Cloud Platform Console.

    Visit the Kubernetes Engine menu

  2. Select the desired cluster.

  3. From Master version under the Cluster menu, click Upgrade available.
  4. Select the desired version, then click Change.

What's next

Was this page helpful? Let us know how we did:

Send feedback about...

Kubernetes Engine