Auto-upgrading nodes

This page shows you how to configure node auto-upgrades in Google Kubernetes Engine.

Overview

Node auto-upgrades help you keep the nodes in your cluster up-to-date with the cluster control plane (master) version when your control plane is updated on your behalf. When you create a new cluster or node pool with Google Cloud Console or the gcloud command, node auto-upgrade is enabled by default.

You can learn more about cluster and node upgrades.

Node auto-upgrades provide several benefits:

  • Lower management overhead: You don't have to manually track and update your nodes when the control plane is upgraded on your behalf.
  • Better security: Sometimes new binaries are released to fix a security issue. With auto-upgrades, GKE automatically ensures that security updates are applied and kept up to date.
  • Ease of use: Provides a simple way to keep your nodes up to date with the latest Kubernetes features.

Node pools with auto-upgrades enabled are scheduled for upgrades when they meet the selection criteria (announced in the release notes). Rollouts are phased across multiple weeks to ensure cluster and fleet stability. When the upgrade is performed, nodes are drained and re-created to match the current control plane version. Modifications on the boot disk of a node VM do not persist across node re-creations. To preserve modifications across node re- creation, use a DaemonSet.

Node auto-upgrade is not available for Alpha clusters. If you are using a cluster with Windows Server node pools, review Upgrading Windows Server node pools before enabling node auto-upgrade.

Checking the state of auto-upgrade for an existing node pool

You can check whether auto-upgrade is enabled or disabled for a node pool using Google Cloud Console or the gcloud command.

gcloud

To check the state of auto-upgrade for a node pool, run the following command:

gcloud container node-pools describe node-pool-name \
  --cluster cluster-name \
  --zone compute-zone

where:

  • node-pool-name is the name of the node pool.
  • cluster-name is the name of the cluster that contains the node pool.
  • compute-zone is the zone for the cluster.

Console

To check the state of auto-upgrade for a node pool, perform the following:

  1. Visit the Google Kubernetes Engine menu in Cloud Console.

    Visit the Google Kubernetes Engine menu

  2. Click the cluster's Edit button, which looks like a pencil.

  3. In the Node pools section, select the node pool you wish to examine.

  4. Under the Management section, view the value for the Auto-upgrade field.

Enabling node auto-upgrades for an existing node pool

When you create a new cluster with Google Cloud Console or the gcloud command, node auto-upgrade is enabled by default.

You can enable node auto-upgrade if it is currently disabled.

gcloud

To enable auto-upgrades for an existing node pool, run the following command:

gcloud container node-pools update node-pool-name --cluster cluster-name \
    --zone compute-zone --enable-autoupgrade

where:

  • node-pool-name is the name of the node pool.
  • cluster-name is the name of the cluster that contains the node pool.
  • compute-zone is the zone for the cluster.

Console

To enable auto-upgrades for an existing node pool, perform the following steps:

  1. Visit the Google Kubernetes Engine menu in Cloud Console.

    Visit the Google Kubernetes Engine menu

  2. Click the cluster's Edit button, which looks like a pencil.

  3. In the Node pools section, click the name of the node pool you want to modify, and then click Edit.

  4. Select Enable auto-upgrade.

  5. Click Save to modify the cluster.

For more control over when nodes can be auto-upgraded, consider configuring maintenance windows and exclusions.

Disabling node auto-upgrades for an existing node pool

You can disable node auto-upgrade for an existing node pool. If you do so, you are responsible for ensuring that the cluster's nodes run a version compatible the cluster's version.

gcloud

To disable auto-upgrades for an existing node pool, run the following command:

gcloud container node-pools update node-pool-name --cluster cluster-name \
    --zone compute-zone --no-enable-autoupgrade

Console

To disable auto-upgrades for an existing node pool, perform the following steps:

  1. Visit the Google Kubernetes Engine menu in Cloud Console.

    Visit the Google Kubernetes Engine menu

  2. Click the cluster's Edit button, which looks like a pencil.

  3. In the Node pools section, click the name of the node pool you want to modify, and then click Edit.

  4. Clear Enable auto-upgrade.

  5. Click Save to modify the cluster.

Creating a cluster or node pool with node auto-upgrades enabled

gcloud

To create a cluster with auto-upgrades enabled for the default node pool, specify the --enable-autoupgrade flag in the gcloud container clusters create command:

gcloud container clusters create cluster-name --zone compute-zone \
    --enable-autoupgrade

To create a node pool with auto-upgrade enabled:

gcloud container node-pools create node-pool-name --cluster cluster-name \
    --zone compute-zone --enable-autoupgrade

Console

Clusters and node pools created with Cloud Console have auto-upgrades enabled by default. Visit Creating a Cluster or the Adding and Managing Node Pools for instructions to create clusters and node pools.

You can disable auto-upgrades for new node pools. From the cluster creation page, click the name of the node pool you want to modify, then clear Enable auto-upgrade.

Changing surge upgrade parameters

Surge Upgrades allow you to change the number of nodes GKE upgrades at one time and the amount of disruption an upgrade makes on your workloads.

The max-surge-upgrade and max-unavailable-upgrade flags are defined for each node pool. For more information on chosing the right parameters, go to Determining your optimal surge configuration.

You can change these settings when creating or updating a cluster or node pool.

The following variables are used in the commands mentioned below:

  • cluster-name is the name of the cluster for the node pool.
  • compute-zone is the zone for the cluster.
  • node-pool-name is the name of the node pool.
  • number-nodes is the number of nodes in the node pool in each of the cluster's zones.
  • surge-nodes is the number of extra (surge) nodes to be created on each upgrade of the node pool.
  • unavailable-nodes is the number of nodes that can be unavailable at the same time on each upgrade of the node pool.

Creating a cluster with specific surge parameters

To create a cluster with specific settings for surge upgrades, use the max-surge-upgrade and max-unavailable-upgrade flags.

gcloud container clusters create cluster-name \
  --max-surge-upgrade=surge-nodes --max-unavailable-upgrade=unavailable-nodes

Creating a cluster with surge upgrade disabled

To create a cluster without surge upgrades, set the value for the max-surge-upgrade flag to 0.

gcloud container clusters create cluster-name \
  --max-surge-upgrade=0 --max-unavailable-upgrade=1

Creating a node pool with specific surge parameters

To create a node pool in an existing cluster with specific settings for surge upgrades, use the max-surge-upgrade and max-unavailable-upgrade flags.

gcloud container node-pools create node-pool-name \
  --num-nodes=number-nodes --cluster=cluster-name \
  --max-surge-upgrade=surge-nodes --max-unavailable-upgrade=unavailable-nodes

Turn on or turn off Surge Upgrade for an existing node pool

To update the upgrade settings of an existing node pool, use the max-surge-upgrade and max-unavailable-upgrade flags. If you set max-surge-upgrade to greater than 0, GKE creates surge nodes. If you set max-surge-upgrade to 0, GKE doesn't create surge nodes.

gcloud beta container node-pools update node-pool-name \
  --cluster=cluster-name \
  --max-surge-upgrade=surge-nodes --max-unavailable-upgrade=unavailable-nodes

Checking if surge upgrades are enabled on a node pool

To see if surge upgrades are enabled on a node pool, use gcloud to describe the cluster's parameters:

gcloud container node-pools describe node-pool-name \
--cluster=cluster-name

Receiving upgrade notifications

GKE publishes upgrade notifications to Pub/Sub, providing you with a channel to receive information from GKE about your clusters.

For more information, see Receiving cluster upgrade notifications.

What's next