Receiving cluster upgrade notifications

Google Kubernetes Engine (GKE) publishes upgrade notifications to Pub/Sub, providing you with a channel to receive information from GKE about your clusters, and therefore better predictability for important updates scheduled for your cluster.

GKE upgrade notifications provide the following benefits:

• You receive proactive information about updates scheduled for your cluster, allowing you to better plan for testing and qualifications, and to help ensure a smooth and predicted upgrade process.
• You are notified when there is a new GKE version that you can upgrade to. Previously, you had to check the GKE release notes or the GKE API to discover when a new GKE version was released.
• You are notified when your cluster upgrades, including both user-initiated and GKE-initiated upgrades, providing you with more visibility into the background operations of your cluster.
• Pub/Sub is highly extensible, giving you flexibility in how you process incoming notifications. For example, you could integrate with Slack to forward notifications to a Slack channel, or initiate Cloud Functions to run custom processes.
• When custom processes are required (for example, orchestrating a staging to production workflow to test and certify an upgrade), you can use the notification to auto-trigger these workflows.

Types of upgrade notifications

GKE sends the following notifications for cluster upgrades: UpgradeAvailableEvent and UpgradeEvent.

Note: If you want to only receive specific notification types, you will need to filter based on the type_url attribute.

UpgradeAvailableEvent

Preview

This product or feature is covered by the Pre-GA Offerings Terms of the Google Cloud Platform Terms of Service. Pre-GA products and features may have limited support, and changes to pre-GA products and features may not be compatible with other pre-GA versions. For more information, see the launch stage descriptions.

When a new version becomes available on a release channel, GKE sends an UpgradeAvailableEvent notification to clusters on that release channel to inform the clusters that a new version is now available. This notification provides one week of advance notice for patch versions and at least 2-4 weeks for minor versions (depending on the channel). For more information, see What versions are available in a channel.

Note: GKE does not send a notification when the release channel available version becomes default.

For clusters not on a release channel, GKE sends notifications for new versions that clusters can upgrade to (patches on the current minor version and the next minor version). Clusters and masters with node auto-upgrades enabled may upgrade shortly after receiving this notification because there is no concept of available/default for clusters not on a release channel.

UpgradeEvent

When you or GKE initiates an upgrade, GKE sends an UpgradeEvent notification, providing you with visibility that an upgrade is occurring. You should already be aware that an upgrade is incoming from the UpgradeAvailableEvent notification, and you can either upgrade in advance or take necessary measures to prepare (or set up maintenance windows, etc.).

The notification is sent at the start of the upgrade operation. The operation ID is passed in the UpgradeEvent.

Enabling upgrade notifications

You can enable GKE upgrade notifications on a new or existing cluster by performing the following tasks:

1. Create a Pub/Sub topic in your project, and specify a topic name.

2. Use one of the following commands to enable GKE upgrade notifications on a cluster:

   For new clusters:

   gcloud container clusters create CLUSTER_NAME \
       --notification-config=pubsub=ENABLED,pubsub-topic=projects/PROJECT/topics/TOPIC_NAME
   

   For existing clusters:

   gcloud container clusters update CLUSTER_NAME \
       --notification-config=pubsub=ENABLED,pubsub-topic=projects/PROJECT/topics/TOPIC_NAME
   

   Replace the following:

   • CLUSTER_NAME: the name of your cluster.
   • PROJECT: the project ID or number of your cluster and topic. The project must be the same for both cluster and topic.
   • TOPIC_NAME: the name of your Pub/Sub topic.

3. Create a Pub/Sub subscription for your topic.

GKE has no restrictions on how you correlate topics with clusters, as long as the topics are in the same project as the clusters. You can choose to create one topic for all clusters, or create multiple topics for groups of clusters.

Verifying setup for notifications

To verify that upgrade notifications are set up correctly for your cluster, perform the following:

1. Upgrade a node pool in your cluster:

   gcloud container clusters upgrade CLUSTER_NAME \
       --cluster-version VERSION \
       --node-pool NODE_POOL_NAME
   

   Replace the following:

   • CLUSTER_NAME: the name of the cluster that contains the node pool.
   • VERSION: the version to upgrade your node pool to.
   • NODE_POOL_NAME: the name of the node pool.

2. Check your Pub/Sub subscription for a notification message.

Disabling upgrade notifications

To disable GKE upgrade notifications, perform the following tasks:

1. Update the cluster to disable notifications:

   gcloud container clusters update CLUSTER_NAME \
       --notification-config=pubsub=DISABLED
   

   Replace the following:

   • CLUSTER_NAME: the name of your existing cluster.

2. Optionally, delete the Pub/Sub topic you created for notifications.

Reading the notification configuration

To read the notification configuration, use the following command:

gcloud container clusters describe CLUSTER_NAME

The output is similar to the following:

name: c1
location: us-central1-c
notificationConfig:
  enabled: True
  topic: AnotherTopic

Consuming notifications

Pub/Sub messages contain two fields: data (string) and attributes (string-to-string map).

For GKE notifications, GKE sends a human-readable message in the data field. The attributes contain generic notification information (for example, project number, cluster name, and cluster location) along with the specific notification type (for example, UpgradeAvailableEvent or UpgradeEvent).

Notifications always contain the following attributes regardless of the payload:

Attribute name	Example	Description
project_number	123456789	The project number that owns the cluster.
cluster_location	us-central1-c	The location of the cluster.
cluster_name	my-cluster	The name of the cluster.
type_url	type.googleapis.com/google.container.v1beta.UpgradeEvent	The type of notification. The example shown is specific to an UpgradeEvent notification.
payload	{"resourceType":"MASTER", "operation":"operation-1595889094437-87b7254a", "operationStartTime":"2020-07-27T22:31:34.437652293Z", "currentVersion":"1.15.12-gke.2", "targetVersion":"1.15.12-gke.9"}	A JSON-parsable string carrying notification-specific information. The example shown is specific to an UpgradeEvent notification.

GKE will always send beta notification types, but the payload can be parsed into the corresponding GA (v1) type, if it is available.

Filtering notifications

You should configure your Pub/Sub subscriptions to filter only the notification types that you would like to consume. For example, you can configure your subscription to only receive UpgradeAvailableEvent notifications by using the following filtering syntax:

attributes.type_url = "type.googleapis.com/google.container.v1beta1.UpgradeAvailableEvent"

For more information, see Pub/Sub filtering.

Note: You can only configure the filter when creating a subscription. After creating the subscription, the filter cannot be modified.

What's next

• Learn about Configuring Pub/Sub notifications.
• Learn how to Configuring cluster upgrade notifications for third-party services.
• Learn about the Pub/Sub message format in Pub/Sub notifications for Cloud Storage.