Receiving cluster upgrade notifications

Google Kubernetes Engine (GKE) publishes upgrade notifications to Pub/Sub, providing you with a channel to receive information from GKE about your clusters, and therefore better predictability for important updates scheduled for your cluster.

GKE upgrade notifications provide the following benefits:

  • You are notified when your cluster upgrades, including both user-initiated and GKE-initiated upgrades, providing you with more visibility into the background operations of your cluster.
  • Pub/Sub is highly extensible, giving you flexibility in how you process incoming notifications. For example, you could integrate with Slack to forward notifications to a Slack channel, or initiate Cloud Functions to run custom processes.

Types of upgrade notifications

GKE sends the following notification for cluster upgrades: UpgradeEvent.

UpgradeEvent

When you or GKE initiates an upgrade, GKE sends an UpgradeEvent notification, providing you with visibility that an upgrade is occurring.

The notification is sent at the start of the upgrade operation. The operation ID is passed in the UpgradeEvent.

Enabling upgrade notifications

You can enable GKE upgrade notifications on a new or existing cluster by performing the following tasks:

  1. Create a Pub/Sub topic in your project, and specify a topic name.

  2. Use one of the following commands to enable GKE upgrade notifications on a cluster:

    For new clusters:

    gcloud beta container clusters create cluster-name \
    --notification-config=pubsub=ENABLED,pubsub-topic=projects/project/topics/topic-name

    For existing clusters:

    gcloud beta container clusters update cluster-name \
    --notification-config=pubsub=ENABLED,pubsub-topic=projects/project/topics/topic-name

    Replace the following:

    • cluster-name: the name of your cluster.
    • project: the project ID or number of your cluster and topic. The project must be the same for both cluster and topic.
    • topic-name: the name of your Pub/Sub topic.

  3. Create a Pub/Sub subscription for your topic.

GKE has no restrictions on how you correlate topics with clusters, as long as the topics are in the same project as the clusters. You can choose to create one topic for all clusters, or create multiple topics for groups of clusters.

Verifying setup for notifications

To verify that upgrade notifications are set up correctly for your cluster, perform the following:

  1. Upgrade a node pool in your cluster:

    gcloud container clusters upgrade cluster-name \
    --cluster-version version \
    --node-pool node-pool-name

    Replace the following:

    • cluster-name: the name of the cluster that contains the node pool.
    • version: the version to upgrade your node pool to.
    • node-pool-name: the name of the node pool.

  2. Check your Pub/Sub subscription for a notification message.

Disabling upgrade notifications

To disable GKE upgrade notifications, perform the following tasks:

  1. Update the cluster to disable notifications:

    gcloud beta container clusters update cluster-name \
    --notification-config=pubsub=DISABLED

    Replace the following:

    • cluster-name: the name of your existing cluster.

  2. Optionally, delete the Pub/Sub topic you created for notifications.

Reading the notification configuration

To read the notification configuration, use the following command:

gcloud beta container clusters describe cluster-name

The output is similar to the following:

name: c1
location: us-central1-c
notificationConfig:
  enabled: True
  topic: AnotherTopic

Consuming notifications

Pub/Sub messages contain two fields: data (string) and attributes (string-to-string map).

For GKE notifications, GKE sends a human-readable message in the data field. The attributes contain generic notification information (for example, project number, cluster name, and cluster location) along with the specific notification type (for example, UpgradeEvent).

Notifications always contain the following attributes regardless of the payload:

Attribute name Example Description
ProjectNumber 123456789 The project number that owns the cluster.
ClusterLocation us-central1-c The location of the cluster.
ClusterName my-cluster The name of the cluster.
TypeURL type.googleapis.com/google.container.v1beta1.UpgradeEvent The type of notification.
Payload {"resourceType":"MASTER", "operation":"operation-1595889094437-87b7254a", "operationStartTime":"2020-07-27T22:31:34.437652293Z", "currentVersion":"1.15.12-gke.2", "targetVersion":"1.15.12-gke.9"} A JSON-parsable string carrying notification-specific information.

What's next