Maintenance windows and exclusions

This page describes maintenance windows and maintenance exclusions, which provide control over when cluster maintenance such as auto-upgrades can and cannot occur on your Google Kubernetes Engine clusters. For example, a retail business could limit maintenance to only occur on weekday evenings, and could prevent automated maintenance during a key industry sales event.

Status of maintenance windows

  • The simplified --maintenance-window flag that allows you to specify a start time for a four-hour daily maintenance window remains generally available.
  • Extended flags for specifying the duration and recurrence of maintenance windows are available as a beta feature.
  • Maintenance exclusions are available as a beta feature.

Overview

Maintenance windows and exclusions now give you fine-grained control over when automatic maintenance can occur on your clusters.

A maintenance window is an arbitrary, repeating window of time during which automatic maintenance are permitted.

A maintenance exclusion is an arbitrary non-repeating window of time during which automatic maintenance is forbidden. A cluster can have up to three maintenance exclusions at a time.

You can configure maintenance windows and maintenance exclusions separately and independently. You can configure multiple maintenance exclusions.

Examples of automatic maintenance

Google performs maintenance tasks on your clusters as needed, or when you make a configuration change that re-creates nodes or networks in the cluster For example:

Some of these types of maintenance, such as cluster and node upgrades, can be difficult to predict and plan for. A zonal cluster cannot be modified while its master is upgraded (including deploying workloads). Each of the other types of changes listed above can cause temporary disruptions while moving workloads off each node as it is re-created.

Maintenance windows

Maintenance windows allow you to control when automatic upgrades of masters and nodes can occur, to mitigate potential transient disruptions to your workloads. Maintenance windows are useful for the following types of scenarios, among others:

  • Off-peak hours: You want to minimize the chance of downtime by scheduling automatic upgrades during off-peak hours when traffic is reduced.
  • On-call: You want to ensure that upgrades happen during working hours so that someone can monitor the upgrades and manage any unanticipated issues.
  • Multi-cluster upgrades: You want to roll out upgrades across multiple clusters in different regions one at a time at specified intervals.

In addition to automatic upgrades, Google may occasionally need to perform other maintenance tasks, and honors a cluster's maintenance window if at all possible.

If tasks run beyond the maintenance window, GKE attempts to pause the operation, and attempts to resume it during the next maintenance window.

GKE reserves the right to roll out unplanned, emergency upgrades outside of maintenance windows. Additionally, mandatory upgrades to upgrade from deprecated or outdated software might automatically occur outside of maintenance windows.

You can configure a maintenance window for a new or existing cluster.

Caveats for maintenance windows

Maintenance windows and exclusions can cause security patches to be delayed. GKE reserves the right to override maintenance policies for critical security vulnerabilities. Before enabling maintenance windows, make sure you understand the following caveats.

Other Google Cloud maintenance

GKE clusters and workloads can also be impacted by automatic maintenance on other services, such as Compute Engine. Maintenance windows and maintenance exclusions do not affect automatic maintenance apart from GKE.

Node re-creation and maintenance windows

When you enable or modify features or options such as those that impact networking between the cluster masters and nodes, the nodes are recreated to apply the new configuration. Some examples of features that cause nodes to be recreated are:

If you use maintenance windows and you enable or modify a feature or option that requires nodes to be recreated, the new configuration is applied to the nodes only during a maintenance window. If you prefer not to wait, you can manually "upgrade" the node pool to the same version it is already using, by setting the --cluster-version flag to the same GKE version the nodes are already running. You must use the gcloud command if you use this workaround.

One maintenance window per cluster

You can only configure a single maintenance window per cluster. Configuring a new maintenance window overwrites the previous one.

Timezones for maintenance windows

When configuring and viewing maintenance windows, times are shown differently depending on the tool you are using:

When configuring maintenance windows

When configuring maintenance windows using the older --maintenance-window flag, you cannot specify a timezone. UTC is used when using the gcloud command or the API, and Google Cloud Console displays times using the local timezone.

When using the more granular flags, such as --maintenance-window-start, you can specify the timezone as part of the value. If you omit the timezone, your local timezone is used. Times are always stored in UTC.

When viewing maintenance windows

When viewing information about your cluster, timestamps for maintenance windows may be shown in UTC or in your local timezone, depending on how you are viewing the information:

  • When using Google Cloud Console to view information about your cluster, times are always displayed in your local timezone.
  • When using gcloud to view information about your cluster, times are always shown in UTC.

Maintenance exclusions

With maintenance exclusions, you can prevent automatic maintenance from occurring during a specific time period. For example, many retail businesses have business guidelines prohibiting infrastructure changes during the end-of-year holidays.

You can add a maximum of three exclusions. You must allow Google adequate time to maintain your clusters in order to remain in a supported configuration.

Exclusions have no recurrence. Instead, create each instance of a periodic exclusion separately.

You can configure a maintenance exclusion for a new or existing cluster.

What's next

Was this page helpful? Let us know how we did:

Send feedback about...

Kubernetes Engine Documentation