Each virtual machine (VM) instance or bare metal instance uses a host maintenance policy to determine the behavior of the instance during a maintenance operation. Some instances offer the additional option of viewing the maintenance schedule ahead of time.
This page explains how to monitor and plan for a host maintenance event on Compute Engine instances.
Before you begin
-
If you haven't already, set up authentication.
Authentication is
the process by which your identity is verified for access to Google Cloud services and APIs.
To run code or samples from a local development environment, you can authenticate to
Compute Engine as follows.
Select the tab for how you plan to use the samples on this page:
Console
When you use the Google Cloud console to access Google Cloud services and APIs, you don't need to set up authentication.
gcloud
-
Install the Google Cloud CLI, then initialize it by running the following command:
gcloud init
- Set a default region and zone.
REST
To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.
Install the Google Cloud CLI, then initialize it by running the following command:
gcloud init
For more information, see Authenticate for using REST in the Google Cloud authentication documentation.
-
Required roles
To get the permissions that you need to create instances and manage instance maintenance, ask your administrator to grant you the following IAM roles on the project:
-
Compute Instance Admin (v1) (
roles/compute.instanceAdmin.v1
) -
For read-only access to System Event audit logs:
Logs Viewer (
roles/logging.viewer
)
For more information about granting roles, see Manage access to projects, folders, and organizations.
These predefined roles contain the permissions required to create instances and manage instance maintenance. To see the exact permissions that are required, expand the Required permissions section:
Required permissions
The following permissions are required to create instances and manage instance maintenance:
-
To get information about an instance, including metadata:
compute.instances.get
You might also be able to get these permissions with custom roles or other predefined roles.
Limitations
To view the maintenance notifications for an upcoming maintenance event for an instance, the instance must be running a machine type from one of the following machine families:
- General-purpose C4 (Preview) VMs
- General-purpose C3 instances
- General-purpose C3D VMs
- Storage-optimized Z3 VMs
- Memory-optimized M1, M2, M3, and X4 instances
Overview of maintenance notifications
Google sends notifications for upcoming host maintenance through several methods. When the maintenance window opens, Google Cloud automatically performs maintenance on your instance. By monitoring your instance's upcoming maintenance windows, you can proactively prepare your workloads to handle upcoming maintenance with minimal disruption.
Compute instances that support maintenance event notifications have the following characteristics:
- Fewer maintenance events: In general, instances with recurrent maintenance intervals should see fewer maintenance events.
- Longer maintenance notification: Get notified of maintenance events well in advance for planning purposes.
- Monitoring and planning: Use Cloud Logging to track your maintenance schedule. Use incidents and alerts to stay informed.
- On-demand maintenance control: Start maintenance during the notification period to update your instances when it fits your schedule.
The information about an upcoming notification event is presented in a manner similar to the following:
upcomingMaintenance:{
"canReschedule":True
"latestWindowStartTime": "2024-12-01T19:00:01Z"
"maintenanceStatus":"PENDING"
"type":"SCHEDULED"
"windowEndTime": "2024-12-01T22:00:00Z"
"windowStartTime": "2024-12-01T19:00:00Z"
}
If there is no upcoming maintenance event, instead you see a message similar to the following:
{ "error": "no notifications have been received yet, try again later" }
Maintenance status definitions
The following status definitions explain the responses to queries about host maintenance for an instance. They provide information related to the maintenance event. The Google Cloud CLI, REST, and the metadata server use these same responses:
- canReschedule: whether the maintenance can be started manually during the
notification period for this instance.
TRUE
: customer-triggered maintenance can be performed during the notification period.FALSE
: customer-triggered maintenance can't be performed on this instance. This often occurs during the period in which the instance is undergoing maintenance or if the instance type doesn't support on-demand maintenance.
- latestWindowStartTime: the latest time the maintenance window can be moved to.
- maintenanceStatus: the current status of the maintenance event.
ONGOING
: the maintenance operation is underway.PENDING
: the maintenance operation is scheduled, but has not yet started.
- type: the type of maintenance to be performed.
NONE
: No maintenance is scheduled for this instance.SCHEDULED
: For disruptive maintenance, Compute Engine provides a minimum of 7 days notice for most instances; X4 instances get approximately 60 days advanced notice.UNSCHEDULED
: Because the maintenance represents critical updates, Compute Engine tries to provide as much advanced notice as possible, but it is usually much less than for scheduled maintenance events.
- windowEndTime: the end of the time window in which maintenance occurs.
- windowStartTime: the start of the time window in which maintenance occurs.
Maintenance status behaviors
When managing maintenance events, check the values for
canReschedule
and maintenanceStatus
. When combined, these fields
indicate what actions you can or can't take with regards to rescheduling a maintenance event:
canReschedule=True
andmaintenanceStatus=Pending
— you can manually start the maintenance event for the instance before the scheduled start time.canReschedule=False
andmaintenanceStatus=Ongoing
—the maintenance is underway and can't be rescheduled.canReschedule=False
andmaintenanceStatus=Pending
—your instance doesn't support manually-triggered maintenance events.
View maintenance notifications
You can find maintenance notifications by querying your compute instances, the metadata server, or by using Cloud Logging.
Check instances for a maintenance event notification
Use the Google Cloud CLI, REST, or query the metadata server to see if there is an upcoming host maintenance event for your instance.
gcloud
To see the upcoming maintenance window for an instance, use the
gcloud compute instances describe
command.
gcloud compute instances describe INSTANCE_NAME \ --zone=ZONE_NAME --format="yaml(upcomingMaintenance)"
Replace the following:
INSTANCE_NAME
: The name of the compute instance.ZONE_NAME
: The zone where the instance resides.
If there is an upcoming maintenance event, then the response contains a section similar to the following:
upcomingMaintenance:{
"canReschedule":True
"latestWindowStartTime": "2024-12-01T19:00:01Z"
"maintenanceStatus":"PENDING"
"type":"SCHEDULED"
"windowEndTime": "2024-12-01T22:00:00Z"
"windowStartTime": "2024-12-01T19:00:00Z"
}
In this response:
- The maintenance is scheduled for the date and time shown in
windowStartTime
. canReschedule
is set toTrue
andmaintenanceStatus
is set toPENDING
. These settings indicate you can manually start the scheduled maintenance event before the date shown inlatestWindowStartTime
.
REST
To see if there is upcoming maintenance for an instance, construct a GET
request using the instances.get
method:
GET https://compute.googleapis.com/compute/v1/projects/PROJECT_NAME/zones/ZONE/instances/INSTANCE_NAME
Replace the following:
PROJECT_NAME
: The name of the project that that contains the compute instance.ZONE
: The zone where the instance is located.INSTANCE_NAME
: The name of the instance.
If there is an upcoming maintenance event, then the response contains a section similar to the following:
upcomingMaintenance:{
"canReschedule":True
"latestWindowStartTime": "2023-12-01T19:00:01Z"
"maintenanceStatus":"PENDING"
"type":"SCHEDULED"
"windowEndTime": "2023-12-01T22:00:00Z"
"windowStartTime": "2023-12-01T19:00:00Z"
}
In this response:
- The maintenance is scheduled for the date and time shown in
windowStartTime
. canReschedule
is set toTrue
andmaintenanceStatus
is set toPENDING
. These settings indicate you can manually start the scheduled maintenance event before the date shown inlatestWindowStartTime
.
Metadata server
From the Guest OS, query the metadata server to see the next maintenance event.
$ curl http://metadata.google.internal/computeMetadata/v1/instance/upcoming-maintenance?alt=json -H "Metadata-Flavor: Google"
Check Cloud Logging for a maintenance event notification
Compute Engine creates system events in the Cloud Audit Logs for an instance for maintenance events. You can view these events before, during, and after a maintenance event using Cloud Logging and Logs Explorer.
Console
To query the audit logs for maintenance notifications for an instance, complete the following steps:
Go to the VM instances page.
Click the Name of the instance for which you want to view maintenance notifications.
The Instance details page opens.
In the Logs section, click the link labeled Logging.
The Logs Explorer query editor page opens. In the Query pane, the
resource.type
and instance ID are already populated for your instance.In the Query pane, add the following line to the query:
operation.producer="compute.instances.upcomingMaintenance" OR "compute.instances.terminateOnHostMaintenance" OR "compute.instances.migrateOnHostMaintenance"
Click Run query. The matching maintenance notification events appear in the query results pane.
In the query results pane you can click Edit time to expand the search timeframe, or to narrow the results to specific dates or times.
Click a log entry to view the maintenance notification details.
- For upcoming maintenance notifications, expand the heading
metadata
to view information such as the current status, type, and scheduled maintenance window start and end times. - Expand the heading
status
to view the descriptive message for the notification.
- For upcoming maintenance notifications, expand the heading
Examples of maintenance notifications
A maintenance event notification for an instance appears in the Logs Explorer with values similar to the following:
methodName
:"compute.instances.upcomingMaintenance"
metadata
:maintenanceStatus
:"PENDING"
windowStartTime
:"2024-07-23T20:00:00Z"
When the maintenance event starts, a new informational event appears in the logs with values similar to the following:
methodName
:"compute.instances.upcomingMaintenance"
metadata
:maintenanceStatus
:"ONGOING"
windowStartTime
:"2024-07-23T20:00:00Z"
During the maintenance event, depending on the host maintenance policy configuration for the instance, one of the following system events is logged to the audit logs:
- For instances configured to use live migration during maintenance events,
a system event with
methodName: "compute.instances.migrateOnHostMaintenance"
. - For instances configured to terminate during maintenance events,
a system event with
methodName: "compute.instances.terminateOnHostMaintenance"
.
When the maintenance event ends, a new informational event appears in the audit logs with values similar to the following:
methodName
:"compute.instances.upcomingMaintenance"
status: {
message: "Maintenance window has completed for this instance. All
maintenance notifications on the instance have been removed."
}
Configure alerts for host maintenance notifications
You can set up a log-based alerting policy to search for specific maintenance notification events and send alerts using a notification channel.
Console
To create an alert for a maintenance event for your instance, complete the following steps:
Go to the VM instances page.
Click the Name of the instance for which you want to create a maintenance event alert.
The Instance details page opens.
In the Logs section, click the link labeled Logging.
The Logs Explorer query editor page opens. In the Query pane, the
resource.type
and instance ID are already populated for your instance.In the Query pane, add the following line to the query:
operation.producer="compute.instances.upcomingMaintenance"
Click Run query. The matching maintenance notification events appear in the query results pane.
In the query results pane, click Edit time.
- On the left-hand side of the edit window, in the Relative time
field, enter
1d
to view the log entries for the past week. - Click Apply.
- On the left-hand side of the edit window, in the Relative time
field, enter
In the header of the Query results pane, click add_alert Create alert. If your viewing window is narrow, the Create alert option might appear on the Actions menu instead.
In the Create logs-based alert policy pane, in the Alert details section, do the following:
- Enter a name for the alert policy, for example
Upcoming maintenance for my-c3d-vm@us-central1-b
. From the Policy severity level menu, select No severity.
In the Documentation field, you can enter a description for your alerting policy. You can also include information that might help the recipient of a notification diagnose the problem. The following string summarizes the reason for the notification:
Log-based alerting policy in project ${project} to monitor upcoming maintenance notifications. See also "Host maintenance alerts" and "onHostMaintenance actions" alerting policies.
For information about how you can format and tailor the content of this field, see Using Markdown and variables in documentation templates.
To advance to the next step, click Next.
- Enter a name for the alert policy, for example
In the Choose logs to include in the alert section, check the query and results by clicking Preview logs.
The query you built in the Query pane is also displayed on this pane. We recommend building the query in the Logs Explorer Query pane first.
You can edit the query in this pane, if necessary. If you edit the query, then check the results by clicking Preview logs.
Click Next.
In the Set notification frequency and autoclose duration pane, do the following:
Select the minimum time between notifications. This value lets you control the number of notifications you get from Monitoring if this condition is met multiple times. For this example, select 1 day from the options.
For the Incident autoclose duration, use the maximum value of 7 days.
Click Next.
If you already have an email notification channel configured, then you can select it from the list. If not, click Manage notification channels and add an email channel. For information about creating notification channels, see Create and manage notification channels.
Click Save.
Your log-based alerting policy is now ready to test as described in Test the example log-based alerting policy
To learn more, read Configure log-based alerts and Create and manage notification channels.
What's next
- Learn how to Set a host maintenance policy.
- Learn how to Get live migration notices.
- Learn how to manually Trigger a host maintenance event
- Learn how to manage log-based alerting policies using the Google Cloud console for Monitoring or by using the Monitoring API.