Getting live migration notices


The metadata server provides information about a virtual machine (VM) instance's scheduling options and settings, through the scheduling/ metadata directory listing and the maintenance-event metadata key. You can use these metadata keys to learn about a VM's scheduling options, and also for notifying you when a maintenance event is about to happen.

By default, all VMs are set to live migrate and the metadata server receives maintenance event notices before a VM instance is live migrated. However, if you choose a different scheduling option the maintenance events and the VM behavior might differ. To learn more about maintenance events and VM behavior during the events, see scheduling options and settings.

Before you begin

Get live migration notices

You can learn when a maintenance event will occur by querying the maintenance-event metadata key periodically.

The maintenance-event metadata key is populated for maintenance events only if you have set your VM's scheduling option to migrate or if your VM has a GPU attached.

The value of this metadata key changes 60 seconds before a maintenance event starts, giving your application code a way to trigger any tasks you want to perform prior to a maintenance event, such as backing up data or updating logs.

In summary, Compute Engine gives the 60-second warning only if:

  • You have set the VM's availability options to live migrate during a maintenance event.

  • You have queried the maintenance-event metadata key at least once since the last maintenance event.

    • If you have never queried the maintenance-event metadata key or have not queried the metadata key since the last migration, Compute Engine assumes that the VM doesn't require advance warning of maintenance events. The maintenance event initiates immediately and skips the 60-second warning.

    • If you don't want to skip the 60-second warning, make sure your client code queries the maintenance-event metadata key at least once between migration events. You must query the maintenance-event metadata key directly for Compute Engine to determine that you are watching this metadata key. Querying a higher level metadata doesn't trigger the advance notice.

For VMs with attached GPUs, the value changes 60 minutes before the VMs are stopped to give you time to shutdown and restart again on another host. VMs with attached GPUs are not live migrated and are instead stopped and optionally restarted. To learn more, see Handling GPU host maintenance events.

Query the maintenance event metadata key

Linux VMs

To query the maintenance-event metadata key on Linux VMs, run the following command:

user@myinst:~$ curl http://metadata.google.internal/computeMetadata/v1/instance/maintenance-event -H "Metadata-Flavor: Google"

The output is similar to the following:

NONE

Windows VMs

To query the maintenance-event metadata key on Windows VMs, run the following command:

PS C:\> 
$value = (Invoke-RestMethod `
         -Headers @{'Metadata-Flavor' = 'Google'} `
         -Uri "http://metadata.google.internal/computeMetadata/v1/instance/maintenance-event")
$value

The output is similar to the following:

NONE

Python

You can use the maintenance-event metadata key with the waiting for updates feature to notify your scripts and applications when a maintenance event is about to start and end. This lets you automate any actions that you might want to run before or after the event.

The following Python sample provides an example of how you might implement these two features together.


import time

import requests


METADATA_URL = 'http://metadata.google.internal/computeMetadata/v1/'
METADATA_HEADERS = {'Metadata-Flavor': 'Google'}


def wait_for_maintenance(callback):
    url = METADATA_URL + 'instance/maintenance-event'
    last_maintenance_event = None
    last_etag = '0'

    while True:
        r = requests.get(
            url,
            params={'last_etag': last_etag, 'wait_for_change': True},
            headers=METADATA_HEADERS)

        # During maintenance the service can return a 503, so these should
        # be retried.
        if r.status_code == 503:
            time.sleep(1)
            continue
        r.raise_for_status()

        last_etag = r.headers['etag']

        if r.text == 'NONE':
            maintenance_event = None
        else:
            maintenance_event = r.text

        if maintenance_event != last_maintenance_event:
            last_maintenance_event = maintenance_event
            callback(maintenance_event)


def maintenance_callback(event):
    if event:
        print('Undergoing host maintenance: {}'.format(event))
    else:
        print('Finished host maintenance')


def main():
    wait_for_maintenance(maintenance_callback)


if __name__ == '__main__':
    main()

Review the outputs

The initial and default value of the maintenance-event metadata key is NONE.

  • For VMs with attached GPUs, during a maintenance event the value changes from NONE to TERMINATE_ON_HOST_MAINTENANCE. This value is updated 60 minutes before the stopping event starts.

  • For non-GPU VMs with a scheduling option of migrate, the maintenance-event value changes as follows:

    1. At the start of the migration event, the value changes from NONE to MIGRATE_ON_HOST_MAINTENANCE. This value is updated 60 seconds before the stopping event starts.
    2. Throughout the duration of the event and while your VM instance is being live migrated, the value remains as MIGRATE_ON_HOST_MAINTENANCE.
    3. After the maintenance event ends, the value returns to NONE.

What's next?