Suspending and resuming an instance

This document describes how to suspend and resume a virtual machine (VM) instance. For information about how to stop and start an instance, see Stopping and starting an instance. For information about the instance lifecycle, see Instance Life Cycle documentation.

Suspending an instance sends an ACPI S3 suspend signal to the instance's operating system. Suspending an instance is analogous to closing the lid of your laptop, putting the instance into a SUSPENDED state. Suspending an instance is ideal for:

  • Development and test environments that are not being fully utilized during off periods, such as during evenings or weekends, for cost savings or faster initialization than creating new VM instances.
  • Applications that require a long period of initialization after the instance has finished booting, but before the application is ready to service its first request, such as virtual developer workstations or complex Java applications.

Suspending an instance differs from stopping an instance in the following ways:

  • Suspended instances preserve the guest OS memory, device state, and application state.
  • Google charges for suspended instances.
  • You can only suspend an instance for up to 60 days. After 60 days, the instance is automatically moved to the TERMINATED state.

With the exception of local SSD data, all resources that are attached to the instance remain attached to the instance and will be charged, including persistent disks and static or reserved external IP addresses. All of these resources are charged according to the price sheet, even if an instance is suspended.

You cannot suspend an instance by using the standard processes that are built into the guest environment. Commands, such as the systemctl suspend command in Ubuntu 16.04 and later, are not available. You can only use the gcloud command-line tool command-line tool or the Compute Engine API to suspend an instance.

If you don't care about restoring an instance's memory and device state when you resume the instance later, you can stop the instance instead, which does not incur additional storage charges.

Before you begin

Beta restrictions

During beta, the following restrictions apply for this feature:

  • You cannot suspend an instance that uses a GPU.
  • You can suspend an instance that has attached local SSD, but the local SSD contents must be discarded using a special flag for discarding local SSD data. For more information, see Suspending a VM instance.
  • You cannot suspend an instance by using the standard processes that are built into the guest environment. Commands, such as the systemctl suspend command in Ubuntu 16.04 and later, are not available. The in-guest signal is ignored.
  • You can only suspend an instance for up to 60 days before the VM is automatically stopped.
  • You cannot suspend instances with more than 120 GB of memory.
  • You can suspend preemptible instances, but the preemptible instance might be terminated before it is successfully suspended.
  • During beta, Compute Engine might discard the preserved memory state of a suspended instance. If Compute Engine does have to discard preserved memory state, Compute Engine will move the instance to the TERMINATED state, and you can reboot the instance normally using the instances.start method.

OS compatibility

Most operating systems offered on Compute Engine support the suspend and resume functionality, but there are a few OSes that do not. Use the following table to determine whether an OS supports suspend and resume.

Image family Versions Status Notes
Supported OS
Ubuntu 16, 18, 19 or later Supported None
Container-Optimized OS 69, 73, 77, 81 or later Supported None
RHEL 7 Supported None
CentOS 7 Supported None
Windows 2012, 2016, 2019 or later Supported None
SUSE (SLES) 12, 15 or later Supported None
Debian 8, 9 Supported, but requires some configuration Instructions for configuring Debian 8, 9
Unsupported OS
CentOS 6, 8 Unsupported None
RHEL 6, 8 Unsupported None
SQL Server on Windows Server All Unsupported None
RHEL for SAP All Unsupported None
Debian 10 Unsupported None
CoreOS All Unsupported None

Pricing

When you suspend an instance, you are charged for:

Suspending an instance

To suspend an instance, use the Cloud Console, the gcloud tool, or the beta API.

You cannot suspend an instance by using the standard processes that are built into the guest environment. You can use the Cloud Console, the gcloud command-line tool, or the API to suspend an instance.

Console

  1. In the Cloud Console, go to the VM instances page.

    Go to the VM instances page

  2. Select one or more instances to suspend.

  3. Click Suspend.

  4. When prompted, if you want to discard local SSD data, select Discard the SSD content. When your instance is resumed, any local SSD data on the instance will have been discarded.

gcloud

To suspend an instance in the gcloud command-line tool:

 gcloud beta compute instances suspend INSTANCE_NAME

After you make a request to suspend an instance, it can take some time for Compute Engine to preserve all the data necessary for the instance. During this time, you continue to be charged for the instance while it remains running.

A suspended instance is marked by SUSPENDED status. Check an instance's status by making a describe request:

gcloud compute instances describe INSTANCE_NAME

To suspend an instance with local SSD data, you must discard the local SSD data by providing the --discard-local-ssd flag:

gcloud beta compute instances suspend INSTANCE_NAME --discard-local-ssd

API

In the API, make a request using the instances.suspend method:

https://www.googleapis.com/compute/beta/projects/PROJECT_ID/zones/ZONE/instances/INSTANCE_NAME/suspend

Replace the following:

  • PROJECT_ID: the project ID
  • ZONE: the zone of the VM
  • INSTANCE_NAME: the instance that you want to suspend

After you make a request to suspend an instance, it can take some time for Compute Engine to preserve all the data necessary for the instance. During this time, you are charged for the instance while it remains running.

Compute Engine marks suspended instance with the SUSPENDED status. Check an instance's status by making a GET request:

GET https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/INSTANCE_NAME

The instance's status is visible in the status field. For example:

...
"zone": "https://content.googleapis.com/compute/v1/projects/example-project/zones/us-central1-a",
"status": "SUSPENDED",
"name": "example-vm",
...

To suspend an instance with local SSD data, you must discard the local SSD data by providing the optional discardLocalSsd query parameter:

https://www.googleapis.com/compute/beta/projects/PROJECT_ID/zones/ZONE/instances/INSTANCE_NAME/suspend?discardLocalSsd=true

Resuming a suspended instance

To resume a suspended instance, use the Cloud Console, the gcloud tool, or the beta API.

Console

  1. In the Google Cloud Console, go to the VM instances page.

    Go to the VM instances page

  2. Select one or more instances to resume.

  3. Click Start/Resume.

gcloud

To resume an instance in the gcloud command-line tool:

 gcloud beta compute instances resume INSTANCE_NAME

After you make a request to resume an instance, it can take some time for Compute Engine to restore all the data necessary for the instance. During this time, you are charged for the instance while it is resuming.

An instance is resumed when it is marked as RUNNING. Check an instance's status by making a describe request:

gcloud compute instances describe INSTANCE_NAME

API

Make a request to the instances.resume method:

https://www.googleapis.com/compute/beta/projects/PROJECT_ID/zones/ZONE/instances/INSTANCE_NAME/resume

Replace the following:

  • PROJECT_ID: the project ID for this request
  • ZONE: the zone of the VM
  • INSTANCE_NAME: the instance to resume

After you make a request to resume an instance, it can take some time for Compute Engine to restore all the data necessary for the instance. During this time, you are charged for the instance while it is resuming

Compute Engine marks the instance with the RUNNING status once the resume is complete. Check an instance's status by making a GET request:

GET https://compute.googleapis.com/compute/v1/projects/example-project/zones/us-central1-a/instances/example-instance

The instance's status is visible in the status field. For example:

...
"zone": "https://content.googleapis.com/compute/v1/projects/example-project/zones/us-central1-a",
"status": "RUNNING",
"name": "example-instance",
...

Resuming instances that have encrypted disks

If the instance you want to resume uses customer-supplied encryption keys, you must provide those keys when trying to resume the instance.

Resume the instance by using the gcloud tool or the API.

gcloud

Provide the key by using the --csek-key-file flag and the name of the instance to start. If you are using an RSA-wrapped key, use the gcloud beta component:

gcloud beta compute instances resume INSTANCE_NAME \
  --csek-key-file ENCRYPTION_KEY_FILE

Replace the following:

  • INSTANCE_NAME: the name of the instance
  • ENCRYPTION_KEY: the relative path to the file that contains the encryption key that you used to encrypt persistent disks that are attached to the instance

API

Construct a POST request to the instances.resume method to resume the instance by using an encryption key.

POST https://compute.googleapis.com/compute/beta/projects/PROJECT_ID/zones/ZONE/instances/INSTANCE_NAME/resume
{
  "disk": [
    {
       "source": "DISK_NAME",
       "diskEncryptionKey": {
         "rsaEncryptedKey": "ENCRYPTION_KEY"
       }
    }
  ]
}

Replace the following:

  • INSTANCE_NAME: the name of the instance
  • PROJECT_ID: the project ID for this request
  • ZONE: the zone for this instance
  • DISK_NAME: the attached disk that is encrypted with a customer-supplied encryption key
  • ENCRYPTION_KEY: the encryption key that you use to encrypt persistent disks that are attached to the instance

Each encrypted disk must be listed, with the matching diskEncryptionKey, because Compute Engine does not store these keys on its servers while the instance is in the SUSPENDED state.

Suspend process

When you make a suspend request, you send an ACPI suspend signal to the VM instance. If the VM does not respond to the ACPI S3 suspend signal within a couple of minutes, Compute Engine cancels the suspend attempt and returns the VM to a RUNNING state.

The following table describes the effect of suspending a VM instance on its associated resources:

Resource Support
Memory Only VMs with less than or equal to 120 GB of memory can be suspended
Local SSD Local SSD data is discarded
Persistent disk Persistent HDD and SSD disks are retained
IP addresses Ephemeral IPs are released during suspension, but static IPs remain attached to the VM instances. If you would like to retain your ephemeral ID, promote it.
VM configuration (such as machine type, metadata, labels, etc. All VM configurations, other than ephemeral IP addresses, are preserved and restored when the instance resumes.

Local SSDs and preemptible VMs

Local SSD content is not preserved when an instance is suspended. By default, suspending an instance that has a local SSD returns an error. You can override this default behavior by specifying a flag to discard local SSD data, which lets the instance be suspended and resumed, but discards the contents of the local SSD in the process. In this scenario, Compute Engine resumes the instance with the same number of local SSD volumes attached, but the volumes will be empty and uninitialized.

You can suspend a preemptible VM but if preemption (not the advance warning that precedes the preemption) occurs before the suspend operation completes, the suspend exits and the instance is preempted.

Configuring a Debian VM to support suspend and resume

VMs running Debian 8 and 9 can suspend and resume, but they must be configured beforehand. To configure your Debian instance, complete one of the following sets of instructions, option A or option B. We recommend configuring the ACPID if possible (option A).

Option A

This option configures the ACPID to handle the sleep button event and adds a shell script for handling the sleep event.

  1. Connect to your VM instance using ssh:

    gcloud compute instances ssh INSTANCE_NAME
    
  2. On the VM instance, create a directory under the acpi folder:

    sudo mkdir -p /etc/acpi/events/
    
  3. Configure ACPID to handle the sleep button event:

    cat <<EOF | sudo tee /etc/acpi/events/sleepbtn-acpi-support
    event=button[ /]sleep
    action=/etc/acpi/sleepbtn-acpi-support.sh
    EOF
    
  4. Create the sleep event handling script:

    cat <<EOF | sudo tee /etc/acpi/sleepbtn-acpi-support.sh
    #!/bin/sh
    echo mem > /sys/power/state
    EOF
    
  5. Set up the permissions for the script:

    sudo chmod 755 /etc/acpi/sleepbtn-acpi-support.sh
    
  6. Restart ACPID:

    sudo systemctl restart acpid.service
    

Option B

  1. Connect to your VM instance using ssh:

    gcloud compute instances ssh INSTANCE_NAME
    
  2. On the VM instance, install dbus:

    sudo apt-get install dbus
    
  3. Restart logind:

    sudo systemctl restart systemd-logind.service
    

What's next