VM instance lifecycle

This document describes the lifecycle of a single virtual machine (VM) instance, starting from its creation through its potential deletion.

A VM instance can transition through many states as part of its lifecycle. When you create a VM, Compute Engine provisions resources to start the VM. Next, the VM moves into staging, where it prepares for first boot. During and after start-up, a VM is considered running. During its lifetime, a running VM can be repeatedly stopped and restarted or suspended and resumed.

A VM can be in one of the following states:

  • PROVISIONING: resources are allocated for the VM. The VM is not running yet.
  • STAGING: resources are acquired, and the VM is preparing for first boot.
  • RUNNING: the VM is booting up or running.
  • STOPPING: the VM is being stopped. You requested a stop, or a failure occurred. This is a temporary status after which the VM enters the TERMINATED status.
  • REPAIRING: the VM is being repaired. Repairing occurs when the VM encounters an internal error or the underlying machine is unavailable due to maintenance. During this time, the VM is unusable. You are not billed when a VM is in repair. VMs are not covered by the Service level agreement (SLA) while they are in repair. If repair succeeds, the VM returns to one of the above states.
  • TERMINATED: the VM is stopped. You stopped the VM, or the VM encountered a failure. You can restart or delete the VM.
  • SUSPENDING: The VM is in the process of being suspended. You suspended the VM.
  • SUSPENDED: The VM is in a suspended state. You can resume the VM or delete it.

The following diagram shows the transition between each VM state:

VM instance lifecycle.

Stopping, suspending, or resetting a VM

You can stop or suspend a VM if you no longer need it but want to keep the VM for future use. Your decision to stop or suspend a VM depends on whether you need to preserve its guest OS memory and application state.

When you suspend or stop a VM, consider the following:

  • You pay for resources that are still attached to a suspended or stopped VM, such as static IPs and persistent disk data. For more information, see the pricing guide.
  • Ephemeral external IP addresses are released from the VM. When you restart or resume the VM, Google assigns new ephemeral external IP addresses. If you depend on an ephemeral external IP address remaining the same through restarts, promote the ephemeral external IP address to a static IP address.
  • Static external IP addresses are retained.

Stopping a VM

Stopping a VM causes Compute Engine to send the ACPI shutdown signal to the VM. Modern guest operating systems are configured to perform a clean shutdown before powering off in response to the power off signal. Compute Engine waits a short time for the guest to finish shutting down and then transitions the VM to the TERMINATED state.

You might want to stop a VM for several reasons:

  • You no longer need the VM but want the resources that are attached to the VM—such as its internal IPs, MAC address, and persistent disk.
  • You don't need to preserve the guest OS memory, device state, or application state.
  • You want to change certain properties of the VM that require you to first stop the VM.

You can restart a terminated VM when you need to use it again.

Suspending a VM

Suspending a VM sends an ACPI S3 suspend signal to the VM's operating system. Suspending a VM is analogous to closing the lid of your laptop or putting the VM into a suspended state. When you suspend a VM, it transitions to the SUSPENDED state.

You might want to suspend a VM for the following reasons:

  • You want to stop paying for the core and memory costs of running a VM and pay the comparatively cheaper cost of storage to preserve the state of your VM instead.
  • You don't need the VM at this time but want to be able to bring it back up quickly with its OS and application state where you left it.

You can resume a suspended VM when you need to use it again.

Resetting a VM

Alternatively, you can reset a VM to wipe the memory contents of the VM and reset the VM to its initial state. Resetting a VM causes an immediate hard reset of the VM; the VM doesn't do a graceful shutdown of the guest OS. However, the VM retains all persistent disk data, and none of the VM properties change. The VM remains in the RUNNING state through the reset.

Comparison table

The following table shows how stopping, suspending, and resetting a VM affects the VM and its resources:

Stopping Suspending Resetting
Billing Google does not charge for VM resources while they are in the TERMINATED state. Google charges for storing the preserve state of suspended VMs. Because the VM remains in the RUNNING state when it is resetting, Google charges for VMs according to standard VM prices.
VM state TERMINATED SUSPENDED RUNNING
Persistent disks Persistent disks are maintained, even when persistent disks are marked for auto-delete.

You are charged for a persistent disk that is associated with a stopped VM, just like you are charged for a persistent disk that is not associated with a VM.		 Persistent disks are maintained, even when persistent disks are marked for auto-delete.

You are charged for a persistent disk that is associated with a stopped VM, just like you are charged for a persistent disk that is not associated with a VM.		 Persistent disks are maintained, even when persistent disks are marked for auto-delete.

You are charged for persistent disks as long as the VM remains in a RUNNING state.
RAM and VM state Reset to power-on state, no data is saved. Guest OS and application state is preserved and restored when the VM is resumed.
You are charged a specific rate for storing preserved data. For more information, see Billing for suspended instances.		 Reset to power-on state, no data is saved.
GPUs GPUs are not charged when a VM is stopped.

A GPU that is attached to a TERMINATED VM doesn't count against your project quota.		 Not supported.

If a GPU is attached to a VM, you cannot suspend the VM.		 You are charged standard GPU prices.
Local SSDs Not supported.

You cannot stop a VM that has a local SSD attached to it.		 Not supported.

You can suspend a VM that has a local SSD, but you must explicitly discard the local SSD data in the process.		 Local SSDs are maintained.
External ephemeral IPs Ephemeral IPs are released when a VM is stopped, but a new ephemeral IP address is acquired when the VM restarts. Ephemeral IPs are released when a VM is suspended, but a new ephemeral IP address is acquired when the VM restarts. Ephemeral IPs are released when a VM is reset, but a new ephemeral IP address is acquired when the VM restarts.
External static IPs Static external IPs are maintained.

Static IPs that are assigned to VMs in the TERMINATED state are charged as if they aren't attached to any VM. For more information, see pricing.		 Static external IPs are maintained.

Static IPs that are assigned to VMs in the SUSPENDED state are charged as if they aren't attached to any VM. For more information, see pricing.		 Static external IPs are maintained.
Internal IPs or MAC addresses Internal IPs and MAC addresses are maintained. Internal IPs and MAC addresses are maintained. Internal IPs are released when a VM is deleted.

MAC addresses are generated based on the internal IP. If you want to reuse the MAC address, set the same internal IP address or use a reserved internal IP.
VM metadata VM metadata is maintained. VM metadata is maintained. VM metadata is maintained.

Checking a VM's status

Every VM reports its status as part of the VM's properties. Check a VM's status in the following circumstances:

  • You are experiencing issues with the VM.
  • You just started the VM and want to confirm that the VM is running.
  • You want to confirm that the VM has been suspended or stopped (that is, in the SUSPENDED or TERMINATED state).

To list all VMs and their statuses, use the following command:

gcloud compute instances list

To describe the status of a single instance, use this command:

gcloud compute instances describe example-instance

Hardware failure

Rarely, a VM might fail due to an unexpected outage, hardware error, or another system issue. Mitigate hardware failure by using persistent disks, routinely backing up your data, and designing your system so that a single VM failure is not catastrophic. For more information, see Designing robust systems.

If a VM fails, it is restarted automatically with the same root persistent disk, metadata, and VM settings. To modify the automatic restart behavior, see Setting instance availability policies.

What's next