Creating Persistent Disk Snapshots

Use snapshots to back up data from your persistent disks. Snapshots are different from public images and custom images, which are used primarily to create instances or configure instance templates. Snapshots are useful for periodic backup of the data on your persistent disks. You can create snapshots from persistent disks even while they are attached to running instances.

Snapshots are incremental, so you can create regular snapshots on a persistent disk faster and at a much lower cost than if you regularly created a full image of the disk. Incremental snapshots work in the following manner:

  • The first successful snapshot of a persistent disk is a full snapshot that contains all the data on the persistent disk.
  • The second snapshot only contains any new data or modified data since the first snapshot. Data that hasn't changed since snapshot 1 isn't included. Instead, snapshot 2 contains references to snapshot 1 for any unchanged data.
  • Snapshot 3 contains any new or changed data since snapshot 2 but won't contain any unchanged data from snapshot 1 or 2. Instead, snapshot 3 contains references to blocks in snapshot 1 and snapshot 2 for any unchanged data.

This repeats for all subsequent snapshots of the persistent disk. Snapshots are always created based on the last successful snapshot taken.

Diagram describing how to create a snapshot

Compute Engine stores multiple copies of each snapshot redundantly across multiple locations with automatic checksums to ensure the integrity of your data. You cannot share snapshots across projects.

To see a list of snapshots available to a project, run the following command:

gcloud compute snapshots list

To list information about a particular snapshot, use the gcloud compute snapshots describe command:

gcloud compute snapshots describe example-snapshot

Before you begin

Understanding snapshot best practices

You can create as many snapshots as you need at any time, but you can create snapshots more quickly and with greater reliability if you use the following best practices:

Prepare your persistent disk for the best snapshot consistency

In most situations, you can create a snapshot from persistent disks even while your applications are writing data to them and still expect the snapshot to have good consistency. The quality of the snapshot depends on the ability of your applications to recover from snapshots that you create during heavy write workloads.

If your applications require strict consistency, you can take one or more steps to ensure that a snapshot is consistent with the desired state of the persistent disk.

  • Pause applications or operating system processes that write data to that persistent disk. Then, flush the disk buffers before you create the snapshot.
  • Unmount the persistent disk completely to ensure that no data is written to it while you create the snapshot. This is usually unnecessary, but it does improve the consistency of the snapshot.
  • If your applications require consistency between multiple persistent disks, you must freeze or unmount all of the file systems on each disk and complete all of the snapshots for those disks before you resume your applications. Compute Engine does not guarantee consistency between simultaneous snapshots running on multiple persistent disks.
  • Use journaling file systems like ext4 to reduce the risk that data is cached without actually being written to the persistent disk.
  • For persistent disks that are attached to Windows Server instances, use VSS Snapshots to help preserve data integrity.

Use existing snapshots as a baseline for subsequent snapshots

If you have existing snapshots on a persistent disk, the system automatically uses them as a baseline for any subsequent snapshots that you create from that same disk.

  • Create a new snapshot from a persistent disk before you delete the previous snapshot from that same persistent disk. The system can create the new snapshot more quickly if it can use the previous snapshot and reads only the new or changed data from the persistent disk.

  • Wait for new snapshots to finish before you take subsequent snapshots from the same persistent disk. If you run two snapshots simultaneously on the same persistent disk, they will both start from the same baseline and duplicate effort. If you wait for the new snapshot to finish, any subsequent snapshots will run more quickly because they need only to obtain the data that has changed since the last snapshot finished.

Schedule snapshots during off-peak hours

If you schedule regular snapshots for your persistent disks, you can reduce the time that it takes to complete each snapshot by creating them during off-peak hours when possible.

  • Schedule automated snapshots during the business day in the zone where your persistent disk is located. Snapshot creation typically peaks at the end of the business day.
  • Schedule automated snapshots early in the morning in the zone where your persistent disk is located rather than immediately at midnight. Snapshot creation typically peaks at midnight.

Organize your data on separate persistent disks

If you create a snapshot of a persistent disk, any data that you store on the disk will be included in the snapshot. Larger amounts of data create larger snapshots, which cost more and take longer to create. To ensure that you are snapshotting only the data that you need, organize your data on separate persistent disks.

  • Store critical data on a secondary persistent disk rather than your boot disk. This allows you to snapshot your boot disks only when necessary or on a less frequent schedule.
  • If you do create snapshots of your boot disks, store swap partitions, pagefiles, cache files, and non-critical logs on a separate persistent disk. These files and partitions change frequently and the snapshot process is likely to identify them as changed data that must be included in an incremental snapshot.
  • Reduce the number of snapshots that you need to create by keeping similar data together on one persistent disk. You do want to keep your operating system and volatile data separate from the data that you want to snapshot, but there is no need to distribute your critical data across multiple persistent disks like you would for a physical machine. One large persistent disk is able to achieve the same performance as multiple smaller persistent disks of the same total size.

Enable the discard option or run fstrim on your persistent disk

On Linux instances, if you did not format and mount your persistent disk with the discard option, run the fstrim command on the instance before you create a snapshot. The command removes blocks that the file system no longer needs so that the system can create the snapshot more quickly and with a smaller size. See formatting and mounting a persistent disk to learn how to configure the discard option on your persistent disks.

Create a snapshot

For instructions on how to create snapshots from disks that are encrypted with your own encryption keys, see Creating a snapshot from an encrypted disk.

For instructions on how to create a Windows persistent disk snapshot, see Creating a Windows persistent disk snapshot.

Prepare the persistent disk for a snapshot

You can create a snapshot of a persistent disk even while your applications write data to the disk. However, you can improve snapshot consistency if you flush the disk buffers and sync your file system before you create a snapshot.

To prepare your persistent disk before you take a snapshot:

  1. Connect to your instance using SSH.
  2. Run an application flush to disk if necessary. For example, MySQL has a FLUSH statement. Other applications might have similar processes.
  3. Stop your applications from writing to your persistent disk.
  4. Run sudo sync.

If you skip this step, only data that is successfully flushed to disk by the application will be included in the snapshot. The application experiences this scenario as if it was a sudden power outage.

Optionally, you can freeze or unmount the filesystem before you take a snapshot. This is the safest and most reliable way to that ensure your disk buffers are cleared, but it is not as convenient as simply flushing the disk buffers.

  1. Connect to your instance using SSH.
  2. Stop any applications that are reading or writing data to the persistent disk.
  3. Either freeze the file system or unmount the file system.
    • Freeze: sudo fsfreeze -f example-disk_location
    • Unmount: sudo umount example-disk_location

You can unfreeze or mount the file system after you complete your snapshot:

+ Unfreeze: `sudo fsfreeze -u example-disk_location`
+ Mount: `sudo mount example-disk_location mount_location`

Create your snapshot

Console

  1. Go to the Create a new snapshot page.
  2. Enter a snapshot name and select an existing disk from which you want to make a snapshot.
  3. Click Create to create the snapshot.

gcloud

To create your snapshot, use the gcloud compute disks snapshot command:

gcloud compute disks snapshot [DISK_NAME]

The gcloud command-line tool waits until the operation returns a status of READY or FAILED, or reaches the maximum timeout and returns the last known details of the snapshot.

API

Make a POST request to the createSnapshot method:

POST https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/[ZONE]/disks/[DISK_NAME]/createSnapshot

where:

  • [PROJECT_ID] is your project ID.
  • [ZONE] is the zone where your instance and new disk are located.
  • [DISK_NAME] is the name of the new disk.

Restoring a snapshot to a larger persistent disk

You can restore a non-root persistent disk snapshot to a larger size than the original snapshot, but you must run some additional commands from within the instance for the additional space to be recognized by the instance.

Depending on your operating system and filesystem type, you might need to use a different filesystem resizing tool. Please refer to your operating system documentation for more information.

First, create a new persistent disk that is larger than your snapshot:

  1. Create a new persistent disk from your non-root snapshot, using the --size flag to specify a disk size that is larger than the snapshot size:

    gcloud compute disks create example-disk \
      --source-snapshot=example-snapshot --size=600GB
    
  2. Attach your persistent disk to an instance:

    gcloud compute instances attach-disk example-instance --disk=example-disk
    
  3. After you create the new persistent disk and attach it to your instance, resize the filesystem on that persistent disk to include the additional disk space.

You can now use the extra persistent disk space to store data.

Restoring a snapshot to a different type of disk

You can restore a snapshot to a different type of persistent disk by running the following command:

gcloud compute disks create example-disk --source-snapshot=example-snapshot --type={pd-standard|pd-ssd}

Deleting a snapshot

Compute Engine uses incremental snapshots so that each snapshot contains only the data that has changed since the previous snapshot. For unchanged data, snapshots use references to the data in previous snapshots. Persistent disk snapshots charge only for the total size of the snapshot.

When you delete a snapshot, Compute Engine immediately marks the snapshot as DELETED in the system. If the snapshot has no dependent snapshots, it is deleted outright. However, if the snapshot does have dependent snapshots:

  1. Any data that is required for restoring other snapshots is moved into the next snapshot, increasing its size.
  2. Any data that is not required for restoring other snapshots is deleted. This lowers the total size of all your snapshots.
  3. The next snapshot no longer references the snapshot marked for deletion, instead referencing the snapshot before it.

Because subsequent snapshots might require information stored in a previous snapshot, keep in mind that deleting a snapshot does not necessarily delete all the data on the snapshot. As mentioned in the first bullet above, if any data on a snapshot that is marked for deletion is needed for restoring subsequent snapshots, that data is moved into the next corresponding snapshot. To definitively delete data from your snapshots, you should delete all snapshots.

The diagram below illustrates the process described above:

Diagram describing the
  process for deleting a snapshot

To delete a snapshot, use the gcloud compute snapshots delete command:

gcloud compute snapshots delete example-snapshot

Monitor your resources on the go

Get the Google Cloud Console app to help you manage your projects.

Send feedback about...

Compute Engine Documentation