Best practices for persistent disk snapshots

You can create persistent disk snapshots at any time, but you can create snapshots more quickly and with greater reliability if you use the following best practices.

Before you begin

Preparing for consistent snapshots

In most situations, you can create a snapshot from persistent disks, and even if your apps are writing data to those disks you can still expect the snapshot to have good consistency. The quality of the snapshot depends on the ability of your apps to recover from snapshots that you create during heavy write workloads.

If your apps require strict consistency, you can take the following steps to ensure that a snapshot is consistent with the desired state of the persistent disk.

Flush the disk's buffers before a snapshot

You can create a snapshot of a persistent disk even while your apps write data to the disk. However, you can improve snapshot consistency if you flush the disk buffers and sync your file system before you create a snapshot.

Pause apps or operating system processes that write data to that persistent disk. Then flush the disk buffers before you create the snapshot.

To prepare your persistent disk before you take a snapshot do the following:

  1. Connect to your instance using SSH.
  2. Run an app flush to disk. For example, MySQL has a FLUSH statement. Use whichever tool is available for your app.
  3. Stop your apps from writing to your persistent disk.
  4. Run sudo sync.

If you skip this step, only data that's successfully flushed to disk by the app is included in the snapshot. The app experiences this scenario as a sudden power outage.

Freeze or unmount your filesystem

An alternative option is to freeze or unmount the filesystem before you take a snapshot. This is the most reliable way to ensure that your disk buffers are cleared, but it is more time consuming and not as convenient as simply flushing the disk buffers.

Unmount the persistent disk completely to ensure that no data is written to it while you create the snapshot. This is usually unnecessary, but it does improve the consistency of the snapshot.

  1. Connect to your instance using SSH.
  2. Stop any apps that are reading or writing data to the persistent disk.
  3. Either freeze the file system or unmount the file system.

    1. Freeze: sudo fsfreeze -f [example-disk_location]
    2. Unmount: sudo umount [example-disk_location]

    You can unfreeze or mount the file system after you complete your snapshot:

    1. Unfreeze: sudo fsfreeze -u [example-disk_location]
    2. Mount: sudo mount [example-disk_location mount_location]

Linux users

If your disk is connected to a Linux instance, unmount the disk from the instance by connecting to your instance and using the umount tool:

sudo umount /dev/disk/by-id/google-[DISK_NAME]

where [DISK_NAME] is the name of the persistent disk.

Windows users

If your disk is connected to a Windows instance, unmount the disk from the instance by connecting to your instance and using the Disk Management tool.

Remount the persistent disk

After you take a snapshot, you must remount the persistent disk. See Formatting and mounting a persistent disk for more information.

If your apps require consistency between multiple persistent disks, you must freeze or unmount all of the file systems on each disk and complete all of the snapshots for those disks before you resume your apps. Compute Engine doesn't guarantee consistency between simultaneous snapshots running on multiple persistent disks.

Use journaling file systems like ext4 to reduce the risk that data is cached without actually being written to the persistent disk.

Persistent disks using Windows Server instances

For persistent disks that are attached to Windows Server instances, use VSS Snapshots to help preserve data integrity.

Creating frequent snapshots efficiently

Use snapshots to manage your data efficiently.

  • Create a snapshot of your data on a regular schedule to minimize data loss due to unexpected failure.

  • Improve performance by eliminating excessive snapshot downloads and by creating an image and reusing it.

  • Set your snapshot schedule to off-peak hours to reduce snapshot time.

Create an image of a frequently-used snapshot

If you are repeatedly using a snapshot in the same zone to create a persistent disk, save networking costs by using the snapshot once and creating an image of that snapshot. Store this image and use it to create your disk and start a VM instance. For instructions, see Creating a custom image.

As a best practice, take a snapshot of the disk once per hour. Avoid taking snapshots more often than that. The easiest way to achieve this is to set up a snapshot schedule.

Use existing snapshots as a baseline for subsequent snapshots

If you have existing snapshots of a persistent disk, the system automatically uses them as a baseline for any subsequent snapshots that you create from that same disk.

  • Create a new snapshot from a persistent disk before you delete the previous snapshot from that same persistent disk. The system can create the new snapshot more quickly if it can use the previous snapshot and reads only the new or changed data from the persistent disk.

  • Wait for new snapshots to finish before you take subsequent snapshots from the same persistent disk. If you run two snapshots simultaneously on the same persistent disk, they both start from the same baseline and duplicate effort. If you wait for the new snapshot to finish, any subsequent snapshots run more quickly because they need only to obtain the data that has changed since the last snapshot finished.

Schedule snapshots during off-peak hours

If you schedule regular snapshots for your persistent disks, you can reduce the time that it takes to complete each snapshot by creating them during off-peak hours when possible.

  • Schedule automated snapshots during the business day in the zone where your persistent disk is located. Snapshot creation typically peaks at the end of the business day.
  • Schedule automated snapshots early in the morning in the zone where your persistent disk is located rather than immediately at midnight. Snapshot creation typically peaks at midnight.

Organize your data on separate persistent disks

If you create a snapshot of a persistent disk, any data that you store on the disk is included in the snapshot. Larger amounts of data create larger snapshots, which cost more and take longer to create. To ensure that you create a snapshot of only the data you need, organize your data on separate persistent disks.

  • Store critical data on a secondary persistent disk rather than your boot disk. This lets you create a snapshot of your boot disks only when necessary or on a less frequent schedule.
  • If you do create snapshots of your boot disks, store swap partitions, pagefiles, cache files, and non-critical logs on a separate persistent disk. These files and partitions change frequently, and the snapshot process is likely to identify them as changed data that must be included in an incremental snapshot.
  • Reduce the number of snapshots that you need to create by keeping similar data together on one persistent disk. Keep operating system and volatile data separate from the data that you want to snapshot, but you don't need to distribute your critical data across multiple persistent disks like you would for a physical machine. One large persistent disk is able to achieve the same performance as multiple smaller persistent disks of the same total size.

Enable the discard option or run fstrim on your persistent disk

On Linux instances, if you didn't format and mount your persistent disk with the discard option, run the fstrim command on the instance before you create a snapshot. The command removes blocks that the file system no longer needs, so that the system can create the snapshot more quickly and with a smaller size. See formatting and mounting a persistent disk to learn how to configure the discard option on your persistent disks.

What's next