You can create application consistent snapshots of persistent disks attached to Linux virtual machine (VM) instances. In general, the quality of your persistent disk snapshot depends on how well your applications can recover from snapshots that you create during heavy write workloads. Application consistent snapshots capture the state of application data at the time of backup with all application transactions completed and all pending writes flushed to the disk.
To create snapshots that are application consistent, pause apps or operating system processes that write data to the persistent disk, flush the disk buffers, and sync the file system before you create the snapshot. Depending on your application, these and other steps might be required to ensure that all application transactions are complete and captured in the backup.
To create an application consistent snapshot of your persistent disks, use the following process:
- To prepare the guest environment for application consistency, create custom shell scripts to run before and after the snapshot is captured
- Configure snapshot settings on your VM (virtual machine) instance.
- Create a snapshot with the
guest-flushoption enabled. The
guest-flushoption starts your pre and post snapshot scripts.
Before you begin
- If you want to use the command-line examples in this guide, do the following:
- Create a Linux VM.
- Update the guest environment.
Permissions required for this task
To perform this task, you must have the following permissions:
compute.disks.createSnapshoton the disk
compute.disks.addResourcePolicieson the disk
compute.snapshots.deleteon the snapshot
compute.resourcePolicies.createon the resource policy
- Application consistency is guaranteed only by the behavior of your custom pre and post snapshot scripts, not by the snapshot operation itself.
- When using the
guest-flushoption in your snapshot creation request, no snapshot is created in the event of a script error or timeout.
- You can only create application consistent snapshots of persistent disks
attached to certain public or custom Linux images. The
following images are supported:
- CentOS 7
- CentOS 8
- Debian 9 Stretch
- Debian 10 Buster
- RHEL 7
- RHEL 8
- SLES 12
- SLES 15
- Ubuntu 18.04
- Ubuntu 20.04
Create pre and post snapshot scripts
Before you proceed, update the guest environment so that you are running the latest software on your linux VM.
To facilitate application consistency, create pre and post snapshot shell scripts to run before and after the snapshot is captured. Use the pre and post scripts for operations such as:
- Pause apps or operating system processes running on the VM that writes data to the persistent disk.
- Flush the disk buffers. For example, MySQL has a
FLUSHstatement. Use whichever tool is available for your app.
- Sync your file system.
The following code example shows a simple pre snapshot script. Note the leading
#! characters. Running
fsfreeze -f blocks any running process that tries to
access the filesystem, so use this with caution if your application is
#!/bin/bash fsfreeze -f [example-disk-location]
The following code example shows a simple post snapshot script. Note the
#!/bin/bash fsfreeze -u [example-disk-location]
You must save your scripts on your VM in the directory
/etc/google/snapshots/. The full path of your pre script must be
/etc/google/snapshots/pre.sh and the full path of your post script must be
Referencing specific disks in your scripts
The first argument passed to your pre and post snapshot scripts is a list of disks that are being snapshotted. You can use this argument in your scripts for various checks. For example, if your VM has multiple disks attached but you only specified one disk in your snapshot request, you can check which disk is being snapshotted.
The argument is formatted as follows:
- SCSI-attached disks: a comma-separated list of
- NVME-attached disks: a comma-separated list of
For example, your boot disk might appear as
1/0 while an additional
disk attached to the VM might appear as
Edit your guest environment configuration file
Configure your application consistent snapshot settings by updating a specific configuration file on your VM.
Open or create your guest environment configuration file:
Add the following section to the configuration file:
[Snapshots] enabled = enabled timeout_in_seconds = timeout
Replace the values according to the descriptions below:
Key Value Default Description
Whether the application consistent snapshot feature is enabled.
Integer [0, 300] 60 Number of seconds the pre or post snapshot script can take to finish running before a timeout error. Note that the number of seconds the entire snapshot operation can take to complete before a timeout error is 300 seconds per disk, and this is not configurable.
Save your configuration settings:
sudo systemctl restart google-guest-agent.service
Create a snapshot with
Using the Google Cloud Console, the
gcloud command-line tool, or the Compute Engine API,
create a snapshot with the
guest-flush option enabled. This starts running
the pre and post snapshot scripts before and after the snapshot is captured.
Run the command
gcloud compute disks snapshot DISK_NAME [DISK_NAME ...] --guest-flush
Replace the following:
DISK_NAMEis the name of one or more persistent disks of which you want to create snapshots.
gcloud command-line tool waits until the operation returns a status of
FAILED, or reaches the maximum timeout and returns the last
known details of the snapshot.
POST request to the
method with the
guestFlush option enabled:
Replace the following:
PROJECT_ID: your project ID.
ZONE: the zone where your VM and disk are located.
DISK_NAME: the name of the persistent disk of which you want to create a snapshot.
Create a snapshot schedule with
Use scheduled snapshots to regularly and automatically back up your zonal and
regional persistent disks. If you want to schedule application consistent
snapshots for your backup, use the
--guest-flush option when you create the
snapshot schedule so that the pre and post snapshot scripts execute before
and after each scheduled snapshot.
gcloud compute resource-policies create snapshot-schedule SCHEDULE_NAME \ --description "MY HOURLY SNAPSHOT SCHEDULE" \ --start-time 22:00 \ --hourly-schedule 4 \ --guest-flush
To learn more, see Creating scheduled snapshots of persistent disks.
Review the logs from your snapshot creation events:
- In the Cloud Console, go to the Google Cloud's operations suite Logging >
Go to the Logs Viewer page
- Navigate to Logs Viewer in the left navigation.
- In the Filter by label or text search list, select Convert to advanced filter.
Replace the filter text box with the following text:
- In the Cloud Console, go to the Google Cloud's operations suite Logging > Logs page:
If no script is found, no snapshot is created. Make sure you've followed the steps in Create pre and post snapshot scripts.
If there is a script error or timeout, no snapshot is created. Review Preparing for consistent snapshots. Note that the maximum timeout period that you can configure in your settings is 300 seconds. Try repeating the entire process using the sample scripts.
A persistent disk must be attached to a VM to capture a snapshot of the disk with
guest-flushenabled. To learn how to create a Linux VM and attach a persistent disk, see Creating a VM.