Overview of backups

Stay organized with collections Save and categorize content based on your preferences.

This page describes what a backup is, how it works, some common use cases, and best practices when creating and using backups. To learn how to create and manage backups, as well as how to restore a Filestore instance from a backup, see Back up data for disaster recovery.

What is a backup?

A Filestore backup is a copy of a file share that includes all file data and metadata of the file share from the point in time when the backup is created. After a backup of a file share is created, the original file share can be modified or deleted without affecting the backup. A file share can be restored from a backup as a new Filestore instance or onto an existing file share.

Backups are regional resources that remain within the region that you specify at the time of creation. You can create backups in the same region as the Filestore instance or in another region for cross-region redundancy. Backups are globally addressable and can be used to restore file shares to any region, but they cannot be shared across projects.

Backup creation

The first backup you create is a complete copy of all file data and metadata on a file share. Each subsequent backup copies any incremental changes made to the data since the previous backup. A group of backups associated with the same instance are called a backup chain. Backup chains reside in a single bucket and region and can be located outside of the region used to store the source instance. This behavior gives you the option of creating a geo-redundant copy of instance data.

Unchanged data contained in previous backups are referenced in, but not copied to, newer backups. If an older backup is deleted, its unique data is copied to the next most recent backup and all internal data references are automatically updated.

Backup creation is instantaneous, but it takes a period that's proportional to the amount of data being copied before the backup is available for use. During this period, the backup transitions through three states:

State Duration Description
Creating A few seconds Capturing the current state of the file share. Any new changes to file share data may or may not be included in the backup. Stable writes acknowledged by the instance before the backup is initiated are included.
Finalizing Depends on size Uploading data to the backup. Any new changes to file share data are not included in the backup.
Ready Until the backup is deleted The backup is ready for use.

After creation, backups are automatically compressed to reduce cost. Instance performance may be reduced while creating a backup for High Scale SSD or Enterprise tier instances. Creating a backup does not affect the availability or performance of Basic tier instances.

Backup consistency

Filestore backups have NFSv3 consistency semantics. Before a backup is initiated, any write that the Filestore instance acknowledges as written to stable storage or that is followed by an acknowledged COMMIT is included in the backup. For details, see NFSv3 RFC-1813 section 3.3.7.

Common use cases

The following sections describe common use cases for backups.

Backing up data for disaster recovery

Imagine that you have a Filestore instance in us-west1-c, and you want to protect your data against disasters that affect this region. You can schedule a job that regularly creates backups of this instance to a remote region, say us- east1. If a disaster occurred involving us-west1-c, you can create a new instance in another location from any previous backup.

Backing up data to protect against accidental changes

If you want to protect your Filestore data against unintended changes, you can schedule a job that regularly creates backups of the instance. If you lose data, you can browse through the list of backups to identify the one with the version of the file needed. Then, you can create a new Filestore instance from the backup, mount it to the same client as the original instance, and copy the file over.

Before copying the file over, you can use the Linux diff command on the two mount points to check the differences between the data on the original instance and the data restored from the backup. After the data is recovered, you can delete the restored instance and create a new backup to preserve your data's present state for future use.

Alternatively, you can do an in-place restore where the backup data is directly restored to the original Filestore instance, replacing all data on it with data from the backup. We recommend that you create a backup of the latest data before performing an in-place restoration because any unbacked data is lost.

Creating clones for development and testing

Imagine you have a database set up on a Filestore instance that serves production traffic. If you want to run a test with a database as an input, you can create a new Filestore instance from a backup of the production instance for the test. In this way, test usage does not interfere with production.

Similarly, you can use backups for offline analysis and investigation without affecting production.

Migrating Data

After you create a Filestore instance, you cannot change its location or service tier. To migrate your data to another region, you can create a backup of it and use the backup to create a new Filestore instance or restore it to an existing instance.

Also, when you create a new Filestore instance from a backup, you can choose between Basic HDD and Basic SSD tiers regardless of the tier of the source instance.

Feature limitations

Filestore Backups is generally available (GA) for Basic HDD and Basic SSD tier instances and is a Preview feature for Enterprise and High Scale SSD tier instances.

While in Preview, the following limitations apply:

  • We recommend creating a new project to enable the feature for Enterprise or High Scale SSD tier instances. Production workloads should always reside in projects separate from Preview workloads.

  • Filestore backups cannot be combined with the Filestore Multishares feature.

  • Once pricing is implemented, relevant fees apply.

  • Backups created in Preview reflect feature behavior available at the time of creation. Existing backups are not updated as feature limitations are removed. Once the feature becomes generally available in your service tier, create a new backup or backups to replace those created in Preview. Backups created in Preview are subject to deletion.

The following sections cover other feature limitations while in Preview.

Performance

  • Numerous changes made through numerous hard-links on the same file (e.g. tens or hundreds of thousands) may result in impacts to performance.

  • Enterprise and High Scale SSD tier instance performance may be reduced by as much as 15% while a backup is uploaded. Basic tier backups do not impact instance performance.

Storage

  • Enterprise and High Scale SSD tier instances support a single backup chain. Such a chain must reside entirely in a single region, though it's not restricted to the same location as the source instance.

    If you want to use a CMEK and store your backup data in a region separate from the source for geo-redundancy, you must use two separate CMEK keys: one for the source instance and one for the backup chain. Some restrictions apply:

    • A CMEK must reside in the same region as the backup chain it encrypts.

    • A single CMEK is applied to the bucket where the backup chain is stored and cannot be combined or replaced.

    • To create a backup using a new CMEK, the entire existing backup chain must first be deleted.

    • If a CMEK exists, it must first be enabled to delete a backup.

    • Basic HDD and SSD tier instances support multiple backup chains. CMEK support is not available for these service tiers.

    For more information, see CMEK support for backup chains.

  • Once a RestoreInstance operation is applied to an Enterprise tier instance, you won't be able to create snapshots with the same names as previous snapshots prior to the operation.

  • After an instance is deleted, its backup cannot be deleted. See Deletion requests for Enterprise and High Scale SSD tier backups for instruction on how to submit a deletion request.

  • Attempts to restore an instance from a backup while either a backup deletion or snapshot deletion are in progress fail.

  • To create a Enterprise or High Scale SSD tier backup in a new location, the entire existing backup chain must first be deleted.

Capacity

Each backup occupies instance capacity. This capacity varies relative to the scope of changes made to the data since the last backup was created.

More specifically, when a backup is created, Filestore creates an internal snapshot of the filesystem which also occupies a portion of available instance capacity.

Snapshot size is also relative to the scope of changes made to data within the share since the last backup was created. This snapshot continues to exist until the next subsequent backup is created and uploaded.

All data referenced by the backup persists in the state as it was when captured and continues to take up capacity from the filesystem. So for example, if you were to delete data from the mounted filesystem, that action itself would not free up capacity. Instead, to do so, you would create a new backup after deleting or overwriting significant amounts of data.

To anticipate sufficient capacity for your workloads, consider applying one of the following:

  • Increase instance capacity for workloads with significant, frequent data changes or a "high change rate".

  • Limit the scope or change rate of data modifications.

Best practices

The following sections cover recommended best practices.

Preparing your file share for the best backup consistency

The quality of a backup depends on the ability of your application to recover from backups that are created during heavy write workloads. In most situations, you can create backups that have good consistency even while your applications write data to the file share. However, if your applications require strict consistency, we recommend doing one or more of the following:

  • Use sync mount. For more information, see "The sync mount option" section in nfs(5). Alternatively, you can open files with the O_DIRECT|O_SYNC flags. For more information, see open(2).
  • Pause applications or operating system processes that write data to the file share and cause them to flush their changes to the file share before initiating the backup. For more information, see fsync(2).
  • If your applications require consistency between multiple shares, pause all applications on all instances that are writing to all file shares and create backups of all file shares before resuming your applications.
  • If you require application level consistency, stop your applications and unmount the file share before creating a backup.

Using existing backups as a baseline for new backups to reduce backup creation time

Existing backups of a file share within a region are used as baselines for creating new backups of the file share, reducing backup creation time. Therefore, we recommend that you do the following:

  • Take a new backup of a file share before you delete the previous backup of that file share.

  • Wait for new backups to be in the Ready state before creating subsequent backups of the same file share.

Scheduling backups during off-peak hours to reduce backup creation time

Creating backups during off-peak hours reduces the time that it takes to create a backup. If you schedule regular backups of your file shares, we recommend scheduling them during off-peak hours when possible.

Peak hours for backups creation are the end of each business day and midnight in the region where the Filestore instance is located. We recommend creating your backups either in the early morning or during the business day.

Organizing your data on separate Filestore instances to maximize efficiency

The more data on the file share, the larger the backup and the more it costs. To back up only the data that you need to back up, we recommend organizing your data on separate file shares, namely:

  • Storing critical data with different write patterns or with different backup requirements on different file shares.
  • Reducing the number of backups that you need to create by keeping similar data in one file share.

Quota

A quota limit exists regarding the number of backups per region for Basic SSD and Basic HDD service tiers.

Backup quota limits do not apply to High Scale SSD and Enterprise service tiers.

For more information, see Service tiers and quota.

Get started with Filestore backups

To get started using the feature, see Backup data for disaster recovery.

What's next