Overview of backups

Stay organized with collections Save and categorize content based on your preferences.

This page describes what a backup is, how it works, some common use cases, and best practices when creating and using backups. To learn how to create and manage backups, as well as how to restore a Filestore instance from a backup, see Back up data for disaster recovery.

What is a backup?

A Filestore backup is a copy of a file share that includes all file data and metadata of the file share from the point in time when the backup is created. After a backup of a file share is created, the original file share can be modified or deleted without affecting the backup. A file share can be restored from a backup as a new Filestore instance or onto an existing file share.

Backups are regional resources that remain within the region that you specify at the time of creation. You can create backups in the same region as the Filestore instance or in another region for cross-region redundancy. Backups are globally addressable and can be used to restore file shares to any region, but they cannot be shared across projects.

Backup creation

The first backup you create is a complete copy of all file data and metadata on a file share. Each subsequent backup copies any incremental changes made to the data since the previous backup. A group of backups associated with the same instance and region are called a backup chain.

Unchanged data contained in previous backups are referenced in, but not copied to, newer backups. If an older backup is deleted, its unique data is copied to the next most recent backup and all internal data references are automatically updated.

Backup creation is instantaneous, but it takes a period that's proportional to the amount of data being copied before the backup is available for use. During this period, the backup transitions through three states:

State Duration Description
Creating A few seconds Capturing the current state of the file share. Any new changes to file share data may or may not be included in the backup. Stable writes acknowledged by the instance before the backup is initiated are included.
Finalizing Depends on size Uploading data to the backup. Any new changes to file share data are not included in the backup.
Ready Until the backup is deleted The backup is ready for use.

After creation, backups are automatically compressed to reduce cost. Instance performance may be reduced while creating a backup for High Scale or Enterprise tier instances. Creating a backup does not affect the availability or performance of Basic tier instances.

Backup consistency

Filestore backups have NFSv3 consistency semantics. Before a backup is initiated, any write that the Filestore instance acknowledges as written to stable storage or that is followed by an acknowledged COMMIT is included in the backup. For details, see NFSv3 RFC-1813 section 3.3.7.

Common use cases

The following sections describe common use cases for backups.

Backing up data for disaster recovery

Imagine that you have a Filestore instance in us-west1-c, and you want to protect your data against disasters that affect this region. You can schedule a job that regularly creates backups of this instance to a remote region, say us- east1. If a disaster occurred involving us-west1-c, you can create a new instance in another location from any previous backup.

Backing up data to protect against accidental changes

If you want to protect your Filestore data against unintended changes, you can schedule a job that regularly creates backups of the instance. If you lose data, you can browse through the list of backups to identify the one with the version of the file needed. Then, you can create a new Filestore instance from the backup, mount it to the same client as the original instance, and copy the file over.

Before copying the file over, you can use the Linux diff command on the two mount points to check the differences between the data on the original instance and the data restored from the backup. After the data is recovered, you can delete the restored instance and create a new backup to preserve your data's present state for future use.

Alternatively, you can do an in-place restore where the backup data is directly restored to the original Filestore instance, replacing all data on it with data from the backup. We recommend that you create a backup of the latest data before performing an in-place restoration because any unbacked data is lost.

Creating clones for development and testing

Imagine you have a database set up on a Filestore instance that serves production traffic. If you want to run a test with a database as an input, you can create a new Filestore instance from a backup of the production instance for the test. In this way, test usage does not interfere with production.

Similarly, you can use backups for offline analysis and investigation without affecting production.

Migrating Data

After you create a Filestore instance, you cannot change its location or service tier. To migrate your data to another region, you can create a backup of it and use the backup to create a new Filestore instance or restore it to an existing instance.

Also, when you create a new Filestore instance from a backup, you can choose between Basic HDD and Basic SSD tiers regardless of the tier of the source instance.

Feature limitations

  • Filestore Backups is generally available (GA) for Basic HDD and Basic SSD tier instances and is a Preview feature for Enterprise and High Scale tier instances.

  • While in Preview, the following limitations apply:

    • Users should create a new project to enable the feature for Enterprise or High Scale tier instances. Production workloads should always reside in projects separate from preview workloads.

    • Filestore backups cannot be combined with the Filestore Multishares feature.

    • Once pricing is implemented, relevant fees apply.

    • With regard to performance, the following limitations apply:

      • Numerous changes made through numerous hard-links on the same file (e.g. tens or hundreds of thousands) may result in impacts to performance.

      • Enterprise and High Scale tier instance performance may be reduced by as much as 15% while a backup is uploaded. Basic tier backups do not impact the performance of the instance.

    • With regard to storage, the following limitations apply:

      • An entire backup chain associated with a High Scale or Enterprise tier instance must reside in a single region.

        • Basic tier instances can have multiple backup chains but each must reside in a separate region.
      • Instances and their associated backup chains can reside in separate regions. Instances and their backup chains can also use separate CMEK keys. The following restrictions apply:

        • A CMEK must reside in the same location as its associated backup.

        • All backups associated with an instance can only use one key at any given point in time.

        • To create a backup for an instance in a new location, or to create a backup using a new CMEK, the entire existing backup chain must first be deleted.

      • If electing to use CMEK and you want to delete an Enterprise or High Scale tier backup, the CMEK must first be enabled.

      • Once a RestoreInstance operation is applied to an Enterprise tier instance, users won't be able to create snapshots with the same names as previous snapshots prior to the operation.

      • After an instance is deleted, its backup cannot be deleted. See Deletion requests for Enterprise and High Scale tier backups for instruction on how to submit a deletion request.

      • Attempts to restore an instance from a backup while either a backup deletion or snapshot deletion are in progress fail.

    • With regard to capacity, the following limitations apply:

      • Each backup occupies instance capacity. This capacity varies relative to the scope of changes made to the data since the last backup was created.

        • More specifically, when a backup is created, Filestore creates an internal snapshot of the filesystem which also occupies a portion of available instance capacity.

          Snapshot size is also relative to the scope of changes made to data within the share since the last backup was created. This snapshot continues to exist until the next subsequent backup is created and uploaded.

          All data referenced by the backup persists in the state as it was when captured and continues to take up capacity from the filesystem. So for example, if you were to delete data from the mounted filesystem, that action itself would not free up capacity. Instead, to do so, you would create a new backup after deleting or overwriting significant amounts of data.

        • To anticipate sufficient capacity for their workloads, users should consider applying one of the following:

          • Increase instance capacity for workloads with significant, frequent data changes or a "high change rate".

          • Limit the scope or change rate of data modifications.

Best practices

The following sections cover recommended best practices.

Preparing your file share for the best backup consistency

The quality of a backup depends on the ability of your application to recover from backups that are created during heavy write workloads. In most situations, you can create backups that have good consistency even while your applications write data to the file share. However, if your applications require strict consistency, we recommend doing one or more of the following:

  • Use sync mount. For more information, see "The sync mount option" section in nfs(5). Alternatively, you can open files with the O_DIRECT|O_SYNC flags. For more information, see open(2).
  • Pause applications or operating system processes that write data to the file share and cause them to flush their changes to the file share before initiating the backup. For more information, see fsync(2).
  • If your applications require consistency between multiple shares, pause all applications on all instances that are writing to all file shares and create backups of all file shares before resuming your applications.
  • If you require application level consistency, stop your applications and unmount the file share before creating a backup.

Using existing backups as a baseline for new backups to reduce backup creation time

Existing backups of a file share within a region are used as baselines for creating new backups of the file share, reducing backup creation time. Therefore, we recommend that you do the following:

  • Take a new backup of a file share before you delete the previous backup of that file share.

  • Wait for new backups to be in the Ready state before creating subsequent backups of the same file share.

Scheduling backups during off-peak hours to reduce backup creation time

Creating backups during off-peak hours reduces the time that it takes to create a backup. If you schedule regular backups of your file shares, we recommend scheduling them during off-peak hours when possible.

Peak hours for backups creation are the end of each business day and midnight in the region where the Filestore instance is located. We recommend creating your backups either in the early morning or during the business day.

Organizing your data on separate Filestore instances to maximize efficiency

The more data on the file share, the larger the backup and the more it costs. To back up only the data that you need to back up, we recommend organizing your data on separate file shares, namely:

  • Storing critical data with different write patterns or with different backup requirements on different file shares.
  • Reducing the number of backups that you need to create by keeping similar data in one file share.

Request Preview access

Filestore Backups for High Scale and Enterprise tier instances are available as a Preview feature for select customers. To request feature access, fill out and submit this form.

Get started with Filestore backups

To get started using the feature, see Backup data for disaster recovery.

What's next