Backups overview

Spanner backup lets you create backups of Spanner databases on demand, and restore them to provide protection against operator and application errors which result in logical data corruption. Backups are highly available, encrypted, and can be retained for up to a year from the time they are created. When you create a backup, the backup resides in the same instance, region, and project as its source database. If you need to restore the backup in a different region or project for compliance or business continuity reasons, you can copy the backup to an instance in a separate region or project. To retain backups longer than a year, we recommend exporting your database. For protection against logical data corruption, Spanner also offers point-in-time recovery. You can also enable database deletion protection to prevent the accidental deletion of databases.

Key features

  • Data consistency: Backups are a transactionally and externally consistent copy of a Spanner database at the version_time of the backup.

  • Replication: Backups reside in the same instance as their source database and are replicated in the same geographic locations. For regional instances, the backup is stored in each of the three read-write zones. For multi-regional instances, the backup is stored in all zones that contain either a read-write or read-only replica. If you need to store the backup of your database in a different region or project, you can copy the completed backup from the source instance to a destination instance located in a different region or project. For more information, see copy a backup.

  • Automatic expiration: All backups have a user-specified expiration date which determines when it will be automatically deleted. Spanner deletes expired backups asynchronously, so there can be a lag between when a backup is expired and when it's actually deleted.

The following table describes several backup strategies, the recommended approach to implement the plan, and the maximum retention time for the approach suggested.

Backup strategy Recommended approach Maximum retention time for suggested approach
Storing the backup of a database in the same instance, region, and project as its source database Create a backup. 1 year
Storing the backup of a database in a different instance, region, or project from its source database (i.e., a cross-region or cross-project backup) Create a backup, then copy it to an instance in a different region or project. 1 year
Storing the backup in Cloud Storage Export the database to a Cloud Storage bucket. For a detailed comparison between backup and export, see Choose between back up and restore or import and export. Unlimited (retained until deletion)
Point-in-time recovery (PITR) To recover data from a point-in-time in the past, choose PITR. You can change the database version_retention_period from the default 1 hour to a maximum of 7 days. 7 days

Access control with Identity and Access Management (IAM)

IAM lets you control access to Spanner resources, which include backups. If you are new to IAM, roles, and permissions, see IAM Overview for an introduction.

Backup resources are organized under instances in the Spanner resource hierarchy. We recommend applying IAM policies at the project level or instance level. If you need finer grain control, IAM policies can also be applied at the backup and database level as well, but this is not recommended due to complexity. Remember that backups do not contain database metadata such as IAM policies, so when you restore a database, the database will initially inherit policies from its parent instance.

This section describes the predefined roles that have access to backup and restore.

The following roles are designed specifically for backup:

  • spanner.backupAdmin: has access to create, view, update, copy, and delete backups. This role can also view and manage a backup's IAM policy. This role cannot restore a database from a backup.
  • spanner.backupWriter: has access to create and copy backups, but cannot update, or delete them. This role is intended to be used by scripts that automate backup creation.

The following roles also have access to Spanner backups:

  • spanner.admin: has full access to backup. This role has complete access to all Spanner resources.
  • owner: has full access to backup.
  • editor: has full access to backup.
  • viewer: has access to view backups and backup operations. This role can't create, update, delete, or copy a backup.

For more information, see Spanner IAM.

How backup creation works

You can create a backup of any Spanner database. These backups are complete, in the sense that they contain all of the data in the database (including the schema and secondary indexes) at the version_time of the backup. Any modifications to the data or schema after the version_time are not included in the backup. Backups include all database options that are set with the ALTER DATABASE SET OPTIONS command, but do not include Identity and Access Management (IAM) policies. When you create a backup, the backup resides in the same instance, region, and project as its source database.

You can create a backup in the following ways:

When you create a backup, you must specify a source database, a name for the backup resource, and an expiration date (up to 1 year from backup creation time). You can also optionally specify a version_time, which lets you back up your database to an earlier point in time. The version_time field is typically used to either synchronize the backups of multiple databases or recover data using point-in-time recovery. If version_time is not specified, then it is set to the create_time of the backup. The system creates a backup resource and a long-running backup operation to track the progress of the backup. The newly created backup resides in the same instance, region, and project as its source database.

To help ensure external consistency of the backup, Spanner pins the contents of the database at create_time. This prevents the garbage collection system from removing the relevant data values for the duration of the backup operation. Then, every read-write and read-only zone in the instance begins copying the data in parallel. If any zone is temporarily unavailable, the backup is not complete until the zone comes back online and finishes. Backups are restorable as soon as the operation is complete. For multi-region instances, all read-write and read-only zones in all regions must complete their backup replicas before the backup is marked as restorable.

Backups also include the schema of a database's change streams, but not any existing change records. Change stream data is meant to be streamed out and consumed near-simultaneously with the changes it describes. As such, Spanner excludes this data from backups.

How backup copying works

Spanner back up and restore lets you copy a backup of your Spanner database from one instance to another instance in a different region or project, to provide additional data protection and compliance capabilities. The copied backup has the same key features as the original backup. Additionally, you can restore a copied backup in the same instance as the copied backup to support cross-region and cross-project backup and restore use cases.

Common cross-region use cases

Some common cross-region use cases for copying a backup include the following:

  • Maintain a backup in another region for compliance and regulatory requirements.

    For example, you can copy a backup of your database to an instance in a region that is at a minimum distance away from your production data to meet compliance requirements.

  • Maintain a backup in a separate region for disaster recovery and business continuity purposes.

    For example, you can copy a backup database to a destination instance for disaster recovery purposes with non-zero Recovery Time Objective (RTO) and Recovery Point Objective (RPO). Then, when needed, you can restore the database from the copied backup in the destination instance. (If your application has zero-RTO and zero-RPO requirements, we recommend Spanner multi-region configurations for your disaster recovery plans.)

Common cross-project use cases

Some common cross-project use cases for copying a backup include the following:

  • Maintain a backup copy in a separate project to meet operational, security, or compliance requirements.

  • Copy and move data among development, test, and production projects.

    For example, if you want to move data from your production project to a test project, you can create a backup of your production data, then copy your backup to the test project. Once the copy operation is complete, you can restore the copied backup to an instance in the test project.

  • Move your database from one project to another (Note that there might be some downtime during the migration).

You can copy a backup to a destination instance in a different region or project by specifying a source backup, a destination backup, and an expiration date up to one year from the source backup creation time. This means that the value for expiration_date must be at least six hours from the time the current copy request is processed and at most 366 days after the source backup create_time.

At the beginning of the copy backup request, Spanner creates a backup resource and a long-running backup operation to help track the progress of the backup. The backup is copied to every read-write and read-only zone in the destination instance. If a zone is temporarily unavailable, the backup copy does not complete until the zone comes back online. You are not able to delete the destination instance during the copy. To track the progress and completion status of the copy backup operation, follow the steps in Show backup progress. After the copy is complete, you can delete the source backup if you don't need it anymore. Once the copying completes, you can use operations such as GetBackup, UpdateBackup, and DeleteBackup with the copied backup.

Prerequisites for initiating copy of a backup

If you are copying a backup to an instance in a different region or project, you need to set up and configure the destination instance first. The destination instance is the instance where the copy of your backup resides. It can be as small as 100 processing units, and it does not need to have the same instance configuration as the source instance (the instance where your source backup lives). Before restoring, make sure that the destination instance has enough nodes or processing units provisioned to support the database size according to the 4TB per node storage limit (e.g. You need at least 2 nodes to restore a 8TB backup). To create a new destination instance, see Create and manage instances.

Additional considerations

Additional considerations include the following:

  • When you copy a backup from a source instance to a destination instance, the copied backup exists independently from its source backup. Once the copy operation is complete, there is one backup in the source instance and one backup in the destination instance. If you don't need the backup in the source instance anymore, you can delete it.
  • When you copy a backup to a regional instance, backup data is copied to each of the three read-write zones in the destination instance.
  • When you copy a backup to a multi-regional instance, the backup data is copied to each zone in the instance that contains either a read-write or read-only replica.
  • You can copy multiple backups at the same time.
  • You can update or delete the destination backup while a copying process is still in progress. If you delete the destination backup, the ongoing copy operation will cancel as a result.
  • You can restore a backup in the source instance while there is a copying operation in progress.
  • You can cancel a copy operation before it completes.

The following operations are not allowed during the copying process:

  • You cannot delete the source backup while a copy operation is in progress.
  • You cannot initiate a new copy or restore on the destination copied backup while the copy is still in progress. Once a copy has been completed, it can be copied again or restored.

Where Spanner backups are stored

Backups are resources in Spanner. Each backup resource is organized under the same instance as its source database in the resource hierarchy and has a resource path in the form projects/<project>/instances/<instance>/backups/<backup>. A backup continues to exist even after its source database has been deleted, but it cannot outlive its parent instance. To prevent accidental deletion of backups, you cannot delete a Spanner instance if there are backups. If you want to delete the instance, we recommend restoring the backup, then exporting the restored database, before deleting the backup and the instance.

Encryption

Spanner backups, like databases, are encrypted by either Google-managed or customer-managed encryption. By default, a backup uses the same encryption configuration as its database, but you can override this behavior by specifying a different encryption configuration when creating the backup. If the backup is CMEK-enabled, it is encrypted using the primary version of the KMS key at the time of backup creation. Once the backup is created, its key and key version cannot be modified, even if the KMS key is rotated. For more information, see create a CMEK-enabled backup.

By default, a copied backup uses the same encryption configuration, either Google-managed or customer-managed (CMEK), as its source backup encryption. You can override this behavior by specifying a different encryption configuration when copying the backup. If you want the copied backup to be encrypted with CMEK when copying across regions, specify the KMS key corresponding to the destination region.

Performance

This section describes optimal backup performance in Spanner.

Performance when backing up

When performing a backup, Spanner creates a backup job to copy data directly from the database to backup storage, and sizes this job based on the size of the database. This backup job does not use CPU resources allocated to the database's instance so it does not affect the instance's performance. Moreover, compute load on the database's instance does not affect the speed of the backup operation. To track the progress and completion of a backup operation, see Show backup progress.

Generally, most backups take 1-4 hours. Some backups might take longer due to their size or because there is an internal queueing for resources. If a backup is taking longer than usual when no other factors have changed, it might be due to a delay in scheduling the backup task in a zone. This can sometimes take up to 30 minutes. We recommend that you do not cancel and restart the backup, as it's likely you'll encounter the same scheduling delay with the new backup operation.

Performance when copying a backup

The time it takes to copy a backup depends on factors such as the size of the source backup and the destination region chosen for the copied backup. Generally, most copies complete in 1-4 hours. Some copies might take longer depending on the backup size and destination region. Copying a backup does not have any performance implications on the source instance or database. You can make multiple concurrent copies of the source backup to instances in different regions without any performance implications issues.

Pricing

You are billed based on the amount of storage used by your backups per unit time. Billing begins once the backup operation is complete and will continue until the backup has been deleted. A completed backup is billed for a minimum of 24 hours. If you create a backup, then delete it a minute after it finishes, you are still billed for 24 hours.

A copy of a backup is subject to the same storage costs as an original backup. If you create a copy between two instances that occupy different regions, then outbound data transfer costs apply.

For example, if you copy your database from the source multi-region instance configuration nam7 to the destination multi-region instance configuration nam-eur-asia3, the following charges apply:

  • No charge for overlapping us-central1 region
  • No charge for witness us-central2 region
  • Inter-continental data transfer charge apply twice: once for each new continent (Europe and Asia)
  • Data transfer between regions within the same continent charge apply once for us-east1
  • Data transfer between regions within the same continent charge apply once in Europe

Spanner optimizes the copying process to minimize the number of cross-region transfers. This helps to minimize the data transfer costs while providing a fast copy backup experience.

Backups are stored and billed separately. Backup storage does not affect billing for database storage or database storage limits. For more information, also see Storage utilization metrics.

For more complete information on backup costs, see Spanner pricing.

What's next