Backing up and Restoring Databases using gcloud

This page describes how to use Cloud Spanner Backup and Restore with the Google Cloud CLI.

Prerequisites

The examples in this page make the following assumptions:

  • You have already set up gcloud CLI for use with Cloud Spanner. If you are new to using gcloud with Cloud Spanner, see Getting started with Cloud Spanner using gcloud.

  • You have configured gcloud with a default project. For example:

    gcloud config set core/project my-project

  • You have an instance named test-instance and a database named example-db.

Create a backup

To create a backup named example-db-backup-6 and have it expire in 1 year:

gcloud spanner backups create example-db-backup-6 --instance=test-instance \
    --database=example-db --retention-period=1y --async

Here are some usage notes:

  • To set the backup's expiration date, specify either the --retention-period or --expiration-date flag. For information on date syntax , see gcloud topic datetimes.
  • The backup's name must be unique in the instance.
  • The command returns immediately because of the --async flag. Without the flag, the command will wait for the backup operation to complete.
  • Specify the --help flag to get help for any command.

Here's what it outputs:

Create request issued for: [example-db-backup-6]
Check operation [projects/my-project/instances/test-instance/backups/example-db-backup-6/operations/_auto_op_234567] for status.

In the next section, we'll use the backup operation to check in on the backup's progress.

For more details about this command, see gcloud spanner backups create.

Show backup progress

  1. To track the progress of a backup operation, use gcloud spanner operations describe:

    gcloud spanner operations describe _auto_op_234567 \
    --instance=test-instance --backup=example-db-backup-6

    Here's the output:

    done: true
metadata:
  '@type': type.googleapis.com/google.spanner.admin.database.v1.CreateBackupMetadata
  database: projects/my-project/instances/test-instance/databases/example-db
  name: projects/my-project/instances/test-instance/backups/example-db-backup-6
  progress:
    endTime: '2020-02-04T02:07:59.089820Z'
    progressPercent: 100
    startTime: '2020-02-04T02:05:43.920377Z'
name: projects/my-project/instances/test-instance/backups/example-db-backup-6/operations/_auto_op_234567
response:
  '@type': type.googleapis.com/google.spanner.admin.database.v1.Backup
  createTime: '2020-02-04T02:05:43.920377Z'
  database: projects/my-project/instances/test-instance/databases/example-db
  expireTime: '2021-02-04T02:05:43.268327Z'
  name: projects/my-project/instances/test-instance/backups/example-db-backup-6
  sizeBytes: '1000000000'
  state: READY

  2. If you don't have the operation's name, find it by using gcloud spanner operations list:

    gcloud spanner operations list --instance=test-instance --type=BACKUP

    Usage notes:

    • To limit the list, specify the --filter flag.

      • For example, --filter="metadata.name:example-db" lists just the backup operations on a specific database.
      • For example, --filter="error:*" lists just the backup operations that failed.

      For information on filter syntax, see gcloud topic filters. For information on filtering backup operations, see the filter field in ListBackupOperationsRequest.

    • The --type flag is not case sensitive.

    Here's the output:

    OPERATION_ID               DONE  @TYPE                 BACKUP               SOURCE_DATABASE  START_TIME                   END_TIME
_auto_op_123456  True  CreateBackupMetadata  example-db-backup-7  example-db       2020-02-04T02:12:38.075515Z  2020-02-04T02:22:40.581170Z
_auto_op_234567  True  CreateBackupMetadata  example-db-backup-6  example-db       2020-02-04T02:05:43.920377Z  2020-02-04T02:07:59.089820Z

Restore a database from a backup

To restore a database:

gcloud spanner databases restore --async \
    --destination-instance=test-instance --destination-database=example-db-restored \
    --source-instance=test-instance --source-backup=example-db-backup-6

Usage notes:

  • The command returns immediately because of the --async flag. Without the flag, the command will wait for the restore operation to complete.
  • If the source and destination instances are different, they must have the same instance configuration.
  • If the destination database already exists, the operation will fail.

Here's the output:

Restore database in progress. Operation name=projects/my-project/instances/test-instance/databases/example-db-restored-1/operations/_auto_op_bb8e360b256b04bf

For more details about this command, see gcloud spanner databases restore.

Show restore and optimize progress

  1. To track the progress of a restore operation, use gcloud spanner operations describe:

    gcloud spanner operations describe  _auto_op_bb8e360b256b04bf \
    --instance=test-instance --database=example-db-restored

    Here's the output:

    done: true
metadata:
  '@type': type.googleapis.com/google.spanner.admin.database.v1.RestoreDatabaseMetadata
  backupInfo:
    backup: projects/my-project/instances/test-instance/backups/example-db-backup-6
    createTime: '2020-02-03T23:48:06.259296Z'
    sourceDatabase: projects/my-project/instances/test-instance/databases/example-db
  name: projects/my-project/instances/test-instance/databases/example-db-restored
  optimizeDatabaseOperationName: projects/my-project/instances/test-instance/databases/example-db-restored/operations/_auto_op_7702f3f19fa99616
  progress:
    endTime: '2020-02-04T02:40:56.424382Z'
    progressPercent: 100
    startTime: '2020-02-04T02:35:30.683330Z'
  sourceType: BACKUP
name: projects/my-project/instances/test-instance/databases/example-db-restored/operations/_auto_op_91ff1f270832a22b
response:
  '@type': type.googleapis.com/google.spanner.admin.database.v1.Database
  createTime: '2020-02-04T02:35:30.683330Z'
  name: projects/my-project/instances/test-instance/databases/example-db-restored
  restoreInfo:
    backupInfo:
      backup: projects/my-project/instances/test-instance/backups/example-db-backup-6
      createTime: '2020-02-03T23:48:06.259296Z'
      sourceDatabase: projects/my-project/instances/test-instance/databases/example-db
    sourceType: BACKUP
  state: READY_OPTIMIZING

    From the output, we can see that the restore operation is complete and is now in the READY_OPTIMIZING state as described in How restore works. We are also given optimizeDatabaseOperationName, which we can use to track the progress of the optimize operation.

  2. To track the progress of the optimize operation, use gcloud spanner operations describe:

    gcloud-testing spanner operations describe _auto_op_7702f3f19fa99616 \
    --instance=test-instance --database=example-db-restored

    Here's the output:

    done: true
metadata:
  '@type': type.googleapis.com/google.spanner.admin.database.v1.OptimizeRestoredDatabaseMetadata
  name: projects/my-project/instances/test-instance/databases/example-db-restored
  progress:
    endTime: '2020-02-04T22:11:21.731739Z'
    progressPercent: 100
    startTime: '2020-02-04T22:10:48.308090Z'
name: projects/my-project/instances/test-instance/databases/example-db-restored/operations/_auto_op_7702f3f19fa99616
response:
  '@type': type.googleapis.com/google.spanner.admin.database.v1.Database
  createTime: '2020-02-04T22:03:34.282186Z'
  name: projects/my-project/instances/test-instance/databases/example-db-restored
  restoreInfo:
    backupInfo:
      backup: projects/my-project/instances/test-instance/backups/example-db-backup-6
      createTime: '2020-02-03T23:48:06.259296Z'
      sourceDatabase: projects/my-project/instances/test-instance/databases/example-db
    sourceType: BACKUP
  state: READY

  3. If you don't have the name of the restore or optimize operation, find it by using gcloud spanner operations list. If the restored database is not in the same instance as the backup, make sure you specify the destination instance in the instance flag:

    gcloud spanner operations list --instance=test-instance \
    --type=DATABASE_RESTORE

    Here's the output:

    OPERATION_ID               DONE  @TYPE                             RESTORED_DATABASE    SOURCE_BACKUP        START_TIME                   END_TIME
_auto_op_7702f3f19fa99616  True  OptimizeRestoredDatabaseMetadata  example-db-restored                       2020-02-04T02:40:56.424382Z  2020-02-04T02:41:37.765157Z
_auto_op_91ff1f270832a22b  True  RestoreDatabaseMetadata           example-db-restored  example-db-backup-6  2020-02-04T02:35:30.683330Z  2020-02-04T02:40:56.424382Z

Get backup information

To get information about a backup:

gcloud spanner backups describe example-db-backup-6 --instance=test-instance

Here's the output:

createTime: '2020-02-04T02:05:43.920377Z'
database: projects/my-project/instances/test-instance/databases/example-db
expireTime: '2021-02-04T02:05:43.268327Z'
name: projects/my-project/instances/test-instance/backups/example-db-backup-6
sizeBytes: '1000000000'
state: READY

For more details about this command, see gcloud spanner backups describe.

List backups

To list all the backups in an instance:

gcloud spanner backups list --instance=test-instance

Here's the output:

BACKUP               SOURCE_DATABASE  CREATION_TIME                EXPIRATION_TIME              STATE  BACKUP_SIZE_IN_BYTES  IN_USE_BY
example-db-backup-6  example-db       2020-02-04T02:05:43.920377Z  2021-02-04T02:05:43.268327Z  CREATING
example-db-backup-4  example-db       2020-02-04T01:21:20.873839Z  2021-02-04T01:21:20.530151Z  READY  32
example-db-backup-3  example-db       2020-02-03T23:59:18.936433Z  2021-02-03T23:59:18.203083Z  READY  32
example-db-backup-5  example-db       2020-02-03T23:48:06.259296Z  2021-02-03T23:48:05.830937Z  READY  32
example-db-backup-2  example-db       2020-01-30T19:49:00.616338Z  2021-01-30T19:49:00.283917Z  READY  32
example-db-backup-1  example-db       2020-01-30T19:47:09.492551Z  2021-01-30T19:47:09.097804Z  READY  32

To limit the list, specify the --filter flag. For example --filter="state:creating" lists just the backups that are still being created. For information on filter syntax, see gcloud topic filters. For information on filtering backups, see the filter field in ListBackupsRequest.

For more details about this command, see gcloud spanner backups list.

Update backup metadata

To update a backup's expiration date:

gcloud spanner backups update-metadata example-db-backup-6 \
    --instance=test-instance --expiration-date=2020-05-05T00:00:00Z

Here's the output:

createTime: '2020-02-04T02:05:43.920377Z'
database: projects/my-project/instances/test-instance/databases/example-db
expireTime: '2020-05-05T00:00:00Z'
name: projects/my-project/instances/test-instance/backups/example-db-backup-6
sizeBytes: '1000000000'
state: READY

For more details about this command, see gcloud spanner backups update-metadata.

Delete backup

To delete a backup:

gcloud spanner backups delete example-db-backup-1 --instance=test-instance

You must confirm the operation:

You are about to delete backup [example-db-backup-1]

Do you want to continue (Y/n)?  Y

Deleted backup [example-db-backup-1].

For more details about this command, see gcloud spanner backups delete.