Backing up metadata from a service

A backup takes a snapshot of both the metadata stored in the Dataproc Metastore service and the service configuration. You can recover it in a service by restoring.

This page explains how to create a backup of an existing Dataproc Metastore service using the Google Cloud Console opened in a local browser or by using the Dataproc Metastore API method backups.create.

Access control

  • To backup metadata, you must be granted an IAM role containing the metastore.backups.create IAM permission. The Dataproc Metastore specific roles roles/metastore.admin, roles/metastore.editor, and roles/metastore.metadataOperator can be used to grant backup permission.

  • You can also give backup permission to users or groups by using the roles/owner and roles/editor legacy roles.

For more information, see Dataproc Metastore IAM and access control.

Backing up metadata from a service

You can backup metadata from a service by starting a backup on the Service detail page opened in a local browser or by issuing a Dataproc Metastore API method backups.create.

A backup takes a snapshot of your Dataproc Metastore service's stored metadata and configuration. The backups are fully managed by Dataproc Metastore.

You can create and store up to 7 backups at a time for each Dataproc Metastore service.

While a backup is running, no updates can be made to the service. You can still use the service while it's undergoing a backup.

You can backup metadata from a Dataproc Metastore service using the Google Cloud Console or the Dataproc Metastore API:

Console

  1. In the Cloud Console, open the Dataproc Metastore page:

    Open Dataproc Metastore in the Cloud Console

  2. On the Dataproc Metastore page, click the name of the service you'd like to backup metadata from. The Service detail page opens.

    Service detail page

  3. At the top of the page, click Backup. The Backup page opens.

  4. Enter the Backup name.

  5. Optional: Enter a Description of the backup.

  6. Click Backup to start the backup.

  7. Verify that you have returned to the Service detail page, and that your backup appears under Backups on the Backup/Restore tab.

REST

Follow the API instructions to backup metadata from a service by using the APIs Explorer.

When the backup is done, the service automatically enters active state regardless of whether or not it succeeded.

To view a service's backups, refer to the Backup/Restore tab on the Service detail page in the Cloud Console.

Backup caveats

  • A list of backups is available on the UI. Deleting the service itself will delete all backups under that service.

  • If there are already 7 backups in a service, then you must first manually delete a backup before creating a new one. You can delete existing backups from the Backup/Restore tab.

Common failures

  • The user creating the backup doesn't have metastore.backups.create permission.

What's next?