Back up a Dataproc Metastore service

Stay organized with collections Save and categorize content based on your preferences.

This page explains how to create a backup of a Dataproc Metastore service.

A backup takes a snapshot of your service saves its current configuration settings and all stored metadata.

After you create a backup, you can use the Restore from a backup feature to populate a new Dataproc Metastore service with the data saved in the snapshot.

Before you begin

Required roles

To get the permission that you need to back up a Dataproc Metastore service, ask your administrator to grant you the following IAM roles on your project, based on the principle of least privilege:

  • Grant full control of Dataproc Metastore resources (roles/metastore.editor)
  • Grant full access to all Dataproc Metastore resources, including IAM policy administration (roles/metastore.admin)
  • Grant read and modify access to specific metadata of databases and tables (roles/metastore.metadataOperator)

For more information about granting roles, see Manage access.

This predefined role contains the metastore.backups.create permission, which is required to back up a Dataproc Metastore service. You might also be able to get this permission with custom roles or other predefined roles.

For more information about specific Dataproc Metastore roles and permissions, see Dataproc Metastore IAM overview.

Back up a service

Create a backup

Before running a backup operation, note the following considerations:

  • For each Dataproc Metastore service, you can create and store up to seven backups at a time. If you try to exceed seven backups, the backup process fails. If you want to create another backup, you must first manually delete one of your stored backup files.
  • While a backup operation is running, you can't update your Dataproc Metastore service — for example changing configuration settings. However, you can still use it for normal operations, such as using it to access its metadata from attached Dataproc or self-managed clusters.

To back up a Dataproc Metastore service, complete the following steps:

Console

  1. In the Google Cloud console, open the Dataproc Metastore page:

    Open Dataproc Metastore

  2. On the Dataproc Metastore page, click the name of the service you want to back up.

    The Service detail page opens.

    Service detail page

  3. At the top of the page, click Backup.

    The Backup page opens.

  4. Enter the Backup name.

  5. Optional: Enter a Description of the backup.

  6. To start the backup operation, click Backup.

    Return to the Dataproc Metastore page, and verify that your service was successfully backed up.

    When the backup completes, Dataproc Metastore automatically returns to the active state regardless of whether or not the backup succeeded.

gcloud CLI

  1. To back up a Dataproc Metastore service, run the following gcloud metastore services backups create command:

    gcloud metastore services backups create BACKUP \
    --location=LOCATION \
    --service=SERVICE \
    --description=DESCRIPTION

    Replace the following:

    • BACKUP: the ID or fully qualified identifier for the backup.
    • LOCATION: the Google Cloud region in which your Dataproc Metastore service resides.
    • SERVICE: the name of your Dataproc Metastore service.
    • DESCRIPTION: a description of your backup.

  2. Verify that your service was successfully backed up.

    When the backup completes, Dataproc Metastore automatically returns to the active state regardless of whether or not the backup succeeded.

REST

Follow the API instructions to back up metadata from a service by using the APIs Explorer.

When the backup completes, Dataproc Metastore automatically returns to the active state regardless of whether or not the backup succeeded.

View backup history

To view the backup history of a Dataproc Metastore service in the Google Cloud console, complete the following steps:

  1. In the Google Cloud console console, open the Dataproc Metastore page.

  2. In the navigation bar, click Backup/Restore.

    Your backup history appear in a table under Backups.

    The history displays up to the latest 7 backups.

    Deleting a Dataproc Metastore service also deletes all associated backup history.

Delete a backup

To delete a Dataproc Metastore backup in the Google Cloud console, complete the following steps:

  1. In the Google Cloud console, open the Dataproc Metastore page.
  2. In the navigation bar, click Backup/Restore.
  3. Find the backup you want to delete and click the settings button.
  4. Click Delete.

Troubleshoot common issues

What's next