Use soft-deleted objects

Overview Setup

This page describes how to list, access, and restore soft-deleted objects.

Before you begin

To complete the tasks on this page, make sure that you have the Storage Admin (roles/storage.admin) IAM role on the bucket or the project that contains the bucket. This predefined role includes the permissions required to manage and restore soft delete on a bucket.

To see the exact permissions that are required, expand the Required permissions section:

Required permissions

  • storage.buckets.get
  • storage.buckets.update
  • storage.buckets.list
    • This permission is required if you plan to use Google Cloud console to perform the instructions on this page.
  • storage.objects.restore
  • storage.objects.create
  • storage.objects.delete
    • This permission is required if restoring an object causes an existing object to be overwritten.
  • storage.buckets.restore
    • This permission is required if you want to bulk restore the soft-deleted objects in a bucket.
  • storage.bucketOperations.*
    • This permission is required for long-running operations.

For information about granting roles on buckets, see Use IAM with buckets. For information about granting roles on projects, see Manage access to projects.

List all of the soft-deleted objects in a bucket

You can list all of the soft-deleted objects in a bucket.

Console

  1. In the Google Cloud console, go to the Cloud Storage Buckets page.

    Go to Buckets

  2. In the list of buckets, click the name of the bucket you want to view the soft-deleted objects of. The Bucket details page opens with the Objects tab selected.

  3. In the Show list, select Soft-deleted objects only. A list of soft-deleted objects is displayed.

Command line

To display a list of all the soft-deleted objects in a bucket, use the gcloud storage ls command with the --soft-deleted and --recursive flags:

  gcloud storage ls gs://BUCKET_NAME --soft-deleted --recursive

Replace the following:

  • BUCKET_NAME: the name of the bucket. For example, my-bucket.

REST APIs

JSON API

  1. Have gcloud CLI installed and initialized, in order to generate an access token for the Authorization header.

    Alternatively, you can create an access token using the OAuth 2.0 Playground and include it in the Authorization header.

  2. Use cURL to call the JSON API with a request to list objects that uses the ?softDeleted query parameter:

    curl -X GET \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  "https://storage.googleapis.com/storage/v1/b/BUCKET_NAME/o?softDeleted=true"

    Where BUCKET_NAME is the name of the relevant bucket. For example, my-bucket.

View all of the soft-deleted versions of an object

You can view all the soft-deleted versions of a specific object in your bucket.

Console

  1. In the Google Cloud console, go to the Cloud Storage Buckets page.

    Go to Buckets

  2. In the list of buckets, click the name of the bucket that contains the object you want to view soft-deleted versions of. The Bucket details page opens with the Objects tab selected.

  3. Click the object that you want to view all the soft-deleted versions of. The Object details page displays.

  4. Click the Version History tab.

  5. In the Show list, select Soft-deleted objects only. A list of all the soft-deleted versions for that object displays.

Command line

To display a list of all the soft-deleted versions of an object in a bucket, use the gcloud storage ls command:

  gcloud storage ls gs://BUCKET_NAME/OBJECT_NAME --soft-deleted

Replace the following:

  • BUCKET_NAME: the name of the bucket. For example, my-bucket.
  • OBJECT_NAME: the name of the object you want to view all the versions of.

REST APIs

JSON API

  1. Have gcloud CLI installed and initialized, in order to generate an access token for the Authorization header.

    Alternatively, you can create an access token using the OAuth 2.0 Playground and include it in the Authorization header.

  2. Use cURL to call the JSON API with a request to list objects that uses the ?softDeleted query parameter:

    curl -X GET \
  -H "Authorization: Bearer OAUTH2_TOKEN" \
  -H "Content-Type: application/json" \
  "https://storage.googleapis.com/storage/v1/b/BUCKET_NAME/o?softDeleted=true"

    Where BUCKET_NAME is the name of the relevant bucket. For example, my-bucket.

Restore a soft-deleted object

You can restore a specific version of a soft-deleted object in your bucket.

Console

  1. In the Google Cloud console, go to the Cloud Storage Buckets page.

    Go to Buckets

  2. In the list of buckets, click the name of the bucket that contains the soft-deleted objects that you want to restore. The Bucket details page opens with the Objects tab selected.

  3. In the Show list, select Soft-deleted objects only.

  4. Navigate to the object, which may be located in a folder.

  5. Click the soft-deleted object. The Object details page displays.

  6. In the Show list, select Soft-deleted objects only. A list of all the soft-deleted versions of that object displays.

  7. To restore a specific soft-deleted version of an object, click Restore next to that object version. The restore object version pane opens.

  8. Click Confirm.

Command line

To restore a soft-deleted version of an object, use the gcloud storage restore command:

  gcloud storage restore gs://BUCKET_NAME/OBJECT_NAME#GENERATION_NUMBER

Replace the following:

  • BUCKET_NAME: the name of the bucket. For example, my-bucket.
  • OBJECT_NAME: the name of the object for which you want to view all the versions.
  • GENERATION_NUMBER: the generation number of the soft-deleted object you want to restore. If you don't specify the generation number, the latest version is restored. For example, 1560468815691234.

For additional options for restoring multiple objects in bulk, including restoring all versions of an object with the all-versions flag, see the gcloud storage restore reference.

REST APIs

JSON API

  1. Have gcloud CLI installed and initialized, in order to generate an access token for the Authorization header.

    Alternatively, you can create an access token using the OAuth 2.0 Playground and include it in the Authorization header.

  2. Use cURL to call the JSON API with a POST Object request:

    curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  "https://storage.googleapis.com/storage/v1/b/BUCKET_NAME/o/OBJECT_NAME/restore?generation=GENERATION_NUMBER"

    Where:

    • BUCKET_NAME is the name of the relevant bucket. For example, my-bucket.
    • OBJECT_NAME is the name of the soft-deleted object you want to restore. For example, cat.jpeg.
    • GENERATION_NUMBER is the generation number of the soft-deleted object you want to restore. If you don't specify the generation number, the latest version is restored. For example, 1560468815691234.

Bulk restore soft-deleted objects in a bucket

If you want to restore a large number of objects, or don't know the specific objects that you want to restore, use a bulk restore operation. Note that bulk restore operations occur asynchronously and can, in some cases, take an hour or more to begin.

Bulk restore operation initiates a long-running operation on a bucket. You can use the Google Cloud CLI to get and list long-running operations so you can view the details and status of the bulk restore operation while it's in progress.

Console

  1. In the Google Cloud console, go to the Cloud Storage Buckets page.

    Go to Buckets

  2. In the list of buckets, find the bucket that contains the objects you want to restore.

  3. Click the Bucket overflow menu () associated with the bucket and select Create restore job.

    The Create a restore job pane opens.

  4. In the Choose which soft-deleted objects to restore section, select the deletion period from which you want to restore objects.

  5. (Optional) Select Filter by glob pattern and enter a glob pattern to restore only objects that meet the glob pattern criteria.

  6. (Optional) Toggle additional restore behaviors in the Restore options section.

  7. Click Create.

You can track the progress of your bulk restore operation by clicking on the Notifications button () found in the Google Cloud console header.

Command line

To use the bulk restore operation to restore all objects in a bucket, use the gcloud storage restore command.

The following command performs bulk restore operation to asynchronously restore all the objects of the bucket that were deleted during a specific time period:

  gcloud storage restore gs://BUCKET_NAME/** \
      --async --deleted-after-time=DELETED_AFTER_TIME \
      --deleted-before-time=DELETED_BEFORE_TIME

Replace the following:

  • BUCKET_NAME: the name of the bucket. For example, my-bucket.
  • DELETED_AFTER_TIME: an optional argument to specify the date after which the objects are restored. For example, 2022-12-01.

  • DELETED_BEFORE_TIME: an optional argument to specify the date before which the objects are restored. For example, 2022-12-21.

    If successful, the command returns the ID of the long-running operation associated with the bulk restore operation. You can get details about the long-running operation or cancel the long-running operation to stop the bulk restore operation before it completes.

REST APIs

JSON API

  1. Have gcloud CLI installed and initialized, in order to generate an access token for the Authorization header.

    Alternatively, you can create an access token using the OAuth 2.0 Playground and include it in the Authorization header.

  2. Use cURL to call the JSON API with a POST Object request:

    curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  "https://storage.googleapis.com/storage/v1/b/BUCKET_NAME/o/bulkRestore"

    Where BUCKET_NAME is the name of the relevant bucket. For example, my-bucket.

If successful, the command returns the ID of the long-running operation associated with the bulk restore operation. You can get details about the long-running operation or cancel the long-running operation to stop the bulk restore operation before it completes.

Manage long-running operations for bulk restore

Long-running operations in the Google Cloud CLI are identified by an operation name, which is a combination of the bucket name and the operation ID.

The following example shows a long-running operation that's returned from a bulk restore operation:

done: false
{
  "kind": "storage@operation",
  "name": "projects/_/buckets/my-bucket/operations/CiQyODRlMTY0My0wNGExLTQ3MWItOTE2Zi1hZmQ0ZmM4ZDc4MjQQAg",
  "metadata": {
    "@type": "type.googleapis.com/google.storage.v2.BulkRestoreObjectsMetadata",
    "commonMetadata": {
      "type": "bulk-restore-objects",
      "createTime": "2023-10-20T21:08:11.289Z",
      "updateTime": "2023-10-20T21:18:37.583Z",
      "endTime": "2023-10-20T21:18:37.583Z",
      "requestedCancellation": false,
      "progressPercent": -1,
    },
    "allow_overwrite": false,
    "matchGlobs": ["*"],
    "succeededCount": "0",
    "failedCount": "0",
    "skippedCount": "0",
  },
  "done": true,
  "response": {
    "@type": "type.googleapis.com/google.storage.v2.BulkRestoreObjectsResponse"
  }
}

For general information on how to manage long-running operations, see the gcloud storage operations documentation.

Get the details of a long-running operation

Use the gcloud storage operations describe command:

gcloud storage operations describe projects/_/buckets/BUCKET_NAME/operations/OPERATION_ID

Replace the following:

  • BUCKET_NAME: the name of the bucket that contains the long running operation. For example, my-bucket.

  • OPERATION_ID: the ID of the long-running operation returned in the response of bulk restore operation.

List the long-running operations in a bucket

Use the gcloud storage operations list command:

gcloud storage operations list gs://BUCKET_NAME

Replace the following:

  • BUCKET_NAME: the name of the bucket that contains the long-running operations. For example, my-bucket.

Cancel a long-running operation

Use the gcloud storage operations cancel command:

gcloud storage operations cancel projects/_/buckets/BUCKET_NAME/operations/OPERATION_ID

Replace the following:

  • BUCKET_NAME: the name of the bucket that contains the long-running operation. For example, my-bucket.

  • OPERATION_ID: the ID of the long-running operation returned in the response of bulk restore operation.

Error handling

Long-running operations are returned from asynchronous APIs, which require different error-handling practices than synchronous APIs. Unlike synchronous APIs, APIs that return long-running operations succeed even if the underlying operation fails. When using APIs that return long-running operations, you respond to error codes returned in successful responses, instead of responding to error codes returned in failed responses.

For example, if you make a soft delete bulk restore request, it returns a successful HTTP status code (200 OK) even if an error occurs over the course of the operation. To check whether the bulk restore operation was successful, query the status of the long-running operation.

Note that the long-running operation APIs (Get, Create, Cancel) are synchronous and return normal errors.

Retries

Asynchronous soft delete APIs are idempotent by default. Therefore, you can safely retry asynchronous soft delete API calls without concern about modifying resources unintentionally.

What's next