Soft Delete

Setup Usage

This page describes the soft delete feature, which preserves objects that get deleted or overwritten and the buckets that contain them for a specified period of time. Soft delete helps protect your data from accidental or malicious deletion by retaining deleted objects in a soft-deleted state, during which the object cannot be permanently deleted. Soft delete is enabled by default on all buckets and has a retention duration of seven days unless you or your organization have chosen a different policy.

Overview

When you enable soft delete on a bucket, objects that get deleted from the bucket enter a soft-deleted state instead of being permanently deleted. Soft-deleted objects behave in the following ways:

To enable soft delete on a bucket, you create a soft delete policy that specifies a retention duration controlling how long soft-deleted objects are retained before getting permanently deleted.

To learn how to create and manage a soft delete policy to enable or disable soft delete on a bucket, see Use soft delete.

Soft delete policies

A soft delete policy can be created, deleted, or modified during a bucket creation or update. Once a soft delete policy is created, you can use it to enable or disable soft delete for the bucket.

Soft delete policies behave in the following ways:

  • Updating a bucket's soft delete policy only applies to objects that you delete after the soft delete policy takes effect. Objects that you deleted prior to the update are retained for the duration that was in effect when they were deleted.

    For example, say you enable a soft delete policy in your bucket with the default retention duration of seven days, and then you delete the object cat.png. In this scenario, cat.png is retained as a soft-deleted object for the next seven days, and then it's permanently deleted. This occurs even if you later change or remove the soft delete policy for your bucket.

  • If you add a new soft delete retention duration to an existing bucket, it won't apply to any of the objects deleted before the soft delete policy takes effect.

  • If you delete your project, you cannot use soft delete to restore the buckets or objects in it, even if they had soft delete enabled. To ensure you don't lose your data in case of malicious or accidental deletion, we recommend limiting access to project-level deletion by placing a lien on your projects or backing up business-critical data to buckets in a different project.

  • If a bucket is deleted, the objects within it are not permanently deleted but cannot be listed or restored unless the soft-deleted bucket is restored first. When you restore a soft-deleted bucket, the bucket is restored without live objects in it. You'll need to either perform an object restore operation or add new objects in it. To learn more about using soft-deleted buckets, see Restore a soft-deleted bucket.

Soft delete retention duration

When you create a bucket, a default soft delete policy is added to the bucket with a retention duration of 7 days, which is the minimum retention duration for a soft delete policy. For improved protection, you can select any retention duration up to a maximum of 90 days. Alternatively, you can set the retention duration to 0 days, which disables soft delete for the bucket.

Within a soft delete policy's retention duration, you can restore deleted objects, but after the duration ends, Cloud Storage permanently deletes the objects. The soft delete retention duration is measured in seconds. However, some tools, like the Google Cloud console and the Google Cloud CLI, enable you to set and view the retention duration using other units of time for convenience.

  • One day is considered to be 86,400 seconds.

  • One month is considered to be 31 days, which is 2,678,400 seconds.

For the gcloud CLI, when specifying a retention period, you must specify an integer and a unit, where the unit can be s, d, or m to signify seconds, days, or months, respectively. For example, 7d43200s sets a retention period of 7 days and 43,200 seconds (seven and a half days).

You can set a maximum retention duration of 7,776,000 seconds (90 days) or a minimum retention duration is 604,800 seconds (7 days). When using REST APIs, you can also set the retention duration to a value of 0, which disables the soft delete policy.

Restore behavior

When you restore a soft-deleted object, Cloud Storage creates a copy of the soft-deleted object in the same bucket the object was deleted from. The restored object's metadata is the same as the metadata of the deleted object. At the end of the soft delete retention duration, Cloud Storage permanently deletes the soft-deleted object.

You can restore the soft-deleted objects before the soft delete retention duration ends.

Restore behavior for soft delete can be described as the following:

If you delete your project, Cloud Storage permanently deletes all the buckets and objects within that project. In such a scenario, you cannot restore objects and buckets using the soft delete feature. Therefore, it is important to take steps to limit access to project-level deletions, such as placing a lien on projects or backing up business-critical data to buckets in a different project.

  • A restored object always replaces its live version: if a live version of your object already exists, the soft-deleted version replaces the live version, and the pre-existing live version becomes soft-deleted. In such a scenario, your bucket contains the following objects:

    • The replaced live object that is in the soft-deleted state.

    • Two copies of the previously soft-deleted object, one live copy and one still-soft-deleted copy.

  • Object copies incur charges until permanently deleted: object copies incur storage charges until the soft-deleted objects are permanently deleted after the retention duration is over. For more information about costs associated with soft-deleted objects, see Cloud Storage pricing.

  • Soft-deleted objects can be restored using multiple methods: you can restore the soft-deleted objects synchronously by specifying a list of objects, or you can create a long-running operation to bulk restore objects deleted between two timestamps.

  • Attempting to restore buckets that were soft-deleted before August 6, 2024 results in an error stating The specified bucket does not exist and you must contact Cloud Customer Care to perform restoration.

Restore and retrieve metadata of objects in buckets with hierarchical namespace enabled

In Cloud Storage buckets with hierarchical namespace enabled, duplicate soft-deleted objects can occur. The duplicate objects share the same name and generation values, causing potential ambiguity during restoration.

How duplicate objects occur

To understand how duplicate objects with identical name and generation values can occur, consider two objects folderA/my-object.txt and folderB/my-object.txt and carry out the following sequence of actions on the objects:

  1. Soft-delete object folderA/my-object.txt.
  2. Delete object folderA/my-object.txt parent folder, folderA.
  3. Rename folderB to folderA, after which object folderB/my-object.txt becomes folderA/my-object.txt.
  4. Soft-delete object folderA/my-object.txt which has the same name as the previously deleted object.

Each version of an object in Cloud Storage has a unique generation value. However, if objects folderA/my-object.txt and folderB/my-object.txt are created independently and are unrelated to each other, they might coincidentally have the same generation value.

As a result, it's possible to have two soft-deleted objects that have the same name (folderA/my-object.txt) and generation value.

To make sure you restore or retrieve the metadata of the intended my-object.txt object (since there are now two objects with the same name and generation value), you need to provide a unique identifier.

Unique object identification with restoreToken

The restoreToken uniquely identifies the correct object that you want to restore or retrieve. To get the value of the restoreToken, you can list the soft-deleted objects in a bucket. For details, see listing soft-deleted objects.

Considerations and errors

Consider the following information and potential errors that you might encounter when working with duplicate soft-deleted objects in buckets with hierarchical namespace enabled:

  • The occurrence of two unrelated objects having the same generation value is extremely unlikely, and the specific sequence of actions required for them to also end up with the same name makes this situation even rarer.

  • As the occurrence is rare, the restoreToken parameter might not be necessary for the vast majority of use cases. However, you can use the restoreToken for accurate object retrieval when duplicate soft-deleted objects with the same name and generation value occur.

  • If you try to restore or retrieve the metadata of soft-deleted objects without specifying the restoreToken and there are multiple objects matching the specified name and generation value, then you get an error indicating that you should specify the restoretoken.

  • If you try to restore or retrieve the metadata of soft-deleted objects by providing incorrect values of restoreToken, name, or generation, then you get an error indicating that the specified key does not exist.

Considerations with other features

When soft delete is enabled, it has the following interactions with other Cloud Storage features:

  • Object Lifecycle Management

    • Object Lifecycle Management rules don't affect soft-deleted objects. It is not possible to use Object Lifecycle Management to change the storage class of soft-deleted objects or to permanently delete then.

    • Objects deleted by Object Lifecycle Management become soft-deleted. If you also have Object Versioning enabled on your bucket, deleted live objects become noncurrent and deleted noncurrent objects become soft-deleted.

  • Object Versioning

    When you delete a noncurrent object, it becomes soft-deleted.

  • XML API multipart uploads

    XML API multipart upload parts are not protected by soft delete.

  • Autoclass

    • Autoclass management fees are not assessed for soft-deleted objects.

    • Autoclass does not change the storage class of soft-deleted objects.

    • When you restore a soft-deleted object, the resulting object is set to the Standard storage class.

  • Bucket Lock and Object Retention Lock features

    These features will prevent you from deleting objects until they fulfill their retention policies. Soft delete then provides an additional layer of protection by retaining deleted objects for a separate soft delete retention duration.

  • Pub/Sub notifications

    Restoring a soft-deleted version of an object triggers an OBJECT_FINALIZE event.

  • Managed folders: Soft delete cannot restore the IAM policies on managed folders. If you soft delete an object and delete a managed folder that grants IAM policies for the object, you might have to recreate those IAM policies before you have the necessary permissions to restore the soft-deleted object.

  • Tags: If you create a bucket without including a soft delete setting yourself, the Cloud Storage default is to create the bucket with a seven day soft delete retention duration. You can change this default by using a tag. The tag can change the default retention duration for new buckets to be any value between 7 and 90 days, or it can change the default to disable soft delete on new buckets. You can also use tags to disable soft delete by default.

  • Recommender: You can enable the Recommender API to receive suggestions and insights on enabling or disabling soft delete based on Cloud Billing costs and usage. To learn more, see soft delete recommender.

What's next