Soft Delete

Setup Usage

This page describes the soft delete feature, which preserves objects that get deleted or overwritten for a specified period of time. Soft delete helps protect your data from accidental or malicious deletion by retaining deleted objects in a soft-deleted state, during which the object cannot be permanently deleted.

Overview

When you enable soft delete on a bucket, objects that get deleted from the bucket enter a soft-deleted state instead of being permanently deleted. Soft-deleted objects behave in the following ways:

To enable soft delete on a bucket, you create a soft delete policy that specifies a retention duration controlling how long soft-deleted objects are retained before getting permanently deleted.

To learn how to create and manage a soft delete policy to enable or disable soft delete on a bucket, see Use soft delete.

Soft delete policies

A soft delete policy can be created, deleted, or modified during a bucket creation or update. Once a soft delete policy is created, you can use it to enable or disable soft delete for the bucket.

Soft delete policies behave in the following ways:

  • Updating a bucket's soft delete policy only applies to objects that you delete after the soft delete policy takes effect. Objects that you deleted prior to the update are retained for the duration that was in effect when they were deleted.

    For example, say you enable a soft delete policy in your bucket with the default retention duration of seven days, and then you delete the object cat.png. In this scenario, cat.png is retained as a soft-deleted object for the next seven days, and then it's permanently deleted. This occurs even if you later change or remove the soft delete policy for your bucket.

    • If you disable a soft delete policy from your bucket that has soft-deleted objects in it during the time of disablement, Cloud Storage retains the soft-deleted objects for the specified retention duration before permanent deletion.

    • After disabling a soft delete policy on your bucket, Cloud Storage doesn't retain deleted data once a delete operation is performed.

    • You cannot force delete a soft-deleted object. This is to prevent malicious or accidental deletion. To prevent your bucket from further retaining soft-deleted data, we recommend disabling your bucket's soft delete policy.

Soft delete retention duration

When you create a bucket, a default soft delete policy is added to the bucket with a retention duration of 7 days, which is the minimum retention duration for a soft delete policy. You can set the retention duration to a maximum of 90 days or to 0 days, which disables soft delete for the bucket.

Within a soft delete policy's retention duration, you can restore deleted objects, but after the duration ends, Cloud Storage permanently deletes the objects. The soft delete retention duration is measured in seconds. However, some tools, like the Google Cloud console and the Google Cloud CLI, enable you to set and view the retention duration using other units of time for convenience.

The following conversions apply in such cases:

  • A day is considered to be 86,400 seconds.
  • A month is considered to be 31 days, which is 2,678,400 seconds.

For the gcloud CLI, when specifying a retention period, you specify an integer and a unit, where the unit can be s, d, or m to signify seconds, days, or months, respectively. For example, 7d43200s sets a retention period of 7 days and 43,200 seconds (seven and a half days).

You can set a maximum retention duration of 7,776,000 seconds (90 days) or a minimum retention duration is 604,800 seconds (7 days). When using REST APIs, you can also set the retention duration to a value of 0, which disables the soft delete policy.

Restore behavior

When you restore a soft-deleted object, Cloud Storage creates a copy of the soft-deleted object in the same bucket the object was deleted from. The restored object's metadata is the same as the metadata of the deleted object. At the end of the soft delete retention duration, Cloud Storage permanently deletes the soft-deleted object.

You can restore the soft-deleted objects before the soft delete retention duration ends.

  • For the restore to be successful, you must start the restore early enough to be able to complete it within the soft delete retention duration, as objects are permanently deleted after that duration ends.
  • You can restore the soft-deleted objects synchronously by specifying a list of objects, or you can create a long-running operation to bulk restore objects deleted between two timestamps.
  • Cloud Storage always restores objects to the same bucket from where they were originally deleted.

If you delete your project, Cloud Storage permanently deletes all the buckets and objects within that project. In such a scenario, you cannot restore objects and buckets using the soft delete feature. Therefore, it is important to take steps to limit access to project-level deletions, such as placing a lien on projects or backing up business-critical data to buckets in a different project.

When you restore a soft-deleted object version, Cloud Storage restores the copy of the soft-deleted object to a live version. If a live version already exists, the copy of the soft-deleted version replaces the live version, and the pre-existing live version becomes soft-deleted. In such a scenario, your bucket contains the following objects:

  • The replaced live object that is in the soft-deleted state
  • Two copies of the previously soft-deleted object – one live copy and one still-soft-deleted copy

All these object copies incur storage charges until the soft-deleted objects are permanently deleted after the retention duration is over. For more information about costs associated with soft-deleted objects, see Cloud Storage pricing.

Cloud Storage also copies the metadata of restored soft-deleted objects when replacing the live object. This means that object-level security metadata and settings like storage class are set to the last state of that specific object version upon restore.

Soft-deleted objects are not permanently deleted upon bucket deletion. However, the correct bucket version must be restored before it is possible to restore the soft-deleted objects inside it. To restore a soft-deleted bucket, contact Google Cloud Support.

Restore and retrieve metadata of objects in buckets with hierarchical namespace enabled

In Cloud Storage buckets with hierarchical namespace enabled, duplicate soft-deleted objects can occur. The duplicate objects share the same name and generation values, causing potential ambiguity during restoration.

How duplicate objects occur

To understand how duplicate objects with identical name and generation values can occur, consider two objects folderA/my-object.txt and folderB/my-object.txt and carry out the following sequence of actions on the objects:

  1. Soft-delete object folderA/my-object.txt.
  2. Delete object folderA/my-object.txt parent folder, folderA.
  3. Rename folderB to folderA, after which object folderB/my-object.txt becomes folderA/my-object.txt.
  4. Soft-delete object folderA/my-object.txt which has the same name as the previously deleted object.

Each version of an object in Cloud Storage has a unique generation value. However, if objects folderA/my-object.txt and folderB/my-object.txt are created independently and are unrelated to each other, they might coincidentally have the same generation value.

As a result, it's possible to have two soft-deleted objects that have the same name (folderA/my-object.txt) and generation value.

To make sure you restore or retrieve the metadata of the intended my-object.txt object (since there are now two objects with the same name and generation value), you need to provide a unique identifier.

Unique object identification with restoreToken

The restoreToken uniquely identifies the correct object that you want to restore or retrieve. To get the value of the restoreToken, you can list the soft-deleted objects in a bucket. For details, see listing soft-deleted objects.

Considerations and errors

Consider the following information and potential errors that you might encounter when working with duplicate soft-deleted objects in buckets with hierarchical namespace enabled:

  • The occurrence of two unrelated objects having the same generation value is extremely unlikely, and the specific sequence of actions required for them to also end up with the same name makes this situation even rarer.

  • As the occurrence is rare, the restoreToken parameter might not be necessary for the vast majority of use cases. However, you can use the restoreToken for accurate object retrieval when duplicate soft-deleted objects with the same name and generation value occur.

  • If you try to restore or retrieve the metadata of soft-deleted objects without specifying the restoreToken and there are multiple objects matching the specified name and generation value, then you get an error indicating that you should specify the restoretoken.

  • If you try to restore or retrieve the metadata of soft-deleted objects by providing incorrect values of restoreToken, name, or generation, then you get an error indicating that the specified key does not exist.

Considerations with other features

When soft delete is enabled, it has the following interactions with other Cloud Storage features:

  • Object Lifecycle Management

    • Object Lifecycle Management rules don't affect soft-deleted objects.

    • Objects deleted by Object Lifecycle Management become soft-deleted.

  • Object Versioning

    When you delete a noncurrent object, it becomes soft-deleted.

  • XML API multipart uploads

    XML API multipart upload parts are not protected by soft delete.

  • Autoclass

    • Autoclass management fees are not assessed for soft-deleted objects.

    • Autoclass does not change the storage class of soft-deleted objects.

    • When you restore of a soft-deleted object, the resulting object is set to the Standard storage class.

  • Bucket Lock and Object Retention Lock features

    You can't delete objects until they fulfill their retention policies, even if soft delete would retain the object for a sufficient duration.

  • Pub/Sub notifications

    Restoring a soft-deleted version of an object triggers an OBJECT_FINALIZE event.

  • Managed folders

    • soft delete doesn't apply to managed folders.

    • If you soft delete an object and delete a managed folder that grants IAM policies for the object, you might not have permission to restore the soft-deleted object until you have recreated the IAM policies associated with the managed folder.

  • Managed folders: Soft delete cannot restore the IAM policies on managed folders. If you soft delete an object and delete a managed folder that grants IAM policies for the object, you might have to recreate those IAM policies before you have the necessary permissions to restore the soft-deleted object.

  • Tags: If you create a bucket without including a soft delete setting yourself, the Cloud Storage default is to create the bucket with a seven day soft delete retention duration. You can change this default by using a tag. The tag can change the default retention duration for new buckets to be any value between 7 and 90 days, or it can change the default to disable soft delete on new buckets. You can also use tags to disable soft delete by default.

  • Recommender: You can enable the Recommender API to receive suggestions and insights on enabling or disabling soft delete based on Cloud Billing costs and usage. To learn more, see soft delete recommender.

What's next