Bucket Lock

Setup

This page discusses the Bucket Lock feature, which lets you configure a Cloud Storage bucket's retention policy. This policy governs how long objects in the bucket must be retained. The feature also lets you lock the bucket's retention policy, permanently preventing the policy from being reduced or removed.

This feature can provide immutable storage on Cloud Storage. In conjunction with Detailed audit logging mode, which logs Cloud Storage request and response details, Bucket Lock can help with regulatory and compliance requirements, such as those associated with FINRA, SEC, and CFTC. Bucket Lock can also help you address certain health care industry retention regulations.

Overview

  • You can add a retention policy to a bucket to specify a retention period.

    • When a bucket retention policy is set, objects in the bucket can only be deleted or replaced once their age is greater than the retention period.

    • The policy retroactively applies to existing objects in the bucket as well as new objects added to the bucket. This differs from the Object Retention Lock feature, which lets you define data retention requirements on a per-object basis.

  • You can lock a bucket's retention policy to permanently set it on the bucket.

    • Once you lock a policy, you cannot remove it or reduce the retention period it has.

    • You cannot delete a bucket with a locked policy unless every object in the bucket has met the retention period.

    • You can increase the retention period of a locked policy.

Bucket retention policies

You can include a retention policy when creating a new bucket, or you can add a retention policy to an existing bucket. Placing a retention policy on a bucket ensures that all current and future objects in the bucket cannot be deleted or replaced until they reach the age you define in the policy. Attempts to delete or replace objects whose age is less than the retention period fail with a 403 - retentionPolicyNotMet error.

For example, say you have a bucket with two objects in it: Object A you added a month ago, and Object B you added two years ago. If you apply a retention policy to your bucket that has a retention period of 1 year, you cannot delete or replace Object A for another 11 months: it is currently 1 month old, but must be a least 1 year old to delete or replace. Object B, on the other hand, can be deleted or replaced immediately, since its age is greater than the retention period. If you decided to replace Object B, this new version of Object B has an age that restarts at 0 years.

To help track when individual objects are eligible for deletion, objects in a bucket with a retention policy each have retention expiration time metadata. This piece of metadata shows the date and time when an object fulfills the retention period.

General considerations

When working with retention policies, keep in mind the following:

  • Unless a bucket's retention policy is locked, you can increase, decrease, or remove the policy.

  • An object's editable metadata is not subject to a bucket's retention policy and can be modified even when the object itself cannot be.

  • A bucket's retention policy contains an effective time, the time after which all objects in the bucket are guaranteed to be in compliance with the retention period.

  • To see the earliest date when a given object is eligible for deletion in a bucket with a retention policy, view the retention expiration date portion of the object's metadata.

Considerations with other features

The following are interactions that retention policies have with other Cloud Storage features:

  • In buckets that use Object Versioning, a live object version that has a retention expiration date in the future can still be made noncurrent, and any versioned objects that exist in the bucket at the time you apply a retention policy are also protected by the policy.

  • An object that is subject to an event-based hold cannot be deleted while the hold applies to it. Once the event-based hold is removed from the object, the object's retention period is reset.

  • An individual object can be subject to the bucket's retention policy and to its own, individual retention configuration. If an object is subject to both, it is retained until both retentions have been satisfied.

  • You cannot destroy Cloud Key Management Service key versions that encrypt objects in locked buckets if the objects haven't met their retention expiration times. For more information, see Key versions used to encrypt locked objects.

  • You can use Object Lifecycle Management to automatically delete objects in a bucket, including in a bucket with a locked policy. A lifecycle rule won't delete an object until after the object fulfills the retention policy.

  • You should not perform parallel composite uploads if your bucket has a retention policy, because the component pieces cannot be deleted until each has met the bucket's minimum retention period.

  • Attempting to complete an XML API multipart upload fails if the resulting object would overwrite an object that has not yet met its retention period.

  • You can use the retention policy constraint in your organization policies to require that bucket retention policies with specific retention periods be included as part of creating a new bucket or as part of adding/updating the retention policy on an existing bucket.

Retention periods

Retention periods are measured in seconds; however, some tools, like the Google Cloud console and the Google Cloud CLI allow you to set and view retention periods with other units of time for convenience. The following conversions apply in such cases:

  • A day is considered to be 86,400 seconds.
  • A month is considered to be 31 days, which is 2,678,400 seconds.
  • A year is considered to be 365.25 days, which is 31,557,600 seconds.

You can set a maximum retention period of 3,155,760,000 seconds (100 years).

For the gcloud CLI, when specifying a retention period, you specify an integer and a unit, where the unit can be s, d, m, or y to signify seconds, days, months, or years, respectively. For example, 1d43200s sets a retention period of 1 day and 43,200 seconds (one and a half days).

Retention policy locks

When you lock a bucket's retention policy, you prevent the policy from ever being removed or the retention period from ever being reduced (although you can still increase the retention period). If you try to remove or reduce the policy duration of a locked bucket, you get a 400 BadRequestException error. Once a retention policy is locked, you cannot delete the bucket until every object in the bucket has met the retention period.

Locking a bucket's retention policy is irreversible, and you should be familiar with the implications of doing so prior to using this feature. When you use an unlocked policy, you have the ability to remove the policy, allowing you to still delete objects when desired. When you lock a policy, you must delete the entire bucket in order to "remove" the policy. However, you can't delete the bucket if there are objects in it that haven't fulfilled their retention period. Thus, to "remove" a locked policy, you have to wait until every object in the bucket has fulfilled its retention period, at which point you can delete the bucket.

Additionally, when you lock a retention policy, Cloud Storage automatically applies a lien to the projects.delete permission for the project that contains the bucket. While in place, the lien prevents the project from being deleted. To delete the project, you must first remove all such liens. Note that removing a lien requires the resourcemanager.projects.updateLiens permission, which is part of the roles/owner and roles/resourcemanager.lienModifier roles.

For information on how locking a retention policy can help your data comply with record retention regulations, see the compliance page.

What's next