Object Lifecycle Management

Go to examples

To support common use cases like setting a Time to Live (TTL) for objects, retaining noncurrent versions of objects, or "downgrading" storage classes of objects to help manage costs, Cloud Storage offers the Object Lifecycle Management feature.

This page describes the feature as well as the options available when using it. For the general format of a lifecycle configuration file, see the bucket resource representation for JSON or the lifecycle configuration format for XML.

Introduction

You can assign a lifecycle management configuration to a bucket. The configuration contains a set of rules which apply to current and future objects in the bucket. When an object meets the criteria of one of the rules, Cloud Storage automatically performs a specified action on the object. Here are some example use cases:

  • Downgrade the storage class of objects older than 365 days to Coldline Storage.
  • Delete objects created before January 1, 2013.
  • Keep only the 3 most recent versions of each object in a bucket with versioning enabled.

Lifecycle configuration

Each lifecycle management configuration contains a set of rules. When defining a rule, you can specify any set of conditions for any action. If you specify multiple conditions in a rule, an object has to match all of the conditions for the action to be taken. If you specify multiple rules that contain the same action, the action is taken when an object matches the condition(s) in any of the rules. Each rule should contain only one action.

If multiple rules have their conditions satisfied simultaneously for a single object, Cloud Storage performs the action associated with only one of the rules, based on the following considerations:

  • The Delete action takes precedence over any SetStorageClass action.
  • The SetStorageClass action that switches the object to the storage class with the lowest at-rest storage pricing takes precedence.

Once an action occurs, the object is re-evaluated before any additional actions are taken.

So, for example, if you have one rule that deletes an object and another rule that changes the object's storage class, but both rules use the exact same condition, the delete action always occurs when the condition is met. If you have one rule that changes the object's class to Nearline Storage and another rule that changes the object's class to Coldline Storage, but both rules use the exact same condition, the object's class always changes to Coldline Storage when the condition is met.

See Managing object lifecycles for examples of using lifecycle configurations.

Lifecycle actions

A lifecycle rule specifies either a Delete action or a SetStorageClass action.

Delete

The Delete action deletes an object when the object meets all conditions specified in the lifecycle rule.

Exception: In buckets with Object Versioning enabled, deleting the live version of an object causes it to become a noncurrent version, while deleting a noncurrent version deletes that version permanently. See the configuration for deleting objects for an example of using the Delete action along with Object Versioning.

SetStorageClass

The SetStorageClass action changes the storage class of an object when the object meets all conditions specified in the lifecycle rule.

SetStorageClass supports the following storage class transitions:

Original storage class New storage class
Durable Reduced Availability (DRA) Storage Nearline Storage
Coldline Storage
Archive Storage
Multi-Regional Storage/Regional Storage1
Standard Storage, Multi-Regional Storage, or Regional Storage Nearline Storage
Coldline Storage
Archive Storage
Nearline Storage Coldline Storage
Archive Storage
Coldline Storage Archive Storage

1 For buckets in a region, the new storage class cannot be Multi-Regional Storage. For buckets in a multi-region or dual-region, the new storage class cannot be Regional Storage.

Cloud Storage does not validate correctness of the storage class transition. This means that you can specify a storage class transition not listed in the above table, but the transition will not occur. You should verify that your lifecycle rules use one of the listed storage class transitions.

Lifecycle conditions

A lifecycle rule includes conditions which an object must meet before the action defined in the rule occurs on the object. Lifecycle rules support the following conditions:

All conditions are optional, but at least one condition is required. If you attempt to set an invalid lifecycle configuration, such as by using an action or condition that does not exist, you receive a 400 Bad request error response, and any existing lifecycle configuration remains in place.

Age

The Age condition is satisfied when an object reaches the specified age (in days). Age is measured from the object's creation time. For example, if an object's creation time is 2019/01/10 10:00 UTC and the Age condition is 10 days, then the condition is satisfied for the object on and after 2019/01/20 10:00 UTC. This is true even if the object becomes noncurrent through Object Versioning sometime after its creation.

CreatedBefore

The CreatedBefore condition is satisfied when an object is created before midnight of the specified date in UTC.

CustomTimeBefore

The CustomTimeBefore condition is satisfied when the date portion of an object's Custom-Time metadata is earlier than the date specified in this condition. This condition is set using the date format YYYY-MM-DD. CustomTimeBefore is never satisfied for an object with no Custom-Time metadata set.

DaysSinceCustomTime

The DaysSinceCustomTime condition is satisfied when the specified number of days have passed since the date and time specified in an object's Custom-Time metadata field. For example, if an object's Custom-Time is 2020-05-16T10:00:00Z and the DaysSinceCustomTime condition is 10 days, then the condition is satisfied for the object on and after 2020/05/26 10:00 UTC.

DaysSinceCustomTime is never satisfied for an object with no Custom-Time metadata set.

DaysSinceNoncurrentTime

The DaysSinceNoncurrentTime condition is typically only used in conjunction with Object Versioning. The condition is satisfied when the specified number of days have passed since the object became noncurrent, either because the live version was deleted or replaced. For example, if an object became noncurrent at 2020/07/08 15:00 UTC and the DaysSinceNoncurrentTime condition is 10 days, then the condition is satisfied for the object on and after 2020/07/18 15:00 UTC.

IsLive

The IsLive condition is typically only used in conjunction with Object Versioning. When set to false, this condition is satisfied for any noncurrent version of an object. When set to true, this condition is satisfied for the live version of an object. If you don't use versioning, all your objects are considered live and match when IsLive is true.

MatchesStorageClass

The MatchesStorageClass condition is satisfied when an object in the bucket is stored as the specified storage class. You can use the following values for MatchesStorageClass: STANDARD, NEARLINE, COLDLINE, ARCHIVE, MULTI_REGIONAL, REGIONAL, and DURABLE_REDUCED_AVAILABILITY.

Generally, if you intend to use the MatchesStorageClass condition on Standard Storage objects, you should also include the following:

  • If the bucket is in a region, include REGIONAL and DURABLE_REDUCED_AVAILABILITY in the condition.

  • If the bucket is in a multi-region or dual-region, include MULTI_REGIONAL and DURABLE_REDUCED_AVAILABILITY in the condition.

Including these additional classes ensures the lifecycle rule covers older objects in your buckets which may be set to legacy storage classes.

NoncurrentTimeBefore

The NoncurrentTimeBefore condition is typically only used in conjunction with Object Versioning. The condition is satisfied for objects that became noncurrent on a date prior to the one specified in this condition. The condition is set using the date format YYYY-MM-DD. NoncurrentTimeBefore is never satisfied for a live object.

NumberOfNewerVersions

The NumberOfNewerVersions condition is typically only used in conjunction with Object Versioning. If the value of this condition is set to N, an object version satisfies the condition when there are at least N versions (including the live version) newer than it. For a live object version, the number of newer versions is considered to be 0. For the most recent noncurrent version, the number of newer versions is 1 (or 0 if there is no live object version), and so on.

Object lifecycle behavior

  • If a lifecycle configuration rule deletes millions of objects in a bucket at the same time, object listing performance could be severely degraded after the deletions occur. Reach out to Google Cloud Support or your Account Manager before setting up such a policy.

  • Cloud Storage regularly inspects all the objects in a bucket for which Object Lifecycle Management is configured and performs all actions applicable according to the bucket's rules. Cloud Storage performs an action asynchronously, so there can be a lag between when the conditions are satisfied and when the action is taken.

    For example, if an object meets the conditions for deletion, the object might not be deleted right away. Therefore, you see the object until the lifecycle action is executed on the object. Applicable charges still apply while the object exists, with one exception: at-rest storage costs are waived if the object is subject to a Delete action because of a rule that has only an Age condition.

  • Updates to your lifecycle configuration may take up to 24 hours to go into effect. This means that when you change your lifecycle configuration, Object Lifecycle Management may still perform actions based on the old configuration for up to 24 hours.

    For example, if you change an Age condition from 10 days to 20 days, an object that is 11 days old could be deleted by Object Lifecycle Management up to 24 hours later, due to the criteria of the old configuration.

  • An object lifecycle Delete action will not take effect on an object while the object either has an object hold placed on it or a retention policy that it has not yet fulfilled. Any Delete action that would occur while an object had a hold or retention policy restriction instead occurs after the restrictions no longer apply to the object.

  • An object lifecycle SetStorageClass action is not affected by the existence of object holds or retention policies.

SetStorageClass cost considerations

Similar to changing an object's storage class manually, using SetStorageClass counts as a Class A operation and is billed accordingly.

Unlike changing an object's storage class manually, using SetStorageClass does not rewrite an object. This gives Object Lifecycle Management certain pricing advantages:

  • There are no retrieval fees or early deletion fees associated with the storage class change, even when the object is originally set to Nearline Storage or Coldline Storage.

  • The object's time spent set at the original storage class counts towards any minimum storage duration that applies for the new storage class.

For example, say you upload an object as Nearline Storage, and 20 days later your lifecycle configuration changes the storage class of the object to Coldline Storage. This change incurs no retrieval or early deletion fees. If you then delete the object 60 days after the storage class change, there is only a 10-day early deletion charge, since Coldline Storage has a 90-day minimum storage duration, and the object existed for a total of 80 days.

In comparison, say you upload an object as Nearline Storage, and 20 days later change the storage class using a rewrite (again to Coldline Storage). This change incurs both a retrieval fee and a 10-day early deletion charge. If you then delete the object 60 days after the rewrite, there is a 30-day early deletion charge.

Expiration time metadata

If a Delete action is specified for a bucket with the Age condition (and no other conditions besides matchesStorageClass), then some objects may be tagged with expiration time metadata. An object's expiration time indicates the time at which the object becomes (or became) eligible for deletion by Object Lifecycle Management. The expiration time may change as the bucket's lifecycle configuration or retention policy change.

Note that the absence of expiration time metadata does not necessarily mean the object will not be deleted, but rather that not enough information is available to determine when or if it will be deleted. For example, if the object creation time is 2020/01/10 10:00 UTC and the Age condition is set to 10 days, then the object expiration time is 2020/01/20 10:00 UTC. However, the expiration time is not available for the object if:

  • There are other conditions specified in the Delete rule, with the exception of matchesStorageClass.

  • You use a matchesStorageClass condition that does not include the object's storage class.

  • The object is under a hold, because Cloud Storage cannot know when the hold will be removed.

You are not charged for storage after the object expiration time even if the object is not deleted immediately. You can continue to access the object before it is deleted and are responsible for other charges (request, network bandwidth). If the expiration time is not available for an object, the object is charged for storage until the time it is deleted.

When working with expiration times, keep in mind the following:

  • If the bucket has a retention policy, the expiration time is the later of the Object Lifecycle Management Age condition and the time the object satisfies the retention period specified by the retention policy.

  • If there are multiple conflicting expiration times applicable for an object due to different lifecycle management rules, then the earliest applicable expiration time is used.

Options for tracking Lifecycle actions

To track the lifecycle management actions that Cloud Storage takes, use one of the following options:

  • Use Cloud Storage usage logs. This feature logs both the action and who performed the action. A value of GCS Lifecycle Management in the cs_user_agent field of the log entry indicates the action was taken by Cloud Storage in accordance with a lifecycle configuration.

  • Enable Pub/Sub Notifications for Cloud Storage for your bucket. This feature sends notifications to a Pub/Sub topic of your choice when specified actions occur. Note that this feature does not record who performed the actions.

What's next