To support common use cases like setting a Time to Live (TTL) for objects, archiving older versions of objects, or "downgrading" storage classes of objects to help manage costs, Google Cloud Storage offers the Object Lifecycle Management feature. This page describes the feature as well as the options available when using it. To learn how to enable Object Lifecycle Management, and for examples of lifecycle policies, see Managing Lifecycles.
You can assign a lifecycle management configuration to a bucket. The configuration contains a set of rules which apply to all the objects in the bucket. When an object meets the criteria of one of the rules, Google Cloud Storage automatically performs a specified action on the object. Here are some example use cases:
- Downgrade the storage class of objects older than 365 days to Coldline Storage.
- Delete objects created before January 1, 2013.
- Keep only the 3 most recent versions of each object in a bucket with versioning enabled.
Each lifecycle management configuration contains a set of rules. When defining a rule, you can specify any set of conditions for any action. If you specify multiple conditions in a rule, an object has to match all of the conditions for the action to be taken. If you specify multiple rules that contain the same action, the action is taken when an object matches the condition(s) in any of the rules. Each rule should contain only one action.
If a single object is subject to multiple actions, Cloud Storage performs only one of the actions, and the object will be re-evaluated before any additional actions are taken. A Delete action takes precedence over a SetStorageClass action. If multiple SetStorageClass actions are specified, the action switching to the storage class with the lowest at-rest storage pricing is chosen.
So, for example, if you have one rule that deletes an object and another rule that changes the object's storage class, but both rules use the exact same condition, the delete action always occurs when the condition is met. If you have one rule that changes the object's class to Nearline Storage and another rule that changes the object's class to Coldline Storage, but both rules use the exact same condition, the object's class always changes to Coldline Storage when the condition is met.
The following actions are supported for a lifecycle rule:
- Delete: Delete live and/or archived objects. (A live object is one that is not an archived generation. See object versioning for details.) This action can be applied to both versioned and non-versioned objects. In a bucket with versioning enabled, deleting a live object creates an archived object, while deleting an archived object deletes the object permanently.
- SetStorageClass: Change the storage class of live and/or archived objects. This action can be applied to both versioned and non-versioned objects. This action can change the storage class of the object either to Nearline Storage or Coldline Storage, but cannot be used on objects whose current storage class is Coldline Storage.
The following conditions are supported for a lifecycle rule:
- Age: This condition is satisfied when an object reaches
the specified age (in days).
When you specify the
Agecondition, you are specifying a Time to Live (TTL) for objects in a bucket with lifecycle management configured. The time when the
Agecondition is considered to be satisfied is calculated by adding the specified value to the object creation time. For example, if the object creation time is 2013/01/10 10:00 UTC and the
Agecondition is 10 days, then the condition is satisfied on and after 2013/01/20 10:00 UTC.
- CreatedBefore: This condition is satisfied when an object is created before midnight of the specified date in UTC.
- IsLive: Relevant only for versioned objects. If the value
true, this condition matches the live objects; if the value is
false, it matches archived objects.
- MatchesStorageClass: This condition is
satisfied when an object in the bucket is stored as the specified storage
class. Generally, if you intend to use this condition on Multi-Regional
Storage or Regional Storage objects, you should also include
DURABLE_REDUCED_AVAILABILITYin the condition to ensure all objects of similar storage class are covered.
- NumberOfNewerVersions: Relevant only for versioned objects. If the value of this condition is set to N, an object satisfies the condition when there are at least N versions (including the live version) newer than it. For live objects, the number of newer versions is considered to be 0. For the most recent archived version, the number of newer versions is 1 (or 0 if there is no live object), and so on.
The lifecycle configuration is specified as an XML document for requests to the XML API, and as a JSON document for requests to the JSON API or made with the gsutil tool. Here is the RELAX NG compact syntax schema of the lifecycle configuration XML.
Object lifecycle behavior
Google Cloud Storage regularly inspects all the objects in a bucket for which Object Lifecycle Management is configured and performs all actions applicable according to the bucket's rules. Cloud Storage performs the action asynchronously, so there can be a lag between when the conditions are satisfied and when the action is taken.
For example, if an object meets the conditions for deletion, the object might not be deleted right away. Therefore, you see the object until the lifecycle action is executed on the object. You are not billed for the storage of the object, but accessing the object incurs any applicable operation and bandwidth charges as described in pricing.
Updates to your lifecycle configuration may take up to 24 hours to go into effect. This means that when you change your lifecycle configuration, Object Lifecycle Management may still perform actions based on the old configuration for up to 24 hours.
For example, if you change an
Age condition from 10 days to 20 days, an object
that is 11 days old could be deleted by Object Lifecycle Management up
to 24 hours later, due to the criteria of the old configuration.
Early deletion of Nearline Storage and Coldline Storage objects
Object Lifecycle Management does not rewrite an object when changing its
storage class. This means that when an object is transitioned to
Nearline Storage or Coldline Storage using the
feature, any subsequent early deletion and associated charges are
based on the original creation time of the object, regardless of when
the storage class changed.
For example, say you upload an object as Regional Storage, and 20 days later your lifecycle configuration changes the storage class of the object to Nearline Storage. If you then delete the object immediately, there is a 10-day early deletion charge, since the object existed for 20 days. If you delete the object 10 days after changing its storage class to Nearline Storage, there is no early deletion charge, since the object existed for 30 days.
In comparison, say you upload an object as Regional Storage and 20 days later change the storage class using a rewrite (again to Nearline Storage). If you then delete the object immediately afterward, you would incur a full 30-day early deletion charge, since the rewriting time becomes the new creation time. Similarly, if you waited 10 days after the rewrite to delete the object, you would incur a 20-day early deletion charge.
Expiration time metadata
Delete action is specified for a bucket with the
Age condition (and no
then some objects may be tagged with expiration time metadata. An object's expiration
time indicates the time at which the object becomes (or became) eligible for deletion
by Object Lifecycle Management. The expiration time may change as the bucket's lifecycle
Note that the absence of expiration time metadata does not necessarily mean
the object will not be deleted, but rather that not enough information is
available to determine when or if it will be deleted. For example, if the object
creation time is 2013/01/10 10:00 UTC and the
Age condition is set
to 10 days, then the object expiration time is 2013/01/20 10:00 UTC. However,
the expiration time will not be available for the object if:
NumberOfNewerVersionscondition is also specified. In this case, older versions of the object may still be deleted if new versions are added.
CreatedBeforecondition is also specified and set to “2013-01-01”, because the object doesn’t satisfy this condition.
You are not charged for storage after the object expiration time even if the object is not deleted immediately. You can continue to access the object before it is deleted and are responsible for other charges (request, network bandwidth). If the expiration time is not available for an object, the object is charged for storage until the time it is deleted.
If there are multiple conflicting expiration times applicable for an object due to different lifecycle management rules, then the earliest applicable expiration time is used.
You can access an object's expiration time in its metadata if it is
available. The REST API returns the object's expiration time in the
x-goog-expiration response header.
To find out what lifecycle management actions have been taken, you can enable access logs for your bucket. A value of "GCS Lifecycle Management" in the “cs_user_agent” field in the log entry indicates the action was taken by Google Cloud Storage based on the lifecycle configuration. For more information, see Access Logs & Storage Data.