Jump to Content
Storage & Data Transfer

New prefix and suffix lifecycle rules for Cloud Storage

August 22, 2022
Dominic Zippilli

Product Manager, Google Cloud Storage

Sam Smits

Engineering Manager, Google Cloud Storage

Google Cloud Storage is very happy to launch powerful new object lifecycle rules for GCS to help our customers control their storage costs in new ways. Two new features are included in this launch.

Prefix/suffix lifecycle conditions

Before this launch, customers already had several lifecycle conditions to choose from, such as the age of an object, its version history, a custom timestamp, and more. Now, customers can add conditions on the names of objects; specifically, matching a prefix or a suffix.  

Prefix and suffix conditions are helpful for a number of cases. Here are two:

Managing common object prefixes separately. It's quite common to group objects using a common prefix, such as in a dataset. Now, lifecycle conditions can act on those groups using a MatchesPrefix rule.

Managing categories of objects separately. It's also common to use "extensions" on object keys to denote the format of the data; .mp4, .zip, .csv, and so on. Customers often have a mix of these within a bucket, and would like to use the extension to manage them separately. A great example is when customers have large binary objects that are read seldomly, but metadata about them with a different extension that is read often (think, .mov and .xml). Now, it is easy to keep the metadata "hot" while saving costs with the bulk of your data "cold" using a MatchesSuffix rule.

Cleanup of incomplete multipart uploads

In 2021, we launched the Multipart Upload protocol in our XML API. This provided our customers with an even smoother migration path by greatly improving our S3 compatibility. One common pain point with multipart uploads is that they sometimes get abandoned. Part storage isn't free, so pruning abandoned uploads is critical housekeeping. Now, customers can "set it and forget it" with GCS Lifecycle support for incomplete multipart uploads.

These new lifecycle features are now generally available to all of our customers. They are supported through the GCS API, GCS GUI, gsutil, gcloud storage, and the client libraries.

Posted in