To support the retrieval of objects that are deleted or overwritten, Cloud Storage offers the Object Versioning feature. This page describes the feature and the options available when using it. To learn how to enable and use Object Versioning see Using Object Versioning.
Enable Object Versioning to protect your Cloud Storage data from being overwritten or accidentally deleted. Enabling Object Versioning increases storage costs, which can be partially mitigated by configuring Object Lifecycle Management to delete older object versions.
You enable Object Versioning for a bucket. Once enabled, Cloud Storage creates an archived version of an object each time the live version of the object is overwritten or deleted. The archived version retains the name of the object but is uniquely identified by a generation number. While all objects have generation numbers associated with them, only archived objects require generation numbers in order to identify them.
When Object Versioning is enabled, you can list archived versions of an object, restore the live version of an object to an older state, or permanently delete an archived version, as needed. You can turn versioning on or off for a bucket at any time. Turning off versioning leaves existing object versions in place and causes the bucket to stop accumulating new archived object versions.
Object Versioning details
Cloud Storage uses two properties that together identify the version of an object. One property identifies the version of the object's data; the other property identifies the version of the object's metadata. These properties are always present with every version of the object, even if Object Versioning is not enabled. These properties can be used as preconditions for conditional updates to enforce ordering of updates.
Cloud Storage marks every object using the following properties:
||Identifies the content (data) generation, and updates when the content of an object is overwritten. There is no relationship between the generation numbers of unrelated objects, even if the objects are in the same bucket.|
||Identifies the metadata generation, and increases every time the metadata for a given content generation is updated.
Archived object metadata
Archived objects have their own metadata, which may differ from the metadata of live objects. Most importantly, an archived version retains its ACLs and does not necessarily have the same permissions as the live version of the object.
Each object, whether live or versioned, has one set of metadata; only the latest metageneration number refers to metadata. Older metageneration numbers cannot be used to access metadata that has since been changed.
You can update metadata for an archived object by specifying its
in your request. To ensure safe read-modify-write semantics, you can use
a metageneration-match precondition. Using this precondition causes
the update to fail if the metadata you are attempting to update was
changed between the time you read the metadata and sent the update.
Object Versioning example
This example shows what happens to the file
cat.jpg in a bucket
with Object Versioning enabled as you overwrite, update, and delete the file.
- You upload a new image
When you first upload
cat.jpgto Cloud Storage, it receives a
generationnumber and a
metagenerationnumber. In this example, the generation number is
1360887697105000. Because the object is new, the
metagenerationnumbers even though Object Versioning is not enabled. You can view these numbers by using the
ls -Lcommand in gsutil. For instructions, see viewing the object metadata.
- You enable Object Versioning
At this point, you decide to enable Object Versioning for your bucket. Doing so does not affect the
- You change the metadata of the image
You update the metadata for
cat.jpgby adding custom metadata:
color:black. Updating metadata causes the
cat.jpgto increase, in this case from
2. However, the object itself remains unchanged, so Cloud Storage continues to store only one version of
cat.jpg, and the version continues to have a
- You upload a new version of the image
You upload a new version of
cat.jpgto your Cloud Storage bucket. When you do so, Object Versioning moves the existing
cat.jpgobject into an archived state. The archived version retains the same storage class and metadata it previously had. The archived version appears only if you perform a versioned listing: it does not appear in normal listing commands. The archived version is now referenced as:
Meanwhile, the newly uploaded
cat.jpgbecomes the live version of the object. This new
cat.jpggets its own
generationnumber, in this example
1360887759327000. It also gets its own metadata and a
1, which means it does not contain the
color:blackmetadata unless you specify it. When you access or modify
cat.jpg,this is the version that is used. You can alternatively refer to this version of
generationnumber. For example, when using the gsutil tool you would refer to it as
- You delete the live version of the image
You now delete
cat.jpg. When you do this, the version that had generation number
1360887759327000becomes archived. Your bucket now contains two archived versions of
cat.jpgand no live versions. You can still refer to either archived version by using its
generationnumber, but if you try to access
generationnumber, it fails.
Similarly, a normal object listing of the bucket will not show
cat.jpgas one of the objects in the bucket. For information on listing archived versions of objects, see Listing archived object versions.
- You disable Object Versioning
You disable Object Versioning, which stops future object archiving. Existing archived versions of objects remain in Cloud Storage. Even though Object Versioning is disabled,
cat.jpg#1360887759327000remain stored in your bucket until you delete them, either manually or through using Object Lifecycle Management.
- You restore one of the archived versions
Even with Object Versioning disabled, you can restore one of the existing archived versions by making a copy of it. To do so, simply name the copy you make
cat.jpg. Once you do this, your bucket has three versions of
cat.jpg: the two archived versions and the live version that came from making a copy.
This section discusses tips to help you work with Object Versioning more effectively.
- The gsutil tool has comprehensive support for working with versioned objects
that makes many tasks involving Object Versioning easier. For example, you
can work with archived versions of objects by appending
generationnumber to the object name. For more information on using gsutil with Object Versioning, see Object Versioning and Concurrency Control.
- Consider using
metagenerationnumbers for conditional updates instead of ETags. Together,
metagenerationnumbers keep track of all object updates, including metadata changes, providing a stronger guarantee than ETags.
Understand file deletion and restoration behavior
You can copy an archived object version to the current live version. See Copying archived object versions for a step-by-step guide to copying archived objects.
When you do this with Object Versioning enabled, if there already exists a live version of the object in your bucket, Cloud Storage overwrites it but also creates a new, archived version of the overwritten object. In such a case, your bucket subsequently contains the overwritten object (now archived) and two copies of the object that was previously archived (one live copy and one still-archived copy), all of which incur storage charges. To prevent unnecessary charges, delete the archived version that you used to make the current live copy.
If you send a delete request without specifying a generation, Cloud Storage archives the current live object and causes it to appear missing to subsequent version-unaware requests.
If you send a delete request with a
generationthat corresponds to the currently live object, Cloud Storage deletes the object without making an archived copy.
When you delete an archived object, Cloud Storage deletes that version of the object permanently.
Use generation-match preconditions when mutating live versions of objects
When you use generation numbers, a request succeeds as long as there is an object with that name and generation number, regardless of whether it is live or archived. If no such object exists, Cloud Storage returns
404 Not Found.
When you use generation-match preconditions, a request succeeds only if the live version of the requested object has the specified generation number. If no such object exists, or is archived, Cloud Storage returns
412 Precondition Failed.
You should avoid using a generation-match precondition at the same time as a generation number in the object name. If you use both and the numbers match, the use of the precondition is redundant. If the numbers do not match, the request always fails.
If you make several concurrent mutation requests with a generation-match precondition, Cloud Storage's strong consistency allows only one of those requests to succeed. This feature is useful if your objects are updated from several sources and you need to ensure that users don't accidentally overwrite them.
If you set a generation-match precondition to
0when uploading an object, Cloud Storage performs the specified request only if there is no live version of the object. For example, if you perform a
PUTrequest with the XML API to create a new object with the header
x-goog-if-generation-match:0, the request succeeds if the object does not exist, or if there are only archived versions of the object. If there is a live version of the object, Cloud Storage aborts the update with a status code of
412 Precondition Failed.