Cloud Storage consistency

This page explains which Cloud Storage operations are strongly consistent and which are eventually consistent. In the case of cacheable, publicly readable objects, you control the degree to which operations on the objects are consistent.

Strongly consistent operations

Cloud Storage provides strong global consistency for the following operations:

  • Bucket listing
  • Bucket read-after-create
  • Bucket read-after-metadata-update
  • Bucket read-after-delete
  • Object read-after-write
  • Object read-after-metadata-update
  • Object read-after-delete
  • Object listing

When you write an object to Cloud Storage, such as when you upload, compose, or copy an object, the object is immediately available for reading and metadata operations as soon as you receive a success response to your write request. This is true for all buckets and for all storage classes, and this applies to both creating new objects and replacing existing objects. Cloud Storage also offers read-after-create, read-after-metadata-update, read-after-delete, and listing consistency for resources such as folders and managed folders.

Because writes are strongly consistent, you never receive a 404 Not Found response or stale data for an object read-after-write or object read-after-metadata-update operation, even for buckets located in dual-regions or multi-regions. In the rare event where your data has not yet been replicated across regions, but the location your object was first written to becomes unavailable, attempts to access the object return a retryable 500 error response.

Strong global consistency also extends to deletion operations on objects. If a deletion request succeeds, an immediate attempt to download the object or its metadata results in a 404 Not Found status code. You get the 404 error because the object no longer exists after the delete operation succeeds.

Bucket listing and object listing are strongly consistent: when you create a bucket or object and then immediately perform the relevant list operation, the newly created bucket or object appears in the returned list.

For buckets, while metadata updates are strongly consistent for read-after-metadata-update operations, the resulting configuration changes may take time to propagate. For example, if you enable object versioning on a bucket, you should wait at least 30 seconds before deleting or replacing objects.

Similarly for HMAC keys, there is a delay of up to 3 minutes between when you request to change the key state and when the state change takes effect. For example, if you disable an HMAC key, you should wait at least 3 minutes before making a request to delete the key, because attempts to do so in the first 3 minutes could fail.

Eventually consistent operations

The following operations are eventually consistent:

  • Granting access to or revoking access from resources.
  • Recreating buckets after deletion.

It typically takes about a minute for these operations to take effect. In some cases, it might take several minutes longer.

As an example of behavior that can arise from eventual consistency, if you remove a user's access to a bucket, this change is immediately reflected in the metadata for the bucket; however, the user may still have access to the bucket for a short period of time.

Buckets that are recreated after deletion might take several minutes to become accessible.

Cache control and consistency

Cached objects that are publicly readable might not exhibit strong consistency. If you allow an object to be cached, and the object is in the cache when it is updated or deleted, the cached object is not updated or deleted until its cache lifetime expires.

The cache lifetime of an object is defined by the Cache-Control metadata associated with the object. The Cache-Control metadata can be set using a Cache-Control request header included in the initial upload of the object, or in a subsequent update to the metadata of the object. Because you control the Cache-Control metadata, you also control the degree to which cached objects are consistent. Moreover, while requests for the object can include their own Cache-Control header, these headers are ignored by Cloud Storage if they're set to avoid cached content.

Atomic operations

Cloud Storage provides atomicity guarantees for most operations involving individual objects, such as uploading an object (including overwriting an existing object), updating an object's metadata, replacing an object, and deleting an object.

Batch requests aren't guaranteed to be atomic. While single operations within a batch request can be atomic, all the operations within the batch request cannot be guaranteed to be atomic as a group because it's possible for some of the operations to succeed while others fail.

In some situations, you might end up fetching an older version of an object, such as when cached data becomes stale or when you perform ranged reads without specifying a generation number, where the object you intend to fetch becomes overwritten. As a best practice, use preconditions to make sure you're fetching the correct object version.

What's next