About Cloud Storage objects

This page describes objects, a resource in Cloud Storage. For a general overview of how Cloud Storage works, see the Cloud Storage product overview.

Objects

Objects are the individual pieces of data that you store in Cloud Storage.

  • There is no limit on the number of objects that you can create in a bucket.

Objects have two components: object data and object metadata. Object data is typically a file that you want to store in Cloud Storage and is completely opaque to Cloud Storage. Object metadata is a collection of name-value pairs that describe various object qualities.

Two important pieces of object metadata common to all objects are the object's name and its generation number. When you add an object to a Cloud Storage bucket, you specify the object name and Cloud Storage assigns the generation number. Together, the name and generation uniquely identify the object within that bucket.

Object names

The name you give to an object must meet the following requirements:

  • Object names can contain any sequence of valid Unicode characters, of length 1-1024 bytes when UTF-8 encoded.
  • Object names cannot contain Carriage Return or Line Feed characters.
  • Object names cannot start with .well-known/acme-challenge/.
  • Objects cannot be named . or ...

It is strongly recommended that you avoid the following in your object names:

  • Control characters that are illegal in XML 1.0 (#x7F–#x84 and #x86–#x9F): these characters cause XML listing issues when you try to list your objects.
  • The # character: Google Cloud CLI commands interpret object names ending with #<numeric string> as version identifiers, so including # in object names can make it difficult or impossible to perform operations on such versioned objects using the gcloud CLI.
  • The [, ], *, or ? characters: gcloud storage and gsutil interpret these characters as wildcards, so including them in object names can make it difficult or impossible to perform wildcard operations with those tools. Additionally, * and ? are not valid characters for file names in Windows.
  • The :, ", <, >, or | characters: These are not valid characters for file names in Windows, so attempts to download an object that uses such characters in its name to a Windows file fail unless the download method includes renaming the resulting Windows file. The / character, while also not a valid character for file names in Windows, is typically OK to use in object names for mimicking a directory structure; tools such as gcloud storage and gsutil automatically convert the character to `` when downloading to a Windows environment.
  • Sensitive or personally identifiable information (PII): object names are more broadly visible than object data. For example, object names appear in URLs for the object and when listing objects in a bucket.

Existing objects cannot be directly renamed, but you can indirectly rename an object by copying and deleting the original object.

Object namespace

Object names reside in a flat namespace within a bucket. This means that:

  • Different buckets can have objects with the same name.
  • Objects do not reside within subdirectories in a bucket.

For convenience, tools such as Google Cloud console and the Google Cloud CLI work with objects that use the slash (/) character as if they were stored in a virtual hierarchy.

For example, you can name an object /europe/france/paris.jpg to make it appear that paris.jpg resides in the subdirectory /europe/france, but to Cloud Storage, the object simply exists in the bucket and has the name /europe/france/paris.jpg. When using the Google Cloud console, you can then navigate to these objects as if they were in a hierarchical directory structure under the folders europe and france.

Note that because objects reside in a flat namespace, deeply nested, directory-like structures don't have the performance that a native filesystem has when listing deeply nested sub-directories.

See Request rate best practices for recommendations on how to optimize performance by avoiding sequential names during large-scale uploads. Objects uploaded with sequential names are likely to reach the same backend server and constrain performance.

Object immutability

Objects are immutable, which means that an uploaded object cannot change throughout its storage lifetime. An object's storage lifetime is the time between successful object creation, such as uploading, and successful object deletion. In practice, this means that you cannot make incremental changes to objects, such as append operations or truncate operations. However, it is possible to replace objects that are stored in Cloud Storage, and doing so happens atomically: until the new upload completes, the old version of the object is served to readers, and after the upload completes the new version of the object is served to readers. So a single replacement operation simply marks the end of one immutable object's lifetime and the beginning of a new immutable object's lifetime.

The generation number for an object changes each time you replace the object's data. Thus, the generation number uniquely identifies an immutable object.

Note that there is a once-per-second limit for rapidly replacing the same object. Replacing the same object more frequently might result in 429 Too Many Requests errors. You should design your application to upload data for a particular object no more than once per second and handle occasional 429 Too Many Requests errors using an exponential backoff retry strategy.

What's next