Key Terms

To use Google Cloud Storage effectively, you need to understand some of the concepts on which it is built. These concepts define how your data is stored in Google Cloud Storage.

Projects

All data in Google Cloud Storage belongs inside a project. A project consists of a set of users, a set of APIs, and billing, authentication, and monitoring settings for those APIs. You can have one project or multiple projects.

Buckets

Buckets are the basic containers that hold your data. Everything that you store in Google Cloud Storage must be contained in a bucket. You can use buckets to organize your data and control access to your data, but unlike directories and folders, you cannot nest buckets. There is a per-project rate limit on bucket create/delete operations of approximately 1 operation every 2 seconds; however there is no rate limit on object create/delete operations. Therefore, we recommend that you design your storage applications to favor intensive object operations and relatively few buckets operations.

When you create a bucket, you can specify a name, default storage class, and geographic location for the bucket. The storage class you choose will determine the availability, throughput, and storage cost of objects uploaded to the bucket. After you create a bucket, you can still change its default storage class to any class that is allowable, given the bucket's location; however, the bucket name and location can only be changed by deleting and re-creating the bucket.

Bucket names

Bucket names have more restrictions than object names, because every bucket resides in a single Google Cloud Storage namespace. Also, bucket names can be used with a CNAME redirect, which means they need to conform to DNS naming conventions. For more information, see the bucket naming guidelines.

Bucket labels

Bucket labels are key:value metadata pairs that allow you to group your buckets along with other Google Cloud Platform resources such as virtual machine instances and persistent disks. For example, you can use labels to create a team key that has values alpha, beta, and delta, and apply the team:alpha, team:beta, and team:delta labels to different buckets in order to indicate which team is associated with those buckets.

You can apply multiple labels to each bucket, with a maximum of 64 labels per bucket.

Objects

Objects are the individual pieces of data that you store in Google Cloud Storage. A single object can be up to 5 TB in size. Objects have two components: object data and object metadata. The object data component is usually a file that you want to store in Google Cloud Storage. The object metadata component is a collection of name-value pairs that describe various object qualities. There is no limit on the number of objects that you can create in a bucket.

Object names

An object name is just metadata to Google Cloud Storage. Object names can contain any combination of Unicode characters (UTF-8 encoded) less than 1024 bytes in length. A common character to include in file names is a slash (/). By using slashes in an object name, you can make objects appear as though they're stored in a hierarchical structure. For example, you could name one object /europe/france/paris.jpg and another object /europe/france/cannes.jpg. When you list these objects, they appear to be in a hierarchical directory structure based on location; however, Google Cloud Storage sees the objects as independent objects with no hierarchical relationship whatsoever.

For more information, including how to rename an object, see the object naming guidelines.

Generation numbers

A generation number is a number associated with each object. Each time the object is overwritten, the generation number for the object changes. When your bucket contains multiple versions of the same object, such as when Object Versioning is enabled, you can use the generation number to refer to specific versions.

Resources

A resource is an entity within Google Cloud Platform. Each project, bucket, and object in Google Cloud Platform is a resource, as are things such as Google Compute Engine instances.

Resource names

Each resource has a unique name that identifies it, much like a filename. Buckets have a resource name in the form of projects/_/buckets/[BUCKET_NAME], where [BUCKET_NAME] is the ID of the bucket. Objects have a resource name in the form of projects/_/buckets/[BUCKET_NAME]/objects/[OBJECT_NAME], where [OBJECT_NAME] is the ID of the object.

A #[NUMBER] appended to the end of the resource name indicates a specific generation of the object. #0 is a special identifier for the most recent version of an object. #0 is a useful to add when the name of the object ends in a string that would otherwise be interpreted as a generation number.

Data opacity

An object's data component is completely opaque to Google Cloud Storage. It is just a chunk of data to Google Cloud Storage.

Object immutability

Objects are immutable, which means that an uploaded object cannot change throughout its storage lifetime. An object's storage lifetime is the time between successful object creation (upload) and successful object deletion. In practice, this means that you cannot make incremental changes to objects, such as append operations or truncate operations. However, it is possible to overwrite objects that are stored in Google Cloud Storage, and doing so happens atomically — until the new upload completes the old version of the object will be served to readers, and after the upload completes the new version of the object will be served to readers. So a single overwrite operation simply marks the end of one immutable object's lifetime and the beginning of a new immutable object's lifetime.

There is no limit to how quickly you can create or update different objects in a bucket. However, a single particular object can only be updated or overwritten up to once per second. For example, if you have an object bar in bucket foo, then you should only upload a new copy of foo/bar about once per second. Updating the same object faster than once per second may result in 503 Service Unavailable errors.

You should retry failed requests using truncated exponential backoff.

Hierarchy

Google Cloud Storage uses a flat namespace to store objects. However, some tools (for example, Google Cloud Platform Console and gsutil) can work with objects as if they are stored in a virtual hierarchy, as a convenience.

Namespace

There is only one Google Cloud Storage namespace, which means every bucket must have a unique name across the entire Google Cloud Storage namespace. Object names must be unique only within a given bucket.

Send feedback about...

Cloud Storage Documentation