To use Google Cloud Storage effectively, you need to understand some of the concepts on which it is built. These concepts define how your data is stored in Google Cloud Storage.
All data in Google Cloud Storage belongs inside a project. A project consists of a set of users, a set of APIs, and billing, authentication, and monitoring settings for those APIs. You can have one project or multiple projects.
Buckets are the basic containers that hold your data. Everything that you store in Google Cloud Storage must be contained in a bucket. You can use buckets to organize your data and control access to your data, but unlike directories and folders, you cannot nest buckets. There is a per-project rate limit on bucket create/delete operations of approximately 1 operation every 2 seconds; however there is no rate limit on object create/delete operations. Therefore, we recommend that you design your storage applications to favor intensive object operations and relatively few buckets operations.
When you create a bucket, you can specify a name, default storage class, and geographic location for the bucket. The storage class you choose will determine the availability, throughput, and storage cost of objects uploaded to the bucket. After you create a bucket, you can still change its default storage class to any class that is allowable, given the bucket's location; however, the bucket name and location can only be changed by deleting and re-creating the bucket.
Bucket names have more restrictions than object names, because every bucket resides in a single Google Cloud Storage namespace. Also, bucket names can be used with a CNAME redirect, which means they need to conform to DNS naming conventions. For more information, see the bucket naming guidelines.
Bucket labels are key:value metadata pairs that allow you to group
your buckets along with other Google Cloud Platform resources such as
virtual machine instances and persistent disks. For example, you can
use labels to create a
team key that has values
delta, and apply the
to different buckets in order to indicate which team is associated with those
You can apply multiple labels to each bucket, with a maximum of 64 labels per bucket.
- Keys and values cannot be longer than 63 characters each.
- Keys and values can only contain lowercase letters, numeric characters, underscores, and dashes. International characters are allowed.
- Label keys must start with a lowercase letter and international characters are allowed.
- Label keys cannot be empty.
Objects are the individual pieces of data that you store in Google Cloud Storage. A single object can be up to 5 TB in size. Objects have two components: object data and object metadata. The object data component is usually a file that you want to store in Google Cloud Storage. The object metadata component is a collection of name-value pairs that describe various object qualities. There is no limit on the number of objects that you can create in a bucket.
An object name is just metadata to Google Cloud Storage. Object names can contain any
combination of Unicode characters (UTF-8 encoded) less than 1024 bytes in
length. A common character to include in file names is a slash (/). By using
slashes in an object name, you can make objects appear as though they're
stored in a hierarchical structure. For example, you could name one object
/europe/france/paris.jpg and another object
When you list these objects, they appear to be in a hierarchical directory
structure based on location; however, Google Cloud Storage sees the objects as
independent objects with no hierarchical relationship whatsoever.
For more information, including how to rename an object, see the object naming guidelines.
A generation number is a number associated with each object. Each time the object is overwritten, the generation number for the object changes. When your bucket contains multiple versions of the same object, such as when Object Versioning is enabled, you can use the generation number to refer to specific versions.
A resource is an entity within Google Cloud Platform. Each project, bucket, and object in Google Cloud Platform is a resource, as are things such as Google Compute Engine instances.
Each resource has a unique name that identifies it, much like a filename. Buckets
have a resource name in the form of
[BUCKET_NAME] is the ID of the bucket. Objects have a resource name in the
[OBJECT_NAME] is the ID of the object.
#[NUMBER] appended to the end of the resource name indicates a specific
generation of the object.
#0 is a special identifier for the most recent
version of an object.
#0 is a useful to add when the name of the object ends
in a string that would otherwise be interpreted as a generation number.
An object's data component is completely opaque to Google Cloud Storage. It is just a chunk of data to Google Cloud Storage.
Objects are immutable, which means that an uploaded object cannot change throughout its storage lifetime. An object's storage lifetime is the time between successful object creation (upload) and successful object deletion. In practice, this means that you cannot make incremental changes to objects, such as append operations or truncate operations. However, it is possible to overwrite objects that are stored in Google Cloud Storage, and doing so happens atomically — until the new upload completes the old version of the object will be served to readers, and after the upload completes the new version of the object will be served to readers. So a single overwrite operation simply marks the end of one immutable object's lifetime and the beginning of a new immutable object's lifetime.
There is no limit to how quickly you can create or update different objects
in a bucket. However, a single particular object can only be updated or
overwritten up to once per second. For example, if you have an object
foo, then you should only upload a new copy of
once per second. Updating the same object faster than once per second may
503 Service Unavailable errors.
You should retry failed requests using truncated exponential backoff.
Google Cloud Storage uses a flat namespace to store objects. However, some tools (for example, Google Cloud Platform Console and gsutil) can work with objects as if they are stored in a virtual hierarchy, as a convenience.
There is only one Google Cloud Storage namespace, which means every bucket must have a unique name across the entire Google Cloud Storage namespace. Object names must be unique only within a given bucket.