Product overview of Cloud Storage

This page provides an overview of Cloud Storage and how it works.

Cloud Storage is a service for storing your objects in Google Cloud. An object is an immutable piece of data consisting of a file of any format. You store objects in containers called buckets.

All buckets are associated with a project, and you can group your projects under an organization. Each project, bucket, managed folder, and object in Google Cloud is a resource in Google Cloud, as are things such as Compute Engine instances.

After you create a project, you can create Cloud Storage buckets, upload objects to your buckets, and download objects from your buckets. You can also grant permissions to make your data accessible to principals you specify or accessible to everyone on the public internet.

The Google Cloud hierarchy

Here's how the Cloud Storage structure can apply to a real-world case:

  • Organization: Your company, called Example Inc., creates a Google Cloud organization called exampleinc.org.

  • Project: Example Inc. is building several applications, and each one is associated with a project. Each project has its own set of Cloud Storage APIs, as well as other resources.

  • Bucket: Each project can contain multiple buckets, which are containers to store your objects. For example, you might create a photos bucket for all the image files your app generates and a separate videos bucket.

  • Object: Each bucket can contain essentially unlimited individual objects, such as an image called puppy.png.

  • Managed folder: Each bucket can also contain multiple managed folders, which grant additional access beyond the access granted to the overall bucket. This additional access applies to objects that have a prefix matching the managed folder name. For example, you might create an animals/ managed folder, which gives designated users additional access to objects such as animals/puppy.png and animals/kitten.png.

  • Hierarchical namespace enabled buckets: Buckets can have hierarchical namespace enabled. Buckets with hierarchical namespace enabled can store your data in a logical file system structure. The folder management operations provide reliability and management capabilities, including creating, deleting, listing, and renaming folders. Hierarchical namespace is beneficial for data-intensive tasks like analytics, AI, and ML workloads as they offer up to 8 times higher initial QPS limits for reading and writing objects compared to buckets without hierarchical namespace enabled.

Basic tools for Cloud Storage

Here are some basic ways you can interact with Cloud Storage:

  • Console: The Google Cloud console provides a visual interface for you to manage your data in a browser.

  • Google Cloud CLI: The gcloud CLI allows you to interact with Cloud Storage through a terminal using gcloud storage commands.

  • Client libraries: The Cloud Storage client libraries allow you to manage your data using one of your preferred languages, including C++, C#, Go, Java, Node.js, PHP, Python, and Ruby.

  • REST APIs: Manage your data using the JSON or XML API.

  • Terraform: Terraform is an infrastructure-as-code (IaC) tool that you can use to provision the infrastructure for Cloud Storage.

  • Cloud Storage FUSE: Cloud Storage FUSE lets you mount Cloud Storage buckets to your local file system. This enables your applications to read from a bucket or write to a bucket by using standard file system semantics.

  • gRPC: gRPC lets you interact with Cloud Storage. gRPC is a high performance, open source universal RPC framework developed by Google that you can use to define your services using Protocol Buffers.

Securing your data

Once you upload your objects to Cloud Storage, you have fine-grained control over how you secure and share your data. Here are some ways to secure the data you upload to Cloud Storage:

  • Identity and Access Management: Use IAM to control who has access to the resources in your Google Cloud project. Resources include Cloud Storage buckets and objects, as well as other Google Cloud entities such as Compute Engine instances. You can grant principals certain types of access to buckets and objects, such as update, create, or delete.

  • Data encryption: Cloud Storage uses server-side encryption to encrypt your data by default. You can also use supplemental data encryption options such as customer-managed encryption keys and customer-supplied encryption keys.

  • Authentication: Ensure that anyone who accesses your data has proper credentials.

  • Bucket Lock: Govern how long objects in buckets must be retained by specifying a retention policy.

  • Soft delete: Prevent permanent loss of data against accidental or malicious deletion by retaining recently deleted objects and buckets. By default, Cloud Storage enables soft delete for all buckets with a seven day retention period.

  • Object Versioning: When a live version of an object is replaced or deleted, it can be retained as a noncurrent version if you enable Object Versioning.

  • Bucket IP filtering: With bucket IP filtering, you can restrict access to a bucket based on the source IP address of the request and secure your data from unauthorized access from specific IP addresses or Virtual Private Cloud (VPC).

Resource names

Each resource has a unique name that identifies it, much like a filename. Buckets have a resource name in the form of projects/_/buckets/BUCKET_NAME, where BUCKET_NAME is the ID of the bucket. Objects have a resource name in the form of projects/_/buckets/BUCKET_NAME/objects/OBJECT_NAME, where OBJECT_NAME is the ID of the object.

A #NUMBER appended to the end of the resource name indicates a specific generation of the object. #0 is a special identifier for the most recent version of an object. #0 is useful to add when the name of the object ends in a string that would otherwise be interpreted as a generation number.

Quickstart guides

To learn the fundamentals of using Cloud Storage, visit the following guides:

What's next