Discover object storage with the gsutil tool

Discover object storage with the gsutil tool

Important: gsutil is a set of legacy CLI commands used for interacting with Cloud Storage. New users should use gcloud storage commands instead of gsutil commands.

This page shows you how to perform basic tasks in Cloud Storage using the gsutil command-line tool, which is installed as part of the Google Cloud CLI.

Costs that you incur in Cloud Storage are based on the resources you use. This quickstart typically uses less than $0.01 USD worth of Cloud Storage resources.

Before you begin

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
2. Install the Google Cloud CLI.

3. To initialize the gcloud CLI, run the following command:

   gcloud init

4. Create a Google Cloud project.

   gcloud projects create PROJECT_ID

5. Make sure that billing is enabled for your Google Cloud project.

6. Install the Google Cloud CLI.

7. To initialize the gcloud CLI, run the following command:

   gcloud init

8. Create a Google Cloud project.

   gcloud projects create PROJECT_ID

9. Make sure that billing is enabled for your Google Cloud project.

10. If you're using a local development environment, install Python 3.8.

    If you are using Windows and you left the relevant checkbox selected when you installed the gcloud CLI, this was done automatically.

Note: If you are using Cloud Shell or a Compute Engine instance, Python and gsutil are pre-installed and authenticated.

Note: If you use Windows and did not install gsutil as part of the gcloud CLI, you need to preface each gsutil command with python (for example, python gsutil mb gs://my-awesome-bucket). For more information, see the Python on Windows FAQ.

Create a bucket

Buckets are the basic containers that hold your data in Cloud Storage.

To create a bucket:

1. Open a terminal window.

2. Use the gsutil mb command and a unique name to create a bucket:

   gsutil mb -b on -l us-east1 gs://my-awesome-bucket/

   This uses a bucket named "my-awesome-bucket." You must choose your own, globally-unique, bucket name.

   See bucket naming requirements.

   • Bucket names can only contain lowercase letters, numeric characters, dashes (-), underscores (_), and dots (.). Spaces are not allowed. Names containing dots require verification.
   • Bucket names must start and end with a number or letter.
   • Bucket names must contain 3-63 characters. Names containing dots can contain up to 222 characters, but each dot-separated component can be no longer than 63 characters.
   • Bucket names cannot be represented as an IP address in dotted-decimal notation (for example, 192.168.5.4).
   • Bucket names cannot begin with the "goog" prefix.
   • Bucket names cannot contain "google" or close misspellings, such as "g00gle".
   Caution: Do not include sensitive information in the bucket name, since the bucket namespace is global and publicly visible.

   If successful, the command returns:

   Creating gs://my-awesome-bucket/...

You've just created a bucket where you can store your data!

Note: If the bucket name you chose is already in use, either by you or someone else, the command returns:

Creating gs://my-awesome-bucket/...
ServiceException: 409 Bucket my-awesome-bucket already exists.

Try again with a different bucket name.

Upload an object into your bucket

1. Right-click the image above and save it somewhere on your computer, such as on the desktop.

   If you are using Cloud Shell or a Compute Engine instance, download the image using the following command: wget https://cloud.google.com/storage/images/kitten.png

2. Use the gsutil cp command to copy the image from the location where you saved it to the bucket you created:

   gsutil cp Desktop/kitten.png gs://my-awesome-bucket

   If successful, the command returns:

   Copying file://Desktop/kitten.png [Content-Type=image/png]...
   Uploading   gs://my-awesome-bucket/kitten.png:       0 B/164.3 KiB
   Uploading   gs://my-awesome-bucket/kitten.png:       164.3 KiB/164.3 KiB

   You've just stored an object in your bucket.

Note: When typing your bucket name, you can use the tab key to autocomplete it.

Download the object from your bucket

1. Use the gsutil cp command to download the image you stored in your bucket to somewhere on your computer, such as the desktop:

   gsutil cp gs://my-awesome-bucket/kitten.png Desktop/kitten2.png

   If successful, the command returns:

   Copying gs://my-awesome-bucket/kitten.png...
   Downloading file://Desktop/kitten2.png:               0 B/164.3 KiB
   Downloading file://Desktop/kitten2.png:               164.3 KiB/164.3 KiB

   You've just downloaded something from your bucket.

Copy the object to a folder in the bucket

1. Use the gsutil cp command to create a folder and copy the image into it:

   gsutil cp gs://my-awesome-bucket/kitten.png gs://my-awesome-bucket/just-a-folder/kitten3.png

   Note: Folders in Cloud Storage have limitations compared to local filesystems, but many of the same operations are supported.

   If successful, the command returns:

   Copying gs://my-awesome-bucket/kitten.png [Content-Type=image/png]...
   Copying     ...my-awesome-bucket/just-a-folder/kitten3.png: 164.3 KiB/164.3 KiB

   You've just copied your image into a new folder in your bucket.

List contents of a bucket or folder

1. Use the gsutil ls command to list the contents at the top level of your bucket:

   gsutil ls gs://my-awesome-bucket

   If successful, the command returns a message similar to:

   gs://my-awesome-bucket/kitten.png
   gs://my-awesome-bucket/just-a-folder/

   You've just seen the contents at the top level of your bucket.

List details for an object

1. Use the gsutil ls command with the -l flag to get some details about the image:

   gsutil ls -l gs://my-awesome-bucket/kitten.png

   If successful, the command returns a message similar to:

   2638  2016-02-26T23:05:14Z  gs://my-awesome-bucket/kitten.png
   TOTAL: 1 objects, 168243.2 bytes (164.3 KiB)

   You've just obtained information about the image's size and date of creation.

Make the objects publicly accessible

1. Use the gsutil iam ch command to grant all users permission to read the images stored in your bucket:

   gsutil iam ch allUsers:objectViewer gs://my-awesome-bucket

   The command is successful if no error is returned.

   Now anyone can get your images.

2. To remove this access, use the command:

   gsutil iam ch -d allUsers:objectViewer gs://my-awesome-bucket

   The command is successful if no error is returned.

   You have removed public access to the images in your bucket.

Note: After you remove public access, you may still be able to access a cached version of one of your images for a period of time.

Give someone access to your bucket

1. Use the gsutil iam ch command to give a specific email address permission to read and write objects in your bucket:

   gsutil iam ch user:[e-mail]:objectCreator,objectViewer gs://my-awesome-bucket

   The command is successful if no error is returned.

   Now someone else can put things into and view what's in your bucket.

2. To remove this permission, use the command:

   gsutil iam ch -d user:[e-mail]:objectCreator,objectViewer gs://my-awesome-bucket

   The command is successful if no error is returned.

   You have removed the user's access to this bucket.

Delete an object

1. Use the gsutil rm command to delete one of your images:

   gsutil rm gs://my-awesome-bucket/kitten.png

   If successful, the command returns:

   Removing gs://my-awesome-bucket/kitten.png...

   This copy of the image is no longer stored on Cloud Storage (though the copy you made in the folder just-a-folder/ still exists).

Clean up

To avoid incurring charges to your Google Cloud account for the resources used on this page, delete the Google Cloud project with the resources.

1. Open a terminal window (if not already open).

2. Use the gsutil rm command with the -r flag to delete the bucket and anything inside of it:

   gsutil rm -r gs://my-awesome-bucket

   If successful, the command returns a message similar to:

   Removing gs://my-awesome-bucket/just-a-folder/cloud-storage.logo.png#1456530077282000...
   Removing gs://my-awesome-bucket/...

   Your bucket and its contents are deleted.

What's next

• Work through Cloud Storage Quickstart using the Google Cloud console.
• Read the Cloud Storage product overview.
• Learn about Cloud Storage pricing.
• See the reference pages for gsutil commands, such as make bucket (mb), copy (cp), list (ls), identity access management (iam), and remove (rm).