The page provides a quick and easy way to get started with Google Cloud Storage using the gsutil tool. It shows you how to:
- Create and delete a bucket
- Upload, move, download, and delete objects
- List your buckets and objects
- Share objects and buckets
To follow a similar exercise using the Google Cloud Platform Console, see Using the Developers Console.
Note: This exercise requires that you set up gsutil and authenticate to the Google Cloud Storage service. If you are downloading or accessing publicly-accessible objects, you do not need to activate Google Cloud Storage, turn on billing, or authenticate to the Google Cloud Storage service. Rather, you only need to install gsutil and then you can download publicly-accessible objects.
Before you start, make sure you have:
Access to Google Cloud Storage to complete these exercises.
You can access Google Cloud Storage if you were added as a project team member to an existing Google Account or if you have recently created a Google Cloud Storage account (for example, if you followed the steps in Sign Up).
The latest version of gsutil installed.
gsutil is a Python application that lets you access Google Cloud Storage. The current version of gsutil only supports Python 2.6 or Python 2.7.
First, create two buckets in your project with unique bucket names. You can do
this by using the gsutil
mb command. For example, the following command
creates buckets named
gsutil mb gs://cats gs://dogs
Note: If you are running gsutil on a Windows operating system you must
first call the Python interpreter. For example, to create two buckets named
cats and dogs you must type
c:\Python27\python \path\to\gsutil\gsutil mb gs://cats gs://dogs at the
command prompt (assuming you have Python installed at the specified location). For
tips on running Python without specifying the full path see the Python on Windows FAQ
Choose bucket names that are unique when you create these buckets because it is
likely that if you try to create buckets named
dogs, you will
receive the following error:
ServiceException: 409 Bucket example-bucket already exists.
This error means that the bucket names
dogs are already taken.
Google Cloud Storage has a single bucket namespace so your bucket names must be unique
across the entire Google Cloud Storage system. Someone else has already created
dogs, so you won't be allowed to create buckets with
those names. You'll have to modify the bucket names so that they are unique,
and you must also follow the bucket naming guidelines.
It is also possible to specify a location constraint so that your bucket
is created in a geographic location by including the
option. For more information, see Specifying Bucket Locations.
Upload objects to a bucket
Now that you have some buckets, try uploading a couple of objects to one of
them. To do this, create a new local directory, add two image files to the
directory, and name the files
collie.jpg. To upload the
files, change to the directory you just created and use the gsutil
cp command as
gsutil cp *.jpg gs://dogs
cp command behaves much like the Linux
with the recursion (
-R) option, allowing you to copy whole
directories or just the contents of directories. gsutil also supports
wildcards, which makes it easy for you to copy or move collections of files.
Depending on how you upload your objects, gsutil creates new objects with names that follow certain guidelines. Expand to find out more.
- If you use the
-Roption to recursively upload a directory, gsutil creates your object names based on the directory where it starts recursive processing.
For example, if you have a directory located at
client1/low_priority/and you recursively copy the directory using
gsutil cp -R client1/low_priority gs://dst_uri, gsutil creates new objects with
low_priorityin the object name, because
low_priorityis the directory where the recursive processing begins:
-Roption also copies subdirectories and includes subdirectory names in the object name. For example, if
client1/low_prioritycontained another directory named datasheets,
gsutil cp -R client1/low_priority gs://dst_uriwill create objects named:
gs://dst_uri/low_priority/building.cad gs://dst_uri/low_priority/schematics.cad gs://dst_uri/low_priority/datasheets/assets.txt gs://dst_uri/low_priority/datasheets/expenses.txt
This can be useful for organizational purposes but if you aren't careful, you could also embed unintended paths in file names (for example, embedding
low_priorityfor a customer to see in the example above).
- If you need to copy or upload a directory without prepending the directory
name, you can use wildcards.
For example, if you copy files using
gsutil cp client1/low_priority/* gs://dst_uri, gsutil copies the objects in the directory without prepending the directory name:
List the buckets and objects
Next, list the buckets and objects that you created. You can use the gsutil
command to do this. For example, the following command lists all buckets
in your default project:
The following command lists the objects that are stored in the bucket
gsutil ls gs://dogs
To lists all the objects and their sizes inside the
dogs bucket, as well as
the total size of all objects:
gsutil ls -l gs://dogs
To provide a complete list of all your objects and their sizes in all buckets in your default project, as well the total size of all these objects:
gsutil ls -l gs://*
Note: You can also use the
-L option to provide more detailed information
about your objects and buckets. However, this option uses more
bandwidth to retrieve the information it needs.
For more information, see the ls command.
Move an object
You can move objects from one bucket to another by using the gsutil
To move all of the files that are in the
dogs bucket to the
use the following command:
gsutil mv gs://dogs/*.jpg gs://cats/
mv command can also be used to rename an object.
poodle.jpg, which you just moved to the
you use the following command:
gsutil mv gs://cats/poodle.jpg gs://cats/siamese.jpg
Download an object
You can download objects by using the gsutil
cp command. To download
the objects that are in the
cats bucket and save them in an existing
local directory named
pets, use the following command:
gsutil cp gs://cats/*.jpg pets/
Sharing objects and buckets
Access Control Lists (ACLs) define who has permission to access
an object or bucket. Google Cloud Storage lets you grant permissions for a wide
range of entities, such as individual users, Google Apps domains, and
Google groups. You can share objects and buckets with other people by
using the gsutil
acl ch command to modify the ACLs that are applied to objects
or buckets. For example, using the following command you can let Jane download
siamese.jpg object and you can let everyone who is a member of the
group email@example.com download the
gsutil acl ch -g firstname.lastname@example.org:R -u email@example.com:R gs://cats/siamese.jpg
If you simply want to apply a predefined ACL (also known as a canned ACL) to a bucket or object, you can use the following command:
gsutil acl set bucket-owner-full-control gs://cats/siamese.jpg
To learn more about sharing data and modifying ACLs (including the available predefined ACLs), see Access Control.
Delete buckets and objects
You can delete objects with the gsutil
rm command, and you can delete
buckets with the gsutil
rb command. For example, the following command
collie.jpg object from the
gsutil rm gs://cats/collie.jpg
The following command deletes the
gsutil rb gs://dogs
You must remove all objects from a bucket before you delete it. You can do
this by using the gsutil
rm command or by using the gsutil
command to move objects from one bucket to another (like you did when you
moved the objects from the dogs bucket to the cats bucket).