Google Cloud Platform
Cloud Storage

Install gsutil

This page describes the installation and setup of gsutil, a tool that enables you to access Google Cloud Storage from the command-line.

Contents

Simply installing gsutil gives you immediate read and/or write access to public data. Authenticating to the Cloud Storage service gives you read and/or write access to protected data that has been shared with you. Enabling billing gives you the ability to create and manage your own buckets.

System requirements

The gsutil tool runs on Linux/Unix, Mac OS, and Windows (XP or later). If you plan to use composite objects, you need to install compiled crcmod, which is only available for 32-bit Python on Windows.

Installing gsutil

The officially supported installation and update method for gsutil is as part of the Google Cloud SDK.

Installing gsutil as part of the Google Cloud SDK

Follow the instructions for your operating system to install gsutil as a part of the Google Cloud SDK:

Linux
  1. Enter the following at a command prompt:
    $ curl https://sdk.cloud.google.com | bash
  2. Restart your shell:
    $ exec -l $SHELL
  3. Run gcloud init to initialize the gcloud environment:
    $ gcloud init
Debian/Ubuntu

The Cloud SDK is available in package format for installation on Debian and Ubuntu systems. This package contains the gcloud, gsutil and bq commands only.

You can install the Cloud SDK from a Debian/Ubuntu package for all official Ubuntu releases that have not reached end of life, and Debian stable releases from Wheezy forward.

The SDK deb package does not include kubectl or the App Engine extensions required to deploy an application using gcloud commands. If you want these components, you must install them separately.

# Create an environment variable for the correct distribution
export CLOUD_SDK_REPO="cloud-sdk-$(lsb_release -c -s)"

# Add the Cloud SDK distribution URI as a package source
echo "deb http://packages.cloud.google.com/apt $CLOUD_SDK_REPO main" | sudo tee /etc/apt/sources.list.d/google-cloud-sdk.list

# Import the Google Cloud public key
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

# Update and install the Cloud SDK
sudo apt-get update && sudo apt-get install google-cloud-sdk

# Run gcloud init to get started
gcloud init
Mac OS X
  1. Enter the following at a command prompt:
    $ curl https://sdk.cloud.google.com | bash
  2. Restart your shell:
    $ exec -l $SHELL
  3. Run gcloud init to initialize the gcloud environment:
    $ gcloud init
Windows
  1. Download the Google Cloud SDK installer. The installer is signed by Google Inc.

  2. Launch the installer and follow the prompts. If Python 2.7 is not installed on your system, make sure the option to install Bundled Python is checked.

  3. After installation has completed, accept the following options:

    • Start Google Cloud SDK Shell
    • Run gcloud init

  4. The installer starts a terminal window and runs the gcloud init command.

Note: On Google Compute Engine instances running Windows Server, Internet Explorer is set for Enhanced Security Configuration. This setting prevents you from running the installer above. Disable Enhanced Security Configuration first, restart Internet Explorer and then run the installer. For instructions on disabling Enhanced Security on your Windows Server Instance, open Windows Help and Support and search for "IE ESC".

Perform updates with the components update command: gcloud components update.

To learn how to use gsutil, see Quickstart: Using the gsutil Tool exercise, or run gsutil help.

Alternative installation methods

There are several ways to install gsutil as a stand-alone product. You may prefer one of these methods if you do not want any of the other components that come with the Cloud SDK, or if you are managing packages with PyPI.

If you are installing gsutil as a standalone, an additional system requirement is that you must have Python 2.6.x or 2.7.x installed on your computer. gsutil does not currently run on Python 3.x. Python is installed by default on most distributions of Linux and Mac OS X. Python is not installed by default on Windows; you must install Python before you can run gsutil on Windows.

Expand the desired method for instructions.

Setting Up Credentials to Access Protected Data

Installing gsutil allows you to download and access publicly-accessible objects, but in order to access protected data or write to a protected bucket, you need to set up credentials (authenticate). For example, if someone else has created a Cloud Storage account and uploaded data that is only accessible to you or other specific individuals, you must set up your credentials to the Cloud Storage service to be able to access this data.

If you installed gsutil as part of the Google Cloud SDK, follow these instructions. Otherwise, see the Authenticate stand-alone gsutil section.

Authenticate Google Cloud SDK

When using gsutil as part of the Google Cloud SDK, OAuth2 is used to authenticate and authorize access.

  1. Open a command prompt instance.

  2. Run gcloud init at the command prompt.

    You receive the following output:

    Welcome! This command will take you through the configuration of gcloud.
     
    Your current configuration has been set to: [default]
     
    To continue, you must login. Would you like to login (Y/n)?
    

  3. Type "Y" and press enter to login.

    • The command prints a URL and tries to open a browser window to request access to your project. If a browser window can be opened, you receive the following output:

      Your browser has been opened to visit:
       
      https://accounts.google.com/o/oauth2/auth?redirect_uri=http%3A%2F%2F...
      

      Go to the next step.

    • If you are working on a local machine and your browser doesn't automatically load the URL, retry the gcloud init command with the --console-only flag:

      gcloud init --console-only
      

      If you use this command, or if the Cloud SDK detects that a browser can not be opened (such as if you are working on a remote machine), you receive the following output:

      Go to the following link in your browser:
       
      https://accounts.google.com/o/oauth2/auth?redirect_uri=urn%3Aietf%3Awg%3A...
       
      Enter verification code:
      

      Open a browser and navigate to the URL provided.

  4. If prompted, sign in with the Google account associated with your Cloud Storage data in the browser window.

  5. Grant access.

    • If the browser window opened automatically, review the application permissions and click Accept when you are ready. The verification code is then automatically sent to the command line tool.

    • If you are working on a remote machine or used the --console-only flag, copy the verification code from the URL and paste it to the terminal command line, after Enter verification code:.

  6. Choose the default project for this configuration.

    After setting up your credentials, gcloud prompts for a default project for this configuration and provides a list of available projects. Select a project ID from the list.

    When you set this property, all of your gsutil commands use the default project ID unless you override them with the --project flag or set the CLOUDSDK_CORE_PROJECT environment variable. If you do not set a default project or environment variable, you must include a --project flag in each gsutil command that you run.

That's it. You're ready to access protected data. To see a listing of gsutil commands, type gsutil at the command prompt.

Authenticate stand-alone gsutil

Authenticate with HMAC

While OAuth 2.0 is the recommended way to authenticate gsutil, you can also use HMAC authentication and developer keys. To do so, use the -a flag in the gsutil config command:

gsutil config -a

Using this command takes you through a similar process as authenticating the stand-alone gsutil (see previous section), except you are prompted for your google access key ID and your google secret access key.

If you are using the Google Cloud SDK, you also should disable credential passing using the command:

gcloud config set pass_credentials_to_gsutil false