Install gsutil

This page describes the installation and setup of gsutil, a tool that enables you to access Cloud Storage from the command-line using HTTPS.

Simply installing gsutil gives you immediate read and/or write access to public data. Authenticating to the Cloud Storage service gives you read and/or write access to protected data that has been shared with you. Enabling billing gives you the ability to create and manage your own buckets.

System requirements

  • The gsutil tool runs on Linux/Unix, Mac OS, and Windows (XP or later).

  • gsutil versions 5.0 and up require Python 3.

  • If you plan to use composite objects, you need to install compiled crcmod. On Windows, this is only available for 32-bit Python.

Installing gsutil

The officially supported installation and update method for gsutil is as part of the Google Cloud SDK.

Installing gsutil as part of the Google Cloud SDK

Follow the instructions for your operating system to install gsutil as a part of the Google Cloud SDK:

Linux
  1. Confirm that you have a supported version of Python. Cloud SDK requires Python 3 (3.7 to 3.8 preferred) or Python 2 (2.7.9 or higher). For information on how to choose and configure your Python interpreter, see gcloud topic startup.
  2. Download one of the following:
    Platform Package Size SHA256 Checksum
    Linux 64-bit

    (x86_64)

    google-cloud-sdk-365.0.0-linux-x86_64.tar.gz 92.7 MB 81d0ad64dca3e97d02e873d386bafcc77b416d7c9b45e5ec2387e5075b133fc0
    Linux 64-bit

    (arm)

    google-cloud-sdk-365.0.0-linux-arm.tar.gz 90.1 MB 0d5074e2566d0617fae408b0f40c82be2487a95de7e84a772ad43b66da4c224a
    Linux 32-bit

    (x86)

    google-cloud-sdk-365.0.0-linux-x86.tar.gz 90.1 MB 26de2a21039ddf9713b71e090244f0ae57f252ec8d0ab14c2d594c8a5bcc6c4b

    Alternatively, to download the Linux 64-bit archive file, at the command line, run:

    curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-365.0.0-linux-x86_64.tar.gz

    For the 64-bit arm archive file, run:

    curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-365.0.0-linux-arm.tar.gz

    For the 32-bit archive file, run:

    curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-365.0.0-linux-x86.tar.gz
  3. Extract the contents of the file to any location on your file system (preferably your Home directory). To replace an existing installation, remove the existing google-cloud-sdk directory and extract the archive to the same location.
  4. If you want to send anonymous usage statistics to help improve Cloud SDK, answer Y when prompted.
  5. To add Cloud SDK command-line tools to your PATH and enable command completion, answer Y when prompted.
  6. Optional. Use the install script to add Cloud SDK tools to your PATH. You can also opt-in to command-completion for your shell and usage statistics collection.

    Run the script (from the root of the folder you extracted to) using the following command:

    ./google-cloud-sdk/install.sh
    
    This can also be done non-interactively (for example, using a script) and by providing preferences as flags. These are described in:
    ./google-cloud-sdk/install.sh --help
    
    Open a new terminal so that the changes take effect.
  7. To initialize the SDK, run gcloud init:
  8. ./google-cloud-sdk/bin/gcloud init
    
  9. Optional. Install additional components using the component manager.
Debian/Ubuntu

Package contents

Cloud SDK is available in package format for installation on Debian and Ubuntu systems. This package contains the gcloud, gcloud alpha, gcloud beta, gsutil, and bq commands only. It doesn't include kubectl or the App Engine extensions required to deploy an application using gcloud commands. If you want these components, you must install them separately.

Before you begin

Before you install Cloud SDK, make sure that your operating system meets the following requirements:

  • An Ubuntu release that hasn't reached end-of-life or a Debian stable release that hasn't reached end-of-life
  • apt-transport-https is installed:
    sudo apt-get install apt-transport-https ca-certificates gnupg
Installation
  1. Add the Cloud SDK distribution URI as a package source. If your distribution supports the signed-by option, run the following command:
    echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list

    If your distribution doesn't support the signed-by option, run the following command:

    echo "deb https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list
  2. Import the Google Cloud public key. If your distribution's apt-key command supports the --keyring argument, run the following command:
    curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key --keyring /usr/share/keyrings/cloud.google.gpg add -

    If your distribution's apt-key command doesn't support the --keyring argument, run the following command:

    curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

    If you can't get latest updates due to an expired key, obtain the latest apt-get.gpg key file.

  3. Update and install the Cloud SDK:
    sudo apt-get update && sudo apt-get install google-cloud-sdk
    For additional apt-get options, such as disabling prompts or dry runs, refer to the apt-get man pages.

    Docker Tip: If installing the Cloud SDK inside a Docker image, use a single RUN step instead:

    RUN echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key --keyring /usr/share/keyrings/cloud.google.gpg  add - && apt-get update -y && apt-get install google-cloud-sdk -y
          
  4. Optionally, install any of the following additional components:
    • google-cloud-sdk-app-engine-python
    • google-cloud-sdk-app-engine-python-extras
    • google-cloud-sdk-app-engine-java
    • google-cloud-sdk-app-engine-go
    • google-cloud-sdk-bigtable-emulator
    • google-cloud-sdk-cbt
    • google-cloud-sdk-cloud-build-local
    • google-cloud-sdk-datalab
    • google-cloud-sdk-datastore-emulator
    • google-cloud-sdk-firestore-emulator
    • google-cloud-sdk-pubsub-emulator
    • kubectl

    For example, the google-cloud-sdk-app-engine-java component can be installed as follows:

    sudo apt-get install google-cloud-sdk-app-engine-java
  5. Run gcloud init to get started:
    gcloud init

Downgrading Cloud SDK versions

To revert to a specific version of Cloud SDK, where VERSION is of the form 123.0.0, run the following command:

sudo apt-get update && sudo apt-get install google-cloud-sdk=123.0.0-0

The most recent ten releases are always available in the repo.

Red Hat/Fedora/CentOS

Package contents

Cloud SDK is available in package format for installation on Red Hat Enterprise Linux 7, Red Hat Enterprise Linux 8, Fedora 33, and CentOS 7 and 8 systems. This package contains the gcloud, gcloud alpha, gcloud beta, gsutil, and bq commands only. It doesn't include kubectl or the App Engine extensions required to deploy an application using gcloud commands, which can be installed separately as described later in this section.

Installation
  1. Update DNF with Cloud SDK repo information. The following sample command is for a Red Hat Enterprise Linux 8-compatible installation. For a Red Hat Enterprise Linux 7-compatible installation, replace el8 with el7 in the value for baseUrl.
    sudo tee -a /etc/yum.repos.d/google-cloud-sdk.repo << EOM
    [google-cloud-sdk]
    name=Google Cloud SDK
    baseurl=https://packages.cloud.google.com/yum/repos/cloud-sdk-el8-x86_64
    enabled=1
    gpgcheck=1
    repo_gpgcheck=0
    gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg
           https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
    EOM
    
  2. Install the Cloud SDK:
    sudo dnf install google-cloud-sdk
  3. Optionally, install any of these additional components:
    • google-cloud-sdk-app-engine-python
    • google-cloud-sdk-app-engine-python-extras
    • google-cloud-sdk-app-engine-java
    • google-cloud-sdk-app-engine-go
    • google-cloud-sdk-bigtable-emulator
    • google-cloud-sdk-cbt
    • google-cloud-sdk-cloud-build-local
    • google-cloud-sdk-datalab
    • google-cloud-sdk-datastore-emulator
    • google-cloud-sdk-firestore-emulator
    • google-cloud-sdk-pubsub-emulator
    • kubectl

    For example, the google-cloud-sdk-app-engine-java component can be installed as follows:

    sudo dnf install google-cloud-sdk-app-engine-java
  4. Run gcloud init to get started:
    gcloud init

Downgrading Cloud SDK versions

If you'd like to revert to a specific version of Cloud SDK, where VERSION is of the form 123.0.0, run: sudo dnf downgrade google-cloud-sdk-VERSION The most recent ten releases will always be available in the repo.

macOS
  1. Confirm that you have a supported version of Python:
    • Supported versions are Python 3 (3.5 to 3.8, 3.7 recommended) and Python 2 (2.7.9 or higher).
    • Modern versions of macOS include the appropriate version of Python required for the Cloud SDK. To check your current Python version, run python -V.
    • For Cloud SDK release version 352.0.0 and above, the main install script offers to install CPython's Python 3.7 on Intel-based Macs.
    • For more information on how to choose and configure your Python interpreter, refer to gcloud topic startup.
  2. Download one of the following:
  3. Platform Package Size SHA256 Checksum
    macOS 64-bit

    (x86_64)

    google-cloud-sdk-365.0.0-darwin-x86_64.tar.gz 93.2 MB 9e05d06a5067361d3c47866d293ee23c2e9d87f2fb505b6cf39a535910bb4810
    macOS 64-bit

    (arm64, Apple M1 silicon)

    google-cloud-sdk-365.0.0-darwin-arm.tar.gz 93.2 MB 477886b3a4fe7444bd3aa89afdd1b7d39977dea09699bd8b8b8a884cbc49e9e5
    macOS 32-bit

    (x86)

    google-cloud-sdk-365.0.0-darwin-x86.tar.gz 97.0 MB ce776030bd83d6e6a27ec89fffddad61ede8a55f51c30fec3869c335be9dc5ea
    1. Extract the archive to any location on your file system (preferably your Home directory). On macOS, this can be achieved by opening the downloaded .tar.gz archive file in the preferred location.

      To replace an existing installation, remove the existing google-cloud-sdk directory and then extract the archive to the same location.

    2. Optional. Use the install script to add Cloud SDK tools to your PATH.

      You can also opt-in to command-completion for your shell and usage statistics collection, and optionally install Python 3.7.

      Run the script (from the root of the folder you extracted in the last step) using this command:

      ./google-cloud-sdk/install.sh
      
      This can also be done non-interactively (for example, using a script) by providing preferences as flags. These are described in:
      ./google-cloud-sdk/install.sh --help
      
      To run the install script with screen reader mode on:
      ./google-cloud-sdk/install.sh --screen-reader=true
      
      Open a new terminal so that the changes take effect.
    3. Run gcloud init to initialize the SDK:
    4. ./google-cloud-sdk/bin/gcloud init
      
    5. Optional. Install additional components using the component manager.
Windows
  1. Download the Cloud SDK installer.

    Alternatively, open a PowerShell terminal and run the following PowerShell commands:

    (New-Object Net.WebClient).DownloadFile("https://dl.google.com/dl/cloudsdk/channels/rapid/GoogleCloudSDKInstaller.exe", "$env:Temp\GoogleCloudSDKInstaller.exe")
    
    & $env:Temp\GoogleCloudSDKInstaller.exe
        
  2. Launch the installer and follow the prompts. The installer is signed by Google LLC.

    If you're using a screen reader, check the Turn on screen reader mode checkbox. This option configures gcloud to use status trackers instead of unicode spinners, display progress as a percentage, and flatten tables. For more information, see the Accessibility features guide.

  3. Cloud SDK requires Python; supported versions are Python 3 (preferred, 3.5 to 3.8) and Python 2 (2.7.9 or higher). Cloud SDK comes bundled with Python 3 by default. To use Cloud SDK, your operating system must be able to run a supported version of Python.

    The installer installs all necessary dependencies, including the needed Python version. While Cloud SDK installs and manages Python 3 by default, you can use an existing Python installation if necessary by unchecking the option to Install Bundled Python. See gcloud topic startup to learn how to use an existing Python installation.

  4. After installation is complete, the installer gives you the option to create Start Menu and Desktop shortcuts, start Cloud SDK shell, and configure the Cloud SDK. Make sure that you leave the options to start the shell and configure your installation selected. The installer starts a terminal window and runs the gcloud init command.

  5. The default installation doesn't include the App Engine extensions required to deploy an application using gcloud commands. These components can be installed using the Cloud SDK component manager.
Troubleshooting tips:
  • If your installation is unsuccessful due to the find command not being recognized, ensure your PATH environment variable is set to include the folder containing find. Usually, this is C:\WINDOWS\system32;.
  • If you have just uninstalled Cloud SDK, you must reboot your system before installing Cloud SDK again.
  • If unzipping fails, run the installer as an administrator.

Perform updates with the components update command: gcloud components update.

To learn how to use gsutil, see Quickstart: Using the gsutil Tool exercise, or run gsutil help.

Alternative installation methods

There are several ways to install gsutil as a stand-alone product. You may prefer one of these methods if you do not want any of the other components that come with the Cloud SDK, or if you are managing packages with PyPI.

If you are installing gsutil 5.0 or higher, you must have Python 3 installed on your computer, with Python 3.5 to 3.8 preferred. Python 3 is installed by default on most distributions of Linux and macOS, however:

  • Python 3 is not installed by default on Windows, but can be installed manually.
  • Python 3 might not be installed by default on CentOS or RHEL. If it is not currently installed, it can be installed manually.

Expand the desired method for instructions to install gsutil.

Setting Up Credentials to Access Protected Data

Installing gsutil allows you to download and access publicly-accessible data, but in order to access protected data or write to a protected bucket, you need to set up credentials (authenticate). For example, if someone else has created a Cloud Storage account and uploaded data that is only accessible to you or other specific individuals, you must set up your credentials to the Cloud Storage service to be able to access this data.

If you installed gsutil as part of the Google Cloud SDK, follow these instructions. Otherwise, see the Authenticate stand-alone gsutil section.

Authenticate Google Cloud SDK

When using gsutil as part of the Google Cloud SDK, OAuth2 is used to authenticate and authorize access. Note that you may have already performed the following steps if you performed Installing gsutil as part of the Google Cloud SDK above. If you ran gcloud init previously, you are asked if you want to re-initialize the configuration or create a new one.

  1. Open a command prompt instance.

  2. Run gcloud init at the command prompt.

    You receive the following output:

    Welcome! This command will take you through the configuration of gcloud.
     
    Your current configuration has been set to: [default]
     
    To continue, you must login. Would you like to login (Y/n)?
    

  3. Type "Y" and press enter to login.

    • The command prints a URL and tries to open a browser window to request access to your project. If a browser window can be opened, you receive the following output:

      Your browser has been opened to visit:
       
      https://accounts.google.com/o/oauth2/auth?redirect_uri=http%3A%2F%2F...
      

      Go to the next step.

    • If you are working on a local machine and your browser doesn't automatically load the URL, retry the gcloud init command with the --console-only flag:

      gcloud init --console-only
      

      If you use this command, or if the Cloud SDK detects that a browser can not be opened (such as if you are working on a remote machine), you receive the following output:

      Go to the following link in your browser:
       
      https://accounts.google.com/o/oauth2/auth?redirect_uri=urn%3Aietf%3Awg%3A...
       
      Enter verification code:
      

      Open a browser and navigate to the URL provided.

  4. If prompted, sign in with the Google account associated with your Cloud Storage data in the browser window.

  5. Grant access.

    • If the browser window opened automatically, review the application permissions and click Accept when you are ready. The verification code is then automatically sent to the command line tool.

    • If you are working on a remote machine or used the --console-only flag, copy the verification code from the URL and paste it to the terminal command line, after Enter verification code:.

  6. Choose the default project for this configuration.

    After setting up your credentials, the gcloud command-line tool prompts you for a default project for this configuration and provides a list of available projects. Select a project ID from the list.

    When you set this property, gsutil commands that require a project, such as gsutil mb, use the default project ID unless you override them with the -p flag or set the CLOUDSDK_CORE_PROJECT environment variable.

That's it. You're ready to access protected data. To see a listing of gsutil commands, type gsutil at the command prompt.

Authenticate stand-alone gsutil

Authenticate with HMAC

While OAuth 2.0 is the recommended way to authenticate gsutil, you can also use HMAC keys for your credentials. To authenticate with HMAC, use the -a flag in the gsutil config command:

gsutil config -a

Using this command takes you through a similar process as authenticating the stand-alone gsutil (see previous section), except you are prompted for the access ID and secret associated with your HMAC key.

If you are using the Google Cloud SDK, you also should disable credential passing using the command:

gcloud config set pass_credentials_to_gsutil false

Enable mTLS

You may also want to enable mutual TLS (mTLS). When mTLS is enabled on your device, your device attempts to connect to storage.mtls.googleapis.com. Before the connection is allowed, Cloud Storage verifies the certificate on your device.

The simplest way to to obtain a certificate is through Google Cloud SDK. You can set one manually in the .boto file by setting the following values under "Credentials":

  1. use_client_certificate: A flag controlling whether or not to use mTLS.
  2. cert_provider_command: A shell command that prints a certificate to stdout for gsutil to read.

Try it for yourself

If you're new to Google Cloud, create an account to evaluate how Cloud Storage performs in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

Try Cloud Storage free