This page describes the installation and setup of gsutil, a tool that enables you to access Cloud Storage from the command-line using HTTPS.
Simply installing gsutil gives you immediate read and/or write access to public data. Authenticating to the Cloud Storage service gives you read and/or write access to protected data that has been shared with you. Enabling billing gives you the ability to create and manage your own buckets.
System requirements
The gsutil tool runs on Linux/Unix, Mac OS, and Windows (XP or later).
If you plan to use composite objects, you need to install compiled crcmod. On Windows, this is only available for 32-bit Python.
Installing gsutil
The officially supported installation and update method for gsutil is as part of the Google Cloud SDK.
Installing gsutil as part of the Google Cloud SDK
Follow the instructions for your operating system to install gsutil as a part of the Google Cloud SDK:
Cloud SDK requires Python; supported versions are Python 3 (preferred, 3.5 to 3.8) and Python 2 (2.7.9 or higher).
python --version
For more information on how to choose and configure your Python interpreter, refer togcloud topic startup
.- Download one of the following:
-
Alternatively, to download the Linux 64-bit archive file from your
command-line, run:
curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-329.0.0-linux-x86_64.tar.gz
curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-329.0.0-linux-x86.tar.gz
- Extract the contents of the file to any location on your file system (preferably
your Home directory). If
you would like to replace an existing installation, remove the existing
google-cloud-sdk
directory and extract the archive to the same location. - Optional. Use the install script to add Cloud SDK tools to your
PATH
. You'll also be able to opt-in to command-completion for your shell and usage statistics collection.Run the script (from the root of the folder you extracted in the last step) using this command:
./google-cloud-sdk/install.sh
This can also be done non-interactively (for example, using a script) as well by providing preferences as flags. These are described in:./google-cloud-sdk/install.sh --help
Open a new terminal so that the changes take effect. - Run gcloud init to initialize the SDK:
- Optional. Install additional components using the component manager.
Platform | Package | Size | SHA256 Checksum |
---|---|---|---|
Linux 64-bit (x86_64) |
google-cloud-sdk-329.0.0-linux-x86_64.tar.gz | 112.3 MB | bab65ba4997632a85bbf7f5d7c7cfceb851edd0fa00114151dea2492b8045854 |
Linux 32-bit (x86) |
google-cloud-sdk-329.0.0-linux-x86.tar.gz | 86.8 MB | 141b394133104d149bf3ede72a35c050d6ef43af4f9ed055e5dbf2f3120941ee |
./google-cloud-sdk/bin/gcloud init
Package contents
Cloud SDK is available in package format for installation on Debian and Ubuntu systems. This
package contains the gcloud
, gcloud alpha
, gcloud beta
,
gsutil
, and bq
commands only. It does not include kubectl
or the App Engine extensions required to deploy an application using gcloud
commands.
If you want these components, you must install them separately as described
later in this section.
Before you install Cloud SDK, make sure that your operating system is one of the following:
- Ubuntu release that has not reached end-of-life
- Debian stable release from Wheezy forward
- Add the Cloud SDK distribution URI as a package source:
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list
Make sure you have apt-transport-https installed:sudo apt-get install apt-transport-https ca-certificates gnupg
- Import the Google Cloud public key:
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key --keyring /usr/share/keyrings/cloud.google.gpg add -
- Update and install the Cloud SDK:
sudo apt-get update && sudo apt-get install google-cloud-sdk
For additionalapt-get
options, such as disabling prompts or dry runs, refer to theapt-get
man pages.Docker Tip: If installing the Cloud SDK inside a Docker image, use a single RUN step instead:
RUN echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key --keyring /usr/share/keyrings/cloud.google.gpg add - && apt-get update -y && apt-get install google-cloud-sdk -y
- Optionally, install any of these
additional components:
google-cloud-sdk-app-engine-python
google-cloud-sdk-app-engine-python-extras
google-cloud-sdk-app-engine-java
google-cloud-sdk-app-engine-go
google-cloud-sdk-bigtable-emulator
google-cloud-sdk-cbt
google-cloud-sdk-cloud-build-local
google-cloud-sdk-datalab
google-cloud-sdk-datastore-emulator
google-cloud-sdk-firestore-emulator
google-cloud-sdk-pubsub-emulator
kubectl
For example, the
google-cloud-sdk-app-engine-java
component can be installed as follows:sudo apt-get install google-cloud-sdk-app-engine-java
- Run
gcloud init
to get started:gcloud init
Downgrading Cloud SDK versions
If you'd like to revert to a specific version of Cloud SDK, where VERSION
is
of the form 123.0.0
, run:
sudo apt-get update && sudo apt-get install google-cloud-sdk=123.0.0-0
The most recent ten releases will always be available in the repo.
Package contents
Cloud SDK is available in package format for installation on Red Hat Enterprise Linux 7, Red Hat
Enterprise Linux 8, Fedora 32, and CentOS 7 systems. This package contains the gcloud
,
gcloud alpha
, gcloud beta
, gsutil
, and bq
commands only. It does not include kubectl
or the App Engine extensions required to
deploy an application using gcloud
commands, which can be
installed separately as described later in this section.
- Update DNF with Cloud SDK repo information:
sudo tee -a /etc/yum.repos.d/google-cloud-sdk.repo << EOM [google-cloud-sdk] name=Google Cloud SDK baseurl=https://packages.cloud.google.com/yum/repos/cloud-sdk-el7-x86_64 enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg EOM
- Install the Cloud SDK:
sudo dnf install google-cloud-sdk
- Optionally, install any of these
additional components:
google-cloud-sdk-app-engine-python
google-cloud-sdk-app-engine-python-extras
google-cloud-sdk-app-engine-java
google-cloud-sdk-app-engine-go
google-cloud-sdk-bigtable-emulator
google-cloud-sdk-cbt
google-cloud-sdk-cloud-build-local
google-cloud-sdk-datalab
google-cloud-sdk-datastore-emulator
google-cloud-sdk-firestore-emulator
google-cloud-sdk-pubsub-emulator
kubectl
For example, the
google-cloud-sdk-app-engine-java
component can be installed as follows:sudo dnf install google-cloud-sdk-app-engine-java
- Run
gcloud init
to get started:gcloud init
Downgrading Cloud SDK versions
If you'd like to revert to a specific version of Cloud SDK, where VERSION
is
of the form 123.0.0
, run:
sudo dnf downgrade google-cloud-sdk-VERSION
The most recent ten releases will always be available in the repo.
- Cloud SDK requires Python; supported versions are Python 3 (preferred, 3.5 to 3.8) and
Python 2 (2.7.9 or higher). Modern versions of macOS include the appropriate
version of Python required for the Cloud SDK.
python -V
For more information on how to choose and configure your Python interpreter, refer togcloud topic startup
. - Download one of the following:
- Extract the archive to any location on your file system (preferably your
Home directory). On macOS, this can be achieved by opening the
downloaded
.tar.gz
archive file in the preferred location.If you would like to replace an existing installation, remove the existing
google-cloud-sdk
directory and extract the archive to the same location. - Optional. Use the install script to add Cloud SDK tools to your
PATH
. You'll also be able to opt-in to command-completion for your shell and usage statistics collection.Run the script (from the root of the folder you extracted in the last step) using this command:
./google-cloud-sdk/install.sh
This can also be done non-interactively (for example, using a script) as well by providing preferences as flags. These are described in:./google-cloud-sdk/install.sh --help
If you'd like to run the install script with screen reader mode on:./google-cloud-sdk/install.sh --screen-reader=true
Open a new terminal so that the changes take effect. - Run gcloud init to initialize the SDK:
- Optional. Install additional components using the component manager.
Platform | Package | Size | SHA256 Checksum |
---|---|---|---|
macOS 64-bit (x86_64) |
google-cloud-sdk-329.0.0-darwin-x86_64.tar.gz | 110.4 MB | 1edda5cc0168e8c350e56ffbec1b5b78638e2c0a48e5dd88699d89ccfb35d688 |
macOS 32-bit (x86) |
google-cloud-sdk-329.0.0-darwin-x86.tar.gz | 87.9 MB | da72313f5762ddc56ce121e2f32c5af2fdf7c93a3667104ccd2e769d0f44d689 |
./google-cloud-sdk/bin/gcloud init
-
Download the Cloud SDK installer.
Alternatively, open a PowerShell terminal and run the following PowerShell commands.
(New-Object Net.WebClient).DownloadFile("https://dl.google.com/dl/cloudsdk/channels/rapid/GoogleCloudSDKInstaller.exe", "$env:Temp\GoogleCloudSDKInstaller.exe") & $env:Temp\GoogleCloudSDKInstaller.exe
-
Launch the installer and follow the prompts. The installer is signed by Google LLC.
If you'd like to enable screen reader mode, select the Turn on screen reader mode option for a more streamlined screen reader experience. To read more about the Cloud SDK screen reader experience, refer to the Accessibility features guide.
-
Cloud SDK requires Python; supported versions are Python 3 (preferred, 3.5 to 3.8) and Python 2 (2.7.9 or higher).
The installer will install all necessary dependencies, including the needed Python version. While Cloud SDK currently uses Python 3 by default, you can use an existing Python installation if necessary by unchecking the option to Install Bundled Python.
After installation has completed, the installer presents several options:
Make sure that the following are selected:
- Start Google Cloud SDK Shell
- Run
gcloud init
The installer starts a terminal window and runs the
gcloud init
command.- The default installation does not include the App Engine extensions required to deploy an
application using
gcloud
commands. These components can be installed using the Cloud SDK component manager.
- If the Cloud SDK fails to run after installing version 274.0.0, please refer to this tracking bug for the latest workarounds.
- If your installation is unsuccessful
due to the
find
command not being recognized, ensure yourPATH
environment variable is set to include the folder containingfind
. Usually, this isC:\WINDOWS\system32;
. - If you have just uninstalled Cloud SDK, you will need to reboot your system before installing Cloud SDK again.
Perform updates with the components update
command: gcloud components update
.
To learn how to use gsutil, see Quickstart: Using the gsutil Tool exercise,
or run gsutil help
.
Alternative installation methods
There are several ways to install gsutil as a stand-alone product. You may prefer one of these methods if you do not want any of the other components that come with the Cloud SDK, or if you are managing packages with PyPI.
If you are installing gsutil as a standalone, an additional system requirement is that you must have Python installed on your computer. gsutil officially supports Python 3 (preferred, 3.5 to 3.8) and Python 2 (2.7.9 or higher). Python is installed by default on most distributions of Linux and macOS, but not on Windows; you must install Python before you can run gsutil on Windows.
Expand the desired method for instructions.
Setting Up Credentials to Access Protected Data
Installing gsutil allows you to download and access publicly-accessible objects, but in order to access protected data or write to a protected bucket, you need to set up credentials (authenticate). For example, if someone else has created a Cloud Storage account and uploaded data that is only accessible to you or other specific individuals, you must set up your credentials to the Cloud Storage service to be able to access this data.
If you installed gsutil as part of the Google Cloud SDK, follow these instructions. Otherwise, see the Authenticate stand-alone gsutil section.
Authenticate Google Cloud SDK
When using gsutil as part of the Google Cloud SDK, OAuth2 is used to
authenticate and authorize access. Note that you may have already performed
the following steps if you performed
Installing gsutil as part of the Google Cloud SDK above. If you
ran gcloud init
previously, you are asked if you want to re-initialize the
configuration or create a new one.
Open a command prompt instance.
Run
gcloud init
at the command prompt.You receive the following output:
Welcome! This command will take you through the configuration of gcloud. Your current configuration has been set to: [default] To continue, you must login. Would you like to login (Y/n)?
Type "Y" and press enter to login.
The command prints a URL and tries to open a browser window to request access to your project. If a browser window can be opened, you receive the following output:
Your browser has been opened to visit: https://accounts.google.com/o/oauth2/auth?redirect_uri=http%3A%2F%2F...
Go to the next step.
If you are working on a local machine and your browser doesn't automatically load the URL, retry the
gcloud init
command with the--console-only
flag:gcloud init --console-only
If you use this command, or if the Cloud SDK detects that a browser can not be opened (such as if you are working on a remote machine), you receive the following output:
Go to the following link in your browser: https://accounts.google.com/o/oauth2/auth?redirect_uri=urn%3Aietf%3Awg%3A... Enter verification code:
Open a browser and navigate to the URL provided.
If prompted, sign in with the Google account associated with your Cloud Storage data in the browser window.
Grant access.
If the browser window opened automatically, review the application permissions and click Accept when you are ready. The verification code is then automatically sent to the command line tool.
If you are working on a remote machine or used the
--console-only
flag, copy the verification code from the URL and paste it to the terminal command line, after Enter verification code:.
Choose the default project for this configuration.
After setting up your credentials, the
gcloud
command-line tool prompts you for a default project for this configuration and provides a list of available projects. Select a project ID from the list.When you set this property, gsutil commands that require a project, such as
gsutil mb
, use the default project ID unless you override them with the-p
flag or set theCLOUDSDK_CORE_PROJECT
environment variable.
That's it. You're ready to access protected data. To see a listing of gsutil
commands, type gsutil
at the command prompt.
Authenticate stand-alone gsutil
Authenticate with HMAC
While OAuth 2.0 is the recommended way to authenticate gsutil, you can also
use HMAC keys for your credentials. To authenticate with HMAC,
use the -a
flag in the gsutil config
command:
gsutil config -a
Using this command takes you through a similar process as authenticating the stand-alone gsutil (see previous section), except you are prompted for the access ID and secret associated with your HMAC key.
If you are using the Google Cloud SDK, you also should disable credential passing using the command:
gcloud config set pass_credentials_to_gsutil false
Enable mTLS
You may also want to enable mutual TLS (mTLS). When mTLS is enabled on your
device, your device attempts to connect to
storage.mtls.googleapis.com
. Before the connection is allowed,
Cloud Storage verifies the certificate on your device.
The simplest way to to obtain a certificate is through Google Cloud SDK. You can set one manually in the .boto file by setting the following values under "Credentials":
-
use_client_certificate
: A flag controlling whether or not to use mTLS. -
cert_provider_command
: A shell command that prints a certificate to stdout for gsutil to read.
Try it for yourself
If you're new to Google Cloud, create an account to evaluate how Cloud Storage performs in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
Try Cloud Storage free