Using Boto

This page describes how gsutil uses a boto configuration file as well as how to configure boto to use OAuth 2.0 authentication.

Using boto with Google Cloud Storage

  • When gsutil has been installed as part of the Google Cloud SDK:

    The recommended way of installing gsutil is as part of the Google Cloud SDK. When you do so, the boto/gsutil configuration file contains values that control how gsutil behaves, such as which API gsutil preferentially uses (with the prefer_api variable). These variables can be changed by editing the file directly. The file does not contain credentials, and it does not control the default project ID, since these are handled by the Cloud SDK.

  • When gsutil has been installed as a standalone tool:

    When gsutil has been installed as a standalone tool, the boto/gsutil configuration file contains values that control how gsutil behaves, such as which API gsutil preferentially uses and the default project ID gsutil uses (with the prefer_api and default_project_id variables, respectively). These variables can be changed by editing the file directly. The file also controls credentials and OAuth2 settings, which you typically create by using the gsutil config command.

The default location for the boto configuration file is in the user home directory, ~/.boto, for Linux and Mac OS X, and in %HOMEDRIVE%%HOMEPATH%, for Windows. You can get the location of the configuration file by running the command gsutil version -l.

You can override where the config file is used by setting the BOTO_CONFIG environment variable. You can also set up a path of boto config files to load by setting the BOTO_PATH environment variable. This could be useful, for example, in data sharing and collaboration scenarios.

For a complete listing of what the boto configuration file contains, see gsutil config.

Configuring boto to use OAuth 2.0 authentication

To use boto with Google Cloud Storage, we recommend that you configure boto to use OAuth 2.0 authentication. To do so, follow the instructions below for your operating system.

Linux and Mac OS X

  1. Set up your boto configuration file to use OAuth 2.0.

    You can do so using gsutil and running the command gsutil config. If you don't have gsutil installed, see the gsutil installation instructions.

  2. Include the gsutil, boto, and the gcs-oauth2-boto-plugin third_party library in your PYTHONPATH.

    Add the following line to your .bashrc or .bash_profile.

    export PYTHONPATH=${PYTHONPATH}:$HOME/gsutil/third_party/boto:$HOME/gsutil/third_party/gcs-oauth2-boto-plugin:$HOME/gsutil

    If you installed gsutil in a directory other than your home directory, replace $HOME with the directory where your gsutil directory is located.

  3. Import the oauth2_plugin.

    The following statement imports the oauth2_plugin:

    from gcs_oauth2_boto_plugin import oauth2_plugin

That's it! You can now use boto with OAuth 2.0 authentication. The following example uses boto with OAuth 2.0 authentication to request the contents of a text file named rose.txt:

from boto import storage_uri
from gcs_oauth2_boto_plugin import oauth2_plugin
content = storage_uri('gs://pub/shakespeare/rose.txt').get_contents_as_string()
print(content)

Windows

  1. Set up your boto configuration file to use OAuth 2.0.

    You can do so using gsutil and running the command python gsutil config. If you don't have gsutil installed, see the gsutil installation instructions.

  2. Include the gsutil, boto, and third_party libraries in your PYTHONPATH.

    1. Open System properties in Control Panel, click the Advanced tab, and then click Environment Variables.

    2. Create a new variable named PYTHONPATH and assign it the following value: C:\gsutil\third_party\boto;C:\gsutil\third_party\gcs-oauth2-boto-plugin.

    3. Reboot for the environment variable change to take effect.

  3. Import the oauth2_plugin.

    The following statement imports the oauth2_plugin:

    from gcs_oauth2_boto_plugin import oauth2_plugin

Thats it! You can now use boto with OAuth 2.0 authentication. The following example uses boto with OAuth 2.0 authentication to request the contents of a text file named rose.txt:

from boto import storage_uri
from gcs_oauth2_boto_plugin import oauth2_plugin
content = storage_uri('gs://pub/shakespeare/rose.txt').get_contents_as_string()
print(content)

Send feedback about...

Cloud Storage Documentation