Most of the operations you perform in Cloud Storage must be authenticated.
The only exceptions are operations on objects that allow anonymous
access. Objects are anonymously accessible if the
READ permission. The
allUsers group includes
anyone on the Internet.
- gsutil authentication
- Client library authentication
- API authentication
- Cookie-based authentication
- Service account credentials
- User account credentials
Google Cloud Storage uses OAuth 2.0 for API authentication and authorization. Authentication is the process of determining the identity of a client.The details of authentication vary depending on how you are accessing Google Cloud Storage, but fall into two general types:
A server-centric flow allows an application to directly hold the credentials of a service account to complete authentication. Use this flow if your application works with its own data rather than user data. Google Cloud Platform projects have default service accounts you can use, or you can create new ones.
A user-centric flow allows an application to obtain credentials from an end user. The user signs in to complete authentication. Use this flow if your application needs to access user data. See the User account credentials section later in this page for scenarios where a user-centric flow is appropriate.
Keep in mind that you can use both types of authentication together in an application. For more background information about authentication, see the Google Cloud Platform Auth Guide.
Authorization is the process of determining what permissions an authenticated
identity has on a set of specified resources. OAuth uses scopes to
determine if an authenticated identity is authorized. Applications use a
credential (obtained from a user-centric or server-centric authentication flow)
together with one or more scopes to request an access token from a Google
authorization server to access protected resources. For example, application A
with an access token with
read-only scope can only read, while application B
with an access token with
read-write scope can read and modify data. Neither
application can read or modify access control lists on objects and buckets;
only an application with
full-control scope can do so.
||Only allows access to read data, including listing buckets.||
||Allows access to read and change data, but not metadata like ACLs.||
||Allows full control over data, including the ability to modify ACLs.||
||View your data across Google Cloud Platform services. For Google Cloud Storage,
this is the same as
||View and manage data across all Google Cloud Platform services. For
Google Cloud Storage, this is the same as
Using service account credentials
Using user account credentials
gcloud beta auth application-default login to authenticate
with user account credentials:
$ gcloud beta auth application-default login
gcloud auth uses the
cloud-platform scope when getting an
If you installed gsutil independent of the Cloud SDK, then see the gsutil install page for information about how to authenticate with a user or service account.
Client library authentication
Client libraries can use Application Default Credentials to easily authenticate with Google APIs and send requests to those APIs. With Application Default Credentials, you can test your application locally and deploy it without changing the underlying code. For more information, including code samples, see Google Cloud Platform Auth Guide.
Application Default Credentials are part of the client libraries you can use to access Google Cloud Storage. Default credentials identify your application with either a user credential or a default service account. When working with a library, it's typical that you'll choose credentials based on the type of environment where you run your code.
Initialize your environment
- Local Development Environment
To initialize your local development environment, you can use the Google Cloud SDK to authenticate with user credentials. On Linux/Mac OS X, use the script:
$ curl https://sdk.cloud.google.com | bashOnce installed, you can use your own user account credentials as a proxy to test code calling APIs from your local machine by running:
$ gcloud beta auth application-default loginThis command requests access tokens with the
- Google Cloud Platform
If you're running your application on Google App Engine or Google Compute Engine, the environment already provides a service account's authentication information, so no further setup is required. For Compute Engine, the service account scope depends on how you created the instance. See Setting the scope of service account access for instances. For App Engine, the
cloud-platformscope is used.
- Other Environments
For production environments outside the Google Cloud Platform, you provide a service account's authentication information through an environment variable and the scope is set in code. For more information, see How the Application Default Credentials work. Creating a service account is discussed later in this page.
Examples of using Application Default Credentials
To make requests using OAuth 2.0 to either the Google Cloud Storage XML API or
JSON API, include your application's access token in the
Authorization header in every request that requires authentication.
Authorization: Bearer <oauth2_token>
The following is an example of a request that lists objects in a bucket.
Use the list method of the Objects resource.
GET /storage/v1/b/example-bucket/o HTTP/1.1 Host: www.googleapis.com Authorization: Bearer ya29.AHES6ZRVmB7fkLtd1XTmq6mo0S1wqZZi3-Lh_s-6Uw7p8vtgSwg
Use a List objects request.
GET / HTTP/1.1 Host: example-bucket.storage.googleapis.com Authorization: Bearer ya29.AHES6ZRVmB7fkLtd1XTmq6mo0S1wqZZi3-Lh_s-6Uw7p8vtgSwg
Due to the complexity of managing and refreshing access tokens and the security risk when dealing directly with cryptographic applications, we strongly encourage you to use a verified client library.
If you looking for developer keys to use with the XML API for interoperable access with Amazon S3, see Managing developer keys for a simple migration.
Cloud Storage lets you provide browser-based authenticated downloads to users who have Google accounts but do not necessarily have Cloud Storage accounts. To do this, apply Google account-based Access Control Lists (ACLs) to the object, then provide users with a URL that is scoped to the object. The URL for browser-based authenticated downloads is:
For example, to let Jane download an object named
in a bucket named
example-travel-maps, you grant
READ permission to
firstname.lastname@example.org for the object
/europe/france/paris.jpg. Then you provide
Jane with the following URL:
When Jane clicks the URL in her browser, she's automatically prompted to sign in to her Google account (if she's not already logged in). After she is authenticated, and her browser has acquired a cookie with an encapsulated identity token, she is redirected to the object in the Cloud Storage repository. Cloud Storage then verifies that Jane is allowed to read the object, and the object is downloaded to Jane's computer.
The following figure shows how the authentication process for browser-based authenticated downloads works.
Service account credentials
Service accounts are special accounts that represent software rather than people. They are the most common way applications authenticate with Google Cloud Storage. Every project has service accounts associated with it, which may be used for different authentication scenarios, as well as to enable advanced features such as Signed URLs and browser uploads using POST.
When you use a service account to authenticate your application, you do not need a user to authenticate to get an access token. Instead, you obtain a private key from the Google Cloud Platform Console, which you then use to send a signed request for an access token. You can then use the access token like you normally would. For more information see the Google Cloud Platform Auth Guide.
Generating a service account credential
You can create a private key in the Cloud Platform Console by creating an OAuth Client ID for a service account. You can get your private key in JSON and PKCS12 format:
- JSON keys are required if you are using Application Default Credentials in a production environment outside of Google Cloud Platform. JSON keys cannot be converted to other formats.
- PKCS12 is supported by many different programming languages and libraries. If needed, you can convert the key into other formats using OpenSSL (see Converting the private key to other formats). However, PKCS12 keys cannot be converted to JSON format.
To generate a private key in JSON or PKCS12 format:
- Open the list of credentials in the Google Cloud Platform Console.
- Click Create credentials.
- Select Service account key.
A Create service account key window opens.
- Click the drop-down box below Service account, then click New service account.
- Enter a name for the service account in Name.
- Use the default Service account ID or generate a different one.
- Select the Key type: JSON or P12.
- Click Create.
A Service account created window is displayed and the private key for the Key type you selected is downloaded automatically. If you selected a P12 key, the private key's password ("notasecret") is displayed.
- Click Close.
For more information about security and identity in projects, see the Google Cloud Platform Console Help.
Service credential details
The OAuth client ID of a service account uniquely identifies the account being used for the authentication. After you create a Client ID for a service account (see Generating a service account credential), you will see the following information:
- Client ID
- Email address
- Certificate fingerprints
Notice that there are two identifiers of the service account, the Client ID and the Email address. You must use the Email address for RSA signatures, for example, for Signed URLs (Query String Authentication).
Converting the service account credential to other formats
After you create a Client ID for a service account (see Generating a service account credential) and download a private key in PKCS12 format, you can convert the key to other formats if necessary by using OpenSSL. For example, to convert the key to PKCS1 (PEM):
# Convert the key from pkcs12 to pkcs1 (PEM). $ cat /path/to/xxxx-privatekey.p12 | openssl pkcs12 -nodes -nocerts -passin pass:notasecret | openssl rsa > /path/to/secret.pem
You can then use openssl to generate a signature digest for your data:
# Generate a signature digest using the PEM key. $ openssl dgst -sign path/to/key.pem -sha256 path/to/data_to_sign | base64 -w0
You can also use openssl to convert the PEM file format to DER format. Some libraries (for example, PyCrypto) require that keys be in DER format.
# Convert the key from pkcs1 (PEM) to DER. $ openssl rsa -in path/to/key.pem -inform PEM -out path/to/key.der -outform DER
JSON private keys cannot be converted to other formats.
User account credentials
Use user account credentials for authentication when your application requires access to data on a user's behalf; otherwise, use service account credentials. Here are examples of scenarios where user account credentials can be used:
- Web server applications
- Installed and desktop applications
- Applications on limited-input devices
For more information on these scenarios, see OAuth scenarios.
If you are designing an application to support multiple authentication options for end users, then use Google Identity Toolkit, which supports password authentication as well as federated sign in with identity providers like Google, Facebook, Yahoo, and Microsoft.
When an application is granted an access token in a user-centric auth flow by
an end-user, that access token will only have the permissions available to the
user who grants the token. For example, if email@example.com has
example-bucket, an application which Jane has granted
to will be unable to write to
example-bucket on her behalf.