Most of the operations you perform in Google Cloud Storage must be authenticated.
The only exceptions are operations on objects that allow anonymous
access. Objects are anonymously accessible if the
READ permission. The
allUsers group includes
anyone on the Internet.
- gsutil authentication
- Client library authentication
- API authentication
- Cookie-based authentication
- Service account credentials
- User account credentials
Google Cloud Storage uses OAuth 2.0 for API authentication and authorization. Authentication is the process of determining the identity of a client.The details of authentication vary depending on how you are accessing Google Cloud Storage, but fall into two general types:
A server-centric flow allows an application to directly hold the credentials of a service account to complete authentication. Use this flow if your application works with its own data rather than user data. Google Cloud Platform projects have default service accounts you can use, or you can create new ones.
A user-centric flow allows an application to obtain credentials from an end user. The user signs in to complete authentication. Use this flow if your application needs to access user data. See the User account credentials section later in this page for scenarios where a user-centric flow is appropriate.
Keep in mind that you can use both types of authentication together in an application. For more background information about authentication, see the Google Cloud Platform Auth Guide.
Authorization is the process of determining what permissions an authenticated
identity has on a set of specified resources. OAuth uses scopes to
determine if an authenticated identity is authorized. Applications use a
credential (obtained from a user-centric or server-centric authentication flow)
together with one or more scopes to request an access token from a Google
authorization server to access protected resources. For example, application A
with an access token with
read-only scope can only read, while application B
with an access token with
read-write scope can read and modify data. Neither
application can read or modify access control lists on objects and buckets;
only an application with
full-control scope can do so.
||Only allows access to read data, including listing buckets.||
||Allows access to read and change data, but not metadata like ACLs.||
||Allows full control over data, including the ability to modify ACLs.||
||View your data across Google Cloud Platform services. For Google Cloud Storage,
this is the same as
||View and manage data across all Google Cloud Platform services. For
Google Cloud Storage, this is the same as
Using service account credentials
Using user account credentials
gcloud auth login to authenticate with user account
$ gcloud auth login
gcloud auth uses the
cloud-platform scope when getting an access token.
If you installed gsutil independent of the Cloud SDK, then see the gsutil install page for information about how to authenticate with a user or service account.
Client library authentication
Application Default Credentials are part of the client libraries you can use to access Google Cloud Storage. Default credentials identify your application with either a user credential or a default service account. When working with a library, it's typical that you'll choose credentials based on the type of environment where you run your code.
Initialize your environment
- Local Development Environment
To initialize your local development environment, you can use the Google Cloud SDK to authenticate with user credentials. On Linux/Mac OS X, use the script:
$ curl https://sdk.cloud.google.com | bashOnce installed, you can use your own user account credentials as a proxy to test code calling APIs from your local machine by running:
$ gcloud auth loginThis command requests access tokens with the
- Google Cloud Platform
If you're running your application on Google App Engine or Google Compute Engine, the environment already provides a service account's authentication information, so no further setup is required. For Compute Engine, the service account scope depends on how you created the instance. See Setting the scope of service account access for instances. For App Engine, the
cloud-platformscope is used.
- Other Environments
For production environments outside the Google Cloud Platform, you provide a service account's authentication information through an environment variable and the scope is set in code. For more information, see How the Application Default Credentials work. Creating a service account is discussed later in this page.
Examples of using Application Default Credentials
To make requests using OAuth 2.0 to either the Google Cloud Storage XML API or
JSON API, include your application's access token in the
Authorization header in every request that requires authentication.
Authorization: Bearer <oauth access token>
The following is an example of a request that lists objects in a bucket.
Use the list method of the Objects resource.
GET /storage/v1/b/example-bucket/o HTTP/1.1 Host: www.googleapis.com Authorization: Bearer ya29.AHES6ZRVmB7fkLtd1XTmq6mo0S1wqZZi3-Lh_s-6Uw7p8vtgSwg
Use a List objects request.
GET / HTTP/1.1 Host: example-bucket.storage.googleapis.com Authorization: Bearer ya29.AHES6ZRVmB7fkLtd1XTmq6mo0S1wqZZi3-Lh_s-6Uw7p8vtgSwg
Due to the complexity of managing and refreshing access tokens and the security risk when dealing directly with cryptographic applications, we strongly encourage you to use a verified client library.
If you looking for developer keys to use with the XML API for interoperable access with Amazon S3, see Managing developer keys for a simple migration.
Google Cloud Storage lets you provide browser-based authenticated downloads to users who do not have Google Cloud Storage accounts. To do this, apply Google account- based Access Control Lists (ACLs) to the object and then provide users with a URL that is scoped to the object. The URL for browser-based authenticated downloads is:
For example, to let Jane download an object named /europe/france/paris.jpg
that's in a bucket named example-travel-maps, you grant email@example.com
READ permission for the object /europe/france/paris.jpg. Then you
provide Jane with the following URL:
When Jane clicks the URL in her browser she's automatically prompted to sign in to her Google account (if she's not already logged in). After she is authenticated, and her browser has acquired a cookie with an encapsulated identity token, she is redirected to the object in the Google Cloud Storage repository. Google Cloud Storage then verifies that Jane is allowed to read the object, and the object is downloaded to Jane's computer.
The following figure shows how the authentication process for browser-based authenticated downloads works.
Service account credentials
Service accounts are special accounts that represent software rather than people. They are the most common way applications authenticate with Google Cloud Storage. Every project has service accounts associated with it, which may be used for different authentication scenarios, as well as to enable advanced features such as Signed URLs and browser uploads using POST.
When you use a service account to authenticate your application, you do not need a user to authenticate to get an access token. Instead, you obtain a private key from the Google Developers Console, which you then use to send a signed request for an access token. You can then use the access token like you normally would. For more information see the Google Cloud Platform Auth Guide.
Generating a service account credential
You can create a private key in the Developers Console by creating an OAuth Client ID for a service account.
To generate a private key in JSON or PKCS12 format:
- Go to the Google Developers Console.
- Select a project to which the service account will be associated.
- In the left sidebar, select APIs & auth > Credentials.
- Click Add credentials and select Service account.
- In the Create service account window, select the Key type, either JSON or P12.
- Click Create.
A New public/private key pair window is displayed and the private key for the Key type you selected is downloaded automatically. If you selected a P12 key, the private key's password ("notasecret") is displayed.
- Click Close.
You can get your private key in JSON and PKCS12 format. For more information about security and identity in projects, see the Google Developers Console Help.
PKCS12 is supported by many different programming languages and libraries, and if needed, you can convert the key into other formats using OpenSSL (see Converting the private key to other formats).
Service credential details
The OAuth client ID of a service account uniquely identifies the account being used for the authentication. After you create a Client ID for a service account (see Generating a private key), you will see information for your client ID similar to the following example:
Notice that there are two identifiers of the service account, the
Client ID and the Email address. You must use
the Email address for RSA signatures, for example, for Signed URLs (Query String
Authentication). The two identifiers are closely related, you can convert
from the Client ID identifier to the Email
address identifier by replacing
@developer.gserviceaccount.com. For example, this Client
Converting the service account credential to other formats
After you create a Client ID for a service account (see Generating a private key) and download the private key, you might need to convert the key to other formats. The key is provided in PKCS12 format, which is supported by many different programming languages and libraries. However, it can be converted into other formats using OpenSSL.
# Convert the key from pkcs12 to pkcs1 (PEM). $ cat /path/to/xxxx-privatekey.p12 | openssl pkcs12 -nodes -nocerts -passin pass:notasecret | openssl rsa > /path/to/secret.pem
You can use openssl to generate a digest of the data to sign:
# Generate a signature digest using the PEM key. $ openssl dgst -sign path/to/key.pem -sha256 path/to/data_to_sign | base64 -w0
You can also use openssl to convert the PEM file format to DER format. Some libraries (e.g., PyCrypto) require keys be in DER format.
# Convert the key from pkcs1 (PEM) to DER. $ openssl rsa -in path/to/key.pem -inform PEM -out path/to/key.der -outform DER
User account credentials
Use user account credentials for authentication when your application requires access to data on a user's behalf; otherwise, use service account credentials. Here are examples of scenarios where user account credentials can be used:
- Web server applications
- Installed and desktop applications
- Applications on limited-input devices
For more information on these scenarios, see, OAuth scenarios.
If you are designing an application to support multiple authentication options for end users, then use Google Identity Toolkit, which supports password authentication as well as federated sign in with identity providers like Google, Facebook, Yahoo, and Microsoft.
When an application is granted an access token in a user-centric auth flow by
an end-user, that access token will only have the permissions available to the
user who grants the token. For example, if firstname.lastname@example.org has
example-bucket, an application which Jane has granted
to will be unable to write to
example-bucket on her behalf.