Google Cloud Platform Auth Guide

Google Cloud Platform provides a variety of services to customers, such as Cloud Storage and BigQuery. Typically, customers access these services through their APIs (or indirectly through a web interface or command-line tools). For security and privacy reasons, all Google Cloud Platform APIs require authentication and authorization. This guide walks you through the common scenarios of using Cloud Platform APIs, and introduces Application Default Credentials as a simple way to authenticate and authorize across different environments.

Key concepts

User accounts and service accounts

Google supports different types of accounts. The most important ones for Google Cloud Platform are User Accounts and Service Accounts. User accounts represent end users who authorize access to their data, while service accounts authorize server-side applications to access Google APIs. Both are identified by an email address. For example, you need to use user accounts with Gmail and the Google Cloud Platform Console, but you should use service accounts to run your servers, so they don't stop working when you change your password.

Projects and resources

Google Cloud Platform uses projects as the unit of resource ownership. All Cloud Platform resources—such as App Engine apps, Compute Engine instances, Cloud Storage buckets, API keys—are owned by projects. A project can be shared by multiple users for collaboration.

Authentication and authorization

Authentication is the process of determining the identity of a client, which is typically a user account or a service account. Authorization is the process of determining what permissions an authenticated identity has on a set of resources.

For Google APIs, there can be no authorization without authentication, therefore, you may see the term authentication (auth) used to refer to both authentication and authorization.

OAuth

OAuth 2.0 is an industry standard for authentication and authorization with network services. Google Cloud Platform uses OAuth 2.0 for API authentication and authorization. The most commonly used scenarios are the user-centric flow and the server-centric flow.

The user-centric flow allows an application to obtain credentials to an authentication provider from an end user. Because there are 3 parties involved (application, authentication provider, and end user), this process is also known as 3-legged OAuth (3LO).

The user-centric flow allows an application to obtain credentials to
    an authentication provider from an end user.
Your app
User consent
User data
Figure 1: User-centric authentication.

The server-centric flow allows your application to directly hold credentials of a service account to an authentication provider. Because there are 2 parties involved (application and the authentication provider), this process is also known as 2-legged OAuth (2LO).

The server-centric flow allows your
    application to directly hold credentials of a service account to an
    authentication provider.
Your service
Authorization
Google service
Figure 2: Server-centric authentication.

Once your application has credentials, it can contact the authentication provider to obtain an OAuth access token for certain services and use the token to access these services, such as making an API call.

OAuth scopes

Scopes is an OAuth feature that limits the permissions of an OAuth credential. A user account or a service account may have a wide range of permissions on a resource, for example, Storage Read and Storage Write on a Cloud Storage bucket. For security and privacy reasons, your application may not need all these permissions. The OAuth scope feature lets you limit the credential to a subset of these permissions/scopes, for example, Storage Read. You can find the list of scopes that a Google API method requires in its reference documentation. See the Google Cloud Pub/Sub projects.topics.list method for an example.

Application Default Credentials

Application Default Credentials (ADC) is part of Google APIs client libraries that simplifies authentication to Google Cloud Platform APIs. It abstracts authentication across different environments, allowing you to authenticate with just a single line of code. It is designed for use cases where an application works with one authenticated identity and a fixed set of OAuth scopes.

Developer workflow

All scenarios boil down to the same workflow. Your application obtains an OAuth credential and then makes a request to a Google Cloud Platform API, passing in an access token derived from the credential. The API uses the access token to decide whether to allow your request.

Preparation

To get started using Google APIs, you will need to satisfy the following prerequisites in the Google Cloud Platform Console:

  1. If you don't already have a project, you need to create one.

  2. In API Manager, enable the APIs that you plan to use in your project. By enabling these APIs, you are subject to the API's Terms of Service, Privacy Policy, quota limits, and billing charges.

Getting credentials for server-centric flow

Here are the recommended ways to obtain credentials in various scenarios. Application Default Credentials is the recommended method for getting credentials because of its ease of use and versatility across environments.

The advantage of using Application Default Credentials is that your code will run with little to no modification when you deploy your code into production environments.

Local development

To run your code locally during development and testing, use the Google Cloud SDK (gcloud) along with Application Default Credentials. After you authenticate the SDK using the gcloud beta auth application-default login command, API client libraries that use the application default credentials automatically pick up the created credentials. You only need to run the command once per local user environment.

Service running in Compute Engine

To run your code on Compute Engine, you should use Application Default Credentials. API client libraries that use application default credentials automatically obtain the built-in service account credentials from the Compute Engine metadata service at runtime.

Service running in App Engine

To run your code on App Engine, you should use Application Default Credentials. API client libraries that use application default credentials automatically obtain the built-in service account credentials from the App Engine host at runtime.

Service running on-premises

To run your code outside Google Cloud Platform, such as in your own private datacenter or in another public cloud, you should use Application Default Credentials with explicitly created service accounts:

  1. Use the Google Cloud Platform Console to create a service account, and download its credentials JSON file to your servers. Make sure to keep the file secure.

  2. Set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the path of your downloaded credentials JSON file.

User-centric applications

To build applications that request access to user data, you need to use user-centric OAuth flow. See details at Using OAuth 2.0 to Access Google APIs.

Token lifecycle management

OAuth tokens, API keys and other credentials provide access to sensitive data. The following practices ensure protecting access to these resources.

  • Do not embed secrets related to authentication in source code, such as API keys, OAuth tokens, and service account credentials. You may use an environment variable pointing to credentials outside of the application's source code, such as a secure key manager.
  • Use different credentials in different contexts, such as in testing and production environments.
  • Only transfer credentials over HTTPS to prevent a third party from intercepting your credentials. Never transfer in clear text or as part of the URL.
  • Never embed long-lived credentials into your client-side app. For example, do not embed service account credentials into a mobile app. Client-side apps can be examined and credentials can easily be found and used by a third party.
  • If you no longer need a token, revoke it.

Learn more about OAuth 2.0

Code samples

Application Default Credentials is distributed with the client libraries for Google Cloud Platform APIs, and is currently available in Java, Python, Node, Ruby, Go & .Net. These code samples work for all the environments shown above.

Java

Application Default Credentials are available as of version 1.19 of the Google APIs Client Library for Java. They can be used to access Google Cloud Platform APIs, such as the Cloud Storage API, as follows:

public class StorageFactory {
  private static Storage instance = null;

  public static synchronized Storage getService() throws IOException, GeneralSecurityException {
    if (instance == null) {
      instance = buildService();
    }
    return instance;
  }

  private static Storage buildService() throws IOException, GeneralSecurityException {
    HttpTransport transport = GoogleNetHttpTransport.newTrustedTransport();
    JsonFactory jsonFactory = new JacksonFactory();
    GoogleCredential credential = GoogleCredential.getApplicationDefault(transport, jsonFactory);

    // Depending on the environment that provides the default credentials (for
    // example: Compute Engine, App Engine), the credentials may require us to
    // specify the scopes we need explicitly.  Check for this case, and inject
    // the Cloud Storage scope if required.
    if (credential.createScopedRequired()) {
      Collection<String> scopes = StorageScopes.all();
      credential = credential.createScoped(scopes);
    }

    return new Storage.Builder(transport, jsonFactory, credential)
        .setApplicationName("GCS Samples")
        .build();
  }
}
/**
 * Fetch a list of the objects within the given bucket.
 *
 * @param bucketName the name of the bucket to list.
 * @return a list of the contents of the specified bucket.
 */
public static List<StorageObject> listBucket(String bucketName)
    throws IOException, GeneralSecurityException {
  Storage client = StorageFactory.getService();
  Storage.Objects.List listRequest = client.objects().list(bucketName);

  List<StorageObject> results = new ArrayList<StorageObject>();
  Objects objects;

  // Iterate through each page of results, and add them to our results list.
  do {
    objects = listRequest.execute();
    // Add the items in this page of results to the list we'll return.
    results.addAll(objects.getItems());

    // Get the next page, in the next iteration of this loop.
    listRequest.setPageToken(objects.getNextPageToken());
  } while (null != objects.getNextPageToken());

  return results;
}

Python

Application Default Credentials are available as of version 1.3 of the Google APIs Client Library for Python. They can be used to access Google Cloud Platform APIs, such as the Cloud Storage API, as follows:

def create_service():
    """Creates the service object for calling the Cloud Storage API."""
    # Get the application default credentials. When running locally, these are
    # available after running `gcloud init`. When running on compute
    # engine, these are available from the environment.
    credentials = GoogleCredentials.get_application_default()

    # Construct the service object for interacting with the Cloud Storage API -
    # the 'storage' service, at version 'v1'.
    # You can browse other available api services and versions here:
    #     https://developers.google.com/api-client-library/python/apis/
    return discovery.build('storage', 'v1', credentials=credentials)
...
def list_bucket(bucket):
    """Returns a list of metadata of the objects within the given bucket."""
    service = create_service()

    # Create a request to objects.list to retrieve a list of objects.
    fields_to_return = \
        'nextPageToken,items(name,size,contentType,metadata(my-key))'
    req = service.objects().list(bucket=bucket, fields=fields_to_return)

    all_objects = []
    # If you have too many items to list in one request, list_next() will
    # automatically handle paging with the pageToken.
    while req:
        resp = req.execute()
        all_objects.extend(resp.get('items', []))
        req = service.objects().list_next(req, resp)
    return all_objects

Node.js

Application Default Credentials are available in the Google Cloud Client Library for Node.js. They can be used to access Google Cloud Platform APIs, such as the Cloud Storage API, as follows:

const Storage = require('@google-cloud/storage');
function listBuckets () {
  // Instantiates a client
  const storage = Storage();

  // Lists all buckets in the current project
  return storage.getBuckets()
    .then((results) => {
      const buckets = results[0];

      console.log('Buckets:');
      buckets.forEach((bucket) => console.log(bucket.name));

      return buckets;
    });
}

Ruby

Application Default Credentials are available in the Google Cloud Client Library for Ruby. They can be used to access Google Cloud Platform APIs, such as the Cloud Storage API, as follows:

# project_id = "Your Google Cloud project ID"

require "google/cloud/storage"

storage = Google::Cloud::Storage.new project: project_id

storage.buckets.each do |bucket|
  puts bucket.name
end

Go

Application Default Credentials are available as of 2015-03-18 in the Go OAuth2 package. They can be used, in conjunction with the Google APIs Client Library for Go, to access Google Cloud Platform APIs, such as the Cloud Storage API, as follows:

// ListBuckets returns a slice of all the buckets for a given project.
func ListBuckets(projectID string) ([]*storage.Bucket, error) {
	ctx := context.Background()

	// Create the client that uses Application Default Credentials
	// See https://developers.google.com/identity/protocols/application-default-credentials
	client, err := google.DefaultClient(ctx, storage.DevstorageReadOnlyScope)
	if err != nil {
		return nil, err
	}

	// Create the Google Cloud Storage service
	service, err := storage.New(client)
	if err != nil {
		return nil, err
	}

	buckets, err := service.Buckets.List(projectID).Do()
	if err != nil {
		return nil, err
	}

	return buckets.Items, nil
}

.Net

Application Default Credentials are available as of version 1.9.3 of the .Net Client Library for Google APIs. They can be used to access Google Cloud Platform APIs, such as the Cloud Storage API, as follows:

/// <summary>
/// Creates an authorized Cloud Storage client service using Application 
/// Default Credentials.
/// </summary>
/// <returns>an authorized Cloud Storage client.</returns>
public StorageService CreateAuthorizedClient()
{
    GoogleCredential credential =
        GoogleCredential.GetApplicationDefaultAsync().Result;
    // Inject the Cloud Storage scope if required.
    if (credential.IsCreateScopedRequired)
    {
        credential = credential.CreateScoped(new[]
        {
            StorageService.Scope.DevstorageReadOnly
        });
    }
    return new StorageService(new BaseClientService.Initializer()
    {
        HttpClientInitializer = credential,
        ApplicationName = "DotNet Google Cloud Platform Auth Sample",
    });
}
/// <summary>
/// List the contents of a Cloud Storage bucket.
/// </summary>
/// <param name="bucket">the name of the Cloud Storage bucket.</param>
///<returns>a list of the contents of the specified bucket.</returns>
public Objects ListBucketContents(
    StorageService storage, string bucket)
{
    var request = new
        Google.Apis.Storage.v1.ObjectsResource.ListRequest(storage,
        bucket);
    var requestResult = request.Execute();
    return requestResult;
}

Error handling

Use the following error codes descriptions to troubleshoot failing API requests.

Error Code Description
400 Bad request. Typically some request parameters are invalid.
401 Request not authenticated. Missing or invalid or expired authentication token.
403 Permission denied. The authentication token doesn't have the right OAuth scopes, the API is not enabled for the client project, or the client identity does not have sufficient permissions to some resources.
404 Resource not found, or client does not have permissions to access it. For privacy reasons, 404 might be used instead of 403 for permission errors.
409 Conflict. Request failed due to concurrency conflict on the resource being operated upon.
429 Out of API quota. Use the Google Cloud Platform Console to review quota usage for the API the request tried to access. Minimum recommended retry delay is 30s.
500 Server internal error. Client should retry with exponential back off. Minimum recommended retry delay is 1s.
503 Server currently unavailable. Client should retry with exponential back off. Minimum recommended retry delay is 1s.
504 Deadline exceeded. If this happens often, client should reduce the complexity of requests to meet the deadline requirement.
529 Request timed out. Client should retry with exponential back off.

Send feedback about...

Documentation