Google Cloud Platform Auth Guide

Google Cloud Platform provides a variety of services to customers, such as Cloud Storage and BigQuery. Typically, customers access these services through their APIs (or indirectly through a web interface or command-line tools). For security and privacy reasons, all Google Cloud Platform APIs require authentication and authorization. This guide walks you through the common scenarios of using Cloud Platform APIs, and introduces Application Default Credentials as a simple way to authenticate and authorize across different environments.

Key concepts

User accounts and service accounts

Google supports different types of accounts. The most important ones for Google Cloud Platform are User Accounts and Service Accounts. User accounts represent end users who authorize access to their data, while service accounts authorize server-side applications to access Google APIs. Both are identified by an email address. For example, you need to use user accounts with Gmail and the Google Cloud Platform Console, but you should use service accounts to run your servers, so they don't stop working when you change your password.

Projects and resources

Google Cloud Platform uses projects as the unit of resource ownership. All Cloud Platform resources—such as App Engine apps, Compute Engine instances, Cloud Storage buckets, API keys—are owned by projects. A project can be shared by multiple users for collaboration.

Authentication and authorization

Authentication is the process of determining the identity of a client, which is typically a user account or a service account. Authorization is the process of determining what permissions an authenticated identity has on a set of resources.

For Google APIs, there can be no authorization without authentication, therefore, you might see the term authentication (auth) used to refer to both authentication and authorization.

OAuth

OAuth 2.0 is an industry standard for authentication and authorization with network services. Google Cloud Platform uses OAuth 2.0 for API authentication and authorization. The most commonly used scenarios are the user-centric flow and the server-centric flow.

The user-centric flow allows an application to obtain credentials to an authentication provider from an end user. Because there are 3 parties involved (application, authentication provider, and end user), this process is also known as 3-legged OAuth (3LO). This flow is typically used for client applications that access data on behalf of a user, such as access to Google Calendar data.

The user-centric flow allows an application to obtain credentials to
    an authentication provider from an end user.
Your app
User consent
User data
Figure 1: User-centric authentication.

The server-centric flow allows an application to directly hold credentials of a service account to an authentication provider. Because there are 2 parties involved (application and the authentication provider), this process is also known as 2-legged OAuth (2LO). This flow is typically used for server applications where data is accessed on behalf of the application rather than on behalf of a specific user.

The server-centric flow allows your
    application to directly hold credentials of a service account to an
    authentication provider.
Your service
Authorization
Google service
Figure 2: Server-centric authentication.

After your application has the approriate OAuth credentials for your scenario, the application can retrieve an access token from Google to use to access the Google APIs. For more information, see OAuth 2.0

OAuth scopes

OAuth scopes are an OAuth feature that limit the permissions of an OAuth credential. A user account or a service account can have a wide range of permissions, such as read and write a database. An application might not need all these permissions. The OAuth scope feature lets you limit the credential to a subset of the permissions, such as read a database. You can find the required OAuth scopes of an API method in its reference documentation, such as the Cloud Pub/Sub projects.topics.list method.

Application Default Credentials

Application Default Credentials (ADC) is part of Google APIs client libraries that simplifies authentication to Google Cloud Platform APIs. It abstracts authentication across different environments, allowing you to authenticate with just a single line of code. It is designed for use cases where an application works with one authenticated identity and a fixed set of OAuth scopes.

Developer workflow

All authentication scenarios boil down to the same workflow. Your application obtains an OAuth credential, then makes an API request with an OAuth access token derived from the credential. The API server uses the token to determine whether to allow the request.

Preparation

To get started using Google APIs, you will need to satisfy the following prerequisites in the Google Cloud Platform Console:

  1. If you don't already have a project, you need to create one.

  2. In API Manager, enable the APIs that you plan to use in your project. By enabling these APIs, you are subject to the API's Terms of Service, Privacy Policy, quota limits, and billing charges.

Getting credentials for server-centric flow

Here are the recommended ways to obtain credentials in various scenarios. Application Default Credentials is the recommended method for getting credentials because of its ease of use and versatility across environments.

The advantage of using Application Default Credentials is that your code will run with little to no modification when you deploy your code into production environments.

Local development

To run your code locally during development and testing, use the Google Cloud SDK (gcloud) along with Application Default Credentials. After you authenticate the SDK using the gcloud beta auth application-default login command, API client libraries that use the application default credentials automatically pick up the created credentials. You only need to run the command once per local user environment.

Service running in Compute Engine

To run your code on Compute Engine, you should use Application Default Credentials. API client libraries that use application default credentials automatically obtain the built-in service account credentials from the Compute Engine metadata service at runtime.

Service running in App Engine

To run your code on App Engine, you should use Application Default Credentials. API client libraries that use application default credentials automatically obtain the built-in service account credentials from the App Engine host at runtime.

Service running on-premises

To run your code outside Google Cloud Platform, such as in your own private datacenter or in another public cloud, you should use Application Default Credentials with explicitly created service accounts:

  1. Use the Google Cloud Platform Console to create a service account, and download its credentials JSON file to your servers. Make sure to keep the file secure.

  2. Set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the path of your downloaded credentials JSON file.

User-centric applications

To build applications that request access to user data, you need to use user-centric OAuth flow. See details at Using OAuth 2.0 to Access Google APIs.

Token lifecycle management

OAuth tokens, API keys and other credentials provide access to sensitive data. The following practices ensure protecting access to these resources.

  • Do not embed secrets related to authentication in source code, such as API keys, OAuth tokens, and service account credentials. You can use an environment variable pointing to credentials outside of the application's source code, such as a secure key manager.
  • Use different credentials in different contexts, such as in testing and production environments.
  • Only transfer credentials over HTTPS to prevent a third party from intercepting your credentials. Never transfer in clear text or as part of the URL.
  • Never embed long-lived credentials into your client-side app. For example, do not embed service account credentials into a mobile app. Client-side apps can be examined and credentials can easily be found and used by a third party.
  • If you no longer need a token, revoke it.

Code samples

Application Default Credentials is distributed with the client libraries for Google Cloud Platform APIs, and is currently available in Java, Python, Node, Ruby, Go & .Net. These code samples work for all the environments shown above.

C#

Application Default Credentials are available as of version 1.9.3 of the .Net Client Library for Google APIs. They can be used to access Google Cloud Platform APIs, such as the Cloud Storage API, as follows:

/// <summary>
/// Creates an authorized Cloud Storage client service using Application 
/// Default Credentials.
/// </summary>
/// <returns>an authorized Cloud Storage client.</returns>
public StorageService CreateAuthorizedClient()
{
    GoogleCredential credential =
        GoogleCredential.GetApplicationDefaultAsync().Result;
    // Inject the Cloud Storage scope if required.
    if (credential.IsCreateScopedRequired)
    {
        credential = credential.CreateScoped(new[]
        {
            StorageService.Scope.DevstorageReadOnly
        });
    }
    return new StorageService(new BaseClientService.Initializer()
    {
        HttpClientInitializer = credential,
        ApplicationName = "DotNet Google Cloud Platform Auth Sample",
    });
}
/// <summary>
/// List the contents of a Cloud Storage bucket.
/// </summary>
/// <param name="bucket">the name of the Cloud Storage bucket.</param>
///<returns>a list of the contents of the specified bucket.</returns>
public Objects ListBucketContents(
    StorageService storage, string bucket)
{
    var request = new
        Google.Apis.Storage.v1.ObjectsResource.ListRequest(storage,
        bucket);
    var requestResult = request.Execute();
    return requestResult;
}

Go

Application Default Credentials are available as of 2015-03-18 in the Go OAuth2 package. They can be used, in conjunction with the Google APIs Client Library for Go, to access Google Cloud Platform APIs, such as the Cloud Storage API, as follows:

// ListBuckets returns a slice of all the buckets for a given project.
func ListBuckets(projectID string) ([]*storage.Bucket, error) {
	ctx := context.Background()

	// Create the client that uses Application Default Credentials
	// See https://developers.google.com/identity/protocols/application-default-credentials
	client, err := google.DefaultClient(ctx, storage.DevstorageReadOnlyScope)
	if err != nil {
		return nil, err
	}

	// Create the Google Cloud Storage service
	service, err := storage.New(client)
	if err != nil {
		return nil, err
	}

	buckets, err := service.Buckets.List(projectID).Do()
	if err != nil {
		return nil, err
	}

	return buckets.Items, nil
}

Java

Application Default Credentials are available as of version 1.19 of the Google APIs Client Library for Java. They can be used to access Google Cloud Platform APIs, such as the Cloud Storage API, as follows:

public class StorageFactory {
  private static Storage instance = null;

  public static synchronized Storage getService() throws IOException, GeneralSecurityException {
    if (instance == null) {
      instance = buildService();
    }
    return instance;
  }

  private static Storage buildService() throws IOException, GeneralSecurityException {
    HttpTransport transport = GoogleNetHttpTransport.newTrustedTransport();
    JsonFactory jsonFactory = new JacksonFactory();
    GoogleCredential credential = GoogleCredential.getApplicationDefault(transport, jsonFactory);

    // Depending on the environment that provides the default credentials (for
    // example: Compute Engine, App Engine), the credentials may require us to
    // specify the scopes we need explicitly.  Check for this case, and inject
    // the Cloud Storage scope if required.
    if (credential.createScopedRequired()) {
      Collection<String> scopes = StorageScopes.all();
      credential = credential.createScoped(scopes);
    }

    return new Storage.Builder(transport, jsonFactory, credential)
        .setApplicationName("GCS Samples")
        .build();
  }
}
/**
 * Fetch a list of the objects within the given bucket.
 *
 * @param bucketName the name of the bucket to list.
 * @return a list of the contents of the specified bucket.
 */
public static List<StorageObject> listBucket(String bucketName)
    throws IOException, GeneralSecurityException {
  Storage client = StorageFactory.getService();
  Storage.Objects.List listRequest = client.objects().list(bucketName);

  List<StorageObject> results = new ArrayList<StorageObject>();
  Objects objects;

  // Iterate through each page of results, and add them to our results list.
  do {
    objects = listRequest.execute();
    // Add the items in this page of results to the list we'll return.
    results.addAll(objects.getItems());

    // Get the next page, in the next iteration of this loop.
    listRequest.setPageToken(objects.getNextPageToken());
  } while (null != objects.getNextPageToken());

  return results;
}

Node.js

Application Default Credentials are available in the Google Cloud Client Library for Node.js. They can be used to access Google Cloud Platform APIs, such as the Cloud Storage API, as follows:

const Storage = require('@google-cloud/storage');
function listBuckets () {
  // Instantiates a client
  const storage = Storage();

  // Lists all buckets in the current project
  return storage.getBuckets()
    .then((results) => {
      const buckets = results[0];

      console.log('Buckets:');
      buckets.forEach((bucket) => console.log(bucket.name));

      return buckets;
    });
}

Python

Application Default Credentials are available as of version 1.3 of the Google APIs Client Library for Python. They can be used to access Google Cloud Platform APIs, such as the Cloud Storage API, as follows:

def create_service():
    """Creates the service object for calling the Cloud Storage API."""
    # Construct the service object for interacting with the Cloud Storage API -
    # the 'storage' service, at version 'v1'.
    # You can browse other available api services and versions here:
    #     https://developers.google.com/api-client-library/python/apis/
    return googleapiclient.discovery.build('storage', 'v1')
...
def list_bucket(bucket):
    """Returns a list of metadata of the objects within the given bucket."""
    service = create_service()

    # Create a request to objects.list to retrieve a list of objects.
    fields_to_return = \
        'nextPageToken,items(name,size,contentType,metadata(my-key))'
    req = service.objects().list(bucket=bucket, fields=fields_to_return)

    all_objects = []
    # If you have too many items to list in one request, list_next() will
    # automatically handle paging with the pageToken.
    while req:
        resp = req.execute()
        all_objects.extend(resp.get('items', []))
        req = service.objects().list_next(req, resp)
    return all_objects

Ruby

Application Default Credentials are available in the Google Cloud Client Library for Ruby. They can be used to access Google Cloud Platform APIs, such as the Cloud Storage API, as follows:

# project_id = "Your Google Cloud project ID"

require "google/cloud/storage"

storage = Google::Cloud::Storage.new project: project_id

storage.buckets.each do |bucket|
  puts bucket.name
end

Error handling

Learn more about how to troubleshoot failed API requests at Cloud APIs errors.

Send feedback about...

Documentation