Setting Up Authentication for Server to Server Production Applications

This guide explains how to set up authentication and authorization for server to server production applications. Authentication refers to the process of determining a client's identity. Authorization refers to the process of determining what permissions an authenticated client has for a specific resource. That is, authentication identifies who you are, and authorization determines what you can do. For more information about supported authentication methods and how to choose them, see Authentication overview.

Google uses credentials to identify your application for quota and billing. Your credentials are also used to authorize access to GCP APIs, resources, and features.

Providing credentials to your application

Finding credentials automatically

GCP client libraries use a strategy called Application Default Credentials (ADC) to find your application's credentials. When your code uses a client library, the strategy checks for your credentials in the following order:

First, ADC checks to see if the environment variable GOOGLE_APPLICATION_CREDENTIALS is set. If the variable is set, ADC uses the service account file that the variable points to. The next section describes how to set the environment variable.
If the environment variable isn't set, ADC uses the default service account that Compute Engine, Kubernetes Engine, Cloud Run, App Engine, and Cloud Functions provide, for applications that run on those services.
If ADC can't use either of the above credentials, an error occurs.

The following code example illustrates this strategy. The example doesn't explicitly specify the application credentials. However, ADC is able to implicitly find the credentials as long as the GOOGLE_APPLICATION_CREDENTIALS environment variable is set, or as long as the application is running on Compute Engine, Kubernetes Engine, App Engine, or Cloud Functions.

You must install the Cloud Storage client library to run the following example.

C#

View on GitHub

public object AuthImplicit(string projectId)
{
    // If you don't specify credentials when constructing the client, the
    // client library will look for credentials in the environment.
    var credential = GoogleCredential.GetApplicationDefault();
    var storage = StorageClient.Create(credential);
    // Make an authenticated API request.
    var buckets = storage.ListBuckets(projectId);
    foreach (var bucket in buckets)
    {
        Console.WriteLine(bucket.Name);
    }
    return null;
}

Go

View on GitHub


// implicit uses Application Default Credentials to authenticate.
func implicit() {
	ctx := context.Background()

	// For API packages whose import path is starting with "cloud.google.com/go",
	// such as cloud.google.com/go/storage in this case, if there are no credentials
	// provided, the client library will look for credentials in the environment.
	storageClient, err := storage.NewClient(ctx)
	if err != nil {
		log.Fatal(err)
	}

	it := storageClient.Buckets(ctx, "project-id")
	for {
		bucketAttrs, err := it.Next()
		if err == iterator.Done {
			break
		}
		if err != nil {
			log.Fatal(err)
		}
		fmt.Println(bucketAttrs.Name)
	}

	// For packages whose import path is starting with "google.golang.org/api",
	// such as google.golang.org/api/cloudkms/v1, use NewService to create the client.
	kmsService, err := cloudkms.NewService(ctx)
	if err != nil {
		log.Fatal(err)
	}

	_ = kmsService
}

Java

View on GitHub

static void authImplicit() {
  // If you don't specify credentials when constructing the client, the client library will
  // look for credentials via the environment variable GOOGLE_APPLICATION_CREDENTIALS.
  Storage storage = StorageOptions.getDefaultInstance().getService();

  System.out.println("Buckets:");
  Page<Bucket> buckets = storage.list();
  for (Bucket bucket : buckets.iterateAll()) {
    System.out.println(bucket.toString());
  }
}

Node.js

View on GitHub

// Imports the Google Cloud client library.
const {Storage} = require('@google-cloud/storage');

// Instantiates a client. If you don't specify credentials when constructing
// the client, the client library will look for credentials in the
// environment.
const storage = new Storage();

try {
  // Makes an authenticated API request.
  const results = await storage.getBuckets();

  const [buckets] = results;

  console.log('Buckets:');
  buckets.forEach(bucket => {
    console.log(bucket.name);
  });
} catch (err) {
  console.error('ERROR:', err);
}

PHP

View on GitHub

// Imports the Cloud Storage client library.
use Google\Cloud\Storage\StorageClient;

function auth_cloud_implicit($projectId)
{
    $config = [
        'projectId' => $projectId,
    ];

    # If you don't specify credentials when constructing the client, the
    # client library will look for credentials in the environment.
    $storage = new StorageClient($config);

    # Make an authenticated API request (listing storage buckets)
    foreach ($storage->buckets() as $bucket) {
        printf('Bucket: %s' . PHP_EOL, $bucket->name());
    }
}

Python

View on GitHub

def implicit():
    from google.cloud import storage

    # If you don't specify credentials when constructing the client, the
    # client library will look for credentials in the environment.
    storage_client = storage.Client()

    # Make an authenticated API request
    buckets = list(storage_client.list_buckets())
    print(buckets)

Ruby

View on GitHub

# project_id = "Your Google Cloud project ID"

require "google/cloud/storage"

# If you don't specify credentials when constructing the client, the client
# library will look for credentials in the environment.
storage = Google::Cloud::Storage.new project: project_id

# Make an authenticated API request
storage.buckets.each do |bucket|
  puts bucket.name
end

This strategy is useful when testing and experimenting, but can make it hard to tell which credentials your application is using. We recommend explicitly specifying which credentials the application should use, as described in the following section.

Obtaining and providing service account credentials manually

If you're developing code locally, deploying your application on-premises, or deploying to another public cloud, you can create and obtain service account credentials manually.

Creating a service account

The following steps describe how to create a service account. However, instead of setting owner-level permissions, you should restrict access to the permissions, as described in the restricting access section below.

Cloud Console

In the Cloud Console, go to the Create service account key page.
Go to the Create Service Account Key page
From the Service account list, select New service account.
In the Service account name field, enter a name.
From the Role list, select Project > Owner.

Note: The Role field authorizes your service account to access resources. You can view and change this field later by using the Cloud Console. If you are developing a production app, specify more granular permissions than Project > Owner. For more information, see granting roles to service accounts.
Click Create. A JSON file that contains your key downloads to your computer.

Command line

You can run the following commands using the Cloud SDK on your local machine, or in Cloud Shell.

Create the service account. Replace [NAME] with a name for the service account.
```
gcloud iam service-accounts create [NAME]
```
Grant permissions to the service account. Replace [PROJECT_ID] with your project ID.
```
gcloud projects add-iam-policy-binding [PROJECT_ID] --member "serviceAccount:[NAME]@[PROJECT_ID].iam.gserviceaccount.com" --role "roles/owner"
```
Note: The Role field authorizes your service account to access resources. You can view and change this field later by using Cloud Console. If you are developing a production app, specify more granular permissions than Project > Owner. For more information, see granting roles to service accounts.

Generate the key file. Replace [FILE_NAME] with a name for the key file.

gcloud iam service-accounts keys create [FILE_NAME].json --iam-account [NAME]@[PROJECT_ID].iam.gserviceaccount.com

Providing service account credentials

After creating a service account, you have two choices for providing the credentials to your application. You can either set the GOOGLE_APPLICATION_CREDENTIALS environment variable explicitly, or you can pass the path to the service account key in code.

Setting the environment variable

Provide authentication credentials to your application code by setting the environment variable GOOGLE_APPLICATION_CREDENTIALS. Replace [PATH] with the file path of the JSON file that contains your service account key, and [FILE_NAME] with the filename. This variable only applies to your current shell session, so if you open a new session, set the variable again.

Linux or macOS

export GOOGLE_APPLICATION_CREDENTIALS="[PATH]"

For example:

export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/[FILE_NAME].json"

Windows

With PowerShell:

$env:GOOGLE_APPLICATION_CREDENTIALS="[PATH]"

For example:

$env:GOOGLE_APPLICATION_CREDENTIALS="C:\Users\username\Downloads\[FILE_NAME].json"

With command prompt:

set GOOGLE_APPLICATION_CREDENTIALS=[PATH]

After you've completed the above steps, ADC is able to implicitly determine your credentials, as described in the providing credentials to your application section above. We recommend this approach, because it requires less code. In addition, by using the environment variable, your code is more portable, because you can run the same code in multiple environments with different service accounts.

Passing the path to the service account key in code

You can alternately choose to explicitly point to your service account file in code, as shown in the following code example.

You must install the Cloud Storage client library to run the following example.

C#

View on GitHub

// Some APIs, like Storage, accept a credential in their Create()
// method.
public object AuthExplicit(string projectId, string jsonPath)
{
    // Explicitly use service account credentials by specifying 
    // the private key file.
    var credential = GoogleCredential.FromFile(jsonPath);
    var storage = StorageClient.Create(credential);
    // Make an authenticated API request.
    var buckets = storage.ListBuckets(projectId);
    foreach (var bucket in buckets)
    {
        Console.WriteLine(bucket.Name);
    }
    return null;
}
// Other APIs, like Language, accept a channel in their Create()
// method.
public object AuthExplicit(string projectId, string jsonPath)
{
    var credential = GoogleCredential.FromFile(jsonPath)
        .CreateScoped(LanguageServiceClient.DefaultScopes);
    var channel = new Grpc.Core.Channel(
        LanguageServiceClient.DefaultEndpoint.ToString(),
        credential.ToChannelCredentials());
    var client = LanguageServiceClient.Create(channel);
    AnalyzeSentiment(client);
    return 0;
}

Go

View on GitHub


// explicit reads credentials from the specified path.
func explicit(jsonPath, projectID string) {
	ctx := context.Background()
	client, err := storage.NewClient(ctx, option.WithCredentialsFile(jsonPath))
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println("Buckets:")
	it := client.Buckets(ctx, projectID)
	for {
		battrs, err := it.Next()
		if err == iterator.Done {
			break
		}
		if err != nil {
			log.Fatal(err)
		}
		fmt.Println(battrs.Name)
	}
}

Java

View on GitHub

static void authExplicit(String jsonPath) throws IOException {
  // You can specify a credential file by providing a path to GoogleCredentials.
  // Otherwise credentials are read from the GOOGLE_APPLICATION_CREDENTIALS environment variable.
  GoogleCredentials credentials = GoogleCredentials.fromStream(new FileInputStream(jsonPath))
        .createScoped(Lists.newArrayList("https://www.googleapis.com/auth/cloud-platform"));
  Storage storage = StorageOptions.newBuilder().setCredentials(credentials).build().getService();

  System.out.println("Buckets:");
  Page<Bucket> buckets = storage.list();
  for (Bucket bucket : buckets.iterateAll()) {
    System.out.println(bucket.toString());
  }
}

Node.js

View on GitHub

// Imports the Google Cloud client library.
const {Storage} = require('@google-cloud/storage');

// Instantiates a client. Explicitly use service account credentials by
// specifying the private key file. All clients in google-cloud-node have this
// helper, see https://github.com/GoogleCloudPlatform/google-cloud-node/blob/master/docs/authentication.md
// const projectId = 'project-id'
// const keyFilename = '/path/to/keyfile.json'
const storage = new Storage({projectId, keyFilename});

// Makes an authenticated API request.
try {
  const [buckets] = await storage.getBuckets();

  console.log('Buckets:');
  buckets.forEach(bucket => {
    console.log(bucket.name);
  });
} catch (err) {
  console.error('ERROR:', err);
}

PHP

View on GitHub

namespace Google\Cloud\Samples\Auth;

// Imports the Cloud Storage client library.
use Google\Cloud\Storage\StorageClient;

function auth_cloud_explicit($projectId, $serviceAccountPath)
{
    # Explicitly use service account credentials by specifying the private key
    # file.
    $config = [
        'keyFilePath' => $serviceAccountPath,
        'projectId' => $projectId,
    ];
    $storage = new StorageClient($config);

    # Make an authenticated API request (listing storage buckets)
    foreach ($storage->buckets() as $bucket) {
        printf('Bucket: %s' . PHP_EOL, $bucket->name());
    }
}

Python

View on GitHub

def explicit():
    from google.cloud import storage

    # Explicitly use service account credentials by specifying the private key
    # file.
    storage_client = storage.Client.from_service_account_json(
        'service_account.json')

    # Make an authenticated API request
    buckets = list(storage_client.list_buckets())
    print(buckets)

Ruby

View on GitHub

# project_id = "Your Google Cloud project ID"
# key_file   = "path/to/service-account.json"
require "google/cloud/storage"

# Explicitly use service account credentials by specifying the private key
# file.
storage = Google::Cloud::Storage.new project: project_id, keyfile: key_file

# Make an authenticated API request
storage.buckets.each do |bucket|
  puts bucket.name
end

Obtaining credentials on Compute Engine, Kubernetes Engine, App Engine flexible environment, and Cloud Functions

If your application runs on Compute Engine, Kubernetes Engine, the App Engine flexible environment, or Cloud Functions, you don't need to create your own service account. Compute Engine includes a default service account that is automatically created for you, and you can assign a different service account, per-instance, if needed. When you create a new instance, the instance is automatically enabled to run as the default service account and has a default set of authorization permissions. For more information, see Compute Engine default service account.

After you set up a service account, ADC can implicitly find your credentials without any need to change your code, as described in the section above. If you want to specifically use Compute Engine credentials, you can explicitly do so, as shown in the following code example.

You must install the Cloud Storage client library to run the following example.

C#

View on GitHub

// Some APIs, like Storage, accept a credential in their Create()
// method.
public object AuthExplicitComputeEngine(string projectId)
{
    // Explicitly request service account credentials from the compute
    // engine instance.
    GoogleCredential credential =
        GoogleCredential.FromComputeCredential();
    var storage = StorageClient.Create(credential);
    // Make an authenticated API request.
    var buckets = storage.ListBuckets(projectId);
    foreach (var bucket in buckets)
    {
        Console.WriteLine(bucket.Name);
    }
    return null;
}
// Other APIs, like Language, accept a channel in their Create()
// method.
public object AuthExplicitComputeEngine(string projectId)
{
    var credential = GoogleCredential.FromComputeCredential();
    var channel = new Grpc.Core.Channel(
        LanguageServiceClient.DefaultEndpoint.ToString(),
        credential.ToChannelCredentials());
    var client = LanguageServiceClient.Create(channel);
    AnalyzeSentiment(client);
    return 0;
}

Go

View on GitHub


// explicitDefault finds the default credentials.
//
// It is very uncommon to need to explicitly get the default credentials in Go.
// Most of the time, client libraries can use Application Default Credentials
// without having to pass the credentials in directly. See implicit above.
func explicitDefault(projectID string) {
	ctx := context.Background()

	creds, err := google.FindDefaultCredentials(ctx, storage.ScopeReadOnly)
	if err != nil {
		log.Fatal(err)
	}
	client, err := storage.NewClient(ctx, option.WithCredentials(creds))
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println("Buckets:")
	it := client.Buckets(ctx, projectID)
	for {
		battrs, err := it.Next()
		if err == iterator.Done {
			break
		}
		if err != nil {
			log.Fatal(err)
		}
		fmt.Println(battrs.Name)
	}
}

Java

View on GitHub

static void authCompute() {
  // Explicitly request service account credentials from the compute engine instance.
  GoogleCredentials credentials = ComputeEngineCredentials.create();
  Storage storage = StorageOptions.newBuilder().setCredentials(credentials).build().getService();

  System.out.println("Buckets:");
  Page<Bucket> buckets = storage.list();
  for (Bucket bucket : buckets.iterateAll()) {
    System.out.println(bucket.toString());
  }
}

Node.js

View on GitHub

'use strict';

const {auth, Compute} = require('google-auth-library');

/**
 * This example directly instantiates a Compute client to acquire credentials.
 * Generally, you wouldn't directly create this class, rather call the
 * `auth.getClient()` method to automatically obtain credentials.
 */
async function main() {
  const client = new Compute({
    // Specifying the serviceAccountEmail is optional. It will use the default
    // service account if one is not defined.
    serviceAccountEmail: 'some-service-account@example.com',
  });
  const projectId = await auth.getProjectId();
  const url = `https://dns.googleapis.com/dns/v1/projects/${projectId}`;
  const res = await client.request({url});
  console.log(res.data);
}

main().catch(console.error);

PHP

View on GitHub

namespace Google\Cloud\Samples\Auth;

// Imports GCECredentials and the Cloud Storage client library.
use Google\Auth\Credentials\GCECredentials;
use Google\Cloud\Storage\StorageClient;

function auth_cloud_explicit_compute_engine($projectId)
{
    $gceCredentials = new GCECredentials();
    $config = [
        'projectId' => $projectId,
        'credentialsFetcher' => $gceCredentials,
    ];
    $storage = new StorageClient($config);

    # Make an authenticated API request (listing storage buckets)
    foreach ($storage->buckets() as $bucket) {
        printf('Bucket: %s' . PHP_EOL, $bucket->name());
    }
}

Python

View on GitHub

def explicit_compute_engine(project):
    from google.auth import compute_engine
    from google.cloud import storage

    # Explicitly use Compute Engine credentials. These credentials are
    # available on Compute Engine, App Engine Flexible, and Container Engine.
    credentials = compute_engine.Credentials()

    # Create the client using the credentials and specifying a project ID.
    storage_client = storage.Client(credentials=credentials, project=project)

    # Make an authenticated API request
    buckets = list(storage_client.list_buckets())
    print(buckets)

Ruby

View on GitHub

require "googleauth"
require "google/cloud/env"
require "google/cloud/storage"

# Explicitly use Compute Engine credentials and a project ID to create a new
# Cloud Storage client. These credentials are available on Compute Engine,
# App Engine Flexible, and Container Engine.
storage = Google::Cloud::Storage.new project: Google::Cloud.env.project_id,
                                     keyfile: Google::Auth::GCECredentials.new

# Make an authenticated API request
storage.buckets.each do |bucket|
  puts bucket.name
end

Obtaining credentials on App Engine standard environment

If your application runs on App Engine standard environment, you can use the App Engine App Identity API to obtain credentials.

After you set up a service account, ADC can implicitly find your credentials without any need to change your code, as described in the section above. If you want to specifically use App Engine credentials, you can explicitly do so, as shown in the following code example.

You must install the Cloud Storage client library to run the following example.

PHP

View on GitHub

namespace Google\Cloud\Samples\Auth;

// Imports AppIdentityCredentials and the Cloud Storage client library.
use Google\Auth\Credentials\AppIdentityCredentials;
use Google\Cloud\Storage\StorageClient;

function auth_cloud_explicit_app_engine($projectId)
{
    # Learn more about scopes at https://cloud.google.com/storage/docs/authentication#oauth-scopes
    $scope = 'https://www.googleapis.com/auth/devstorage.read_only';
    $gaeCredentials = new AppIdentityCredentials($scope);
    $config = [
        'projectId' => $projectId,
        'credentialsFetcher' => $gaeCredentials,
    ];
    $storage = new StorageClient($config);

    # Make an authenticated API request (listing storage buckets)
    foreach ($storage->buckets() as $bucket) {
        printf('Bucket: %s' . PHP_EOL, $bucket->name());
    }
}

Go

View on GitHub


// explicitDefault finds the default credentials.
//
// It is very uncommon to need to explicitly get the default credentials in Go.
// Most of the time, client libraries can use Application Default Credentials
// without having to pass the credentials in directly. See implicit above.
func explicitDefault(projectID string) {
	ctx := context.Background()

	creds, err := google.FindDefaultCredentials(ctx, storage.ScopeReadOnly)
	if err != nil {
		log.Fatal(err)
	}
	client, err := storage.NewClient(ctx, option.WithCredentials(creds))
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println("Buckets:")
	it := client.Buckets(ctx, projectID)
	for {
		battrs, err := it.Next()
		if err == iterator.Done {
			break
		}
		if err != nil {
			log.Fatal(err)
		}
		fmt.Println(battrs.Name)
	}
}

Java

View on GitHub

static void authAppEngineStandard() throws IOException {
  // Explicitly request service account credentials from the app engine standard instance.
  GoogleCredentials credentials = AppEngineCredentials.getApplicationDefault();
  Storage storage = StorageOptions.newBuilder().setCredentials(credentials).build().getService();

  System.out.println("Buckets:");
  Page<Bucket> buckets = storage.list();
  for (Bucket bucket : buckets.iterateAll()) {
    System.out.println(bucket.toString());
  }
}

Python

View on GitHub

def explicit_app_engine(project):
    from google.auth import app_engine
    import googleapiclient.discovery

    # Explicitly use App Engine credentials. These credentials are
    # only available when running on App Engine Standard.
    credentials = app_engine.Credentials()

    # Explicitly pass the credentials to the client library.
    storage_client = googleapiclient.discovery.build(
        'storage', 'v1', credentials=credentials)

    # Make an authenticated API request
    buckets = storage_client.buckets().list(project=project).execute()
    print(buckets)

Restricting access

Only grant your application the authorization permissions that it needs to interact with applicable GCP APIs, features, or resources. GCP uses Cloud Identity and Access Management (Cloud IAM) for access control. When you create a service account, you can choose a Cloud IAM role for limiting access. The walkthrough at getting started with authentication instructs you to choose the Owner role when you create a service account. You can change this value at any time. For more information, see granting roles to service accounts.

Best practices for managing credentials

Credentials provide access to sensitive data. The following practices help protect access to these resources.

Do not embed secrets related to authentication in source code, such as API keys, OAuth tokens, and service account credentials. You can use an environment variable pointing to credentials outside of the application's source code, such as Cloud Key Management Service.
Do use different credentials in different contexts, such as in testing and production environments.
Do transfer credentials only over HTTPS to prevent a third party from intercepting your credentials. Never transfer in clear text or as part of the URL.
Never embed long-lived credentials into your client-side app. For example, do not embed service account credentials into a mobile app. Client-side apps can be examined and credentials can easily be found and used by a third party.
Do revoke a token if you no longer need it.

Troubleshooting API errors

Learn more about how to troubleshoot failed API requests at Cloud APIs errors.

What's next

Learn about getting started with authentication.
Learn about setting up authentication as an end user.
Learn about using API keys.

Providing credentials to your application

Finding credentials automatically

C#

Go

Java

Node.js

PHP

Python

Ruby

Obtaining and providing service account credentials manually

Creating a service account

Cloud Console

Command line

Providing service account credentials

Setting the environment variable

Linux or macOS

Windows

Passing the path to the service account key in code

C#

Go

Java

Node.js

PHP

Python

Ruby

Obtaining credentials on Compute Engine, Kubernetes Engine, App Engine flexible environment, and Cloud Functions

C#

Go

Java

Node.js

PHP

Python

Ruby

Obtaining credentials on App Engine standard environment

PHP

Go

Java

Python

Restricting access

Best practices for managing credentials

Troubleshooting API errors

What's next

Send feedback about...