Authenticate workloads to Google Cloud APIs using service accounts

This page describes how to use service accounts to enable apps running on your virtual machine (VM) instances to authenticate to Google Cloud APIs and authorize access to resources.

To use service accounts for authentication, you must first ensure that your VM is configured to use a service account. To do this complete one of the following procedures:

Before you begin

  • Review the Service accounts overview.
  • If you haven't already, set up authentication. Authentication is the process by which your identity is verified for access to Google Cloud services and APIs. To run code or samples from a local development environment, you can authenticate to Compute Engine as follows.

    To use the Python samples on this page from a local development environment, install and initialize the gcloud CLI, and then set up Application Default Credentials with your user credentials.

    1. Install the Google Cloud CLI.

    2. To initialize the gcloud CLI, run the following command: 

      gcloud init

    3. Create local authentication credentials for your Google Account: 

      gcloud auth application-default login

    For more information, see Set up authentication for a local development environment.

Overview

After you have set up a VM instance to run using a service account, an application running on the VM instance can use one of the following methods for authentication:

Authenticating applications using service account credentials

After setting up an instance to run as a service account, you can use service account credentials to authenticate applications running on the instance.

Authenticating applications with a client library

Client libraries can use Application Default Credentials to authenticate with Google APIs and send requests to those APIs. Application Default Credentials lets applications automatically obtain credentials from multiple sources so you can test your application locally and then deploy it to a Compute Engine instance without changing the application code.

For information about setting up Application Default Credentials, see Provide credentials to Application Default Credentials.

This example uses the Python client library to authenticate and make a request to the Cloud Storage API to list the buckets in a project. The example uses the following procedure:

  1. Obtain the necessary authentication credentials for the Cloud Storage API and initialize the Cloud Storage service with the build() method and the credentials.
  2. List buckets in Cloud Storage.

You can run this sample on an instance that has access to manage buckets in Cloud Storage.

import argparse
from typing import List

from google.cloud import storage


def create_client() -> storage.Client:
    """
    Construct a client object for the Storage API using the
    application default credentials.

    Returns:
        Storage API client object.
    """
    # Construct the service object for interacting with the Cloud Storage API -
    # the 'storage' service, at version 'v1'.
    # Authentication is provided by application default credentials.
    # When running locally, these are available after running
    # `gcloud auth application-default login`. When running on Compute
    # Engine, these are available from the environment.
    return storage.Client()


def list_buckets(client: storage.Client, project_id: str) -> List[storage.Bucket]:
    """
    Retrieve bucket list of a project using provided client object.


    Args:
        client: Storage API client object.
        project_id: name of the project to list buckets from.

    Returns:
        List of Buckets found in the project.
    """
    buckets = client.list_buckets()
    return list(buckets)


def main(project_id: str) -> None:
    client = create_client()
    buckets = list_buckets(client, project_id)
    print(buckets)


if __name__ == "__main__":
    parser = argparse.ArgumentParser(
        description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
    )
    parser.add_argument("project_id", help="Your Google Cloud Project ID.")

    args = parser.parse_args()

    main(args.project_id)

Authenticating applications directly with access tokens

For most applications, you can authenticate by using Application Default Credentials, which finds credentials and manages tokens for you. However, if your application requires you to provide an OAuth2 access token, Compute Engine lets you get an access token from its metadata server for use in your application.

There are several options for obtaining and using these access tokens to authenticate your applications. For example, you can use curl to create a simple request, or use a programming language like Python for more flexibility.

cURL

To use curl to request an access token and send a request to an API:

  1. On the instance where your application runs, query the metadata server for an access token by running the following command:

    $ curl "http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token" \
-H "Metadata-Flavor: Google"

    The request returns a response similar to:

    {
      "access_token":"ya29.AHES6ZRN3-HlhAPya30GnW_bHSb_QtAS08i85nHq39HE3C2LTrCARA",
      "expires_in":3599,
      "token_type":"Bearer"
 }

    For API requests you need to include the access_token value, not the entire response. If you have the jq command-line JSON processor installed you can use the following command to extract the access token value from the response:

    $ ACCESS_TOKEN=`curl \
"http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token" \
-H "Metadata-Flavor: Google" | jq -r '.access_token'`

  2. Copy the value of the access_token property from the response and use it to send requests to the API. For example, the following request prints a list of instances in your project from a certain zone:

    $ curl https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances \
-H "Authorization":"Bearer ACCESS_TOKEN"

    Replace the following:

    • PROJECT_ID: the project ID for this request.
    • ZONE: the zone to list VMs from.
    • ACCESS_TOKEN: the access token value you got from the previous step.

    For information about the parameters that you can set in your request, see the System parameters documentation.

Python

This example demonstrates how to request a token to access the Cloud Storage API in a Python application. The example uses the following procedure:

  1. Request an access token from the metadata server.
  2. Extract the access token from the server response.
  3. Use the access token to make a request to Cloud Storage.
  4. If the request is successful, the script prints the response.

import argparse

import requests


METADATA_URL = "http://metadata.google.internal/computeMetadata/v1/"
METADATA_HEADERS = {"Metadata-Flavor": "Google"}
SERVICE_ACCOUNT = "default"


def get_access_token() -> str:
    """
    Retrieves access token from the metadata server.

    Returns:
        The access token.
    """
    url = f"{METADATA_URL}instance/service-accounts/{SERVICE_ACCOUNT}/token"

    # Request an access token from the metadata server.
    r = requests.get(url, headers=METADATA_HEADERS)
    r.raise_for_status()

    # Extract the access token from the response.
    access_token = r.json()["access_token"]

    return access_token


def list_buckets(project_id: str, access_token: str) -> dict:
    """
    Calls Storage API to retrieve a list of buckets.

    Args:
        project_id: name of the project to list buckets from.
        access_token: access token to authenticate with.

    Returns:
        Response from the API.
    """
    url = "https://www.googleapis.com/storage/v1/b"
    params = {"project": project_id}
    headers = {"Authorization": f"Bearer {access_token}"}

    r = requests.get(url, params=params, headers=headers)
    r.raise_for_status()

    return r.json()


def main(project_id: str) -> None:
    """
    Retrieves access token from metadata server and uses it to list
    buckets in a project.

    Args:
        project_id: name of the project to list buckets from.
    """
    access_token = get_access_token()
    buckets = list_buckets(project_id, access_token)
    print(buckets)


if __name__ == "__main__":
    parser = argparse.ArgumentParser(
        description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
    )
    parser.add_argument("project_id", help="Your Google Cloud project ID.")

    args = parser.parse_args()

    main(args.project_id)

Access tokens expire after a short period of time. The metadata server caches access tokens until they have 5 minutes of remaining time before they expire. If tokens are unable to be cached, requests that exceed 50 queries per second might be rate limited. Your applications must have a valid access token for their API calls to succeed.

Authenticating tools on an instance using a service account

Some applications might use commands from the gcloud and gsutil tools, which are included by default in most Compute Engine images. These tools automatically recognize an instance's service account and relevant permissions granted to the service account. Specifically, if you grant the correct roles to the service account, you can use the gcloud and gsutil tools from your instances without having to use gcloud auth login.

This service account recognition happens automatically and applies only to the gcloud and gsutil tools that are included with the instance. If you create new tools or add custom tools, you must authorize your application using a client library or by using access tokens directly in your application.

To take advantage of automatic service account recognition, grant the appropriate IAM roles to the service account and attach the service account to the instance. For example, if you grant a service account the roles/storage.objectAdmin role, the gsutil tool can automatically manage and access Cloud Storage objects.

Likewise, if you enable roles/compute.instanceAdmin.v1 for the service account, the gcloud compute tool can automatically manage instances.

What's next

Try it for yourself

If you're new to Google Cloud, create an account to evaluate how Compute Engine performs in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

Try Compute Engine free