Creating and Enabling Service Accounts for Instances

This page describes how to create and use service accounts with your virtual machine instances. A service account is a special account whose credentials you can use in your application code to access other Google Cloud Platform services.

To learn about service accounts, read the Service Accounts Overview.

Before you begin

Setting up a new service account

You can create and set up a new service account using Google Cloud Identity and Access Management (IAM). After creating an account, grant the account one or more IAM roles, and then authorize a virtual machine instance to run as that service account.

To create a new service account:

  1. Create a new service account as described on the IAM Service Accounts documentation.

  2. Get the service account's email. You need the email to set up an instance to run as this service account. Verify the service account's email in the console:

    1. Go to the Service Accounts page in the Cloud Platform Console.

      Go to the Service Accounts page

    2. If prompted, select a project.
    3. Look for your new service account and make note of the service account email.

    Usually, the service account's email is derived from the service account ID, in the format:

    [SERVICE-ACCOUNT-NAME]@[PROJECT_ID].iam.gserviceaccount.com
    
  3. Grant IAM roles. Follow the instructions to grant IAM roles to the service account. If you don't grant any roles, the service account won't have access to any services. For a full list of IAM roles, see Understanding Roles on the IAM documentation.

  4. Next, set up an instance to run as a service account. Follow the instructions to set up an instance to run as a service account.

Setting up a new instance to run as a service account

After setting up a new service account, you can assign the service account to one or more new virtual machine instances. It is not possible to assign a new service account to an existing virtual machine instance.

You can assign a service account to more than one virtual machine instance but a virtual machine instance can only have one service account identity. If you assign the same service account to multiple virtual machine instances, any subsequent changes you make to the service account will affect instances using the service account. This includes any changes you make to the IAM roles granted to the service account. For example, if you remove a role, all instances using the service account will lose permissions granted by that role.

You can set up a new instance to run as a service account through the Google Cloud Platform Console, gcloud, or directly through the API.

Console

  1. In the Cloud Platform Console, go to the VM Instances page.

    Go to the VM Instances page

  2. Click the Create instance button.
  3. On the Create a new instance page, fill in the desired properties for your instance.
  4. In the Identity and API Access section, choose the service account you want to use from the dropdown list.
  5. Click the Create button to create the instance.

gcloud

To create a new instance and authorize it to run as a custom service account using gcloud, provide the service account email and required access scopes for the instance. Generally, you can just set the cloud-platform access scope. By itself, this scope would authorize access to all Google Cloud Platform services, but when specified with a service account, you limit access to the IAM roles granted to the service account.

Alternatively, you can also choose to set specific scopes that authorizes access to the particular API methods the service will call. For example, to call the instances.insert method requires authorization with either the https://www.googleapis.com/auth/compute scope or the https://www.googleapis.com/auth/cloud-platform scope. You could set the compute scope in place of the cloud-platform scope, which would give the service authorization to call methods in Compute Engine but no authorization to call API methods outside of Compute Engine.

gcloud compute instances create [INSTANCE_NAME] \
    --scopes [SERVICE_ACCOUNT_EMAIL]=https://www.googleapis.com/auth/cloud-platform

where:

  • [INSTANCE_NAME] is the name of the instance.
  • [SERVICE_ACCOUNT_EMAIL] is the email of the custom service account. If you don't know what the email is, learn how to obtain a service account email.

gcloud also offers scope aliases in place of the longer scope URIs. For example, the scope for full access to Google Cloud Storage is https://www.googleapis.com/auth/devstorage.full_control. The alias for this scope is storage-full.

You can see a list of scopes and scope aliases on the instances create page in the description for the --scopes flag. The help for the instances create command also lists these scopes and aliases:

gcloud compute instances create --help

Specify the alias the same way you would specify the normal scope URI. For example:

gcloud compute instances create [INSTANCE_NAME] \
  --scopes [SERVICE_ACCOUNT_EMAIL]=storage-full

API

In the API, construct a standard request to create an instance, but include the serviceAccounts property. Obtain your service account email, and include it the email property, along with the required access scopes for the instance. Generally, you can just set the cloud-platform access scope. By itself, this scope would authorize access to all Google Cloud Platform services, but when specified with a service account, you limit access to the IAM roles granted to the service account.

Alternatively, you can also choose to set specific scopes that authorize access to the particular API methods the service will call. For example, to call the instances.insert method requires authorization with either the https://www.googleapis.com/auth/compute scope or the https://www.googleapis.com/auth/cloud-platform scope. You could set the compute scope in place of the cloud-platform scope, which would give the service authorization to call methods in Compute Engine but no authorization to call API methods outside of Compute Engine.

POST https://www.googleapis.com/compute/v1/projects/zones/[ZONE]/instances

{ "machineType": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/[ZONE]/machineTypes/[MACHINE_TYPE]", "name": "[INSTANCE_NAME]", "serviceAccounts": [ { "email": "[SERVICE_ACCOUNT_EMAIL]", "scopes": ["https://www.googleapis.com/auth/cloud-platform"] } ], ... }

After you have created a new service account and set up an instance to run as the service account, you can use the service account credentials from within an instance in several ways:

Authenticating applications using service account credentials

After setting up an instance to run as a service account, you can use service account credentials to authenticate applications running on the instance.

Authenticating applications with a client library

Client libraries can use Application Default Credentials to authenticate with Google APIs and send requests to those APIs. Application default credentials allow applications to obtain credentials from multiple sources so you can test your application locally and then deploy it to a Compute Engine instance without changing the application code. While you develop your application locally, the application can authenticate using an environment variable or the Google Cloud SDK. When your application runs on an instance, it can authenticate using the service account that has been enabled on the instance.

This example uses the Python Client Library to authenticate and make a request to the Cloud Storage API to create a bucket. The example uses the following procedure:

  1. Obtain the necessary authentication credentials for the Cloud Storage API. On an instance, the get_application_default() method obtains these credentials from the service account.
  2. Initialize the Cloud Storage service with the build() method and the credentials.
  3. List buckets in Cloud Storage.

You can run this sample on an instance that has access to manage buckets in Google Cloud Storage.

import argparse

from googleapiclient import discovery
from oauth2client.client import GoogleCredentials


def create_service():
    # Get the application default credentials. When running locally, these are
    # available after running `gcloud auth`. When running on compute
    # engine, these are available from the environment.
    credentials = GoogleCredentials.get_application_default()

    # Construct the service object for interacting with the Cloud Storage API -
    # the 'storage' service, at version 'v1'.
    return discovery.build('storage', 'v1', credentials=credentials)


def list_buckets(service, project_id):
    buckets = service.buckets().list(project=project_id).execute()
    return buckets


def main(project_id):
    service = create_service()
    buckets = list_buckets(service, project_id)
    print(buckets)


if __name__ == '__main__':
    parser = argparse.ArgumentParser(
        description=__doc__,
        formatter_class=argparse.RawDescriptionHelpFormatter)
    parser.add_argument('project_id', help='Your Google Cloud Project ID.')

    args = parser.parse_args()

    main(args.project_id)

Authenticating applications directly with access tokens

For some applications, you might need to request an OAuth2 access token and use it directly without going through a client library or using the gcloud or gsutil tools. There are several options for obtaining and using these access tokens to authenticate your applications. For example, you can use curl to create a simple request, or use a programming language like Python for more flexibility.

cURL

To use curl to request an access token and send a request to an API:

  1. On the instance where your application runs, query the metadata server for an access token by running the following command:

    $ curl "http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token" \
    -H "Metadata-Flavor: Google"

    The request returns a response similar to:

    {
          "access_token":"ya29.AHES6ZRN3-HlhAPya30GnW_bHSb_QtAS08i85nHq39HE3C2LTrCARA",
          "expires_in":3599,
          "token_type":"Bearer"
     }

  2. Copy the value of the access_token property from the response and use it to send requests to the API. For example, the following request prints a list of instances in your project from a certain zone:

    $ curl https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/[ZONE]/instances \
    -H "Authorization":"Bearer [ACCESS_TOKEN]"
    

    where:

    • [PROJECT_ID] is the project ID for this request.
    • [ZONE] is the zone to list instances from.
    • [ACCESS_TOKEN] is the access token value you got from step 1.

    For information about the parameters that you can set in your request, see the parameters documentation.

Python

This example demonstrates how to request a token to access the Google Cloud Storage API in a Python application. The example uses the following procedure:

  1. Request an access token from the metadata server.
  2. Extract the access token from the server response.
  3. Use the access token to make a request to Google Cloud Storage.
  4. If the request is successful, the script prints the response.

import argparse

import requests


METADATA_URL = 'http://metadata.google.internal/computeMetadata/v1/'
METADATA_HEADERS = {'Metadata-Flavor': 'Google'}
SERVICE_ACCOUNT = 'default'


def get_access_token():
    url = '{}instance/service-accounts/{}/token'.format(
        METADATA_URL, SERVICE_ACCOUNT)

    # Request an access token from the metadata server.
    r = requests.get(url, headers=METADATA_HEADERS)
    r.raise_for_status()

    # Extract the access token from the response.
    access_token = r.json()['access_token']

    return access_token


def list_buckets(project_id, access_token):
    url = 'https://www.googleapis.com/storage/v1/b'
    params = {
        'project': project_id
    }
    headers = {
        'Authentication': 'Bearer {}'.format(access_token)
    }

    r = requests.get(url, params=params, headers=headers)
    r.raise_for_status()

    return r.json()


def main(project_id):
    access_token = get_access_token()
    buckets = list_buckets(project_id, access_token)
    print(buckets)


if __name__ == '__main__':
    parser = argparse.ArgumentParser(
        description=__doc__,
        formatter_class=argparse.RawDescriptionHelpFormatter)
    parser.add_argument('project_id', help='Your Google Cloud project ID.')

    args = parser.parse_args()

    main(args.project_id)

Access tokens expire after a short period of time. The metadata server caches access tokens until they have 60 seconds of remaining time before they expire. You can request new tokens as frequently as you like, but your applications must have a valid access token for their API calls to succeed.

There is a limit to the total number of tokens that your service account can have at any one point in time. Currently, this limit is 600. If this limit is reached, Compute Engine will not be able to create an instance that requires a new token, and you will get a SERVICE_ACCOUNT_TOO_MANY_TOKENS error.

If you reach the token limit and cannot create new tokens, you must delete some tokens that are associated with the service account. Delete Compute Engine instances that are using the service account. This reduces the number of distinct scope sets that the service account uses.

Authenticating tools on an instance using a service account

Some applications might use commands from the gcloud and gsutil tools, which are included by default in most Compute Engine images. These tools automatically recognize an instance's service account and relevant permissions granted to the service account. Specifically, if you grant the correct roles to the service account, you can use gcloud and gsutil from your instances without having to use gcloud auth login.

This service account recognition happens automatically and applies only to the gcloud and gsutil tools that are included with the instance. If you create new tools or add custom tools, you must authorize your application using a client library or by using access tokens directly in your application.

To take advantage of automatic service account recognition, grant the appropriate IAM roles to the service account and set up an instance to run as a service account. For example, if you grant a service account the roles/storage.objectAdmin role, the gsutil tool can automatically manage and access Google Cloud Storage objects.

Likewise, if you enable roles/compute.instanceAdmin for the service account, the gcloud compute tool can automatically manage instances.

Obtaining a service account email

To identify a service account, you need the service account email. Obtain a service account email through one of the following options:

Console

  1. Go to the Service Accounts page in the Cloud Platform Console.

    Go to the Service Accounts page

  2. If prompted, select a project. The service accounts page lists all service accounts for the project and their emails.

gcloud

Use the gcloud compute instances describe command from your local machine:

gcloud compute instances describe [INSTANCE_NAME] --format json

 {
       ...
       "serviceAccounts":[
          {
             "email":"123845678986-compute@developer.gserviceaccount.com",
             "scopes":[
                "https://www.googleapis.com/auth/devstorage.full_control"
             ]
          }
       ]
       ...
    }

If the instance isn't using a service account, you receive a response without the serviceAccounts property.

Metadata Server

Query the metadata server from within the instance itself. Make a request to http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/:

  user@myinst:~$ curl "http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/" \
  -H "Metadata-Flavor: Google"
  

If you enabled one or more service accounts when you created the instance, this curl command returns output similar to the following:

123845678986-compute@developer.gserviceaccount.com/
default/

If the instance isn't using a service account, you receive an empty response.

API

Make a request to the Service Accounts API.

Using the Compute Engine Default Service Account

If you are familiar with the Compute Engine default service account and want to use the credentials provided by the default service account instead of creating new service accounts, you can grant IAM roles to the default service account.

By default, all Compute Engine instances can run as the default service account. When you create an instance using gcloud or the Cloud Platform Console, and omit any service account specifications, the default service account is assigned to the instance.

Before you assign IAM roles to the default service account, note that:

  • Granting an IAM role to the default service account affects all instances that are running as the default service account. For example, if you grant the default service account the roles/storage.objectAdmin role, all instances running as the default service account with the required access scopes will have permissions granted by the roles/storage.objectAdmin role. Likewise, if you limit access by omitting certain roles, it will affect all instances running as the default service account.

  • You must revoke project editor permission for the service account. The default service account is added as a project editor to projects by default. To use IAM roles, you must revoke the project editor permission.

If are unsure you want to grant IAM roles to the default service account, you should create a new service account instead.

Follow these instructions to grant an IAM role to the default service account:

  1. Go to the IAM page in the Cloud Platform Console.

    Go to the IAM page

  2. If prompted, select a project.
  3. Look for the service account named Compute Engine Default Service Account.
  4. In the Role(s) column, expand the drop down menu for the Compute Engine Default Service Account.
  5. Remove Editor access and save your changes.
  6. Next, grant IAM roles to the service account.

Any virtual machine instances that are currently running as the default service account will now have access to other Google Cloud Platform APIs according to the IAM roles you granted to the account.

If you want to set up a new instance to run as the default service account, follow these instructions:

Console

  1. In the Cloud Platform Console, go to the VM Instances page.

    Go to the VM Instances page

  2. Click the Create instance button.
  3. On the Create a new instance page, fill in the desired properties for your instance.
  4. In the Identity and API Access section, choose the Compute Engine Default Service Account from the dropdown list.
  5. Click the Create button to create the instance.

gcloud

To create a new instance and authorize it to have full access to all Google Cloud Platform services using the default service account:

gcloud compute instances create [INSTANCE_NAME] \
     --scopes cloud-platform

API

In the API, construct a standard request to create an instance, but include the serviceAccounts property. Obtain your default service account ID, and include it as the service account's email. Then, set one or more scopes in the scopes property.

POST https://www.googleapis.com/compute/v1/projects/zones/[ZONE]/instances

{ "machineType": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/[ZONE]/machineTypes/[MACHINE_TYPE]", "name": "[INSTANCE_NAME]", "serviceAccounts": [ { "email": "[DEFAULT_SERVICE_ACCOUNT_EMAIL]", "scopes": "scopes": ["https://www.googleapis.com/auth/cloud-platform"] } ], ... }

What's next

Send feedback about...

Compute Engine Documentation