Creating and Enabling Service Accounts for Instances

This page describes how to create and use service accounts to authorize applications that run on your virtual machine instances to other Google Cloud Platform services. To learn more about service accounts for virtual machine instances, read the Service Accounts Overview.

When you write applications that run on Google Compute Engine instances, you might want to connect them to other Google Cloud Platform services, like Google Cloud Storage and BigQuery. If you use service accounts, you can authenticate your applications to access these services without having to provide user credentials in the application code. Service accounts are ideal for server-to-server communication where user interaction is not necessary for an application to run.

Contents

Before you begin

Creating a default service account

By default, all Compute Engine instances have access to the default service account without having to explicitly create the account. If you want to use the default service account, read the Setting up instances to run as a service account section to enable the default service account for instances.

To learn more about the default service account, read the Service Accounts documentation.

Creating a custom service account

Google Cloud Platform offers Identity and Access Management (IAM) which allows you to create custom service accounts in addition to the default service account. For example, you can create a custom service account and grant the service account access to only Google Cloud Storage. Then, you can set up an instance to run as the custom service account to access Google Cloud Storage data.

To create a new custom service account:

  1. Create a new service account using instructions on the IAM Service Accounts documentation. For example, to create a new service account in the Console:

    1. Go to the Service Accounts page in the Cloud Platform Console.

      Go to the Service Accounts page

    2. If prompted, select a project.
    3. Click Create service account.
    4. Enter the Name and the Service account ID and click Create. You can change the name of the service account in the future, but you cannot change the service account ID after the service account is created.

    For other methods to create a service account, read the Creating a service account documentation.

  2. Get the service account's ID. You need the account ID to set up an instance to run as this service account. Verify the service account's ID in the console:

    1. Go to the Service Accounts page in the Cloud Platform Console.

      Go to the Service Accounts page

    2. If prompted, select a project.
    3. Look for your new service account and make note of the service account ID.

    Usually, the service account's ID is derived from the service account name, in the format:

    SERVICE-ACCOUNT-NAME@YOUR-PROJECT.iam.gserviceaccount.com
    
  3. Authorize access and enable a service account for an instance.

Authorize access and enable a service account for an instance

To enable an instance to run as a service account, you must:

  1. (Optional) Assign IAM roles to the service account.
  2. Authorize access by providing access scopes during instance creation.

Assigning IAM roles to a service account is optional. You can use access scopes alone to authorize access. If you want to use just access scopes, skip to Setting up an instance with access scopes.

If you don't know whether you want to use IAM roles, read the documentation on Access Scopes and IAM Roles.

Assign IAM roles to the service account

If you plan to use IAM roles with your service account, assign the IAM roles to the service account before you enable the service account on an instance. If you don't plan to use IAM roles, skip this section and go to Setting up an instance with access scopes.

Using IAM roles, you can grant more granular access to service account beyond the permissions provided by an access scope. For example, instead of blanket read-write access to Google Cloud Storage using the https://www.googleapis.com/auth/devstorage.read_write access scope, you can limit the access scope by granting specific roles like roles/storage.objectAdmin or roles/storage.objectViewer.

There are some important things to note about using IAM roles:

  • Some IAM roles are currently in Beta and not all services support IAM. For a full list of IAM roles and their status, see the IAM roles list.
  • Using IAM roles does not mean you can completely replace access copes. You must still set access scopes when enabling the service account on a instance.

For a detailed discussion about the interactions between access scopes and IAM roles, read the Access scopes and IAM roles documentation.

Granting IAM roles to the default service account

Before you assign IAM roles to the default service account, note that:

  • Granting an IAM role to the default service account affects all instances that are running as the default service account. For example, if you grant the default service account the roles/storage.objectAdmin role, all instances running as the default service account will have permissions granted by the roles/storage.objectAdmin role. Likewise, if you limit access by omiting certain roles, it will affect all instances running as the default service account.

  • You must revoke project editor permission for the service account. The default service account is added as a project editor to projects by default. To use IAM roles, you must revoke the project editor permission.

If are unsure you want to grant IAM roles to the default service account, use access scopes only or create custom service accounts to use with IAM roles instead.

To grant an IAM role to the default service account:

  1. Get the default service account ID.
  2. Revoke the service account's project editor access. The default service account is added as a project editor by default. You must revoke the project editor access before you can use IAM roles.

    1. Go to the IAM & Admin page in the Cloud Platform Console.
    2. Go to the IAM & Admin page

    3. If prompted, select a project.
    4. Click on Editor to expand the list of project editors.
    5. Check the box next to the Compute Engine Default Service Account entry and click Remove.

  3. Follow instruction for Setting an IAM policy to grant IAM roles to the service account.

  4. Finally, set up an instance to use the service account with the Cloud Platform access scope. Provide the follow Cloud Platform access scope when you create the instance:

    https://www.googleapis.com/auth/cloud-platform
    

Granting IAM roles to a custom service account

To grant IAM roles to a custom service account:

  1. Get the service account ID, if necessary.
  2. Follow instructions for Setting an IAM policy to grant IAM roles to the service account.
  3. Finally, set up an instance to use the service account with the Cloud Platform access scope:

    https://www.googleapis.com/auth/cloud-platform
    

Setting up an instance with access scopes

To set up an instance to run as a service account using access scopes, provide the desired scopes when you create the instance.

You can set scopes only when you create a new instance, and cannot change or expand the list of scopes for existing instances. For simplicity, you can choose to enable full access to all Google Cloud Platform APIs with the https://www.googleapis.com/auth/cloud-platform scope. To learn more about access scopes, read the Service Accounts Overview documentation.

If you are using a custom service account or are creating the instance using the API, get the service account's ID.

Set up an instance with access scopes in the Cloud Platform Console, the gcloud tool, or the API.

Console


Note: You can set access scopes in the Google Cloud Platform Console only for the default service account. It is not possible to set access scopes in the console for custom service accounts. Use gcloud and the API instead.

To create an instance and set the scopes to grant to the default service account:

  1. In the Cloud Platform Console, go to the VM Instances page.

    Go to the VM Instances page

  2. Click the Create instance button.
  3. On the Create a new instance page, fill in the desired properties for your instance.
  4. In the Identity and Access section, choose the Compute Engine Default Service Account from the dropdown list.
  5. Select Allow full access to all Cloud APIs to enable the Cloud Platform scope.
  6. If you want to select access to individual APIs instead of granting the Cloud Platform scope:
    1. Select Set access for each API.
    2. Set each scope to the level of access that this instance requires.
  7. Click the Create button to create the instance.

gcloud


To create a new instance and authorize it to have full access to all Google Cloud Platform services using the default service account:

gcloud compute instances create [INSTANCE_NAME] \
     --scopes https://www.googleapis.com/auth/cloud-platform

where [INSTANCE_NAME] is the name of the instance.

To create a new instance and authorize it to run as a custom service account:

gcloud compute instances create [INSTANCE_NAME] \
    --scopes [SERVICE_ACCOUNT_NAME]@[PROJECT_ID].iam.gserviceaccount.com=cloud-platform

where:

  • [INSTANCE_NAME] is the name of the instance.
  • [SERVICE_ACCOUNT_NAME] is the name of the custom service account. If you don't know what your service account name is, obtain your service account ID.
  • [PROJECT_ID] is your project ID.

Alternatively, you can use scope aliases in place of the longer scope URIs. For example, the scope for full access to Google Cloud Storage is https://www.googleapis.com/auth/devstorage.full_control. The alias for this scope is storage-full.

You can see a list of scopes and scope aliases on the instances create page in the description for the --scopes flag. The help for the instances create command also lists these scopes and aliases:

gcloud compute instances create --help

Specify the alias the same way you would specify the normal scope URI. For example:

gcloud compute instances create [INSTANCE_NAME] \
  --scopes storage-full

API


In the API, construct a normal request to create an instance, but include the serviceAccounts property. Obtain your service account ID, and include it the email property. Then, set one or more scopes in the scopes property.

POST https://www.googleapis.com/compute/v1/projects/zones/[ZONE]/instances

{ "machineType": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/[ZONE]/machineTypes/[MACHINE_TYPE]", "name": "[INSTANCE_NAME]", "serviceAccounts": [ { "email": "[SERVICE_ACCOUNT_EMAIL]", "scopes": "scopes": ["https://www.googleapis.com/auth/cloud-platform"] } ], ... }

You can see a list of scopes and scope aliases on the instances create page in the description for the --scopes flag.

After you enable the service account and set the required scopes or IAM roles for your instances:

Authenticating tools through a service account

Some applications might use commands from the gcloud and gsutil tools, which are included by default in most Compute Engine images. These tools automatically recognize an instance's service account and relevant permissions granted to the service account. Specifically, if you have the necessary access scopes defined for the instance's default service account, you can use gcloud and gsutil from your instances without having to use gcloud auth login.

Automatic service account recognition applies only to the gcloud and gsutil tools that are included with the instance. If you create new tools or add custom tools, you must authorize your application using a client library or by using access tokens directly in your application.

Using IAM roles

To enable automatic service account recognition using IAM roles, assign the necessary IAM roles to the service account. For example, if you grant a service account the roles/storage.objectAdmin role, the gsutil tool can automatically manage and access Google Cloud Storage objects.

Likewise, if you enable roles/compute.instanceAdmin for the service account, the gcloud compute tool can automatically manage instances.

Using access scopes

To enable automatic service account recognition, start an instance with the necessary scopes for the instance's service account.

  • For gsutil and access to Google Cloud Storage, enable one of the scopes listed in the Google Cloud Storage OAuth 2.0 guide.
  • For gcloud compute, enable one of the following scopes:

    • Read-write access to Compute Engine methods: https://www.googleapis.com/auth/compute
    • Read-only access to Compute Engine methods: https://www.googleapis.com/auth/compute.readonly

Optionally, you can enable the https://www.googleapis.com/auth/cloud-platform scope, which grants full access to all Google Cloud Platform services and the gcloud and gsutil tools.

Authenticating applications with a client library

Client libraries can use Application Default Credentials to authenticate with Google APIs and send requests to those APIs. Application default credentials allow applications to obtain credentials from multiple sources so you can test your application locally and then deploy it to a Compute Engine instance without changing the application code. While you develop your application locally, the application can authenticate using an environment variable or the Google Cloud SDK. When your application runs on an instance, it can authenticate using the service account that has been enabled on the instance.

This example uses the Python Client Library to authenticate and make a request to the Cloud Storage API to create a bucket. The example uses the following procedure:

  1. Obtain the necessary authentication credentials for the Cloud Storage API. On an instance, the get_application_default() method obtains these credentials from the service account.
  2. Initialize the Cloud Storage service with the build() method and the credentials.
  3. List buckets in Cloud Storage.

You can run this sample on an instance that has access to manage buckets in Google Cloud Storage.

Authenticating applications directly with access tokens

For some applications, you might need to request an OAuth2 access token and use it directly without going through a client library or using the gcloud or gsutil tools. There are several options for obtaining and using these access tokens to authenticate your applications. For example, you can use curl to create a simple request, or use a programming language like Python for more flexibility.

cURL


To use curl to request an access token and send a request to an API:

  1. Enable your instance with either the compute.readonly access scope, or the roles/compute.instanceAdmin IAM role.

  2. On the instance where your application runs, query the metadata server for an access token by running the following command:

    $ curl "http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token" \
    -H "Metadata-Flavor: Google"

    The request returns a response similar to:

    {
          "access_token":"ya29.AHES6ZRN3-HlhAPya30GnW_bHSb_QtAS08i85nHq39HE3C2LTrCARA",
          "expires_in":3599,
          "token_type":"Bearer"
     }

  3. Copy the value of the access_token property from the response and use it to send requests to the API. For example, the following request prints a list of instances in your project from a certain zone:

    $ curl https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/[ZONE]/instances \
    -H "Authorization":"Bearer [ACCESS_TOKEN]"
    

    where:

    • [PROJECT_ID] is the project ID for this request.
    • [ZONE] is the zone to list instances from.
    • [ACCESS_TOKEN] is the access token value you got from step 1.

    For information about the parameters that you can set in your request, see the parameters documentation.

Python


This example demonstrates how to request a token to access the Google Cloud Storage API in a Python application. The example uses the following procedure:

  1. Request an access token from the metadata server.
  2. Extract the access token from the server response.
  3. Use the access token to make a request to Google Cloud Storage.
  4. If the request is successful, the script prints the response.

Access tokens expire after a short period of time. The metadata server caches access tokens until they have 60 seconds of remaining time before they expire. You can request new tokens as frequently as you like, but your applications must have a valid access token for their API calls to succeed.

There is a limit to the total number of tokens that your service account can have at any one point in time. Currently, this limit is 600. If this limit is reached, Compute Engine will not be able to create an instance that requires a new token, and you will get a SERVICE_ACCOUNT_TOO_MANY_TOKENS error.

If you reach the token limit and cannot create new tokens, you must delete some tokens that are associated with the service account. Delete Compute Engine instances that are using the service account. This reduces the number of distinct scope sets that the service account uses.

Obtaining a service account ID

Some API calls require you to include a service account name in the request body. Obtain a service account name through one of the following options:

Console


  1. Go to the Service Accounts page in the Cloud Platform Console.

    Go to the Service Accounts page

  2. If prompted, select a project. The service accounts page lists all service accounts for the project.

gcloud


Use the gcloud compute instances describe command from your local machine:

gcloud compute instances describe [INSTANCE_NAME] --format json

 {
       ...
       "serviceAccounts":[
          {
             "email":"123845678986-compute@developer.gserviceaccount.com",
             "scopes":[
                "https://www.googleapis.com/auth/devstorage.full_control"
             ]
          }
       ]
       ...
    }

If the instance isn't using a service account, you receive a response without the serviceAccounts property.

Metadata Server


Query the metadata server from within the instance itself. Make a request to http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/:

  user@myinst:~$ curl "http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/" \
  -H "Metadata-Flavor: Google"
  

If you enabled one or more service accounts when you created the instance, this curl command returns output similar to the following:

123845678986-compute@developer.gserviceaccount.com/
default/

If the instance isn't using a service account, you receive an empty response.

API


Make a request to the Service Accounts API.

What's next

Send feedback about...

Compute Engine