Google Cloud Platform
Compute Engine

Connecting to Other Google Cloud Platform Services

When you write applications that run on Google Compute Engine instances, you might want to connect them to Google Cloud Storage, BigQuery, and other Google Cloud Platform services. If you use service accounts, you can easily authenticate your applications to access these services.

For example, if you write an application that reads and writes files on Google Cloud Storage, it must first authenticate to the Google Cloud Storage API. Enable the devstorage.read_write access scope on the instance where you plan to run your application and program the application to obtain credentials from the service account. Your application authenticates seamlessly to the API without embedding any secret keys or credentials in your instance, image, or application code.

Prerequisites for this guide


Setting the scope of service account access for instances

Before you authenticate your applications using service accounts, you must set an access scope on your instance. Access scopes specify which APIs your instances can authenticate with, and define the level of access that your instances have with those services. By default, new instances have the following scopes enabled:

  • Read-only access to Google Cloud Storage:

  • Write access to the Compute Engine log:

You can set scopes only when you create a new instance, and cannot change or expand the list of scopes for existing instances. For simplicity, you can choose to enable full access to all Google Cloud Platform APIs with the scope.

Specify scopes for new instances with the Developers Console, the gcloud tool, or the API.


Create an instance and select which scopes you want to allow the instance to access.

  1. In the Developers Console, go to the VM instances page.
  2. Click the New instance button.
  3. On the Create a new instance page, fill in the desired properties for your instance.
  4. Click Management, disk, networking, access & security options to show additional instance settings.
  5. Click Access & security to show all of the available scopes for this instance.
  6. In the Project Access settings, set each scope to the level of access that this instance requires. For example, set the Cloud Platform scope to Enabled so that this instance has full access to all Google Cloud Platform APIs.
  7. Click the Create button to create the instance.


Use the instances create command to create a new instance, and include the --scopes flag with one or more scopes. For example, you can create a new instance and authorize it to have full access to all Google Cloud Platform services:

$ gcloud compute instances create INSTANCE \

Alternatively, you can use scope aliases in place of the longer scope URIs. For example, the scope for full access to Google Cloud Storage is The alias for this scope is storage-full. Specify the alias the same way you would specify the normal scope URI. For example:

$ gcloud compute instances create INSTANCE \
  --scopes storage-full

You can see a list of scopes and scope aliases on the instances create page in the description for the --scopes flag. The help for the instances create command also lists these scopes and aliases:

$ gcloud compute instances create --help


In the API, construct a normal request to create an instance, but include the serviceAccounts property. Obtain your service account name, and include it the email property. Then, set one or more scopes in the scopes property. If you exclude the scopes property or exclude the serviceAccounts property entirely, the instance uses the default scopes.

  "machineType": "",
  "name": "example-instance",
  "serviceAccounts": [
    "email": "",
    "scopes": [

You can see a list of scopes and scope aliases on the instances create page in the description for the --scopes flag.

After you enable the service account and set the required scopes for your instances, you can seamlessly authenticate applications with the following options:

Authenticating applications with a client library

Client libraries can use Application Default Credentials to easily authenticate with Google APIs and send requests to those APIs. This allows applications to obtain credentials from multiple sources so you can test your application locally and then deploy it to a Compute Engine instance without changing the application code. While you develop your application locally, it can authenticate using an environment variable or the Google Cloud SDK. When your application runs on an instance, it can authenticate using a service account.

This example uses the Python Client Library to authenticate and send a query through the BigQuery API. The example uses the following procedure:

  1. Obtain the necessary authentication credentials for the BigQuery API. On an instance, the get_application_default() method obtains these credentials from the service account.
  2. Initialize the BigQuery service with the build() method and the credentials.
  3. Query the samples dataset for the natality table.
  4. Print the lastModifiedTime value from the table.

You can run this script in an instance that has the BigQuery scope enabled.

from oauth2client.client import GoogleCredentials
from googleapiclient.discovery import build

credentials = GoogleCredentials.get_application_default()

DATA_PROJECT_ID = "publicdata"
DATASET = "samples"
TABLE = "natality"

def main():
    bq = build('bigquery', 'v2', credentials=credentials)
    print getLastModTime(bq, DATA_PROJECT_ID, DATASET, TABLE)

def getLastModTime(bq, project, dataset, table):
    data = getTableData(bq, project, dataset, table)
    if data and 'lastModifiedTime' in data:
        return data['lastModifiedTime']
        return None

def getTableData(bq, project, dataset, table):
    tablesCollection = bq.tables()
    request = tablesCollection.get(
    return request.execute()

if __name__ == '__main__':

Authenticating tools through service accounts

Some applications might use commands from the gcloud and gsutil tools, which are included by default in most Compute Engine images. These tools automatically recognize the service account and scopes that are enabled on your instances.

If you have the necessary scopes defined, you can use these tools from your instances without having to use gcloud auth login. For gsutil and access to Google Cloud Storage, enable one of the scopes listed in the Google Cloud Storage developer's guide. For gcloud compute, enable one of the following scopes:

  • Read-write access to Compute Engine methods:

  • Read-only access to Compute Engine methods:

Optionally, you can enable the scope, which grants full access to all Google Cloud Platform services and the gcloud and gsutil tools.

Automatic scope recognition applies only to the gcloud and gsutil tools that are included with the instance. If you create new tools or add custom tools, you must authorize your application using a client library or by using access tokens directly in your application..

Authenticating applications directly with access tokens

For some applications, you might need to request the access token and use it directly without going through a client library or using the gcloud or gsutil tools. There are several options for obtaining and using these access tokens to authenticate your applications. For example, you can use curl to create a simple request, or use a programming language like Python for more flexibility:


Use curl to easily request an access token and send a request to an API.

  1. Ensure that your instance is enabled to use the necessary service account and scopes for your request. For this example, the instance requires the scope at a minimum.

  2. On the instance where your application runs, query the metadata server for an access token:

    $ curl "http://metadata/computeMetadata/v1/instance/service-accounts/default/token" \
      -H "Metadata-Flavor: Google"
  3. Use this access_token when you send requests to the API. You can use the access token only for scopes that you specified when you created the instance. For example, the following request prints a list of instances in your project from zone us-central1-a:

    $ curl \
      -H "Authorization":"Bearer ACCESS_TOKEN"

For information about the parameters that you can set in your request, see the parameters documentation.


This example demonstrates how to request a token to access the Google Cloud Storage API in a Python application. The example uses the following procedure:

  1. Request an access token from the metadata server.
  2. Extract the access token from the server response.
  3. Use the access token to make a request to Google Cloud Storage.
  4. If the request is successful, the script prints the response.

The script uses httplib2 to request the access token and submit the request to Google Cloud Storage, but you can simplify this process with Application Default Credentials and the Python Client Library.


import json
import urllib
import httplib2

METADATA_SERVER = 'http://metadata/computeMetadata/v1/instance/service-accounts'

def main():
  token_uri = '%s/%s/token' % (METADATA_SERVER, SERVICE_ACCOUNT)
  http = httplib2.Http()

  # Request an access token from the metadata server.
  resp, content = http.request(token_uri, method='GET', body=None, headers={'Metadata-Flavor': 'Google'})

  # Extract the access token from the response.
  if resp.status == 200:
    d = json.loads(content)
    access_token = d['access_token'] # Save the access token

    # Construct the request to Google Cloud Storage
    resp, content = http.request('', \
                                  body=None, \
                                  headers={'Authorization': 'OAuth ' + access_token, \
                                           'x-goog-api-version': '2', \
                                           'x-goog-project-id': GOOGLE_STORAGE_PROJECT_NUMBER })

    if resp.status == 200:
       print content
       print resp.status

    print resp.status

if __name__ == '__main__':

Access tokens expire after a short period of time. The metadata server caches access tokens until they have 60 seconds of remaining time to expire. You can request new tokens as frequently as you like, but your applications must have a valid access token for their API calls to succeed.

There is a limit to the total number of tokens that your service account can have at any one point in time. Currently, this limit is 600. If this limit is reached, Compute Engine will not be able to create an instance that requires a new token, and you will get a SERVICE_ACCOUNT_TOO_MANY_TOKENS error.

If you reach the token limit and cannot create new tokens, you must delete some tokens that are associated with the service account. Delete Compute Engine instances that are using the service account. This reduces the number of distinct scope sets that the service account uses.

Obtaining the Compute Engine service account name

Some API calls require you to include the service account name in the request body. Obtain the Compute Engine service account name through one of the following options:

The Service accounts section lists all of the service accounts that are enabled for this project. New Compute Engine projects have service account names with a -compute suffix.

  • Query the metadata server from within the instance itself. Make a request to http://metadata/computeMetadata/v1/instance/service-accounts/:

    user@myinst:~$ curl "http://metadata/computeMetadata/v1/instance/service-accounts/" \
                   -H "Metadata-Flavor: Google"

    If you enabled one or more scopes when you created the instance, this curl command returns output similar to the following:

    If the instance isn't using a Compute Engine service account, you receive an empty response.

  • Use the gcloud compute instances describe command from your local machine:

    $ gcloud compute instances describe INSTANCE --format json

    The response contains the following output:


    If the instance isn't using a Compute Engine service account, you receive a response without the serviceAccounts property.

Service account types

Google offers two different types of service accounts that you can use:

  • Developers Console service accounts let you write applications that can authenticate themselves to access a compatible Google API using a JSON Web Token (JWT). The JWT can acquire access tokens to Google APIs. You can write applications on Google Compute Engine that use Developers Console service accounts. See the Developers Console Service Accounts documentation for more information.

  • Google Compute Engine service accounts work without creating or managing a JWT. Instead, programs running within your Google Compute Engine instances can automatically acquire access tokens with credentials to access any service in your Google API project and any other services that granted access to that service account. New Compute Engine projects have service account names with a -compute suffix.