Accessing resources from AWS

This document shows you how to use identity federation to access Google Cloud resources from Amazon Web Services (AWS).

Traditionally, applications running outside Google Cloud have used service account keys to access Google Cloud resources. Using identity federation, you can allow an AWS user or role to impersonate a service account. This lets your workload access Google Cloud resources directly, using a short-lived access token, and eliminates the maintenance and security burden associated with service account keys.

Before you begin

  1. Enable the IAM, Resource Manager, Service Account Credentials, and Security Token Service (STS) APIs.

    Enable the APIs

  2. Ensure you have the Workload Identity Pool Admin role (roles/iam.workloadIdentityPoolAdmin).

    Alternatively, the IAM Owner (roles/owner) basic role also includes permissions to configure identity federation. You should not grant basic roles in a production environment, but you can grant them in a development or test environment.

  3. Update the organization policy for your organization to allow federation from AWS.

    Optionally, you can also specify which AWS account IDs are allowed to access your Google Cloud resources.

  4. Create an AWS role, and take note of its Amazon Resource Name (ARN).

  5. Create a Google Cloud service account.

  6. Grant the service account access to call the Google Cloud APIs required by your workload.

Creating a workload identity pool

You can use a workload identity pool to organize and manage external identities. Workload identity pools are isolated from each other, but a single pool can impersonate any number of service accounts. In general, we recommend creating a new pool for each of your environments, such as development, staging, or production, which typically means one pool per AWS account.

To create a new workload identity pool, you'll need to provide an ID. You can also provide an optional description and display name.

gcloud

Execute the gcloud iam workload-identity-pools create command to create a workload identity pool:

gcloud iam workload-identity-pools create pool-id \
    --location="global" \
    --description="description" \
    --display-name="display-name"

The response looks like:

Created workload identity pool [pool-id].

REST

The projects.locations.workloadIdentityPools.create method creates a workload identity pool.

HTTP method and URL:

POST https://iam.googleapis.com/v1/projects/project-id/locations/global/workloadIdentityPools?workloadIdentityPoolId=pool-id

Request JSON body:

{
  "description": "description",
  "display-name": "display-name"
}

To send your request, expand one of these options:

The method returns a long-running Operation similar to the following:

{
  "name": "projects/project-number/locations/global/workloadIdentityPools/pool-id/operations/operation-id"
}

Adding AWS as an identity provider

To configure AWS as an identity provider for your workload identity pool, supply at least the following:

  • An ID for the provider.

  • The workload identity pool ID from the previous section in this document.

  • Your AWS account ID.

You can also provide several optional parameters:

  • A display name and description.

  • A list of attribute mappings that map the attributes on an AWS token to the attributes on a Google token. By default, each pool uses the following attribute mappings, which cover most common scenarios:

    Google AWS Description
    google.subject assertion.arn The principal that IAM is authenticating. This is also the subject that appears in Cloud Logging log entries. This mapping is automatically populated with the ARN, using the form arn:aws:sts::account-id:assumed-role/aws-role/aws-session-name.
    attribute.aws_role AWS role The AWS role, using the form arn:aws:sts::account-id:assumed-role/aws-role.

    You can also specify custom mappings, which you can reference in IAM role bindings. Use assertion to refer to the AWS credential, google for Google attributes, and attribute for custom attributes. For example, the following maps attribute.aws_account to assertion.account (in addition to the default mapping for google.subject):

    google.subject=assertion.arn,
    attribute.aws_account=assertion.account
    

    See the GetCallerIdentity() documentation for a list of attributes on AWS tokens that you can reference. Note that the attributes in the AWS documentation use camel case, whereas the attribute mapping uses lower case; for example, Account becomes assertion.account.

    For more complex assertions, you can use Common Expression Language. For example:

    attribute.environment=assertion.arn.contains(":instance-profile/Production") ? "prod" : "test"
    

    To reference a specific part of an attribute in an expression, use the CEL extract() function, which extracts a value from an attribute based on a template you provide. To learn more about extract(), see Extracting values from attributes.

    To check if a credential contains an attribute, use the has() function.

  • An attribute condition specifying attributes that the principal must present. The attribute condition can apply to external and Google credentials. Any request that does not meet the attribute condition is rejected.

    Attribute conditions are formatted as a CEL expression that returns a boolean. For example, the following rejects requests from any identity that does not have a specific AWS role:

    attribute.aws_role == "role-mapping"
    

    To learn more about common use cases, see Attribute conditions.

The following example demonstrates adding AWS as an identity provider:

gcloud

Execute the gcloud iam workload-identity-pools providers create-aws command to add AWS as an identity provider:

gcloud iam workload-identity-pools providers create-aws provider-id \
    --workload-identity-pool="pool-id" \
    --account-id="aws-account-id" \
    --location="global"

The response looks like:

Created workload identity pool provider [provider-id].

REST

The projects.locations.workloadIdentityPools.providers.create method adds AWS as a provider.

HTTP method and URL:

POST https://iam.googleapis.com/v1/projects/project-id/locations/global/workloadIdentityPools/pool-id/providers?workloadIdentityPoolProviderId=provider-id

Request JSON body:

{
  "aws": {
    "accountId": "aws-account-id"
  }
}

To send your request, expand one of these options:

The method returns a long-running Operation similar to the following:

{
  "name": "projects/project-number/locations/global/workloadIdentityPools/pool-id/providers/provider-id/operations/operation-id"
}

Giving permission to impersonate a service account

External identities can't access most Google Cloud resources directly. Instead, you let the identities impersonate a service account by granting them the Workload Identity User role (roles/iam.workloadIdentityUser).

To add this role binding for an AWS role, use the format:

attribute.aws_role/arn:aws:sts::aws-account-id:assumed-role/aws-role-name

For example:

gcloud iam service-accounts add-iam-policy-binding service-account-email \
    --role roles/iam.workloadIdentityUser \
    --member "principalSet://iam.googleapis.com/projects/project-number/locations/global/workloadIdentityPools/pool-id/attribute.aws_role/arn:aws:sts::aws-account-id:assumed-role/aws-role-name"

To add this binding for an AWS user, use the format:

subject/arn:aws:sts::aws-account-id:assumed-role/aws-role-name/aws-session-name

See the AWS documentation on IAM Identifiers for information on how to extract the AWS role session from an AWS ARN.

The following example demonstrates adding a binding for an AWS user:

gcloud iam service-accounts add-iam-policy-binding service-account-email \
    --role roles/iam.workloadIdentityUser \
    --member "principal://iam.googleapis.com/projects/project-number/locations/global/workloadIdentityPools/pool-id/subject/arn:aws:sts::aws-account-id:assumed-role/aws-role-name/aws-session-name"

You can also grant access based on custom attributes. For example:

gcloud iam service-accounts add-iam-policy-binding service-account-email \
    --role="roles/iam.workloadIdentityUser" \
    --member="principalSet://iam.googleapis.com/projects/project-number/locations/global/workloadIdentityPools/pool-id/attribute.custom-attribute/arn:aws:sts::aws-account-id:aws-role-name"

To revoke access, replace add-iam-policy-binding with remove-iam-policy-binding.

You can also add or revoke bindings using the REST API or client libraries. To learn more, see Granting, changing, and revoking access to resources.

Generating Google credentials

If you use a supported client library, you can configure the client library so that it generates Google credentials automatically. Alternatively, you can generate AWS credentials manually, then exchange them for Google credentials.

When possible, we recommend that you generate credentials automatically, so that you do not need to implement the token-exchange process yourself.

Automatically generating credentials

If you access Google Cloud with a client library for one of the following languages, you can configure the client library to automatically generate credentials by using identity federation:

C++

Most of the Google Cloud Client Libraries for C++ support identity federation by using a ChannelCredentials object, which is created by calling grpc::GoogleDefaultCredentials(). To initialize this credential, you must build the client libraries with version 1.36.0 or later of gRPC.

The Cloud Storage Client Library for C++ uses the REST API, not gRPC, so it does not support identity federation.

Go

Client libraries for Go support identity federation if they use version v0.0.0-20210218202405-ba52d332ba99 or later of the golang.org/x/oauth2 module.

To check which version of this module your client library uses, run the following commands:

cd $GOPATH/src/cloud.google.com/go
go list -m golang.org/x/oauth2

Java

Client libraries for Java support identity federation if they use version 0.24.0 or later of the com.google.auth:google-auth-library-oauth2-http artifact.

To check which version of this artifact your client library uses, run the following Maven command in your application directory:

mvn dependency:list -DincludeArtifactIds=google-auth-library-oauth2-http

Node.js

Client libraries for Node.js support identity federation if they use version 7.0.2 or later of the google-auth-library package.

To check which version of this package your client library uses, run the following command in your application directory:

npm list google-auth-library

When you create a GoogleAuth object, you can specify a project ID, or you can allow GoogleAuth to find the project ID automatically. To find the project ID automatically, the service account in the configuration file must have the Browser role (roles/browser), or a role with equivalent permissions, on your project. For details, see the README for the google-auth-library package.

Python

Client libraries for Python support identity federation if they use version 1.27.0 or later of the google-auth package.

To check which version of this package your client library uses, run the following command in the environment where the package is installed:

pip show google-auth

To specify a project ID for the authentication client, you can set the GOOGLE_CLOUD_PROJECT environment variable, or you can allow the client to find the project ID automatically. To find the project ID automatically, the service account in the configuration file must have the Browser role (roles/browser), or a role with equivalent permissions, on your project. For details, see the user guide for the google-auth package.

To configure the client library to generate credentials automatically, run the gcloud iam workload-identity-pools create-cred-config command to generate a JSON configuration file:

gcloud iam workload-identity-pools create-cred-config \
    projects/project-number/locations/global/workloadIdentityPools/pool-id/providers/provider-id \
    --service-account=service-account-email \
    --output-file=filepath \
    --aws

Replace the following values:

  • project-number: The numeric ID for the project.
  • pool-id: The ID for the workload identity pool.
  • provider-id: The ID for the workload identity pool provider.
  • service-account-email: The email address of the service account to impersonate.
  • filepath: The filepath for the configuration file.

After you generate the configuration file, set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the filepath for the configuration file. This environment variable tells the client library to use Application Default Credentials to authenticate. For details, see Finding credentials automatically.

Manually exchanging credentials

Once an AWS role or user has the ability to impersonate a service account, you can manually exchange their AWS credentials for Google credentials.

As part of the exchange process, you pass a GetCallerIdentity token to the Security Token Service. The GetCallerIdentity token contains the information that you would normally include in a request to the AWS GetCallerIdentity() method, as well as the signature that you would normally generate for the request. Google Cloud uses the GetCallerIdentity token to verify the identity of the AWS principal and confirm that the principal has permission to impersonate a service account.

To exchange credentials:

  1. Get temporary AWS credentials.

  2. Create a GetCallerIdentity token. The token contains the information for a request to the AWS GetCallerIdentity() method, as well as the AWS signature for the request information. Use Signature Version 4.

    The request contains the following fields:

    • url: The URL of the AWS STS endpoint for GetCallerIdentity(), with the body of a standard GetCallerIdentity() request appended as query parameters. For example, https://sts.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15. Regional endpoints are also supported.
    • method: The HTTP request method: POST.
    • headers: The HTTP request headers, which must include:

      • Authorization: The request signature.
      • host: The hostname of the url field; for example, sts.amazonaws.com.
      • x-amz-date: The time you will send the request, formatted as an ISO 8601 Basic string. This value is typically set to the current time and is used to help prevent replay attacks.
      • x-goog-cloud-target-resource: The full resource name of the identity provider. For example:
      //iam.googleapis.com/projects/project-number/locations/global/workloadIdentityPools/pool-id/providers/provider-id
      
      • x-amz-security-token: Required only if you are using temporary security credentials. The AWS session token to include.

    A GetCallerIdentity token looks similar to the following:

    {
      "url": "https://sts.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15",
      "method": "POST",
      "headers": [
        {
          "key": "Authorization",
          "value" : "AWS4-HMAC-SHA256 Credential=AKIASOZTBDV4D7ABCDEDF/20200228/us-east-1/sts/aws4_request, SignedHeaders=host;x-amz-date,Signature=abcedefdfedfd"
        },
        {
          "key": "host",
          "value": "sts.amazonaws.com"
        },
        {
          "key": "x-amz-date",
          "value": "20200228T225005Z"
        },
        {
          "key": "x-goog-cloud-target-resource",
          "value": "//iam.googleapis.com/projects/12345678/locations/global/workloadIdentityPools/my-pool/providers/my-aws-provider"
        },
        {
          "key": "x-amz-security-token",
          "value": "GizFWJTqYX...xJ55YoJ8E9HNU="
        }
      ]
    }
    
  3. To exchange the AWS credential for a federated access token, pass the GetCallerIdentity token to the Security Token Service's token() method:

    REST

    The token method exchanges a third-party token for a Google token.

    Before using any of the request data below, make the following replacements:

    • project-number: Your Google Cloud project number.
    • pool-id: The ID of the workload identity pool you created earlier in this tutorial.
    • provider-id: The ID of the AWS identity provider you configured earlier in this tutorial.
    • aws-request: The GetCallerIdentity token, formatted as URL-escaped JSON.

    HTTP method and URL:

    POST https://sts.googleapis.com/v1/token

    Request JSON body:

    {
      "audience": "//iam.googleapis.com/projects/project-number/locations/global/workloadIdentityPools/pool-id/providers/provider-id",
      "grantType": "urn:ietf:params:oauth:grant-type:token-exchange",
      "requestedTokenType": "urn:ietf:params:oauth:token-type:access_token",
      "scope": "https://www.googleapis.com/auth/cloud-platform",
      "subjectTokenType": "urn:ietf:params:aws:token-type:aws4_request",
      "subjectToken": "aws-request"
    }
    

    To send your request, expand one of these options:

     

    The method returns a federated token.

  4. To exchange the federated token for a service account access token, call the generateAccessToken() method. A limited number of Google Cloud APIs support federated tokens; in contrast, all Google Cloud APIs support service account access tokens.

    REST

    The Service Account Credentials API's serviceAccounts.generateAccessToken method generates an OAuth 2.0 access token for a service account.

    Before using any of the request data below, make the following replacements:

    • PROJECT_ID: Your Google Cloud project ID. Project IDs are alphanumeric strings, like my-project.
    • SA_ID: The ID of your service account. This can either be the service account's email address in the form SA_NAME@PROJECT_ID.iam.gserviceaccount.com, or the service account's unique numeric ID.
    • token: The federated access token.

    HTTP method and URL:

    POST https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/SA_NAME@PROJECT_ID.iam.gserviceaccount.com:generateAccessToken

    Request JSON body:

    {
      "scope": [
        "https://www.googleapis.com/auth/cloud-platform"
      ]
    }
    

    To send your request, expand one of these options:

    If the generateAccessToken request was successful, the response body contains an OAuth 2.0 access token and an expiration time. The accessToken can then be used to authenticate a request on behalf of the service account until the expireTime has been reached:

    {
      "accessToken": "eyJ0eXAi...NiJ9",
      "expireTime": "2020-04-07T15:01:23.045123456Z"
    }
    

When you have an access token for a service account, you can use the token to call Google Cloud APIs by including the token in the Authorization header of your requests:

Authorization: Bearer access-token

The request is authorized as the service account.

What's next