Authenticate to APIs and services from fleet workloads

This page shows you how to configure your applications to authenticate to Google Cloud APIs like the Compute Engine API or the AI Platform API by using fleet Workload Identity Federation.

What is fleet Workload Identity Federation?

Workload Identity Federation lets the workloads in your clusters authenticate to Google Cloud without requiring you to download, manually rotate, and generally manage credentials. Instead, workloads authenticate using short-lived tokens generated by Google Cloud.

Workload Identity Federation for GKE provides a project-wide workload identity pool from which applications running in GKE clusters get identities. Fleet Workload Identity Federation extends Workload Identity Federation for GKE to all fleet member clusters, regardless of whether the clusters are in different projects or are outside of Google Cloud. Registered clusters with Workload Identity Federation enabled on their fleet membership get identities using a fleet-wide workload identity pool, which lets you configure authentication to Google Cloud APIs and to other services across your entire fleet, even across multiple projects.

Fleet Workload Identity Federation can also be used by the Connect Agent on some cluster types to authenticate to Google Cloud as part of fleet membership, and is required to use some GKE Enterprise features that work across projects, such as Cloud Service Mesh.

Workload Identity Federation pools and identity sameness

With fleet Workload Identity Federation, each application in your fleet gets a distinct federated identity that can be used to authenticate to Google Cloud and other services that you develop. Applications get a principal identifier that IAM can recognize. This identifier uses the following syntax:

PREFIX://iam.googleapis.com/projects/FLEET_PROJECT_NUMBER/locations/global/workloadIdentityPools/FLEET_PROJECT_ID.svc.id.goog/SELECTOR

This syntax has the following fields:

  • PREFIX: the IAM principal or principalSet depending on the resource being selected.
  • FLEET_PROJECT_ID.svc.id.goog: the workload identity pool for your fleet. Each fleet has a single fixed workload identity pool that's created for you.
  • FLEET_PROJECT_NUMBER: the project number of the fleet host project.
  • SELECTOR: the resource selector. For a list of supported selectors, see supported principal identifiers.

The entire fleet shares a fleet workload identity pool, so that you can give applications anywhere in the fleet, including in other projects or clouds, access to the same resources without needing to manage that access for each cluster. Like other fleet-enabled features, fleet Workload Identity Federation relies on the principle of sameness, in which Kubernetes objects that have the same name and namespace in different clusters are treated as the same thing.

For example, if you have an application with a backend deployed across multiple clusters in the same fleet, and which needs to authenticate to a Google Cloud API, you can configure your application so that all workloads in the backend namespace can access that API. For more information on how fleets use sameness, including identity sameness, see How fleets work.

After you enable fleet Workload Identity Federation, you can reference principals in your fleet in IAM allow policies by specifying the corresponding principal identifier. For example, you could reference a specific ServiceAccount in a specific Kubernetes namespace in your allow policy. All applications that use that ServiceAccount could then access the Google Cloud resource to which the IAM allow policy applies.

Credential flow

To let applications in a specific namespace authenticate using fleet Workload Identity Federation, you do the following:

  1. Deploy a ConfigMap in that namespace that has the following information:

    • The workload identity pool and the identity provider for your cluster.
    • The path in each Pod onto which Kubernetes mounts a ServiceAccount token. This token is a signed JSON Web Token (JWT).

    This ConfigMap functions as the application default credentials (ADC) file for workloads.

  2. Create an IAM allow policy that grants access on specific Google Cloud resources to the principal identifier of the principal in your clusters, like a ServiceAccount in the namespace.

  3. Ensure that your workload in the namespace has the following configurations in the Pod specification:

    • The GOOGLE_APPLICATION_CREDENTIALS environment variable set to the mount path of the ConfigMap in the Pod.
    • A projected volume that contains the ServiceAccount token and the ConfigMap that you created, mounted in the same path that you specify in the GOOGLE_APPLICATION_CREDENTIALS environment variable.
    • A volume mount in the container that references the projected volume.

When the workload makes a Google Cloud API call, the following steps happen:

  1. The Google Cloud authentication libraries use application default credentials (ADC) to find credentials. ADC checks the path that you specified in the GOOGLE_APPLICATION_CREDENTIALS environment variable to look for an authentication token.
  2. The ADC authentication library uses the data in the ConfigMap to exchange the ServiceAccount JWT that you mounted on the Pod for a short-lived federated token from Security Token Service that references the principal identifier of the workload.
  3. ADC includes the federated token with the API request.
  4. The IAM allow policy authorizes the principal identifier to perform the requested operation on the Google Cloud resource.

Supported principal identifiers for fleet Workload Identity Federation

The following table describes the selectors that you can use in IAM allow policies to reference principals in fleets:

Principal identifier type Syntax
All Pods that use a specific Kubernetes ServiceAccount Select the ServiceAccount by name:
principal://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/PROJECT_ID.svc.id.goog/subject/ns/NAMESPACE/sa/SERVICEACCOUNT

Replace the following:

  • PROJECT_NUMBER: your numerical project number. To get the project number, see Identifying projects.
  • PROJECT_ID: your Google Cloud project ID.
  • NAMESPACE: the Kubernetes namespace.
  • SERVICEACCOUNT: the Kubernetes ServiceAccount name.

Select the ServiceAccount by UID:
principal://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/PROJECT_ID.svc.id.goog/kubernetes.serviceaccount.uid/SERVICEACCOUNT_UID

Replace the following:

  • PROJECT_NUMBER: your numerical project number. To get the project number, see Identifying projects.
  • PROJECT_ID: your Fleet project ID.
  • SERVICEACCOUNT_UID: the UID of the ServiceAccount object in the API server.

Before you start

  • Ensure that you have the following command line tools installed:

    • The latest version of the Google Cloud CLI, which includes gcloud, the command line tool for interacting with Google Cloud.
    • kubectl

    If you are using Cloud Shell as your shell environment for interacting with Google Cloud, these tools are installed for you.

  • Ensure that you have initialized the gcloud CLI for use with your project.

Prepare your clusters

Before applications in your fleet can receive a federated identity, the clusters that they run on must be registered to your fleet and properly configured to use fleet Workload Identity Federation. The following sections describe how to set up fleet Workload Identity Federation for different types of clusters.

GKE

For GKE clusters, do the following:

  1. Enable Workload Identity Federation for GKE on your Google Kubernetes Engine cluster, if it is not already enabled.
  2. Register the cluster to the fleet.

You can also enable Workload Identity Federation for GKE during the cluster creation and fleet registration process.

Clusters outside Google Cloud

The following types of clusters automatically enable fleet Workload Identity Federation and are registered to your fleet during cluster creation:

  • Google Distributed Cloud (software only) on VMware
  • Google Distributed Cloud (software only) on bare metal
  • GKE on AWS
  • GKE on Azure

Attached clusters

EKS and AKS attached clusters registered using the GKE Multi-Cloud API are registered with fleet Workload Identity Federation enabled by default. Other attached clusters can be registered with fleet Workload Identity Federation enabled if they meet the necessary requirements. Follow the instructions for your cluster type in Registering a cluster.

Use fleet Workload Identity Federation in applications

The following steps show you how to configure a workload in a registered cluster to use fleet Workload Identity Federation:

  1. Find the name of your cluster workload identity pool and identity provider:

    gcloud container fleet memberships describe MEMBERSHIP_ID \
        --project=FLEET_PROJECT_ID \
        --format="table(authority.identityProvider,authority.workloadIdentityPool,name)"
    

    Replace the following:

    • MEMBERSHIP_ID: the cluster membership name. This is often the name of your cluster.
    • FLEET_PROJECT_ID: the project ID of the fleet host project.

    The output is similar to the following:

    IDENTITY_PROVIDER: IDENTITY_PROVIDER
    WORKLOAD_IDENTITY_POOL: WORKLOAD_IDENTITY_POOL
    NAME: projects/FLEET_PROJECT_ID/locations/MEMBERSHIP_LOCATION/memberships/MEMBERSHIP_ID
    

    This output contains the following information:

    • IDENTITY_PROVIDER: the identity provider for the cluster.
    • MEMBERSHIP_LOCATION: the location of the fleet membership. This is usually the same as your cluster location.
    • WORKLOAD_IDENTITY_POOL: the name of the workload identity pool associated with your fleet. This value has the syntax FLEET_PROJECT_ID.svc.id.goog.
  2. Create a Kubernetes namespace. You can also use any existing namespace, including the default namespace.

    kubectl create namespace NAMESPACE
    

    Replace NAMESPACE with the name of the namespace.

  3. Create a new Kubernetes ServiceAccount in the namespace. You can also use any existing ServiceAccount, including the default ServiceAccount in the namespace.

    kubectl create serviceaccount KSA_NAME \
        --namespace=NAMESPACE
    

    Replace KSA_NAME with the name of the ServiceAccount.

  4. Save the following manifest as adc-config-map.yaml. This ConfigMap contains the ADC configuration for workloads.

    kind: ConfigMap
    apiVersion: v1
    metadata:
      namespace: NAMESPACE
      name: my-cloudsdk-config
    data:
      config: |
        {
          "type": "external_account",
          "audience": "identitynamespace:WORKLOAD_IDENTITY_POOL:IDENTITY_PROVIDER",
          "subject_token_type": "urn:ietf:params:oauth:token-type:jwt",
          "token_url": "https://sts.googleapis.com/v1/token",
          "credential_source": {
            "file": "/var/run/secrets/tokens/gcp-ksa/token"
          }
        }
    
  5. Deploy the ConfigMap:

    kubectl create -f adc-config-map.yaml
    
  6. Save the following manifest as workload-config.yaml:

    apiVersion: v1
    kind: Pod
    metadata:
      name: my-pod
      namespace:  NAMESPACE
    spec:
      serviceAccountName: KSA_NAME
      containers:
      - name: sample-container
        image: google/cloud-sdk:slim
        command: ["sleep","infinity"]
        env:
        - name: GOOGLE_APPLICATION_CREDENTIALS
          value: /var/run/secrets/tokens/gcp-ksa/google-application-credentials.json
        volumeMounts:
        - name: gcp-ksa
          mountPath: /var/run/secrets/tokens/gcp-ksa
          readOnly: true
      volumes:
      - name: gcp-ksa
        projected:
          defaultMode: 420
          sources:
          - serviceAccountToken:
              path: token
              audience: WORKLOAD_IDENTITY_POOL
              expirationSeconds: 172800
          - configMap:
              name: my-cloudsdk-config
              optional: false
              items:
              - key: "config"
                path: "google-application-credentials.json"
    

    When you deploy this workload, the gcp-ksa volume in the Pod contains the following data:

    The container in the Pod mounts the gcp-ksa volume to the /var/run/secrets/tokens/gcp-ksa path and configures ADC to look for the credential configuration JSON file in that path.

  7. Deploy the workload:

    kubectl apply -f workload-config.yaml
    

Alternative: Impersonate an IAM service account

Alternatively, you can configure Kubernetes ServiceAccounts in your clusters to impersonate IAM service accounts and perform any authorized actions that the IAM service accounts can perform. This approach might increase maintenance overhead, because you have to manage pairings of service accounts in both IAM and Kubernetes.

In most scenarios, we recommend that you directly reference Kubernetes principals in IAM allow policies to grant access to Google Cloud resources by following the instructions in Use fleet Workload Identity Federation in applications.

  1. Find the name of your cluster workload identity pool and identity provider:

    gcloud container fleet memberships describe MEMBERSHIP_ID \
        --project=FLEET_PROJECT_ID \
        --format="table(authority.identityProvider,authority.workloadIdentityPool,name)"
    

    Replace the following:

    • MEMBERSHIP_ID: the cluster membership name. This is often the name of your cluster.
    • FLEET_PROJECT_ID: the project ID of the fleet host project.

    The output is similar to the following:

    IDENTITY_PROVIDER: IDENTITY_PROVIDER
    WORKLOAD_IDENTITY_POOL: WORKLOAD_IDENTITY_POOL
    NAME: projects/FLEET_PROJECT_ID/locations/MEMBERSHIP_LOCATION/memberships/MEMBERSHIP_ID
    

    This output contains the following information:

    • IDENTITY_PROVIDER: the identity provider for the cluster.
    • MEMBERSHIP_LOCATION: the location of the membership. This is usually the same as your cluster location.
    • WORKLOAD_IDENTITY_POOL: the name of the workload identity pool associated with your fleet. This value has the syntax FLEET_PROJECT_ID.svc.id.goog.
  2. Create an IAM service account that your application can impersonate. You can also use any existing IAM service account.

    gcloud iam service-accounts create IAM_SA_NAME \
        --project=IAM_SA_PROJECT_ID
    

    Replace the following:

    • IAM_SA_NAME: the name for your IAM service account.
    • IAM_SA_PROJECT_ID: the project ID of the project that contains your IAM service account. This can be different than your fleet host project.
  3. Grant the IAM service account any permissions that it needs to access Google Cloud APIs by adding the necessary IAM allow policies. You can do this by using gcloud iam service-accounts add-iam-policy-binding or another method. You can find out which permissions are required to use Google Cloud APIs in each service's documentation, and see a complete list of predefined roles with the necessary permissions in Understanding roles.

  4. Create a Kubernetes ServiceAccount in a namespace. You can also use an existing Kubernetes ServiceAccount and any namespace, including the default ServiceAccount and the default namespace.

    kubectl create serviceaccount KSA_NAME \
        --namespace=NAMESPACE
    

    Replace the following:

    • KSA_NAME: the name of the ServiceAccount.
    • NAMESPACE: the name of the namespace.
  5. Create an IAM allow policy that lets a Kubernetes ServiceAccount in a specific namespace in your cluster impersonate the IAM service account:

    gcloud iam service-accounts add-iam-policy-binding IAM_SA_NAME@IAM_SA_PROJECT_ID.iam.gserviceaccount.com \
        --project=IAM_SA_PROJECT_ID \
        --role=roles/iam.workloadIdentityUser \
        --member="serviceAccount:WORKLOAD_IDENTITY_POOL[NAMESPACE/KSA_NAME]"
    

    Replace WORKLOAD_IDENTITY_POOL with the name of the workload identity pool.

  6. Save the following manifest as adc-config-map.yaml. This ConfigMap contains the ADC configuration for workloads.

    kind: ConfigMap
    apiVersion: v1
    metadata:
      namespace: K8S_NAMESPACE
      name: my-cloudsdk-config
    data:
      config: |
        {
          "type": "external_account",
          "audience": "identitynamespace:WORKLOAD_IDENTITY_POOL:IDENTITY_PROVIDER",
          "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/IAM_SA_NAME@GSA_PROJECT_ID.iam.gserviceaccount.com:generateAccessToken",
          "subject_token_type": "urn:ietf:params:oauth:token-type:jwt",
          "token_url": "https://sts.googleapis.com/v1/token",
          "credential_source": {
            "file": "/var/run/secrets/tokens/gcp-ksa/token"
          }
        }
    

    Replace the following:

    • IAM_SA_NAME: the name of the IAM service account to impersonate.
    • IAM_SA_PROJECT_ID: the project ID of the IAM service account.
  7. Save the following manifest as workload-config.yaml:

    apiVersion: v1
    kind: Pod
    metadata:
      name: my-pod
      namespace:  K8S_NAMESPACE
    spec:
      serviceAccountName: KSA_NAME
      containers:
      - name: my-container
        image: my-image
        command: ["sleep","infinity"]
        env:
          - name: GOOGLE_APPLICATION_CREDENTIALS
            value: /var/run/secrets/tokens/gcp-ksa/google-application-credentials.json
        volumeMounts:
        - name: gcp-ksa
          mountPath: /var/run/secrets/tokens/gcp-ksa
          readOnly: true
      volumes:
      - name: gcp-ksa
        projected:
          defaultMode: 420
          sources:
          - serviceAccountToken:
              path: token
              audience: WORKLOAD_IDENTITY_POOL
              expirationSeconds: 172800
          - configMap:
              name: my-cloudsdk-config
              optional: false
              items:
                - key: "config"
                  path: "google-application-credentials.json"
    
    

    When you deploy this workload, the gcp-ksa volume in the Pod contains the following data:

    The container in the Pod mounts the gcp-ksa volume to the /var/run/secrets/tokens/gcp-ksa path and configures ADC to look for the credential configuration JSON file in that path.

  8. Deploy the workload:

    kubectl apply -f workload-config.yaml
    

Verify the fleet Workload Identity Federation setup

In this section, you create a Cloud Storage bucket and access it from a Pod that uses fleet Workload Identity Federation. Before you perform these steps, ensure that you configured Workload Identity Federation by following the instructions in the Use fleet Workload Identity Federation in applications section.

This section doesn't show you how to verify Workload Identity Federation using the IAM service account impersonation method.

  1. Find your numerical project number:

    gcloud projects describe FLEET_PROJECT_ID \
        --format="value(projectNumber)"
    

    The output is similar to the following:

    1234567890
    
  2. Create a Cloud Storage bucket:

    gcloud storage buckets create gs://FLEET_PROJECT_ID-test-bucket \
        --location=LOCATION
    

    Replace LOCATION with a Google Cloud location.

  3. Create an IAM allow policy that grants access on the bucket to the service account that you created:

    gcloud storage buckets add-iam-policy-binding gs://FLEET_PROJECT_ID-test-bucket \
        --condition=None \
        --role=roles/storage.objectViewer \
        --member=principal://iam.googleapis.com/projects/FLEET_PROJECT_NUMBER/locations/global/workloadIdentityPools/FLEET_PROJECT_ID.svc.id.goog/subject/ns/NAMESPACE/sa/KSA_NAME
    

    Replace the following:

    • FLEET_PROJECT_NUMBER: your numerical project number.
    • FLEET_PROJECT_ID: your project ID.
    • NAMESPACE: the name of the Kubernetes namespace that runs your Pod from the previous section.
    • KSA_NAME: the name of the Kubernetes ServiceAccount that your Pod from the previous section uses.
  4. Create a shell session in your Pod:

    kubectl exec -it pods/my-pod --namespace=NAMESPACE -- /bin/bash
    
  5. Try to list the objects in the bucket:

    curl -X GET -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
    "https://storage.googleapis.com/storage/v1/b/FLEET_PROJECT_ID-test-bucket/o"
    

    The output is the following:

    {
      "kind": "storage#objects"
    }
    

Authenticate from your code

When you use Cloud Client Libraries, authentication libraries automatically use ADC to look for credentials to authenticate to Google Cloud services. You must use Cloud Client Libraries that support Workload Identity Federation. The following shows the minimum required Cloud Client Libraries versions, as well as instructions for how to check your current version:

C++

Most of the Google Cloud Client Libraries for C++ support identity federation by using a ChannelCredentials object, which is created by calling grpc::GoogleDefaultCredentials(). To initialize this credential, you must build the client libraries with version 1.36.0 or later of gRPC.

The Cloud Storage Client Library for C++ uses the REST API, not gRPC, so it does not support identity federation.

Go

Client libraries for Go support identity federation if they use version v0.0.0-20210218202405-ba52d332ba99 or later of the golang.org/x/oauth2 module.

To check which version of this module your client library uses, run the following commands:

cd $GOPATH/src/cloud.google.com/go
go list -m golang.org/x/oauth2

Java

Client libraries for Java support identity federation if they use version 0.24.0 or later of the com.google.auth:google-auth-library-oauth2-http artifact.

To check which version of this artifact your client library uses, run the following Maven command in your application directory:

mvn dependency:list -DincludeArtifactIds=google-auth-library-oauth2-http

Node.js

Client libraries for Node.js support identity federation if they use version 7.0.2 or later of the google-auth-library package.

To check which version of this package your client library uses, run the following command in your application directory:

npm list google-auth-library

When you create a GoogleAuth object, you can specify a project ID, or you can allow GoogleAuth to find the project ID automatically. To find the project ID automatically, the service account in the configuration file must have the Browser role (roles/browser), or a role with equivalent permissions, on your project. For details, see the README for the google-auth-library package.

Python

Client libraries for Python support identity federation if they use version 1.27.0 or later of the google-auth package.

To check which version of this package your client library uses, run the following command in the environment where the package is installed:

pip show google-auth

To specify a project ID for the authentication client, you can set the GOOGLE_CLOUD_PROJECT environment variable, or you can allow the client to find the project ID automatically. To find the project ID automatically, the service account in the configuration file must have the Browser role (roles/browser), or a role with equivalent permissions, on your project. For details, see the user guide for the google-auth package.

What's next?

Learn best practices for organizing your fleets when using fleet Workload Identity Federation.