Authenticate to APIs and services from fleet workloads

This page shows you how to configure your applications to authenticate to Google Cloud APIs like the Compute Engine API or the AI Platform API by using fleet Workload Identity Federation.

What is fleet Workload Identity Federation?

Workload Identity Federation lets the workloads in your clusters authenticate to Google Cloud without requiring you to download, manually rotate, and generally manage credentials. Instead, workloads authenticate using short-lived tokens generated by Google Cloud.

Workload Identity Federation for GKE provides a project-wide workload identity pool from which applications running in GKE clusters get identities. Fleet Workload Identity Federation extends Workload Identity Federation for GKE to all fleet member clusters, regardless of whether the clusters are in different projects or are outside of Google Cloud. Registered clusters with Workload Identity Federation enabled on their fleet membership get identities using a fleet-wide workload identity pool, which lets you configure authentication to Google Cloud APIs and to other services across your entire fleet, even across multiple projects.

Fleet Workload Identity Federation can also be used by the Connect Agent on some cluster types to authenticate to Google Cloud as part of fleet membership, and is required to use some GKE Enterprise features that work across projects, such as Cloud Service Mesh.

Workload Identity Federation pools and identity sameness

With fleet Workload Identity Federation, each application in your fleet gets a distinct federated identity that can be used to authenticate to Google Cloud and other services that you develop. Applications get a principal identifier that IAM can recognize. This identifier uses the following syntax:

PREFIX://iam.googleapis.com/projects/FLEET_PROJECT_NUMBER/locations/global/workloadIdentityPools/FLEET_PROJECT_ID.svc.id.goog/SELECTOR

This syntax has the following fields:

PREFIX: the IAM principal or principalSet depending on the resource being selected.
FLEET_PROJECT_ID.svc.id.goog: the workload identity pool for your fleet. Each fleet has a single fixed workload identity pool that's created for you.
FLEET_PROJECT_NUMBER: the project number of the fleet host project.
SELECTOR: the resource selector. For a list of supported selectors, see supported principal identifiers.

The entire fleet shares a fleet workload identity pool, so that you can give applications anywhere in the fleet, including in other projects or clouds, access to the same resources without needing to manage that access for each cluster. Like other fleet-enabled features, fleet Workload Identity Federation relies on the principle of sameness, in which Kubernetes objects that have the same name and namespace in different clusters are treated as the same thing.

For example, if you have an application with a backend deployed across multiple clusters in the same fleet, and which needs to authenticate to a Google Cloud API, you can configure your application so that all workloads in the backend namespace can access that API. For more information on how fleets use sameness, including identity sameness, see How fleets work.

After you enable fleet Workload Identity Federation, you can reference principals in your fleet in IAM allow policies by specifying the corresponding principal identifier. For example, you could reference a specific ServiceAccount in a specific Kubernetes namespace in your allow policy. All applications that use that ServiceAccount could then access the Google Cloud resource to which the IAM allow policy applies.

Credential flow

To let applications in a specific namespace authenticate using fleet Workload Identity Federation, you do the following:

Deploy a ConfigMap in that namespace that has the following information:
- The workload identity pool and the identity provider for your cluster.
- The path in each Pod onto which Kubernetes mounts a ServiceAccount token. This token is a signed JSON Web Token (JWT).
This ConfigMap functions as the application default credentials (ADC) file for workloads.
Create an IAM allow policy that grants access on specific Google Cloud resources to the principal identifier of the principal in your clusters, like a ServiceAccount in the namespace.
Ensure that your workload in the namespace has the following configurations in the Pod specification:
- The GOOGLE_APPLICATION_CREDENTIALS environment variable set to the mount path of the ConfigMap in the Pod.
- A projected volume that contains the ServiceAccount token and the ConfigMap that you created, mounted in the same path that you specify in the GOOGLE_APPLICATION_CREDENTIALS environment variable.
- A volume mount in the container that references the projected volume.

When the workload makes a Google Cloud API call, the following steps happen:

The Google Cloud authentication libraries use Application Default Credentials (ADC) to find credentials. ADC checks the path that you specified in the GOOGLE_APPLICATION_CREDENTIALS environment variable to look for an authentication token.
The ADC authentication library uses the data in the ConfigMap to exchange the ServiceAccount JWT that you mounted on the Pod for a short-lived federated access token from Security Token Service that references the principal identifier of the workload.
ADC includes the federated access token with the API request.
The IAM allow policy authorizes the principal identifier to perform the requested operation on the Google Cloud resource.

Supported principal identifiers for fleet Workload Identity Federation

The following table describes the selectors that you can use in IAM allow policies to reference principals in fleets:

Principal identifier type Syntax

All Pods that use a specific Kubernetes ServiceAccount

Principal identifier type	Syntax
All Pods that use a specific Kubernetes ServiceAccount	Select the ServiceAccount by name: principal://iam.googleapis.com/projects/`PROJECT_NUMBER`/locations/global/workloadIdentityPools/`PROJECT_ID`.svc.id.goog/subject/ns/`NAMESPACE`/sa/`SERVICEACCOUNT` Replace the following: `PROJECT_NUMBER`: your numerical project number. To get the project number, see Identifying projects. `PROJECT_ID`: your Google Cloud project ID. `NAMESPACE`: the Kubernetes namespace. `SERVICEACCOUNT`: the Kubernetes ServiceAccount name. Select the ServiceAccount by UID: principal://iam.googleapis.com/projects/`PROJECT_NUMBER`/locations/global/workloadIdentityPools/`PROJECT_ID`.svc.id.goog/kubernetes.serviceaccount.uid/`SERVICEACCOUNT_UID` Replace the following: `PROJECT_NUMBER`: your numerical project number. To get the project number, see Identifying projects. `PROJECT_ID`: your Fleet project ID. `SERVICEACCOUNT_UID`: the UID of the `ServiceAccount` object in the API server.

Select the ServiceAccount by name:

principal://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/PROJECT_ID.svc.id.goog/subject/ns/NAMESPACE/sa/SERVICEACCOUNT

Replace the following:

PROJECT_NUMBER: your numerical project number. To get the project number, see Identifying projects.
PROJECT_ID: your Google Cloud project ID.
NAMESPACE: the Kubernetes namespace.
SERVICEACCOUNT: the Kubernetes ServiceAccount name.

Select the ServiceAccount by UID:

principal://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/PROJECT_ID.svc.id.goog/kubernetes.serviceaccount.uid/SERVICEACCOUNT_UID

Replace the following:

PROJECT_NUMBER: your numerical project number. To get the project number, see Identifying projects.
PROJECT_ID: your Fleet project ID.
SERVICEACCOUNT_UID: the UID of the ServiceAccount object in the API server.

Before you start

Ensure that you have the following command line tools installed:
- The latest version of the Google Cloud CLI, which includes gcloud, the command line tool for interacting with Google Cloud.
- kubectl
If you are using Cloud Shell as your shell environment for interacting with Google Cloud, these tools are installed for you.
Ensure that you have initialized the gcloud CLI for use with your project.

Prepare your clusters

Before applications in your fleet can receive a federated identity, the clusters that they run on must be registered to your fleet and properly configured to use fleet Workload Identity Federation. The following sections describe how to set up fleet Workload Identity Federation for different types of clusters.

GKE

For GKE clusters, do the following:

Enable Workload Identity Federation for GKE on your Google Kubernetes Engine cluster, if it is not already enabled.
Register the cluster to the fleet.

You can also enable Workload Identity Federation for GKE during the cluster creation and fleet registration process.

Clusters outside Google Cloud

The following types of clusters automatically enable fleet Workload Identity Federation and are registered to your fleet during cluster creation:

Google Distributed Cloud (software only) on VMware
Google Distributed Cloud (software only) on bare metal
GKE on AWS
GKE on Azure

Attached clusters

EKS and AKS attached clusters registered using the GKE Multi-Cloud API are registered with fleet Workload Identity Federation enabled by default. Other attached clusters can be registered with fleet Workload Identity Federation enabled if they meet the necessary requirements. Follow the instructions for your cluster type in Registering a cluster.

Use fleet Workload Identity Federation in applications

The following steps show you how to configure a workload in a registered cluster to use fleet Workload Identity Federation:

Find the name of your cluster workload identity pool and identity provider:
```
gcloud container fleet memberships describe MEMBERSHIP_ID \
    --project=FLEET_PROJECT_ID \
    --format="table(authority.identityProvider,authority.workloadIdentityPool,name)"
```
Replace the following:
- MEMBERSHIP_ID: the cluster membership name. This is often the name of your cluster.
- FLEET_PROJECT_ID: the project ID of the fleet host project.
The output is similar to the following:
```
IDENTITY_PROVIDER: IDENTITY_PROVIDER
WORKLOAD_IDENTITY_POOL: WORKLOAD_IDENTITY_POOL
NAME: projects/FLEET_PROJECT_ID/locations/MEMBERSHIP_LOCATION/memberships/MEMBERSHIP_ID
```
This output contains the following information:
- IDENTITY_PROVIDER: the identity provider for the cluster.
- MEMBERSHIP_LOCATION: the location of the fleet membership. This is usually the same as your cluster location.
- WORKLOAD_IDENTITY_POOL: the name of the workload identity pool associated with your fleet. This value has the syntax FLEET_PROJECT_ID.svc.id.goog.
Create a Kubernetes namespace. You can also use any existing namespace, including the default namespace.
```
kubectl create namespace NAMESPACE
```
Replace NAMESPACE with the name of the namespace.
Create a new Kubernetes ServiceAccount in the namespace. You can also use any existing ServiceAccount, including the default ServiceAccount in the namespace.
```
kubectl create serviceaccount KSA_NAME \
    --namespace=NAMESPACE
```
Replace KSA_NAME with the name of the ServiceAccount.

Save the following manifest as adc-config-map.yaml. This ConfigMap contains the ADC configuration for workloads.

kind: ConfigMap
apiVersion: v1
metadata:
  namespace: NAMESPACE
  name: my-cloudsdk-config
data:
  config: |
    {
      "type": "external_account",
      "audience": "identitynamespace:WORKLOAD_IDENTITY_POOL:IDENTITY_PROVIDER",
      "subject_token_type": "urn:ietf:params:oauth:token-type:jwt",
      "token_url": "https://sts.googleapis.com/v1/token",
      "credential_source": {
        "file": "/var/run/secrets/tokens/gcp-ksa/token"
      }
    }

Deploy the ConfigMap:
```
kubectl create -f adc-config-map.yaml
```

Save the following manifest as workload-config.yaml:

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
  namespace:  NAMESPACE
spec:
  serviceAccountName: KSA_NAME
  containers:
  - name: sample-container
    image: google/cloud-sdk:slim
    command: ["sleep","infinity"]
    env:
    - name: GOOGLE_APPLICATION_CREDENTIALS
      value: /var/run/secrets/tokens/gcp-ksa/google-application-credentials.json
    volumeMounts:
    - name: gcp-ksa
      mountPath: /var/run/secrets/tokens/gcp-ksa
      readOnly: true
  volumes:
  - name: gcp-ksa
    projected:
      defaultMode: 420
      sources:
      - serviceAccountToken:
          path: token
          audience: WORKLOAD_IDENTITY_POOL
          expirationSeconds: 172800
      - configMap:
          name: my-cloudsdk-config
          optional: false
          items:
          - key: "config"
            path: "google-application-credentials.json"

When you deploy this workload, the gcp-ksa volume in the Pod contains the following data:

The data in the ConfigMap that you deployed as a file named google-application-credentials.json. This file is an ADC credential configuration file.
The Kubernetes ServiceAccount token as token. Kubernetes mounts a signed JWT for the ServiceAccount as a projected ServiceAccount token file.

The container in the Pod mounts the gcp-ksa volume to the /var/run/secrets/tokens/gcp-ksa path and configures ADC to look for the credential configuration JSON file in that path.

Deploy the workload:
```
kubectl apply -f workload-config.yaml
```

Alternative: Impersonate an IAM service account

Alternatively, you can configure Kubernetes ServiceAccounts in your clusters to impersonate IAM service accounts and perform any authorized actions that the IAM service accounts can perform. This approach might increase maintenance overhead, because you have to manage pairings of service accounts in both IAM and Kubernetes.

In most scenarios, we recommend that you directly reference Kubernetes principals in IAM allow policies to grant access to Google Cloud resources by following the instructions in Use fleet Workload Identity Federation in applications.

Find the name of your cluster workload identity pool and identity provider:
```
gcloud container fleet memberships describe MEMBERSHIP_ID \
    --project=FLEET_PROJECT_ID \
    --format="table(authority.identityProvider,authority.workloadIdentityPool,name)"
```
Replace the following:
- MEMBERSHIP_ID: the cluster membership name. This is often the name of your cluster.
- FLEET_PROJECT_ID: the project ID of the fleet host project.
The output is similar to the following:
```
IDENTITY_PROVIDER: IDENTITY_PROVIDER
WORKLOAD_IDENTITY_POOL: WORKLOAD_IDENTITY_POOL
NAME: projects/FLEET_PROJECT_ID/locations/MEMBERSHIP_LOCATION/memberships/MEMBERSHIP_ID
```
This output contains the following information:
- IDENTITY_PROVIDER: the identity provider for the cluster.
- MEMBERSHIP_LOCATION: the location of the membership. This is usually the same as your cluster location.
- WORKLOAD_IDENTITY_POOL: the name of the workload identity pool associated with your fleet. This value has the syntax FLEET_PROJECT_ID.svc.id.goog.
Create an IAM service account that your application can impersonate. You can also use any existing IAM service account.
```
gcloud iam service-accounts create IAM_SA_NAME \
    --project=IAM_SA_PROJECT_ID
```
Replace the following:
- IAM_SA_NAME: the name for your IAM service account.
- IAM_SA_PROJECT_ID: the project ID of the project that contains your IAM service account. This can be different than your fleet host project.
Grant the IAM service account any permissions that it needs to access Google Cloud APIs by adding the necessary IAM allow policies. You can do this by using gcloud iam service-accounts add-iam-policy-binding or another method. You can find out which permissions are required to use Google Cloud APIs in each service's documentation, and see a complete list of predefined roles with the necessary permissions in Understanding roles.
Create a Kubernetes ServiceAccount in a namespace. You can also use an existing Kubernetes ServiceAccount and any namespace, including the default ServiceAccount and the default namespace.
```
kubectl create serviceaccount KSA_NAME \
    --namespace=NAMESPACE
```
Replace the following:
- KSA_NAME: the name of the ServiceAccount.
- NAMESPACE: the name of the namespace.

Create an IAM allow policy that lets a Kubernetes ServiceAccount in a specific namespace in your cluster impersonate the IAM service account:

gcloud iam service-accounts add-iam-policy-binding IAM_SA_NAME@IAM_SA_PROJECT_ID.iam.gserviceaccount.com \
    --project=IAM_SA_PROJECT_ID \
    --role=roles/iam.workloadIdentityUser \
    --member="serviceAccount:WORKLOAD_IDENTITY_POOL[NAMESPACE/KSA_NAME]"

Replace WORKLOAD_IDENTITY_POOL with the name of the workload identity pool.

Save the following manifest as adc-config-map.yaml. This ConfigMap contains the ADC configuration for workloads.

kind: ConfigMap
apiVersion: v1
metadata:
  namespace: K8S_NAMESPACE
  name: my-cloudsdk-config
data:
  config: |
    {
      "type": "external_account",
      "audience": "identitynamespace:WORKLOAD_IDENTITY_POOL:IDENTITY_PROVIDER",
      "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/IAM_SA_NAME@GSA_PROJECT_ID.iam.gserviceaccount.com:generateAccessToken",
      "subject_token_type": "urn:ietf:params:oauth:token-type:jwt",
      "token_url": "https://sts.googleapis.com/v1/token",
      "credential_source": {
        "file": "/var/run/secrets/tokens/gcp-ksa/token"
      }
    }

Replace the following:

IAM_SA_NAME: the name of the IAM service account to impersonate.
IAM_SA_PROJECT_ID: the project ID of the IAM service account.

Save the following manifest as workload-config.yaml:

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
  namespace:  K8S_NAMESPACE
spec:
  serviceAccountName: KSA_NAME
  containers:
  - name: my-container
    image: my-image
    command: ["sleep","infinity"]
    env:
      - name: GOOGLE_APPLICATION_CREDENTIALS
        value: /var/run/secrets/tokens/gcp-ksa/google-application-credentials.json
    volumeMounts:
    - name: gcp-ksa
      mountPath: /var/run/secrets/tokens/gcp-ksa
      readOnly: true
  volumes:
  - name: gcp-ksa
    projected:
      defaultMode: 420
      sources:
      - serviceAccountToken:
          path: token
          audience: WORKLOAD_IDENTITY_POOL
          expirationSeconds: 172800
      - configMap:
          name: my-cloudsdk-config
          optional: false
          items:
            - key: "config"
              path: "google-application-credentials.json"

When you deploy this workload, the gcp-ksa volume in the Pod contains the following data:

The data in the ConfigMap that you deployed as a file named google-application-credentials.json. This file is an ADC credential configuration file.
The Kubernetes ServiceAccount token as token. Kubernetes mounts a signed JWT for the ServiceAccount as a projected ServiceAccount token file.

The container in the Pod mounts the gcp-ksa volume to the /var/run/secrets/tokens/gcp-ksa path and configures ADC to look for the credential configuration JSON file in that path.

Deploy the workload:
```
kubectl apply -f workload-config.yaml
```

Verify the fleet Workload Identity Federation setup

In this section, you create a Cloud Storage bucket and access it from a Pod that uses fleet Workload Identity Federation. Before you perform these steps, ensure that you configured Workload Identity Federation by following the instructions in the Use fleet Workload Identity Federation in applications section.

This section doesn't show you how to verify Workload Identity Federation using the IAM service account impersonation method.

Find your numerical project number:

gcloud projects describe FLEET_PROJECT_ID \
    --format="value(projectNumber)"

The output is similar to the following:

1234567890

Create a Cloud Storage bucket:

gcloud storage buckets create gs://FLEET_PROJECT_ID-test-bucket \
    --location=LOCATION

Replace LOCATION with a Google Cloud location.

Create an IAM allow policy that grants access on the bucket to the service account that you created:
```
gcloud storage buckets add-iam-policy-binding gs://FLEET_PROJECT_ID-test-bucket \
    --condition=None \
    --role=roles/storage.objectViewer \
    --member=principal://iam.googleapis.com/projects/FLEET_PROJECT_NUMBER/locations/global/workloadIdentityPools/FLEET_PROJECT_ID.svc.id.goog/subject/ns/NAMESPACE/sa/KSA_NAME
```
Replace the following:
- FLEET_PROJECT_NUMBER: your numerical project number.
- FLEET_PROJECT_ID: your project ID.
- NAMESPACE: the name of the Kubernetes namespace that runs your Pod from the previous section.
- KSA_NAME: the name of the Kubernetes ServiceAccount that your Pod from the previous section uses.
Best practice: Create IAM allow policies that give the least amount of permissions on the specific resource that the workload needs to access.

Create a shell session in your Pod:

kubectl exec -it pods/my-pod --namespace=NAMESPACE -- /bin/bash

Try to list the objects in the bucket:

curl -X GET -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
"https://storage.googleapis.com/storage/v1/b/FLEET_PROJECT_ID-test-bucket/o"

The output is the following:

{
  "kind": "storage#objects"
}

Authenticate from your code

When you use Cloud Client Libraries, authentication libraries automatically use ADC to look for credentials to authenticate to Google Cloud services. You must use Cloud Client Libraries that support Workload Identity Federation. The following shows the minimum required Cloud Client Libraries versions, as well as instructions for how to check your current version:

C++

Most of the Google Cloud Client Libraries for C++ support identity federation by using a ChannelCredentials object, which is created by calling grpc::GoogleDefaultCredentials(). To initialize this credential, you must build the client libraries with version 1.36.0 or later of gRPC.

The Cloud Storage Client Library for C++ uses the REST API, not gRPC, so it does not support identity federation.

Go

Client libraries for Go support identity federation if they use version v0.0.0-20210218202405-ba52d332ba99 or later of the golang.org/x/oauth2 module.

To check which version of this module your client library uses, run the following commands:

cd $GOPATH/src/cloud.google.com/go
go list -m golang.org/x/oauth2

Java

Client libraries for Java support identity federation if they use version 0.24.0 or later of the com.google.auth:google-auth-library-oauth2-http artifact.

To check which version of this artifact your client library uses, run the following Maven command in your application directory:

mvn dependency:list -DincludeArtifactIds=google-auth-library-oauth2-http

Node.js

Client libraries for Node.js support identity federation if they use version 7.0.2 or later of the google-auth-library package.

To check which version of this package your client library uses, run the following command in your application directory:

npm list google-auth-library

When you create a GoogleAuth object, you can specify a project ID, or you can allow GoogleAuth to find the project ID automatically. To find the project ID automatically, the service account in the configuration file must have the Browser role (roles/browser), or a role with equivalent permissions, on your project. For details, see the README for the google-auth-library package.

Python

Client libraries for Python support identity federation if they use version 1.27.0 or later of the google-auth package.

To check which version of this package your client library uses, run the following command in the environment where the package is installed:

pip show google-auth

To specify a project ID for the authentication client, you can set the GOOGLE_CLOUD_PROJECT environment variable, or you can allow the client to find the project ID automatically. To find the project ID automatically, the service account in the configuration file must have the Browser role (roles/browser), or a role with equivalent permissions, on your project. For details, see the user guide for the google-auth package.

What's next?

Learn best practices for organizing your fleets when using fleet Workload Identity Federation.