This page shows you how to configure your applications to authenticate to Google Cloud APIs like the Compute Engine API or the AI Platform API by using fleet Workload Identity Federation.
What is fleet Workload Identity Federation?
Workload Identity Federation lets the workloads in your clusters authenticate to Google Cloud without requiring you to download, manually rotate, and generally manage credentials. Instead, workloads authenticate using short-lived tokens generated by Google Cloud.
Workload Identity Federation for GKE provides a project-wide workload identity pool from which applications running in GKE clusters get identities. Fleet Workload Identity Federation extends Workload Identity Federation for GKE to all fleet member clusters, regardless of whether the clusters are in different projects or are outside of Google Cloud. Registered clusters with Workload Identity Federation enabled on their fleet membership get identities using a fleet-wide workload identity pool, which lets you configure authentication to Google Cloud APIs and to other services across your entire fleet, even across multiple projects.
Fleet Workload Identity Federation can also be used by the Connect Agent on some cluster types to authenticate to Google Cloud as part of fleet membership, and is required to use some GKE Enterprise features that work across projects, such as Cloud Service Mesh.
Workload Identity Federation pools and identity sameness
With fleet Workload Identity Federation, each application in your fleet gets a distinct federated identity that can be used to authenticate to Google Cloud and other services that you develop. Applications get a principal identifier that IAM can recognize. This identifier uses the following syntax:
PREFIX://iam.googleapis.com/projects/FLEET_PROJECT_NUMBER/locations/global/workloadIdentityPools/FLEET_PROJECT_ID.svc.id.goog/SELECTOR
This syntax has the following fields:
PREFIX
: the IAMprincipal
orprincipalSet
depending on the resource being selected.FLEET_PROJECT_ID.svc.id.goog
: the workload identity pool for your fleet. Each fleet has a single fixed workload identity pool that's created for you.FLEET_PROJECT_NUMBER
: the project number of the fleet host project.SELECTOR
: the resource selector. For a list of supported selectors, see supported principal identifiers.
The entire fleet shares a fleet workload identity pool, so that you can give applications anywhere in the fleet, including in other projects or clouds, access to the same resources without needing to manage that access for each cluster. Like other fleet-enabled features, fleet Workload Identity Federation relies on the principle of sameness, in which Kubernetes objects that have the same name and namespace in different clusters are treated as the same thing.
For example, if you have an application with a backend deployed across multiple
clusters in the same fleet, and which needs to authenticate to a Google Cloud
API, you can configure your application so that all workloads in the backend
namespace can access that API. For more information on how fleets use sameness,
including identity sameness, see
How fleets work.
After you enable fleet Workload Identity Federation, you can reference principals in your fleet in IAM allow policies by specifying the corresponding principal identifier. For example, you could reference a specific ServiceAccount in a specific Kubernetes namespace in your allow policy. All applications that use that ServiceAccount could then access the Google Cloud resource to which the IAM allow policy applies.
Credential flow
To let applications in a specific namespace authenticate using fleet Workload Identity Federation, you do the following:
Deploy a ConfigMap in that namespace that has the following information:
- The workload identity pool and the identity provider for your cluster.
- The path in each Pod onto which Kubernetes mounts a ServiceAccount token. This token is a signed JSON Web Token (JWT).
This ConfigMap functions as the application default credentials (ADC) file for workloads.
Create an IAM allow policy that grants access on specific Google Cloud resources to the principal identifier of the principal in your clusters, like a ServiceAccount in the namespace.
Ensure that your workload in the namespace has the following configurations in the Pod specification:
- The
GOOGLE_APPLICATION_CREDENTIALS
environment variable set to the mount path of the ConfigMap in the Pod. - A projected volume that contains the ServiceAccount token and the
ConfigMap that you created, mounted in the same path that you specify in
the
GOOGLE_APPLICATION_CREDENTIALS
environment variable. - A volume mount in the container that references the projected volume.
- The
When the workload makes a Google Cloud API call, the following steps happen:
- The Google Cloud authentication libraries use application default
credentials (ADC) to find credentials. ADC checks the path that you
specified in the
GOOGLE_APPLICATION_CREDENTIALS
environment variable to look for an authentication token. - The ADC authentication library uses the data in the ConfigMap to exchange the ServiceAccount JWT that you mounted on the Pod for a short-lived federated token from Security Token Service that references the principal identifier of the workload.
- ADC includes the federated token with the API request.
- The IAM allow policy authorizes the principal identifier to perform the requested operation on the Google Cloud resource.
Supported principal identifiers for fleet Workload Identity Federation
The following table describes the selectors that you can use in IAM allow policies to reference principals in fleets:
Principal identifier type | Syntax |
---|---|
All Pods that use a specific Kubernetes ServiceAccount | Select the ServiceAccount by name:
principal://iam.googleapis.com/projects/ Replace the following:
Select the ServiceAccount by UID: principal://iam.googleapis.com/projects/ Replace the following:
|
Before you start
Ensure that you have the following command line tools installed:
- The latest version of the Google Cloud CLI, which includes
gcloud
, the command line tool for interacting with Google Cloud. kubectl
If you are using Cloud Shell as your shell environment for interacting with Google Cloud, these tools are installed for you.
- The latest version of the Google Cloud CLI, which includes
Ensure that you have initialized the gcloud CLI for use with your project.
Prepare your clusters
Before applications in your fleet can receive a federated identity, the clusters that they run on must be registered to your fleet and properly configured to use fleet Workload Identity Federation. The following sections describe how to set up fleet Workload Identity Federation for different types of clusters.
GKE
For GKE clusters, do the following:
- Enable Workload Identity Federation for GKE on your Google Kubernetes Engine cluster, if it is not already enabled.
- Register the cluster to the fleet.
You can also enable Workload Identity Federation for GKE during the cluster creation and fleet registration process.
Clusters outside Google Cloud
The following types of clusters automatically enable fleet Workload Identity Federation and are registered to your fleet during cluster creation:
- Google Distributed Cloud (software only) on VMware
- Google Distributed Cloud (software only) on bare metal
- GKE on AWS
- GKE on Azure
Attached clusters
EKS and AKS attached clusters registered using the GKE Multi-Cloud API are registered with fleet Workload Identity Federation enabled by default. Other attached clusters can be registered with fleet Workload Identity Federation enabled if they meet the necessary requirements. Follow the instructions for your cluster type in Registering a cluster.
Use fleet Workload Identity Federation in applications
The following steps show you how to configure a workload in a registered cluster to use fleet Workload Identity Federation:
Find the name of your cluster workload identity pool and identity provider:
gcloud container fleet memberships describe MEMBERSHIP_ID \ --project=FLEET_PROJECT_ID \ --format="table(authority.identityProvider,authority.workloadIdentityPool,name)"
Replace the following:
MEMBERSHIP_ID
: the cluster membership name. This is often the name of your cluster.FLEET_PROJECT_ID
: the project ID of the fleet host project.
The output is similar to the following:
IDENTITY_PROVIDER: IDENTITY_PROVIDER WORKLOAD_IDENTITY_POOL: WORKLOAD_IDENTITY_POOL NAME: projects/FLEET_PROJECT_ID/locations/MEMBERSHIP_LOCATION/memberships/MEMBERSHIP_ID
This output contains the following information:
IDENTITY_PROVIDER
: the identity provider for the cluster.MEMBERSHIP_LOCATION
: the location of the fleet membership. This is usually the same as your cluster location.WORKLOAD_IDENTITY_POOL
: the name of the workload identity pool associated with your fleet. This value has the syntaxFLEET_PROJECT_ID.svc.id.goog
.
Create a Kubernetes namespace. You can also use any existing namespace, including the
default
namespace.kubectl create namespace NAMESPACE
Replace
NAMESPACE
with the name of the namespace.Create a new Kubernetes ServiceAccount in the namespace. You can also use any existing ServiceAccount, including the
default
ServiceAccount in the namespace.kubectl create serviceaccount KSA_NAME \ --namespace=NAMESPACE
Replace
KSA_NAME
with the name of the ServiceAccount.Save the following manifest as
adc-config-map.yaml
. This ConfigMap contains the ADC configuration for workloads.kind: ConfigMap apiVersion: v1 metadata: namespace: NAMESPACE name: my-cloudsdk-config data: config: | { "type": "external_account", "audience": "identitynamespace:WORKLOAD_IDENTITY_POOL:IDENTITY_PROVIDER", "subject_token_type": "urn:ietf:params:oauth:token-type:jwt", "token_url": "https://sts.googleapis.com/v1/token", "credential_source": { "file": "/var/run/secrets/tokens/gcp-ksa/token" } }
Deploy the ConfigMap:
kubectl create -f adc-config-map.yaml
Save the following manifest as
workload-config.yaml
:apiVersion: v1 kind: Pod metadata: name: my-pod namespace: NAMESPACE spec: serviceAccountName: KSA_NAME containers: - name: sample-container image: google/cloud-sdk:slim command: ["sleep","infinity"] env: - name: GOOGLE_APPLICATION_CREDENTIALS value: /var/run/secrets/tokens/gcp-ksa/google-application-credentials.json volumeMounts: - name: gcp-ksa mountPath: /var/run/secrets/tokens/gcp-ksa readOnly: true volumes: - name: gcp-ksa projected: defaultMode: 420 sources: - serviceAccountToken: path: token audience: WORKLOAD_IDENTITY_POOL expirationSeconds: 172800 - configMap: name: my-cloudsdk-config optional: false items: - key: "config" path: "google-application-credentials.json"
When you deploy this workload, the
gcp-ksa
volume in the Pod contains the following data:- The data in the ConfigMap that you deployed as a file named
google-application-credentials.json
. This file is an ADC credential configuration file. - The Kubernetes ServiceAccount token as
token
. Kubernetes mounts a signed JWT for the ServiceAccount as a projected ServiceAccount token file.
The container in the Pod mounts the
gcp-ksa
volume to the/var/run/secrets/tokens/gcp-ksa
path and configures ADC to look for the credential configuration JSON file in that path.- The data in the ConfigMap that you deployed as a file named
Deploy the workload:
kubectl apply -f workload-config.yaml
Alternative: Impersonate an IAM service account
Alternatively, you can configure Kubernetes ServiceAccounts in your clusters to impersonate IAM service accounts and perform any authorized actions that the IAM service accounts can perform. This approach might increase maintenance overhead, because you have to manage pairings of service accounts in both IAM and Kubernetes.
In most scenarios, we recommend that you directly reference Kubernetes principals in IAM allow policies to grant access to Google Cloud resources by following the instructions in Use fleet Workload Identity Federation in applications.
Find the name of your cluster workload identity pool and identity provider:
gcloud container fleet memberships describe MEMBERSHIP_ID \ --project=FLEET_PROJECT_ID \ --format="table(authority.identityProvider,authority.workloadIdentityPool,name)"
Replace the following:
MEMBERSHIP_ID
: the cluster membership name. This is often the name of your cluster.FLEET_PROJECT_ID
: the project ID of the fleet host project.
The output is similar to the following:
IDENTITY_PROVIDER: IDENTITY_PROVIDER WORKLOAD_IDENTITY_POOL: WORKLOAD_IDENTITY_POOL NAME: projects/FLEET_PROJECT_ID/locations/MEMBERSHIP_LOCATION/memberships/MEMBERSHIP_ID
This output contains the following information:
IDENTITY_PROVIDER
: the identity provider for the cluster.MEMBERSHIP_LOCATION
: the location of the membership. This is usually the same as your cluster location.WORKLOAD_IDENTITY_POOL
: the name of the workload identity pool associated with your fleet. This value has the syntaxFLEET_PROJECT_ID.svc.id.goog
.
Create an IAM service account that your application can impersonate. You can also use any existing IAM service account.
gcloud iam service-accounts create IAM_SA_NAME \ --project=IAM_SA_PROJECT_ID
Replace the following:
IAM_SA_NAME
: the name for your IAM service account.IAM_SA_PROJECT_ID
: the project ID of the project that contains your IAM service account. This can be different than your fleet host project.
Grant the IAM service account any permissions that it needs to access Google Cloud APIs by adding the necessary IAM allow policies. You can do this by using
gcloud iam service-accounts add-iam-policy-binding
or another method. You can find out which permissions are required to use Google Cloud APIs in each service's documentation, and see a complete list of predefined roles with the necessary permissions in Understanding roles.Create a Kubernetes ServiceAccount in a namespace. You can also use an existing Kubernetes ServiceAccount and any namespace, including the
default
ServiceAccount and thedefault
namespace.kubectl create serviceaccount KSA_NAME \ --namespace=NAMESPACE
Replace the following:
KSA_NAME
: the name of the ServiceAccount.NAMESPACE
: the name of the namespace.
Create an IAM allow policy that lets a Kubernetes ServiceAccount in a specific namespace in your cluster impersonate the IAM service account:
gcloud iam service-accounts add-iam-policy-binding IAM_SA_NAME@IAM_SA_PROJECT_ID.iam.gserviceaccount.com \ --project=IAM_SA_PROJECT_ID \ --role=roles/iam.workloadIdentityUser \ --member="serviceAccount:WORKLOAD_IDENTITY_POOL[NAMESPACE/KSA_NAME]"
Replace
WORKLOAD_IDENTITY_POOL
with the name of the workload identity pool.Save the following manifest as
adc-config-map.yaml
. This ConfigMap contains the ADC configuration for workloads.kind: ConfigMap apiVersion: v1 metadata: namespace: K8S_NAMESPACE name: my-cloudsdk-config data: config: | { "type": "external_account", "audience": "identitynamespace:WORKLOAD_IDENTITY_POOL:IDENTITY_PROVIDER", "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/IAM_SA_NAME@GSA_PROJECT_ID.iam.gserviceaccount.com:generateAccessToken", "subject_token_type": "urn:ietf:params:oauth:token-type:jwt", "token_url": "https://sts.googleapis.com/v1/token", "credential_source": { "file": "/var/run/secrets/tokens/gcp-ksa/token" } }
Replace the following:
IAM_SA_NAME
: the name of the IAM service account to impersonate.IAM_SA_PROJECT_ID
: the project ID of the IAM service account.
Save the following manifest as
workload-config.yaml
:apiVersion: v1 kind: Pod metadata: name: my-pod namespace: K8S_NAMESPACE spec: serviceAccountName: KSA_NAME containers: - name: my-container image: my-image command: ["sleep","infinity"] env: - name: GOOGLE_APPLICATION_CREDENTIALS value: /var/run/secrets/tokens/gcp-ksa/google-application-credentials.json volumeMounts: - name: gcp-ksa mountPath: /var/run/secrets/tokens/gcp-ksa readOnly: true volumes: - name: gcp-ksa projected: defaultMode: 420 sources: - serviceAccountToken: path: token audience: WORKLOAD_IDENTITY_POOL expirationSeconds: 172800 - configMap: name: my-cloudsdk-config optional: false items: - key: "config" path: "google-application-credentials.json"
When you deploy this workload, the
gcp-ksa
volume in the Pod contains the following data:- The data in the ConfigMap that you deployed as a file named
google-application-credentials.json
. This file is an ADC credential configuration file. - The Kubernetes ServiceAccount token as
token
. Kubernetes mounts a signed JWT for the ServiceAccount as a projected ServiceAccount token file.
The container in the Pod mounts the
gcp-ksa
volume to the/var/run/secrets/tokens/gcp-ksa
path and configures ADC to look for the credential configuration JSON file in that path.- The data in the ConfigMap that you deployed as a file named
Deploy the workload:
kubectl apply -f workload-config.yaml
Verify the fleet Workload Identity Federation setup
In this section, you create a Cloud Storage bucket and access it from a Pod that uses fleet Workload Identity Federation. Before you perform these steps, ensure that you configured Workload Identity Federation by following the instructions in the Use fleet Workload Identity Federation in applications section.
This section doesn't show you how to verify Workload Identity Federation using the IAM service account impersonation method.
Find your numerical project number:
gcloud projects describe FLEET_PROJECT_ID \ --format="value(projectNumber)"
The output is similar to the following:
1234567890
Create a Cloud Storage bucket:
gcloud storage buckets create gs://FLEET_PROJECT_ID-test-bucket \ --location=LOCATION
Replace
LOCATION
with a Google Cloud location.Create an IAM allow policy that grants access on the bucket to the service account that you created:
gcloud storage buckets add-iam-policy-binding gs://FLEET_PROJECT_ID-test-bucket \ --condition=None \ --role=roles/storage.objectViewer \ --member=principal://iam.googleapis.com/projects/FLEET_PROJECT_NUMBER/locations/global/workloadIdentityPools/FLEET_PROJECT_ID.svc.id.goog/subject/ns/NAMESPACE/sa/KSA_NAME
Replace the following:
FLEET_PROJECT_NUMBER
: your numerical project number.FLEET_PROJECT_ID
: your project ID.NAMESPACE
: the name of the Kubernetes namespace that runs your Pod from the previous section.KSA_NAME
: the name of the Kubernetes ServiceAccount that your Pod from the previous section uses.
Create a shell session in your Pod:
kubectl exec -it pods/my-pod --namespace=NAMESPACE -- /bin/bash
Try to list the objects in the bucket:
curl -X GET -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \ "https://storage.googleapis.com/storage/v1/b/FLEET_PROJECT_ID-test-bucket/o"
The output is the following:
{ "kind": "storage#objects" }
Authenticate from your code
When you use Cloud Client Libraries, authentication libraries automatically use ADC to look for credentials to authenticate to Google Cloud services. You must use Cloud Client Libraries that support Workload Identity Federation. The following shows the minimum required Cloud Client Libraries versions, as well as instructions for how to check your current version:
C++
Most of the
Google Cloud Client Libraries for C++
support identity federation by using a ChannelCredentials
object, which is
created by calling grpc::GoogleDefaultCredentials()
. To initialize this
credential, you must build the client libraries with version 1.36.0 or later of
gRPC.
The Cloud Storage Client Library for C++ uses the REST API, not gRPC, so it does not support identity federation.
Go
Client libraries for Go support identity federation if they use version
v0.0.0-20210218202405-ba52d332ba99 or later of the golang.org/x/oauth2
module.
To check which version of this module your client library uses, run the following commands:
cd $GOPATH/src/cloud.google.com/go
go list -m golang.org/x/oauth2
Java
Client libraries for Java support identity federation if they use version 0.24.0
or later of the
com.google.auth:google-auth-library-oauth2-http
artifact.
To check which version of this artifact your client library uses, run the following Maven command in your application directory:
mvn dependency:list -DincludeArtifactIds=google-auth-library-oauth2-http
Node.js
Client libraries for Node.js support identity federation if they use version
7.0.2 or later of the
google-auth-library
package.
To check which version of this package your client library uses, run the following command in your application directory:
npm list google-auth-library
When you create a GoogleAuth
object, you can specify a project ID, or you can
allow GoogleAuth
to find the project ID automatically. To find the project ID
automatically, the service account in the configuration file must have the
Browser role (roles/browser
), or a role with equivalent permissions, on your
project. For details, see the
README
for the google-auth-library
package.
Python
Client libraries for Python support identity federation if they use version
1.27.0 or later of the
google-auth
package.
To check which version of this package your client library uses, run the following command in the environment where the package is installed:
pip show google-auth
To specify a project ID for the authentication client, you can set the
GOOGLE_CLOUD_PROJECT
environment variable, or you can allow the client to find
the project ID automatically. To find the project ID automatically, the service
account in the configuration file must have the Browser role (roles/browser
),
or a role with equivalent permissions, on your project. For details, see the
user guide for the google-auth
package.
What's next?
Learn best practices for organizing your fleets when using fleet Workload Identity Federation.