Use Secret Manager add-on with Google Kubernetes Engine

Preview

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

The integration between Secret Manager and Google Kubernetes Engine (GKE) lets you store sensitive data such as passwords and certificates used by GKE clusters as secrets in Secret Manager.

This page explains how you can use the Secret Manager add-on to access the secrets stored in Secret Manager as volumes mounted in Kubernetes Pods.

This process involves the following steps:

1. Install the Secret Manager add-on on a new or existing GKE cluster.
2. Configure applications to authenticate to the Secret Manager API.
3. Define which secrets to mount onto Kubernetes Pods using a SecretProviderClass YAML file.
4. Create a volume where the secrets will be mounted. After the volume is attached, applications in the container can access the data in the container file system.

The Secret Manager add-on is derived from the open source Kubernetes Secrets Store CSI Driver and the Google Secret Manager provider. If you're using the open source Secrets Store CSI Driver to access secrets, you can migrate to the Secret Manager add-on. For information, see Migrate from the existing Secrets Store CSI Driver.

Important: This document uses the term secret to refer to sensitive data stored in Secret Manager, and not to the Secret object in the GKE API.

Benefits

The Secret Manager add-on provides the following benefits:

• You can use a fully managed and supported solution to access Secret Manager secrets from within GKE without any operational overhead.
• You don't have to write custom code to access secrets stored in Secret Manager.
• You can store and manage all your secrets centrally in Secret Manager and selectively access secrets from GKE pods using the Secret Manager add-on. By doing this, you can use features offered by Secret Manager such as CMEK encryption, fine-grained access control, managed rotation, lifecycle management, and audit logs, along with using Kubernetes features such as passing secrets to containers in the form of mounted volumes.
• The Secret Manager add-on is supported on both Standard clusters and Autopilot clusters.
• The Secret Manager add-on supports linux/arm64 and linux/amd64 deployments.

Limitations

This Preview release of the Secret Manager add-on doesn't support the following features that are available in the open source Secrets Store CSI Driver:

• Secret auto rotation
• Sync as Kubernetes Secret

Before you begin

• Enable the Secret Manager and Google Kubernetes Engine APIs.

  Enable the APIs

• If you want to use the Google Cloud CLI for this task, install and then initialize the gcloud CLI. If you previously installed the gcloud CLI, get the latest version by running the gcloud components update command.

  Note: For existing gcloud CLI installations, make sure to set the compute/region and compute/zone properties. By setting default locations, you can avoid errors in gcloud CLI like the following: One of [--zone, --region, --location] must be supplied: Please specify location.

• Ensure that your cluster runs GKE version 1.29 or later with a Linux node image. The Secret Manager add-on doesn't support Windows Server nodes.

Install the Secret Manager add-on

You can install the Secret Manager add-on on both Standard clusters as well as Autopilot clusters. Ensure that Workload Identity Federation for GKE is enabled on the Standard cluster. Workload Identity Federation for GKE is enabled by default on an Autopilot cluster. Kubernetes Pods use workload identity to authenticate to the Secret Manager API.

Install the Secret Manager add-on on a new GKE cluster

To install the Secret Manager add-on on cluster creation, follow these steps:

Standard cluster

• To enable the Secret Manager add-on on a new Standard cluster, run the following command:

  gcloud beta container clusters create CLUSTER_NAME \
      --enable-secret-manager \
      --location=LOCATION \
      --cluster-version=VERSION \
      --workload-pool=PROJECT_ID.svc.id.goog
  

  Replace the following:

  • CLUSTER_NAME: the name of your cluster.
  • LOCATION: the Compute Engine region or zone for the cluster.
  • VERSION: the specific GKE version that you want to use. Ensure that your cluster runs GKE version 1.29 or later. If the default release channel doesn't include this version, use the --release-channel flag to choose a release channel that does.
  • PROJECT_ID: the ID of your Google Cloud project.

Autopilot cluster

• To enable the Secret Manager add-on on a new Autopilot cluster, run the following command:

  gcloud beta container clusters create-auto CLUSTER_NAME \
      --enable-secret-manager \
      --cluster-version=VERSION \
      --location=LOCATION
  

  Replace the following:

  • CLUSTER_NAME: the name of your cluster
  • VERSION: the specific GKE version that you want to use. Ensure that your cluster runs GKE version 1.29 or later. If the default release channel doesn't include this version, use the --release-channel flag to choose a release channel that does.
  • LOCATION: the region for your cluster, such as us-central1.

After you have enabled the Secret Manager add-on, you can use the Secrets Store CSI Driver in Kubernetes volumes using the driver and provisioner name: secrets-store-gke.csi.k8s.io.

Install the Secret Manager add-on on an existing GKE cluster

To enable the Secret Manager add-on on an existing cluster, run the following command:

  gcloud beta container clusters update CLUSTER_NAME \
      --enable-secret-manager \
      --location=LOCATION

Replace the following:

• CLUSTER_NAME: the name of your existing cluster
• LOCATION: the region for your cluster, such as us-central1.

Configure applications to authenticate to the Secret Manager API

The Google Secret Manager provider uses the workload identity of the Pod that a secret is mounted onto when authenticating to the Secret Manager API. To allow your applications to authenticate to the Secret Manager API using Workload Identity Federation for GKE, follow these steps:

• Use an Identity and Access Management (IAM) policy to bind an IAM service account to a Kubernetes ServiceAccount. You can create a new Kubernetes ServiceAccount or use an existing Kubernetes ServiceAccount in any namespace, including the default Kubernetes ServiceAccount.
• Use IAM bindings to grant the IAM service account access to secrets in Secret Manager.

Pods that use the configured Kubernetes ServiceAccount automatically authenticate as the IAM service account when accessing the Secret Manager API.

Bind the Kubernetes ServiceAccount to the IAM service account

Allow the Kubernetes ServiceAccount to impersonate the IAM service account by adding an IAM policy binding between the two service accounts.

Use a new Kubernetes ServiceAccount

1. Save the following manifest as service-account.yaml:

   apiVersion: v1
   kind: ServiceAccount
   metadata:
     name: KSA_NAME
     namespace: NAMESPACE
     annotations:
       iam.gke.io/gcp-service-account: GSA_NAME@PROJECT_ID.iam.gserviceaccount.com
   

   Replace the following:

   • KSA_NAME: the name of your new Kubernetes ServiceAccount
   • NAMESPACE: the name of the Kubernetes namespace for the ServiceAccount
   • GSA_NAME: the name of the IAM service account
   • PROJECT_ID: the project ID of the Google Cloud project for your IAM service account

2. Apply the manifest:

   kubectl apply -f service-account.yaml
   

3. To bind the IAM service account to a new Kubernetes ServiceAccount, run the following command:

   gcloud iam service-accounts add-iam-policy-binding \
       --role roles/iam.workloadIdentityUser \
       --member "serviceAccount:PROJECT_ID.svc.id.goog[NAMESPACE/KSA_NAME]" \
       GSA_NAME@PROJECT_ID.iam.gserviceaccount.com
   

Use an existing Kubernetes ServiceAccount

To bind the IAM service account to an existing Kubernetes ServiceAccount, run the following command:

gcloud iam service-accounts add-iam-policy-binding \
    --role roles/iam.workloadIdentityUser \
    --member "serviceAccount:PROJECT_ID.svc.id.goog[NAMESPACE/KSA_NAME]" \
    GSA_NAME@PROJECT_ID.iam.gserviceaccount.com

Replace the following:

• KSA_NAME: the name of your existing Kubernetes ServiceAccount
• NAMESPACE: the name of the Kubernetes namespace for the ServiceAccount
• GSA_NAME: the name of the IAM service account
• PROJECT_ID: the project ID of the Google Cloud project for your IAM service account

Grant the IAM service account permission to access the secret

To grant the service account permission to access the secret, run the following command:

gcloud secrets add-iam-policy-binding SECRET_NAME \
    --member=serviceAccount:GSA_NAME@PROJECT_ID.iam.gserviceaccount.com \
    --role=roles/secretmanager.secretAccessor

Replace the following:

• SECRET_NAME: the name of the secret in Secret Manager
• GSA_NAME: the name of the IAM service account
• PROJECT_ID: the project ID of the Google Cloud project for your IAM service account

Define which secrets to mount

To specify which secrets to mount as files in the Kubernetes Pod, create a SecretProviderClass YAML manifest and list the secrets to mount and the filename to mount them as. Follow these steps:

1. Save the following manifest as app-secrets.yaml:

   apiVersion: secrets-store.csi.x-k8s.io/v1
   kind: SecretProviderClass
   metadata:
     name: SECRET_PROVIDER_CLASS_NAME
   spec:
     provider: gke
     parameters:
       secrets: |
         - resourceName: "projects/PROJECT_ID/secrets/SECRET_NAME/versions/SECRET_VERSION"
           path: "FILENAME.txt"
   

   Replace the following:

   • SECRET_PROVIDER_CLASS_NAME: the name for your SecretProviderClass object
   • PROJECT_ID: your project ID
   • SECRET_NAME: the secret name
   • SECRET_VERSION: the secret version
   • FILENAME.txt: the filename where the secret value will be mounted. You can create multiple files using the resourceName and path variables.

2. Apply the manifest:

   kubectl apply -f app-secrets.yaml
   

3. Verify that the SecretProviderClass object is created:

   kubectl get SecretProviderClasses
   

Configure a volume where the secrets will be mounted

1. Save the following configuration as my-pod.yaml:

   apiVersion: v1
   kind: Pod
   metadata:
     name: POD_NAME
     namespace: default
   spec:
     serviceAccountName: KSA_NAME
     containers:
     - image: IMAGE_NAME
       imagePullPolicy: IfNotPresent
       name: POD_NAME
       resources:
         requests:
           cpu: 100m
       stdin: true
       stdinOnce: true
       terminationMessagePath: /dev/termination-log
       terminationMessagePolicy: File
       tty: true
       volumeMounts:
         - mountPath: "/var/secrets"
           name: mysecret
     volumes:
     - name: mysecret
       csi:
         driver: secrets-store-gke.csi.k8s.io
         readOnly: true
         volumeAttributes:
           secretProviderClass: SECRET_PROVIDER_CLASS_NAME
   

   Replace the following:

   • POD_NAME: the name of the Kubernetes Pod where the secret is mounted
   • KSA_NAME: the Kubernetes ServiceAccount that you set up in the step Set up the workload identity service account
   • IMAGE_NAME: name of the container image
   • SECRET_PROVIDER_CLASS_NAME: the name for your SecretProviderClass object

2. Apply the configuration to your cluster.

   kubectl apply -f my-pod.yaml
   

This step mounts a volume mysecret at /var/secrets using the CSI driver (secrets-store-gke.csi.k8s.io). This volume references the SecretProviderClass object which acts as the provider.

Migrate from the existing Secrets Store CSI Driver

If you're migrating to the Secret Manager add-on from your existing installation of the Secrets Store CSI Driver, update your Pod manifest as follows:

1. Update the name of your SecretProviderClass and the provider as described in the following manifest:

   apiVersion: secrets-store.csi.x-k8s.io/v1
   kind: SecretProviderClass
   metadata:
     name: app-secrets-gke
   spec:
     provider: gke
     parameters:
       secrets: |
         - resourceName: "projects/<project_id>/secrets/<secret_name>/versions/<secret_version>"
           path: "good1.txt"
   

2. Update the driver and the secretProviderClass for your Kubernetes volume as described in the following manifest:

   volumes:
     - name: mysecret
       csi:
         driver: secrets-store-gke.csi.k8s.io
         readOnly: true
         volumeAttributes:
           secretProviderClass: "app-secrets-gke"
   

Disable the Secret Manager add-on

To disable the Secret Manager add-on on an existing Standard cluster or on an Autopilot cluster, run the following command:

  gcloud beta container clusters update CLUSTER_NAME \
      --no-enable-secret-manager \
      --region=REGION

Replace the following:

• CLUSTER_NAME: the name of your cluster
• REGION: the region of your cluster, such as us-central1