About Backup for GKE CMEK encryption

This page describes how customer-managed encryption key (CMEK) support works in Backup for GKE.

Overview

There are two types of user data artifacts that are produced and stored by Backup for GKE:

  • Config backup: a set of Kubernetes resource descriptions extracted from the API server of the cluster undergoing backup, capturing the cluster state.
  • Volume backups: a set of volume backups that correspond to PersistentVolumeClaim resources found in the config backup.

By default, all backup artifacts produced by Backup for GKE are encrypted at rest using a Google-supplied key.

However, you may choose to have these artifacts encrypted using a customer-managed encryption key (CMEK) managed with the Cloud Key Management Service.

Enable CMEK encryption

Enabling CMEK encryption involves two steps:

  • Designate a key for encrypting backups produced for a BackupPlan.

  • Grant access by the appropriate service accounts to the appropriate keys.

For any particular backup scenario, there are potentially three CMEK keys involved:

  • bplan_key: This is the key that you reference when creating or updating the BackupPlan. When possible, this key that will be used when encrypting all backup artifacts. This key must live in the same region as the BackupPlan itself (see About resource locations).

  • orig_disk_key: If you have encrypted your persistent disk volumes using a CMEK key, then the volume backups that Backup for GKE produces for those volumes will be encrypted with this key, even if a different key is registered with the BackupPlan.

  • new_disk_key: This is the CMEK key you want to use for encrypting volumes you have restored from backup. This is referenced by the StorageClass in the restore's target cluster.

There are five different service accounts that may require access to CMEK keys:

  • agent_wi: If you are running the Preview version of the agent (GKE clusters running version 1.23 or lower), this service account must be granted access to the bplan_key. This service account will have the form: PROJECT_ID.svc.id.goog[gkebackup/agent], where PROJECT_ID is the ID of your Google Cloud project.

  • agent_robot: If your GKE clusters are running version 1.24 or higher, this service account must be granted access to the bplan_key. This service account will have the form: service-PROJECT_NUMBER@container-engine-robot.iam.gserviceaccount.com, where PROJECT_NUMBER is the number of your Google Cloud project.

  • non_cmek_service_agent: When backing up non-CMEK-encrypted volumes, this service account must be granted access to the bplan_key. This service account will have the form: service-PROJECT_NUMBER@gcp-sa-gkebackup.iam.gserviceaccount.com, where PROJECT_NUMBER is the number of your Google Cloud project.

  • cmek_service_agent: When backing up CMEK-encrypted volumes, this service account must be granted access to the orig_disk_key. This service account will have the form: service-TENANT_PROJECT_NUMBER@compute-system.iam.gserviceaccount.com, where TENANT_PROJECT_NUMBER is the number of the tenant project assigned for your BackupPlan.

  • compute_service_agent: This service account is used when creating new encrypted volumes for a cluster and must be granted access to the new_disk_key. This service account will have the form: service-PROJECT_NUMBER@compute-system.iam.gserviceaccount.com, where PROJECT_NUMBER is the number of your Google Cloud project.

If diskEncryptionKey.kmsKeyServiceAccount is set for the disks, you will need to perform the following steps before creating a backup:

  • Disable the organization policy iam.disableCrossProjectServiceAccountUsage to enable service account impersonation across projects:

      gcloud resource-manager org-policies disable-enforce \
      iam.disableCrossProjectServiceAccountUsage
      --project=PROJECT_ID

  • Grant cmek_service_agent the roles/iam.serviceAccountTokenCreator role to create short-lived credentials:

      gcloud iam service-accounts add-iam-policy-binding \
    # Replace the email with the value from
    # `diskEncryptionKey.kmsKeyServiceAccount`
    your-kms-key-service-acount@PROJECT_ID.iam.gserviceaccount.com \
    --member=service-TENANT_PROJECT_NUMBER@compute-system.iam.gserviceaccount.com \
    --role=roles/iam.serviceAccountTokenCreator

The following table summarizes which service accounts must be granted access to which keys in various scenarios:

Artifact Service account Key
config backup, 1.23- cluster agent_wi bplan_key
config backup, 1.24+ cluster agent_robot bplan_key
backup of CMEK-encrypted volume cmek_service_agent orig_disk_key
backup of google-encrypted volume non_cmek_service_agent bplan_key
new CMEK-encrypted volume created during restore compute_service_agent new_disk_key

You can choose to grant access to keys at a project level, which grants access to all keys in that project, or to the individual key.

Example: Grant project-level access

gcloud projects add-iam-policy-binding PROJECT_ID \
    --member serviceAccount:service-PROJECT_NUMBER@gcp-sa-gkebackup.iam.gserviceaccount.com \
    --role roles/cloudkms.cryptoKeyEncrypterDecrypter

Example: Grant key-level access

gcloud kms keys add-iam-policy-binding key \
    --keyring key-ring \
    --location location \
    --member "serviceAccount:PROJECT_ID.svc.id.goog[gkebackup/agent]" \
    --role roles/cloudkms.cryptoKeyEncrypterDecrypter

Usage considerations and limitations

  • If you want to back up a CMEK-encrypted volume, you must grant access to that disk's key, even if you don't enable CMEK encryption in your BackupPlan.

  • CMEK keys must be in the same region as the BackupPlan to ensure that a regional outage won't remove access to the key while backups are still accessible. However, this constraint cannot be enforced for keys shared with encrypted volumes. When encrypted volumes are involved, it is possible that a restoration may fail even when a backup is available, because the disk encryption key may not be stored in the same region as the backup.