Enabling Customer-Managed Encryption Keys (CMEK)

This topic discusses support for Customer-Managed Encryption Keys (CMEK) in Secret Manager.

Overview

Secret Manager provides tools for storing, managing, and accessing sensitive data in your applications.

By default, secrets stored in Secret Manager are encrypted with Google-default encryption. With Google-default encryption, secret payloads are encrypted by keys managed by Google before it is written to persistent storage, with no configuration required. Google-default encryption is the best choice for many organizations.

For organizations that want greater control, CMEK support for Secret Manager allows you to configure the Cloud KMS key that protects data at rest in Secret Manager. If you temporarily disable or permanently destroy the CMEK key, the data encrypted with that key cannot be decrypted and therefore is unaccessible.

How CMEK works in Secret Manager

Before writing a secret version to persistent storage in a particular location, Secret Manager encrypts the data with a unique data encryption key (DEK). This DEK is then encrypted with a replica-specific key, called a key encryption key (KEK) that is owned by the Secret Manager service.

When using CMEK for Secret Manager, the KEK is called a CMEK key and is a symmetric key you manage within Cloud KMS. The CMEK key must be in the same GCP location as the secret version replica it encrypts.

This guide walks through how to configure Secret Manager to use CMEK. For more information about CMEK in general, including when and why to enable it, see the Cloud Key Management Service documentation.

Limitations

CMEK is available only in the Secret Manager v1 API and gCloud.

Before you begin

You may choose to store all resources in the same project or to store secrets and keys in separate projects. Read Cloud KMS Separation of duties to better understand this decision.

Complete the following prerequisites to set up Secret Manager and Cloud KMS:

  • Secret Manager:

    • Create or use an existing project to hold your Secret Manager resources.
    • If necessary, complete the steps in the Configuring Secret Manager section of the Secret Manager quickstart.
  • Cloud KMS:

Set the following variables to the project IDs of your Secret Manager and Cloud KMS projects.

Command-Line

$ export SM_PROJECT_ID="..."
$ export KMS_PROJECT_ID="..."

Authenticate to Google Cloud:

gcloud

$ gcloud auth login

Creating a service account

You need to create a service account for each project that requires customer-managed encryption keys. Currently, you can only use gcloud command-line tool commands to create the type of service account you need for customer-managed encryption keys.

To create a service account with gcloud command-line tool, run the following command:

gcloud

$ gcloud beta services identity create \
    --service "secretmanager.googleapis.com" \
    --project "${SM_PROJECT_ID}"

The previous command returns a service account name, using the following format:

service-[PROJECT_NUMBER]@gcp-sa-secretmanager.iam.gserviceaccount.com

You will grant this service account access to the CMEK Cloud KMS keys used to encrypt and decrypt your secrets.

Save the service account name as an environment variable:

Command-Line

# This is from the output of the command above
$ export SM_SERVICE_ACCOUNT="service-...."

The environment variables for the Secret Manager project, Cloud KMS project, and Secret Manager service account must be set the entire time you are following this procedure.

CMEK with Automatic Replication

This section covers secrets that are configured via an automatic replication policy.

For secrets that use the automatic replication policy, your CMEK key must be located in the global Cloud KMS multi-region.

Create a symmetric Cloud KMS key in the global Cloud KMS region, or use an existing key. This example creates a new key ring called secret-manager-cmek, then creates a new key called my-cmek-key on it.

gcloud

$ gcloud kms keyrings create "secret-manager-cmek" \
    --project "${KMS_PROJECT_ID}" \
    --location "global"

$ gcloud kms keys create "my-cmek-key" \
    --project "${KMS_PROJECT_ID}" \
    --location "global" \
    --keyring "secret-manager-cmek" \
    --purpose "encryption"

Get the key's resource name and set it as an environment variable.

Command-Line

$ export KMS_KEY_NAME="projects/${KMS_PROJECT_ID}/locations/global/keyRings/secret-manager-cmek/cryptoKeys/my-cmek-key"

Grant the service account for Secret Manager access to encrypt and decrypt using the CMEK key. This command grants the Cloud KMS Encrypter / Decrypter role (roles/cloudkms.cryptoKeyEncrypterDecrypter) on the my-cmek-key Cloud KMS key to the service account.

gcloud

$ gcloud kms keys add-iam-policy-binding "my-cmek-key" \
    --project "${KMS_PROJECT_ID}" \
    --location "global" \
    --keyring "secret-manager-cmek" \
    --member "serviceAccount:${SM_SERVICE_ACCOUNT}" \
    --role "roles/cloudkms.cryptoKeyEncrypterDecrypter"

Create a secret with automatic replication. The resource name of the CMEK key is stored as metadata on the secret.

gcloud

$ gcloud beta secrets create my-secret \
    --replication-policy "automatic" \
    --kms-key-name "${KMS_KEY_NAME}" \
    --project "${SM_PROJECT_ID}"

API

Set the value of replication.automatic.customerManagedEncryption.kmsKeyName to the resource name for the CMEK key.

$ curl "https://secretmanager.googleapis.com/v1/projects/${SM_PROJECT_ID}/secrets?secretId=my-secret" \
    --request "POST" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $(gcloud auth print-access-token)" \
    --data-binary @- <<EOF
{
  "replication":{
    "automatic":{
      "customerManagedEncryption":{
        "kmsKeyName": "${KMS_KEY_NAME}"
      }
    }
  }
}
EOF

Now, each time a secret version is created in that secret, the secret version's payload is automatically encrypted using the key before being written to persistent storage, as long as the service account has access to the CMEK key. If the service account loses access or if the key becomes unavailable, an attempt to create a new secret version or access an existing one returns an error.

Add a new secret version. Notice that you don't specify the Cloud KMS key's resource name; it is read from the secret's metadata.

gcloud

$ echo -n "abcd1234" | gcloud beta secrets versions add "my-secret" \
    --project "${SM_PROJECT_ID}" \
    --data-file -

The secret version is created, even if the caller doesn't have direct access to use the CMEK key. The service account for Secret Manager, rather than the caller, is responsible for encrypting and decrypting secrets when reading or writing them.

Similarly, you don't need direct access to the CMEK key in order to access the secret. The service account accesses the key and encrypts or decrypts the secret on your behalf.

Access the secret version you just created:

gcloud

$ gcloud beta secrets versions access latest \
    --project "${SM_PROJECT_ID}" \
    --secret "my-secret"

Updating CMEK Configuration

Create a new symmetric KMS keys in the global Cloud KMS multi-region.

gcloud

$ gcloud kms keys create "my-other-key" \
    --project "${KMS_PROJECT_ID}" \
    --location "global" \
    --keyring "secret-manager-cmek" \
    --purpose "encryption"

Grant the service account for Secret Manager access to encrypt and decrypt using the new CMEK key. This command grants the Cloud KMS Encrypter / Decrypter role (roles/cloudkms.cryptoKeyEncrypterDecrypter) on the my-other-key Cloud KMS key to the service account.

gcloud

$ gcloud kms keys add-iam-policy-binding "my-other-key" \
    --project "${KMS_PROJECT_ID}" \
    --location "global" \
    --keyring "secret-manager-cmek" \
    --member "serviceAccount:${SM_SERVICE_ACCOUNT}" \
    --role "roles/cloudkms.cryptoKeyEncrypterDecrypter"

Modify the CMEK configuration on a secret by updating the replication on the secret with the new Cloud KMS key resource names.

gcloud

$ gcloud beta secrets replication update my-secret \
    --set-kms-key="projects/${KMS_PROJECT_ID}/locations/global/keyRings/secret-manager-cmek/cryptoKeys/my-other-key" \
    --project "${SM_PROJECT_ID}"

API

$ curl "https://secretmanager.googleapis.com/v1/projects/${SM_PROJECT_ID}/secrets/my-secret?updateMask=replication" \
    --request "PATCH" \
    --header "Authorization: Bearer $(gcloud auth print-access-token)" \
    --header "Content-Type: application/json" \
    --data-binary @- <<EOF
{
  "replication": {
    "automatic":{
      "customerManagedEncryption":{
        "kmsKeyName": "projects/${KMS_PROJECT_ID}/locations/global/keyRings/secret-manager-cmek/cryptoKeys/my-other-key"
      }
    }
  }
}
EOF

CMEK with User Managed Replication

This section covers secrets that are configured with a user managed replication policy. With a user managed replication policy, you control the GCP location where the secret is stored. Secrets are always accessible from every GCP location.

Secrets with a user managed replication policy must use Cloud KMS keys that map exactly to the locations in which the secret versions are stored. This guide stores a secret in two separate locations: us-east1, us-central1. Requests to access the secret are routed to one of these locations.

In each of the two regions, create a key ring and a Cloud KMS key with the purpose of encryption, or use an existing key. This example creates a new key ring called secret-manager-cmek, then creates a key called my-cmek-key in each region.

gcloud

# us-east1
$ gcloud kms keyrings create "secret-manager-cmek" \
    --project "${KMS_PROJECT_ID}" \
    --location "us-east1"

$ gcloud kms keys create "my-cmek-key" \
    --project "${KMS_PROJECT_ID}" \
    --location "us-east1" \
    --keyring "secret-manager-cmek" \
    --purpose "encryption"

# us-central1
$ gcloud kms keyrings create "secret-manager-cmek" \
    --project "${KMS_PROJECT_ID}" \
    --location "us-central1"

$ gcloud kms keys create "my-cmek-key" \
    --project "${KMS_PROJECT_ID}" \
    --location "us-central1" \
    --keyring "secret-manager-cmek" \
    --purpose "encryption"

Grant the service account for Secret Manager permission to encrypt and decrypt using the CMEK key by granting the Cloud KMS Encrypter / Decrypter role (roles/cloudkms.cryptoKeyEncrypterDecrypter) for each of the CMEK keys individually or for all keys in the project.

gcloud

# us-east1
$ gcloud kms keys add-iam-policy-binding "my-cmek-key" \
    --project "${KMS_PROJECT_ID}" \
    --location "us-east1" \
    --keyring "secret-manager-cmek" \
    --member "serviceAccount:${SM_SERVICE_ACCOUNT}" \
    --role "roles/cloudkms.cryptoKeyEncrypterDecrypter"

# us-central1
$ gcloud kms keys add-iam-policy-binding "my-cmek-key" \
    --project "${KMS_PROJECT_ID}" \
    --location "us-central1" \
    --keyring "secret-manager-cmek" \
    --member "serviceAccount:${SM_SERVICE_ACCOUNT}" \
    --role "roles/cloudkms.cryptoKeyEncrypterDecrypter"

Create a CMEK enabled secret with user managed replication. The resource name of the CMEK key is stored as metadata on the secret.

gcloud

$ cat <<EOF > replication-policy.json
{
  "userManaged":{
    "replicas":[
      {
        "location":"us-east1",
        "customerManagedEncryption":{
          "kmsKeyName":"projects/${KMS_PROJECT_ID}/locations/us-east1/keyRings/secret-manager-cmek/cryptoKeys/my-cmek-key"
        }
      },
      {
        "location":"us-central1",
        "customerManagedEncryption":{
          "kmsKeyName":"projects/${KMS_PROJECT_ID}/locations/us-central1/keyRings/secret-manager-cmek/cryptoKeys/my-cmek-key"
        }
      }
    ]
  }
}
EOF

$ gcloud beta secrets create my-ummr-secret \
    --replication-policy-file replication-policy.json \
    --project "${SM_PROJECT_ID}"

API

Set the value of replication.userManaged.replicas.customerManagedEncryption.kmsKeyNameto the resource names for the CMEK keys.

$ curl "https://secretmanager.googleapis.com/v1/projects/${SM_PROJECT_ID}/secrets?secretId=my-ummr-secret" \
--request "POST" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $(gcloud auth print-access-token)" \
--data-binary @- <<EOF
{
  "replication":{
    "userManaged":{
      "replicas":[
        {
          "location":"us-east1",
          "customerManagedEncryption":{
            "kmsKeyName":"projects/${KMS_PROJECT_ID}/locations/us-east1/keyRings/secret-manager-cmek/cryptoKeys/my-cmek-key"
          }
        },
        {
          "location":"us-central1",
          "customerManagedEncryption":{
            "kmsKeyName":"projects/${KMS_PROJECT_ID}/locations/us-central1/keyRings/secret-manager-cmek/cryptoKeys/my-cmek-key"
          }
        }
      ]
    }
  }
}
EOF

Now, each time a secret version is created in that secret, the secret version's payload is automatically encrypted using the key before being written to persistent storage, as long as the service account has access to the CMEK key. If the service account loses access or if the key becomes unavailable, an attempt to create a new secret version or access an existing one returns an error.

Add a new secret version. Notice that you don't specify the Cloud KMS key's resource name; it is read from the secret's metadata.

gcloud

$ echo -n "abcd1234" | gcloud beta secrets versions add "my-ummr-secret" \
    --project "${SM_PROJECT_ID}" \
    --data-file -

The secret version is created, even if the caller doesn't have direct access to use the CMEK key. The service account for Secret Manager, rather than the caller, is responsible for encrypting and decrypting secrets when reading or writing them.

Similarly, you don't need direct access to the CMEK key in order to access the secret. The service account accesses the key and encrypts or decrypts the secret on your behalf.

Access the secret version you just created.

gcloud

$ gcloud beta secrets versions access latest \
    --project "${SM_PROJECT_ID}" \
    --secret "my-ummr-secret"

Updating CMEK Configuration

Create two new symmetric KMS keys in the same regions as the secret.

gcloud

# us-east1
$ gcloud kms keys create "my-other-key" \
    --project "${KMS_PROJECT_ID}" \
    --location "us-east1" \
    --keyring "secret-manager-cmek" \
    --purpose "encryption"

# us-central1
$ gcloud kms keys create "my-other-key" \
    --project "${KMS_PROJECT_ID}" \
    --location "us-central1" \
    --keyring "secret-manager-cmek" \
    --purpose "encryption"

Grant the service account for Secret Manager access to encrypt and decrypt using the new CMEK keys. This command grants the Cloud KMS Encrypter / Decrypter role (roles/cloudkms.cryptoKeyEncrypterDecrypter) on the my-other-key Cloud KMS keys to the service account.

gcloud

# us-east1
$ gcloud kms keys add-iam-policy-binding "my-other-key" \
    --project "${KMS_PROJECT_ID}" \
    --location "us-east1" \
    --keyring "secret-manager-cmek" \
    --member "serviceAccount:${SM_SERVICE_ACCOUNT}" \
    --role "roles/cloudkms.cryptoKeyEncrypterDecrypter"

# us-central1
$ gcloud kms keys add-iam-policy-binding "my-other-key" \
    --project "${KMS_PROJECT_ID}" \
    --location "us-central1" \
    --keyring "secret-manager-cmek" \
    --member "serviceAccount:${SM_SERVICE_ACCOUNT}" \
    --role "roles/cloudkms.cryptoKeyEncrypterDecrypter"

Modify the CMEK configuration on a secret by updating the replication on the secret with the new Cloud KMS key resource names.

gcloud

$ gcloud beta secrets replication update my-ummr-secret \
    --set-kms-key="projects/${KMS_PROJECT_ID}/locations/us-east1/keyRings/secret-manager-cmek/cryptoKeys/my-other-key" \
    --location=us-east1 \
    --project "${SM_PROJECT_ID}"

$ gcloud beta secrets replication update my-ummr-secret \
    --set-kms-key="projects/${KMS_PROJECT_ID}/locations/us-central1/keyRings/secret-manager-cmek/cryptoKeys/my-other-key" \
    --location=us-central1 \
    --project "${SM_PROJECT_ID}"

In order to update multiple keys in a secret simultaneously, you may get and set the replication policy via a file.

gcloud

$ gcloud beta secrets replication get my-ummr-secret --project "${SM_PROJECT_ID}" --format=json > replication-policy.json

# update the file to reflect desired CMEK configuration
$ edit replication-policy json

$ gcloud beta secrets replication set my-ummr-secret \
    --replication-policy-file=replication-policy.json \
    --project "${SM_PROJECT_ID}"

API

$ curl "https://secretmanager.googleapis.com/v1/projects/${SM_PROJECT_ID}/secrets/my-ummr-secret?updateMask=replication" \
    --request "PATCH" \
    --header "Authorization: Bearer $(gcloud auth print-access-token)" \
    --header "Content-Type: application/json" \
    --data-binary @- <<EOF
{
  "replication":{
    "userManaged":{
      "replicas":[
        {
          "location":"us-east1",
          "customerManagedEncryption":{
            "kmsKeyName":"projects/${KMS_PROJECT_ID}/locations/us-east1/keyRings/secret-manager-cmek/cryptoKeys/my-other-key"
          }
        },
        {
          "location":"us-central1",
          "customerManagedEncryption":{
            "kmsKeyName":"projects/${KMS_PROJECT_ID}/locations/us-central1/keyRings/secret-manager-cmek/cryptoKeys/my-other-key"
          }
        }]
      }
    }
  }
EOF

Viewing Secret Version CMEK Configuration

To inspect a secret version's metadata, including whether the secret version is CMEK-enabled and the resource name of the CMEK key version, view its metadata.

gcloud

$ gcloud beta secrets versions describe latest --secret my-secret --project "${SM_PROJECT_ID}"

API

$ curl "https://secretmanager.googleapis.com/v1/projects/${SM_PROJECT_ID}/secrets/my-secret/versions/latest" \
    --request "GET" \
    --header "Authorization: Bearer $(gcloud auth print-access-token)" \
    --header "Content-Type: application/json"

This returns the full Cloud KMS resource name of the key version used to encrypt the secret version.

{
  "name": "projects/[PROJECT_NUMBER]/secrets/my-secret/versions/1",
  "createTime": "2020-07-...",
  "state": "ENABLED",
  "replicationStatus": {
    "automatic": {
      "customerManagedEncryption": {
        "kmsKeyVersionName": "projects/[KMS_PROJECT]/locations/global/keyRings/secret-manager-cmek/cryptoKeys/my-cmek-key/cryptoKeyVersions/1"
      }
    }
  }
}

Disabling CMEK

Remove CMEK configuration from a secret by updating the replication policy.

gcloud

$ gcloud beta secrets replication update my-secret --remove-cmek --project "${SM_PROJECT_ID}"

API

$ curl "https://secretmanager.googleapis.com/v1/projects/${SM_PROJECT_ID}/secrets/my-secret?updateMask=replication" \
    --request "PATCH" \
    --header "Authorization: Bearer $(gcloud auth print-access-token)" \
    --header "Content-Type: application/json" \
    --data-binary @- <<EOF
{
  "replication":{
    "automatic":{}
  }
}
EOF

What's next?

  • Learn more about CMEK