Rotate your cluster's security keys

Key Rotation

Key rotation is the act of changing the underlying cryptographic material contained in a key encryption key (KEK). It can be triggered automatically as part of a scheduled rotation, or manually, usually after a security incident where keys might have been compromised. Key rotation replaces only the single field in the key that contains the raw encryption/decryption key data.

Automatic Key Rotation

The AWS Key Management Service (KMS) supports automatic rotation of KMS keys. When enabled, AWS automatically generates new cryptographic key material for your key once a year. No manual actions are required.

After a key rotation, GKE on AWS encrypts each new Secret with the new key. Any previously-created Secret is still decrypted with its original key. For this reason, AWS keeps CMK's older key material in perpetuity, allowing old DEKs to be decrypted when old Secrets are read.

You can see whether a KMS key has automatic rotation enabled or not with the following command:

aws kms get-key-rotation --key-id KMS_KEY_ID

Replace KMS_KEY_ID with your AWS KMS key ID.

You can enable automatic key rotation by running this command:

aws kms enable-key-rotation --key-id KMS_KEY_ID

Manual Key Rotation

This section explains how to manually rotate your control plane or node pool configuration keys.

Control plane configuration key

To manually rotate your control plane configuration key, perform the following steps:

  1. Create a new Cloud Key Management Service key. Save the value of your KMS key ARN. You'll use it later.

  2. Ensure the IAM role associated with the cluster has permissions to encrypt and decrypt using the new key.

  3. Use the gcloud container aws clusters update command to update the encryption key.

     gcloud container aws clusters update CLUSTER_NAME \
        --location=GOOGLE_CLOUD_LOCATION \
        --config-encryption-kms-key-arn=CONFIG_ENCRYPTION_KMS_KEY_ARN
    

    Replace the following:

    • CLUSTER_NAME: your cluster's name
    • GOOGLE_CLOUD_LOCATION: the supported Google Cloud region that manages your cluster—for example, us-west1
    • CONFIG_ENCRYPTION_KMS_KEY_ARN: your new KMS key ARN

Control plane root volume

To manually rotate your control plane root volume key, perform the following steps:

  1. Create a new Cloud Key Management Service key. Save the value of your KMS key ARN. You'll use it later.

  2. Ensure the IAM role associated with the cluster has permissions to encrypt and decrypt using the new key.

  3. Use the gcloud container aws clusters update command to update the encryption key.

     gcloud container aws clusters update CLUSTER_NAME \
        --location=GOOGLE_CLOUD_LOCATION \
        --root-volume-kms-key-arn=ROOT_VOLUME_KMS_KEY_ARN
    

    Replace the following:

    • CLUSTER_NAME: your cluster's name
    • GOOGLE_CLOUD_LOCATION: the supported Google Cloud region that manages your cluster—for example, us-west1
    • ROOT_VOLUME_KMS_KEY_ARN: Amazon Resource Name (ARN) of the AWS KMS key to encrypt the root volume

Node pool configuration key

To manually rotate your Node pool configuration key, perform the following steps:

  1. Create a new Cloud Key Management Service key.

  2. Ensure the IAM role associated with the cluster has permissions to encrypt and decrypt using the new key.

  3. Update the KMS alias in use:

     aws kms update-alias --alias-name KEY_ALIAS \
         --target-key-id CONFIG_ENCRYPTION_KMS_KEY_ARN
    

    Replace the following:

    • KEY_ALIAS: your existing key's alias
    • CONFIG_ENCRYPTION_KMS_KEY_ARN: your new KMS key ARN
  4. Force the cluster to re-encrypt all cluster Secrets using the new encryption key:

     kubectl get secrets --all-namespaces -o json | \
     kubectl annotate --overwrite -f - encryption-key-rotation-time=`date +"%Y%m%d-%H%M%S"`
    
  5. Disable the old AWS KMS key. For more information, see Enabling and disabling keys.

Node pool root volumes

To manually rotate your Node pool root volume key, perform the following steps:

  1. Create a new Cloud Key Management Service key.

  2. Ensure the IAM role associated with the cluster has permissions to encrypt and decrypt using the new key.

  3. Update the KMS alias in use:

     aws kms update-alias --alias-name KEY_ALIAS \
         --target-key-id ROOT_VOLUME_KMS_KEY_ARN
    

    Replace the following:

    • KEY_ALIAS: your existing key's alias
    • ROOT_ENCRYPTION_KMS_KEY_ARN: your new KMS key ARN
  4. Update your node pool:

     gcloud container aws node-pools update NODE_POOL_NAME \
         --root-volume-kms-key-arn=ROOT_VOLUME_KMS_KEY_ARN
    

    Replace the following:

    • NODE_POOL_NAME: name of your node pool
    • ROOT_VOLUME_KMS_KEY_ARN: Amazon Resource Name (ARN) of the AWS KMS key to encrypt the root volume
  5. Disable the old AWS KMS key. For more information, see Enabling and disabling keys.