This document provides instructions for configuring customer-managed encryption keys (CMEK) for logs stored in log buckets. This document also describes how to manage those keys, and limitations associated with using CMEK.
You can configure CMEK as a default resource setting for an organization or a folder. When configured, Cloud Logging ensures that all new log buckets in the organization or folder are encrypted with a customer-managed key. If you don't supply a key when you create the log bucket, then the default key is used. For more information, see Configure CMEK for Cloud Logging.
By default, Cloud Logging encrypts customer content stored at rest. Data stored in log buckets by Logging is encrypted using key-encryption keys, a process known as envelope encryption. Access to your logging data requires access to those key-encryption keys, which Google manages for you without any actions on your part.
Your organization might have regulatory, compliance-related, or advanced encryption requirements that our default encryption at rest doesn't provide. To meet your organization's requirements, instead of Google managing the encryption keys that protect your data, you can manage your keys.
For specific information about CMEK usage, including advantages, and limitations, see Customer-managed encryption keys.
For symmetric encryption, periodically and automatically rotating keys is a recommended security practice. For more information, see Key rotation.
Complete the following steps:
There are some limitations when using CMEK. Before you create a log bucket with CMEK enabled, review the Limitations.
This guide provides instructions using the Google Cloud CLI.
Create or identify the Google Cloud project in which you want to run Cloud KMS.
To get the permissions that you need to create keys, ask your administrator to grant you the Cloud KMS Admin (
roles/cloudkms.admin) IAM role on the project or a parent resource. For more information about granting roles, see Manage access.
Ensure that you have the following Cloud Logging permissions:
Enable the Cloud KMS API for the Google Cloud project that runs Cloud KMS.
Create a key ring and keys for the Google Cloud project that runs Cloud KMS.
To align failure domains, you must use a key ring whose location matches the regional scope of your data:
You can't enable CMEK for log buckets created in the
Identify values for the following variables:
- BUCKET_ID is the unique numeric identifier of the Logging bucket to create.
- BUCKET_PROJECT_ID is the name of the Google Cloud project that contains the new log bucket.
- KMS_PROJECT_ID is the unique alphanumeric identifier, composed of your Google Cloud project name and a randomly assigned number, of the Google Cloud project running Cloud KMS.
- KMS_KEY_LOCATION is the Cloud KMS key's region.
- KMS_KEY_RING is the Cloud KMS key ring's name.
- KMS_KEY_NAME with the Cloud KMS key's name.
It is formatted like this:
For information about locating resource identifiers, see Identifying projects.
After you've completed the prerequisite steps, follow these instructions to enable CMEK for an individual log bucket.
Determine the service account ID
To determine the service account ID associated with the Google Cloud
resource for which CMEK will apply, run the following
gcloud logging settings describe
gcloud logging settings describe --project=BUCKET_PROJECT_ID
Running the previous command generates a service account for the specified
resource, when one doesn't exist already, and returns the ID in the
kmsServiceAccountId: email@example.com name: projects/BUCKET_PROJECT_ID/settings
In this example, the value of SERVICE_ACCT_NAME is
Run the provisioning process only once per resource. Running the
command multiple times returns the same value for the
Assign the Encrypter/Decrypter role
When you're configuring CMEK at the log bucket level, give the service account permission to use your Cloud KMS by assigning the Cloud KMS CryptoKey Encrypter/Decrypter role to the service account:
gcloud kms keys add-iam-policy-binding \ --project=KMS_PROJECT_ID \ --member serviceAccount:SERVICE_ACCT_NAME@gcp-sa-logging.iam.gserviceaccount.com \ --role roles/cloudkms.cryptoKeyEncrypterDecrypter \ --location=KMS_KEY_LOCATION \ --keyring=KMS_KEY_RING \ KMS_KEY_NAME
Set the variables in the previous command as follows:
Replace SERVICE_ACCT_NAME with the
kmsServiceAccountIdvalue you determined in the previous step.
Replace the other variables with the values that you determined in the prerequisite steps.
Create a log bucket and supply the Cloud KMS key
To create a log bucket and enable CMEK for the log bucket,
run the following
gcloud logging buckets create
gcloud logging buckets create BUCKET_ID \ --location=LOCATION \ --cmek-kms-key-name=KMS_KEY_NAME
Verify key enablement
To verify that you've successfully created a log bucket with CMEK enabled, run the following command:
gcloud logging buckets list --project=BUCKET_PROJECT_ID
In the tabular output, you see a column labeled
CMEK. If the value of
CMEK column is
TRUE, then CMEK is enabled for the log bucket.
To view the details for a specific log bucket, including the key's details, run this command:
gcloud logging buckets describe BUCKET_ID
Manage your Cloud KMS key
The following sections describe how to update a log bucket to use the latest primary key version of a Cloud KMS key. They also describe how to change, revoke access for, and disable your Cloud KMS key.
Rotate your Cloud KMS key
If you rotate a Cloud KMS key, then the new key version applies only to log buckets created after the key rotation. If the key is used by an existing log bucket, rotating the key doesn't change how the log bucket protects its data.
For example, suppose that you create a log bucket and enable CMEK, and then you rotate the Cloud KMS key. The log bucket that you created doesn't use the new key version, instead it continues to protect its data with the key version that was marked as primary when the log bucket was created.
To update a log bucket to use the most recent primary key version of a Cloud KMS key, do the following:
- Identify the current Cloud KMS key for the log bucket. For more information, see Verify key enablement.
- Identify another Cloud KMS key that you can use. If your key ring has only one key, then Create a key.
- Change the Cloud KMS key for the log bucket to the Cloud KMS key created in the previous step.
- Change the Cloud KMS key for the log bucket to the original Cloud KMS key.
Change your Cloud KMS key
To change the Cloud KMS key associated with your log bucket, create a key and update the CMEK settings for log bucket:
gcloud logging buckets update --cmek-kms-key-name=NEW_KMS_KEY_NAME
Revoke access to the Cloud KMS key
To revoke Logging's access to the Cloud KMS key at any time, remove the configured service account's IAM permission for that key.
If you remove Logging's access to a key, it can take up to one hour for the change to take effect.
If you have a linked BigQuery dataset, BigQuery can't use this access to apply to key to a new BigQuery table. If you want to use a key on BigQuery table that aren't linked to Logging, follow BigQuery's documentation to do so. If you revoke Logging's access to a key and if you have a linked BigQuery dataset, then you also revoke BigQuery's access to the same key.
You can't revoke BigQuery's access to the linked dataset's key while preserving Logging's access.
For more information about the impact of revoking access, see Limitations.
To remove Logging's access to a key, run the following command:
gcloud kms keys remove-iam-policy-binding \ --project=KMS_PROJECT_ID \ --member serviceAccount:SERVICE_ACCT_NAME@gcp-sa-logging.iam.gserviceaccount.com \ --role roles/cloudkms.cryptoKeyEncrypterDecrypter \ --location=KMS_KEY_LOCATION \ --keyring=KMS_KEY_RING \ KMS_KEY_NAME
The following are known limitations.
CMEK disables Error Reporting
If you want to use Error Reporting, then don't enable customer-managed encryption keys (CMEK) on your log buckets. For more information, see Troubleshooting.
CMEK can't be removed from log buckets
You can't reconfigure log buckets to change or remove CMEK.
Degradation due to Cloud KMS key unavailability
A Cloud KMS key is considered available and accessible by Logging if both of the following are true:
- The key is enabled.
- The Logging service account has encryption and decryption permissions on the key.
Logging strongly recommends ensuring that any keys are properly configured and always available.
Loss of disaster recovery
If there are critical failures in Cloud Logging primary storage, then Logging mirrors the logging data to disaster-recovery files. When CMEK is enabled for a resource, such as a Google Cloud organization, logs belonging to that resource are protected by the configured CMEK key. If the CMEK key isn't accessible, the disaster-recovery files can't be written for that resource.
Loss of disaster-recovery files doesn't affect normal logging operations. However, in the event of a storage disaster, Cloud Logging might be unable to recover logs from resources whose CMEK isn't properly configured.
Cloud Customer Care can't read your resource's logs if its key isn't properly configured or becomes unavailable.
Degraded query performance
When a customer-managed encryption key is inaccessible, Cloud Logging continues to encrypt your data and store data in log buckets. However, Cloud Logging can't perform background optimizations on this data. If key access is restored, the data becomes available; however, the data is initially be stored in an unoptimized state and query performance may suffer.
Degradation due to Cloud EKM key unavailability
When you use a Cloud EKM key, Google has no control over the availability of your externally managed key in the external key-management partner system. For bucket-level CMEK, if an externally managed key is unavailable, Cloud Logging continues to store logs in log buckets but users aren't able to access those logs.
For more considerations, and potential alternatives, when using external keys, see the Cloud External Key Manager documentation.
When you create a log bucket and enable CMEK, you must use a key whose
region matches the regional scope of your data.
You can't configure CMEK for log buckets created in the
Client library availability
Logging client libraries don't provide methods for configuring CMEK.
For details on Logging usage limits, see Quotas and limits.
Troubleshoot configuration errors
For information about troubleshooting CMEK configuration errors, see Troubleshoot CMEK and organization setting errors.