Customer-managed encryption keys (CMEK) overview

By default, all data at rest in Google Cloud, including the data in Spanner, is protected using Google-managed default encryption. Google Cloud handles and manages this default encryption for you without any additional actions on your part.

If you have specific compliance or regulatory requirements related to the keys that protect your data, you can use customer-managed encryption keys (CMEK) for Spanner. Instead of Google managing the encryption keys that protect your data, your Spanner database is protected using a key that you control and manage in Cloud Key Management Service (KMS). This can be a symmetric key, a Cloud HSM key, or a Cloud External Key Manager key.

CMEKs do not support custom instance configurations.

This page describes CMEK for Spanner. For more information about CMEK in general, including when and why to enable it, see the Cloud KMS documentation.

Features

  • Data access control: administrators can rotate, manage access to, and disable or destroy the key used to protect data at rest in Spanner.
  • Auditability: if you enable audit logging for Cloud KMS API in your project, all actions on the key, including those performed by Spanner, are logged and viewable in Cloud Logging. Cloud EKM keys support Key Access Justification, which adds a justification field to all key requests. With select external key management partners, you can automatically approve or deny these requests, based on the justification.
  • Performance: there are no changes to Spanner performance or the service level agreement.

Pricing

Spanner bills CMEK-enabled databases just like any other database; there are no additional Spanner costs. For more information, see Spanner pricing.

You are billed by Cloud KMS for both the cost of the key and any cryptographic operations on that key (whenever Spanner uses the key for encryption/decryption). We expect those costs to be minimal based on the expected number of cryptographic operations generated by Spanner. For more information, see Cloud KMS pricing.

What is protected with CMEK

In a CMEK-enabled database, Spanner uses your Cloud KMS key to protect data at rest. This includes data in a database that is stored on disk or flash.

Some exceptions apply. The following types of data are protected by Google default encryption at rest and not by the CMEK key:

  • A subset of row keys that mark range boundaries
  • Debugging data including core dumps and operational logs
  • Data in transit or in memory
  • Database metadata

In Spanner, there are three layers of encryption. Data at rest is broken into subfile chunks for storage, and each chunk is encrypted at the storage level with an individual encryption key. The key used to encrypt the data in a chunk is called a data encryption key (DEK). Because of the high volume of keys at Google, and the need for low latency and high availability, these keys are stored near the data that they encrypt. The DEKs are encrypted with (or "wrapped" by) a key encryption key (KEK). Finally, each KEK is encrypted with your customer-managed encryption key.

When you rotate the CMEK key, Spanner re-encrypts only the intermediate KEKs with the latest primary version of the CMEK key. Once the re-encryption finishes, disabling or deleting the old versions of the CMEK key will not disable access to the database. You can also view the key versions that are being used to protect a database.

With CMEK

Diagram illustrating encryption with a customer-managed encryption key

Without CMEK

Diagram illustrating encryption with a Google-managed encryption key

Enable CMEK

To use CMEK for Spanner databases, you must create a new database and specify the Cloud KMS key at the time of database creation.

Spanner is able to access the key on your behalf after you grant the Cloud KMS CryptoKey Encrypter/Decrypter (roles/cloudkms.cryptoKeyEncrypterDecrypter) role to a Google-managed Spanner service account.

For detailed instructions, refer to Using CMEK.

Spanner's data access APIs, such as those that are used to manage sessions and execute transactions on data, are exactly the same for both customer-managed and Google-managed keys. Applications do not need to specify keys or encryption configs when reading or writing data. All encryption is handled by the service.

Manage keys

Key management operations are performed using Cloud KMS. Spanner cannot detect or act upon any key changes until they are propagated by Cloud KMS. Some operations, such as disabling or destroying a key, can take up to 3 hours to propagate; changes to permissions usually propagate much faster.

Once the database is created, Spanner calls Cloud KMS about every 5 minutes to make sure that the key is still valid.

If Spanner detects that your Cloud KMS key has been disabled or destroyed, an operation to make your database inaccessible begins immediately. Any subsequent calls to the database, including sessions, reads, and writes, will return a FAILED_PRECONDITION error: KMS key required by the Spanner resource is not accessible.

If Spanner's calls to Cloud KMS detect that a formerly disabled key has been re-enabled, Cloud KMS restores access to the Spanner database automatically.

How an unavailable key status is handled

In rare scenarios, such as during periods when Cloud KMS is unavailable, Spanner may be unable to retrieve the status of your key from Cloud KMS.

If your Spanner database is protected by a key that is enabled at the time at which Spanner is first unable to communicate with Cloud KMS, Spanner continues to support full database operations on a best-effort basis for a period of up to 1 hour, to minimize the impact of any such incident on your workload.

After an hour, if Spanner is still unable to connect with Cloud KMS, Spanner begins taking the database offline as a protective measure. The data in your Spanner database remains inaccessible until your database can reconnect with Cloud KMS and Cloud KMS responds that the key is active.

Conversely, if your Spanner database is protected by a key that is disabled at the time at which Spanner is first unable to communicate with Cloud KMS, your database remains inaccessible until it is able to reconnect to Cloud KMS and you have re-enabled your key.

External key considerations

When you use a Cloud EKM key, Google has no control over the availability of your externally-managed key in the external key management partner system.

If an externally-managed key is unavailable, Spanner continues to support full database operations using a cached version of the key, for up to one hour.

After an hour, if Spanner is still unable to connect with Cloud KMS, Spanner begins taking the database offline as a protective measure. Calls to the database will fail with a FAILED_PRECONDITION error: External key error: Could not find a key resource at the key URI.

See the Cloud External Key Manager documentation for more considerations when using external keys.

Backup and restore

Spanner backups, like databases, can be protected by CMEK or Google-managed encryption. By default, a backup uses the same encryption config as its database, but you can override this behavior by specifying a different encryption configuration when creating the backup. If the backup is CMEK-enabled, it is encrypted using the primary version of the KMS key at the time of backup creation. Once the backup is created, its key and key version cannot be modified, even if the KMS key is rotated. For more information, see Backing up a database.

When you restore a database from a backup, by default, the restored database uses the same encryption config as the backup. You can override this behavior by specifying a different encryption configuration when restoring the database. To restore a CMEK-enabled backup, both the key and key version that was used to encrypt the backup must be available. For more information, see Restoring from a backup.

Logging

You can audit the requests that Spanner sends to Cloud KMS on your behalf in Cloud Logging, if you have enabled audit logging for the Cloud KMS API in your project. These Cloud KMS log entries are visible in Cloud Logging.

Require or limit CMEK within your organization

You can set organization-wide policies regarding the use of CMEK protection across various Google Cloud products, including Spanner. With these policies, you can:

  • Require that new Spanner databases created by your organization use CMEK protection.

  • Limit which of your organization's Cloud KMS keys are available for CMEK protection.

For more information, see CMEK organization policies.