Customer-managed encryption keys (CMEK)

Preview

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

For information about access to this release, see the access request page.

By default, all the data at rest in Firestore in Datastore mode is encrypted using Google's default encryption. Firestore in Datastore mode handles and manages this encryption for you without any additional action on your part.

If you have specific compliance or regulatory requirements related to the keys that protect your data, you can use customer-managed encryption keys (CMEK) for Firestore in Datastore mode. Instead of Google managing the encryption keys that protect your data, your Firestore in Datastore mode database is protected using a key that you control and manage in Cloud Key Management Service (Cloud KMS).

This page describes CMEK for Firestore in Datastore mode. For more information about CMEK in general, including when and why to enable it, see the Cloud KMS documentation. For instructions on performing CMEK-related tasks with Firestore in Datastore mode, see Use CMEK.

Features

• Data control: CMEK lets you manage access to the KMS key. You can rotate, disable, and destroy the key used to encrypt the data at rest in your Firestore in Datastore mode database.
• Performance: CMEK does not impact the Firestore SLA.
• Auditability: If you enable audit logging for Cloud KMS, all the operations on the key are logged and viewable in Cloud Logging.
• Organization policy constraints: You can use CMEK organization policy constraints to specify encryption compliance requirements for Firestore in Datastore mode databases in your organization.

Pricing

Cloud KMS charges for the cost of the key and any cryptographic operations performed using that key. See Cloud KMS pricing for details.

You are billed for the operation costs when Firestore in Datastore mode asks the Cloud KMS key to perform an encryption or decryption operation. Encryption/decryption is not synchronized with your request. It's done every 5 minutes by polling Cloud KMS. Costs are generally low, given the expected number of cryptographic operations generated by Firestore in Datastore mode. Costs for Cloud Audit Logs are an additional expense, but is also expected to be generally low, given the expected number of cryptographic operations.

There are no additional Firestore in Datastore mode costs for using CMEK-protected database and the Firestore in Datastore mode pricing continues to apply.

If you revoke your key to a database, storage cost will be charged based on the size of the last day that the key was available. You will continue to incur storage costs at that database size until the database is deleted or the key becomes available again.

What is protected with CMEK

When you create a Firestore CMEK-protected database, your Cloud KMS key is used to protect data at rest. This includes data that is stored on disk or flash. Some exceptions apply. The following data types are encrypted with Google default encryption and not by the CMEK key:

• Data in transit or in memory
• Database metadata

How an unavailable key status is handled

Encrypt and decrypt operations are not issued on every data request. Instead, the Firestore system polls Cloud Key Management Service every 5 minutes to check if the key is still available and then performs encrypt and decrypt operations if the key is available. If the system detects that the key is unavailable, within 10 minutes, any subsequent calls to the Firestore database, including reads, writes, and queries, return a FAILED_PRECONDITION error with the message The customer-managed encryption key required by the requested resource is not accessible. If the database has time-to-live (TTL) policies, and if any expiration times get exceeded while the key is unavailable, data deletion by TTL will be delayed until the key gets reinstated. If the database has long-running operations in progress, they will be affected as follows:

• Data import or export operations will stop making progress, and be marked as Failed. The failed operations will not be retried if the key gets reinstated.
• Index build operations, and operations enabling new TTL policies will stop making progress. The stopped operations will be retried if the key gets reinstated.

Keys are considered unavailable in any situation that intentionally disallows Firestore from accessing the key. This includes:

• Disabling or destroying the in-use key version. Please be careful when destroying a key version, because this can cause unrecoverable data loss.
• Removing permission to access the key from the Firestore service account.

If the key is reinstated, the polling operation detects that the key is available again. Access is re-enabled, usually within minutes, but it can take up to a few hours in rare cases. Note that some operations on Cloud KMS keys, such as disabling or destroying a key, can take up to 3 hours to propagate. Firestore does not detect any changes until after they take effect in Cloud KMS.

Reinstatement of a key involves the following, depending on the situation:

• Re-enabling a disabled key version.
• Restoring a destroyed key version. Before being permanently destroyed, a key version is scheduled for destruction. You can only restore a key during the period when a key version is scheduled for destruction. You cannot restore a key that has already been permanently destroyed.
• Re-granting the Firestore service agent permission to access the key.

Warning: Do not let your key become unavailable for longer than 30 days. CMEK-protected databases whose keys have been unavailable for more than 30 days are eligible for deletion.

External key considerations

When you use a Cloud EKM key, Google has no control over the availability of your externally-managed key in the external key management partner system.

If an externally-managed key is unavailable, Firestore in Datastore mode continues to support full database operations using a cached version of the key, for up to one hour.

After an hour, if Firestore in Datastore mode is still unable to connect with Cloud KMS, Firestore in Datastore mode begins taking the database offline as a protective measure. Calls to the database will fail with a FAILED_PRECONDITION error that includes additional details.

See the Cloud External Key Manager documentation for more considerations when using external keys.

Caution: If a database remains disabled for more than 30 consecutive days, Firestore in Datastore mode automatically deletes it. To avoid permanent data loss, do not leave external keys in an inaccessible state for an extended time.

Warning: If an external key is deleted and cannot be recovered, any Firestore in Datastore mode database encrypted with that key becomes permanently inaccessible.

Limitations

• Changing a key for a CMEK-protected database is not supported. Key rotation, enablement, and disablement are supported.
• Backup schedules and restore operations are not supported for CMEK-protected databases. You can use Point-in-time recovery (PITR) for disaster recovery.
• CMEK-protected databases support Key Visualizer only for entity and document data, not for index data.
• You cannot enable CMEK on existing databases. You can enable CMEK only on new databases, and you must enable it when you create the database. To migrate data in an existing non-CMEK database to a CMEK-protected database, export your data and import data to a new CMEK-protected database.
• Key tracking is not supported for CMEK-protected database.
• During the Preview, Firestore will support a limited number of CMEK-protected databases.

What's next

• Learn how to use CMEK with Firestore in Datastore mode.