Customer-managed encryption keys

Setup

By default, Cloud Storage encrypts customer content at rest. Cloud Storage handles encryption for you without any additional actions on your part. This option is called Google default encryption.

If you want to control your encryption keys, then you can use customer-managed encryption keys (CMEKs) in Cloud KMS with CMEK-integrated services including Cloud Storage. Using Cloud KMS keys gives you control over their protection level, location, rotation schedule, usage and access permissions, and cryptographic boundaries. Using Cloud KMS also lets you track key usage, view audit logs, and control key life cycles. Instead of Google owning and managing the symmetric key encryption keys (KEKs) that protect your data, you control and manage these keys in Cloud KMS.

After you set up your resources with CMEKs, the experience of accessing your Cloud Storage resources is similar to using Google default encryption. For more information about CMEKs, see Customer-managed encryption keys (CMEK). To learn how to use manually-created CMEKs to protect your Cloud Storage resources, see Use customer-managed encryption keys.

For more information about other encryption options when using Cloud Storage, see Data Encryption Options.

CMEK with Cloud KMS Autokey

You can either create CMEKs manually to protect your Cloud Storage buckets and the objects within them or use Cloud KMS Autokey. With Autokey, key rings and keys are generated on demand as part of resource creation or update in Cloud Storage. Service agents that use the keys for encrypt and decrypt operations are created if they don't already exist and are granted the required Identity and Access Management (IAM) roles. For more information, see Autokey overview.

Autokey doesn't create keys for objects. By default, objects in a bucket use the bucket default key. If you want to encrypt an object using a key other than the bucket default key, you can manually create a CMEK and use that key when creating the object.

To learn how to use CMEKs created by Cloud KMS Autokey to protect your Cloud Storage buckets and the objects within them, see Using Autokey with Cloud Storage resources.

When is the key used?

When you apply a CMEK to an object, Cloud Storage uses the key when encrypting:

• The object's data.
• The object's CRC32C checksum.
• The object's MD5 hash.

Cloud Storage uses standard server-side keys to encrypt the remaining metadata for the object, including the object's name. Thus, if you have sufficient permission, you can perform actions such as reading most metadata, listing objects, and deleting objects even after you've disabled or destroyed the associated CMEK.

Note: Because most metadata can be read regardless of encryption method, do not include sensitive information in your object and bucket metadata or names.

Service agents

Each project has a special Cloud Storage service account called a service agent that performs encryption and decryption with CMEKs. Once you give the service agent access to an encryption key, that service agent encrypts:

• Objects added to a bucket that uses the key as the default key.
• Specific objects that you indicate should be encrypted with that key.

When adding or rewriting an object in Cloud Storage, if you have both a default key set on your bucket and a specific key included in your request, Cloud Storage uses the specific key to encrypt the object.

When a requester wants to read an object encrypted with a CMEK, they simply access the object as they normally would. During such a request, the service agent automatically decrypts the requested object as long as:

• The service agent still has permission to decrypt using the key.
• You have not disabled or destroyed the key.

If one of these conditions is not met, the service agent does not decrypt the data, and the request fails.

Important: There is a delay between when you disable a key and when that key no longer can be used to encrypt and decrypt objects. For information about the typical length of the delay and ways to effectively shorten it, see Cloud KMS Resource Consistency.

Caution: If you destroy a key that encrypts existing objects, you will be unable to recover that data, but you continue to be charged for storage of your objects until you delete them. If you destroy a key, use the object metadata field kmsKeyName to identify objects that used the destroyed key, and delete those objects.

Restrictions

The following restrictions apply when using CMEKs:

• You cannot encrypt an object with a CMEK by updating the object's metadata. Include the key as part of a rewrite of the object instead.

  • gcloud storage uses the objects update command to set encryption keys on objects, but the command rewrites the object as part of the request.

• You must create the Cloud KMS key ring in the same location as the data you intend to encrypt. For example, if your bucket is located in US-EAST1, any key ring used for encrypting objects in that bucket must also be created in US-EAST1.

  • For dual-regions, the Cloud KMS key ring location must match the dual-region's location code. For example, if your bucket is located in the configurable dual-region pair US-EAST1, US-WEST1, any key ring used for encrypting objects in that bucket must be created in the US multi-region, which matches this bucket's location code. If your bucket is located in the predefined dual-region NAM4, then you must create the key ring in the same predefined dual-region, NAM4.

    For available Cloud KMS locations, see Cloud KMS locations.

• Cloud KMS encryption and decryption rates are subject to a quota.

• The CRC32C checksum and MD5 hash of objects encrypted with CMEKs are not returned when listing objects with the JSON API.

  • When appropriate, some tools, such as gcloud storage, perform an additional metadata GET request on each object encrypted with a CMEK in order to retrieve the CRC32C and MD5 information. These additional requests can make listing substantially slower than listing objects encrypted with standard Cloud Storage encryption.

• Only symmetric encryption keys can be used as CMEKs.

Relation to customer-supplied encryption keys

In addition to customer-managed encryption, Cloud Storage offers Customer-Supplied Encryption Keys as a way of controlling your data encryption. You can encrypt different objects in a single bucket with different encryption methods, but note that:

• A single object can only be encrypted by one of these methods at a time.

• If you have a default CMEK set for your bucket and specify a customer-supplied key in a request, Cloud Storage uses the customer-supplied key to encrypt the object.

Key management

This section discusses considerations when rotating keys, replacing keys, and disabling or destroying key versions.

Key rotation

Cloud KMS supports both automatic and manual key rotation to a new version. After rotating a key, Cloud Storage uses the new version for all operations that encrypt using the key, such as:

• Object uploads when the destination bucket uses the key as its default encryption key.

• Object upload, copy, and rewrite operations that specifically use the key in the operation.

Previous versions of the key are not disabled or destroyed, so Cloud Storage can still decrypt existing objects that were previously encrypted using those versions.

Key replacement

Use the following guidelines when replacing the key you use to encrypt Cloud Storage objects with a new key:

1. Check your buckets to see which use the key as their default encryption key. For these buckets, replace the old key with a new key.

   This ensures that all objects written to the bucket use the new key going forward.

2. Inspect your source code to understand which requests use the key in ongoing operations, such as setting bucket configurations and uploading, copying, or rewriting objects. Update these instances to use the new key.

3. Check for objects, in all of your buckets, encrypted with the old key. Use the Rewrite Object method to re-encrypt each object with the new key.

4. Disable all versions of the old key. After disabling old key versions, monitor client and service logs for operations that fail due to a version becoming unavailable.

Disabling or destroying a key version

Caution: Except in certain cases involving locked objects, you can destroy a Cloud KMS key version even if it's actively being used to encrypt Cloud Storage objects. Permanent loss of data can occur if you destroy a key version that is still being used to encrypt objects.

• When you disable or destroy a specific key version, you cannot decrypt any object that is currently encrypted with that key version.

  For example, you cannot download, copy, or rewrite the object, and attempting to do so results in an error.

  • If you disable a key version, you can re-enable it. Once re-enabled, you can access objects that were encrypted by that key version.

  • If you destroy a key version, downloads of objects encrypted with that version are never possible again.

  Before disabling or destroying a key version, you should identify all objects, in all buckets, that were encrypted using the specific key version. Once identified, use the Rewrite Object method to re-encrypt each object using a new key version, a whole new key, or server-side keys.

• When you disable or destroy the primary version of a key, you cannot use the key for encryption until you have a new primary version. For example, without a primary version:

  • You cannot specify the key as part of an object upload, copy, or rewrite.

  • You cannot upload, copy, or rewrite objects to a bucket that has the key set as its default encryption key unless you specify a different, valid key as part of the operation.

  Once you have a primary version for your key, operations that use the key to encrypt objects succeed.

  Before disabling or destroying a key version that is the primary version of the key, you should first stop using it as the primary version. You can do so by either:

  • Replacing it with a new primary version, typically by performing a key rotation.
  • Removing instances where you use the key for encryption. When you do so, Cloud Storage uses server-side keys for encryption instead.

Note: Key disablement and destruction take effect after some time. Cloud KMS documentation contains detailed information about this behavior.

Key versions and locked objects

If a key version encrypts an object that is locked, either because the object is stored in a bucket with a locked retention policy or because the object has its own locked retention configuration, the key version can only be destroyed if the following conditions are met:

• The encrypted object's retention expiration time must be in the past.
• The encrypted object must not have any object holds placed on it.

Once all relevant objects have met these conditions, it's possible to destroy the key version, even without deleting the objects. If you do so, affected object data becomes permanently inaccessible.

What's next

• Set CMEKs on your Cloud Storage buckets and objects.
• Learn more about encryption in Cloud Storage.
• Learn more about Cloud KMS.