Customer-managed encryption keys

Go to examples

This page discusses customer-managed encryption keys and how they are used in Cloud Storage. For other encryption options, see Data Encryption Options.

Overview

As an additional layer on top of Google-managed encryption keys, you can choose to use keys generated by Cloud Key Management Service. Such keys are known as customer-managed encryption keys. If you use a customer-managed encryption key, your encryption keys are stored within Cloud KMS. The project that holds your encryption keys can then be independent from the project that contains your buckets, thus allowing for better separation of duties.

When is the key used?

When you apply a customer-managed encryption key to an object, Cloud Storage uses the key when encrypting:

  • The object's data.
  • The object's CRC32C checksum.
  • The object's MD5 hash.

Cloud Storage uses standard server-side keys to encrypt the remaining metadata for the object, including the object's name. Thus, if you have sufficient permission, you can perform actions such as reading most metadata, listing objects, and deleting objects even after you've disabled or destroyed the associated customer-managed encryption key.

Service accounts

Encryption and decryption with customer-managed encryption keys is accomplished using service accounts. Once you give your Cloud Storage service account access to an encryption key, that service account encrypts:

When adding or rewriting an object in Cloud Storage, if you have both a default key set on your bucket and a specific key included in your request, Cloud Storage uses the specific key to encrypt the object.

When a requester wants to read an object encrypted with a customer-managed encryption key, they simply access the object as they normally would. During such a request, the service account automatically decrypts the requested object as long as:

  • The service account still has permission to decrypt using the key.
  • You have not disabled or destroyed the key.

If one of these conditions is not met, the service account does not decrypt the data, and the request fails.

Key resources

A Cloud KMS key resource has the following format:

projects/[PROJECT_STORING_KEYS]/locations/[LOCATION]/keyRings/[KEY_RING_NAME]/cryptoKeys/[KEY_NAME]

Where:

  • [PROJECT_STORING_KEYS] is the ID of the project associated with the key. For example, my-pet-project.
  • [LOCATION] is the key location. For example, US-EAST1.
  • [KEY_RING_NAME] is the name of the key ring. For example, my-key-ring.
  • [KEY_NAME] is the name of the key. For example, my-key.

Restrictions

The following restrictions apply when using customer-managed encryption keys:

  • Cloud SQL exports to Cloud Storage and Dataflow do not currently support objects encrypted with customer-managed encryption keys.

  • You cannot use the JSON API Compose Object method when one or more of the source objects are encrypted with a customer-managed encryption key.

  • You cannot encrypt an object with a customer-managed encryption key by updating the object's metadata. Include the key as part of a rewrite of the object instead.

  • You must create the Cloud KMS key in the same location as the data you intend to encrypt. For example, if your bucket is located in US-EAST1, any Cloud KMS key encrypting objects in that bucket must also be created in US-EAST1. For available Cloud KMS locations, see Cloud KMS locations.

  • You cannot specify a customer-managed encryption key as part of a Storage Transfer Service transfer, and any such keys on source objects are not applied to the transferred objects. Set a default customer-managed key on your bucket prior to performing the transfer.

Relation to customer-supplied encryption keys

In addition to customer-managed encryption, Cloud Storage offers Customer-Supplied Encryption Keys as a way of controlling your data encryption. You can encrypt different objects in a single bucket with different encryption methods, but note that:

  • A single object can only be encrypted by one of these methods at a time.

  • If you have a default customer-managed key set for your bucket and specify a customer-supplied key in a request, Cloud Storage uses the customer-supplied key to encrypt the object.

  • You can set a default customer-managed key on your bucket, but you cannot set a default customer-supplied key on your bucket.

Key management: rotating, disabling or destroying key versions

Rotation using Cloud Key Management Service

Cloud KMS supports both automatic and manual key rotation. After rotating a key, the new version becomes the primary version used for all operations in Cloud Storage that encrypt using the key, for example:

  • Objects uploaded into buckets using the key as the default encryption key are now encrypted using the new key version.

  • Object upload, copy, rewrite operations that specifically reference the key now use the new key version.

Previous versions of the key are not disabled nor destroyed, therefore existing objects encrypted using those versions can still be decrypted by Cloud Storage.

Disabling or destroying key versions

The actions of disabling and destroying specific key versions have the following effects on Cloud Storage buckets and objects:

If the version is primary, and no new primary version is set, or if all key versions are disabled or destroyed:

  • Uploads to buckets using the key as default result in failure.
  • Object upload, copy or rewrite operations referencing the key result in failure.
  • Downloads of objects encrypted using the affected key version(s) result in failure.

If the version is not primary, or a new primary version has been set:

  • Downloads of objects encrypted using the affected key version(s) result in failure.
  • Uploads to buckets using the key as default will continue to work using the primary key version.
  • Object upload, copy or rewrite operations referencing the key will continue to work, using the primary key version.

If you re-enable a key version, downloads of objects encrypted with that version are possible again.

If a key version is permanently destroyed, downloads of objects encrypted with that version are never possible again.

Replacing an old key with a new key

If you want to stop using an existing key (all of its versions) with Cloud Storage and replace it with a new key, we suggest the following steps:

  • Identify all buckets that use the old key as default, and update them to use the new key. This ensures that all new object uploads into those buckets use the new key going forward.

  • Identify all objects, in all buckets, that were encrypted using the old key. Use the [Rewrite Object method][12] to re-encrypt each object using the new key. This ensures that no existing objects become unaccessible after the old key is disabled or destroyed.

  • Consider disabling the old key first, before destroying it permanently. Monitor client and service logs to check if there are any operations failing due to the old key becoming unavailable.

Understanding the usage of a key with data stored in Cloud Storage

Before you perform key management operations - such as disabling or destroying key versions - we strongly suggest that you fully understand how the affected key is used with your buckets and objects. Keep the following in mind:

  • Buckets can use a key as a default. Identify all buckets, using the console, or programmatically, for references to the key name.

    [LOCATION]/[KEY_RING_NAME]/[KEY_NAME]

  • Objects are encrypted using a specific key version. Identify all objects, using the console, or programmatically, for references to the specific key version.

    [LOCATION]/[KEY_RING_NAME]/[KEY_NAME]/[KEY_VERSION]
  • Inspect your source code and configuration to understand what keys it is referencing for ongoing operations in Cloud Storage including bucket configuration and object upload, copy and rewrite.

What's next