Managing Data Encryption

Google Cloud Storage always encrypts your data on the server side, before it is written to disk, at no additional charge. This page discusses the various ways in which you can encrypt your data when using Cloud Storage.

Overview

Server-side encryption refers to encryption that occurs after Cloud Storage receives your data, but before the data is written to disk and stored.

The default behavior for Cloud Storage is to use its server-side encryption keys to encrypt your data. As an alternative, you can provide your own encryption keys for server-side encryption, which then replace the default encryption keys. Such user-provided encryption keys are called customer-supplied encryption keys.

Client-side encryption refers to encryption that occurs before data is sent to Cloud Storage, which arrives at Cloud Storage already encrypted. You can choose to encrypt data on the client side before you write it to Cloud Storage.

Default server-side encryption

By default, Google Cloud Storage manages server-side encryption keys on your behalf using the same hardened key management systems that we use for our own encrypted data, including strict key access controls and auditing. Cloud Storage encrypts user data at rest using AES-256, and each encryption key is itself encrypted with a regularly rotated set of master keys. There is no setup or configuration required, no need to modify the way you access the service, and no visible performance impact. Data is automatically and transparently decrypted when read by an authorized user.

To protect your data as it travels over the Internet during read and write operations, use Transport Layer Security, commonly known as TLS or HTTPS.

Customer-supplied encryption keys

For examples of using your own keys, see Using Customer-Supplied Encryption Keys.

As an alternative to a Google-managed server-side encryption key, you can choose to provide your own AES-256 key, encoded in standard Base64. This key is known as a customer-supplied encryption key. If you provide a customer-supplied encryption key, Cloud Storage does not permanently store your key on Google's servers or otherwise manage your key. Instead, you provide your key for each Cloud Storage operation, and your key is purged from Google's servers after the operation is complete. Cloud Storage stores only a cryptographic hash of the key so that future requests can be validated against the hash. Your key cannot be recovered from this hash, and the hash cannot be used to decrypt your data.

Customer-supplied encryption keys can apply to operations on an object that read or write data. Operations such as deleting or listing objects can be performed without providing the encryption key. Additionally, metadata for objects is not encrypted, with the exception of the CRC32C checksum and MD5 hash. This means that you can read or update most metadata of an encrypted object without providing the encryption key, though you still must have sufficient permissions on the object in question.

For example, if an object is encrypted with a customer-supplied encryption key, you must use the key to perform operations on the object such as downloading or moving it. If you attempt to read the object's metadata without providing the key, you receive metadata such as the object name and Content-Type, but not the object's CRC32C checksum or MD5 hash. If you do supply your key with the request for the object metadata, the object's CRC32C checksum and MD5 hash are included with the metadata.

If you are performing a compose operation on objects encrypted by customer-supplied encryption keys, the component objects must be encrypted by the same key, and you need to provide the key with the compose request. The resulting composite object is encrypted by the same key.

To protect your data as it travels over the Internet during read and write operations, use Transport Layer Security, commonly known as TLS or HTTPS. TLS is required when you provide an encryption key. If you accidentally use your encryption key over an unencrypted (HTTP) connection, it is possible for an attacker to intercept your key. Because of this possibility, the Cloud Storage API returns an error message warning you that your key may be compromised. If this occurs, you should immediately rotate your keys.

Restrictions

The following restrictions apply when using customer-supplied encryption keys:

  • Cloud Storage Transfer Service, Cloud Dataflow, and Cloud Dataproc do not currently support objects encrypted with customer-supplied encryption keys.

  • You cannot use the Google Cloud Platform Console to download objects that are encrypted with a customer-supplied encryption key. Similarly, when you use the Google Cloud Platform Console to upload an object, you cannot encrypt the object with a customer-supplied encryption key.

  • Customer-supplied encryption keys are available in the following countries:

    Argentina, Austria, Australia, Belgium, Bulgaria, Canada, Chile, Columbia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hong Kong, Hungary, Indonesia, Ireland, Israel, Italy, Japan, Lithuania, Luxembourg, Latvia, Malaysia, Malta, Mexico, Netherlands, New Zealand, Norway, Peru, Poland, Portugal, Romania, Singapore, Slovakia, South Africa, South Korea, Spain, Sweden, Switzerland, Taiwan, Thailand, Turkey, United Kingdom (UK), United States (US), Vietnam

    If your country is not on this list, you can request to add your country.

  • To use customer-supplied encryption keys with gsutil, you must have gsutil 4.18 or later.

Tools for using encryption keys

There are multiple ways that you can use customer-supplied encryption keys with Cloud Storage. These include:

  • Partner companies.
  • The JSON and XML REST APIs.
  • The gsutil command-line tool.

Encryption keys with partner companies

There are several third-party partner options for using customer-supplied encryption keys. These partners make it easier to generate an encryption key and to associate that key with an object in Cloud Storage. Partners that can supply keys for Cloud Storage include Ionic Security and KeyNexus.

To learn more, see the Google Cloud Storage Partners page.

Encryption keys with REST APIs

When you use a customer-supplied encryption key and work directly with the JSON or XML API, you must provide both the AES-256 key and a SHA256 hash of the key. You should store both the AES-256 key and the SHA256 hash of the key securely. Google stores the SHA256 hash of your key in the object's metadata, where you can retrieve it later. This SHA256 hash cannot be used by Google (or anyone else) to decrypt your data. It is stored as a way to uniquely identify the AES-256 key that was used to encrypt a particular object.

Request headers

Include the following HTTP headers in your JSON or XML request:

Header name Value Description
x-goog-encryption-algorithm string The encryption algorithm to use. You must use the value AES256.
x-goog-encryption-key string An RFC 4648 Base64-encoded string of your AES-256 encryption key.
x-goog-encryption-key-sha256 string An RFC 4648 Base64-encoded string of the SHA256 hash of your encryption key.

If you are performing a rewrite operation with the JSON API, the headers listed above are used for encrypting the destination object, and the following headers are used for decrypting the source object:

Header name Value Description
x-goog-copy-source-encryption-algorithm string The encryption algorithm to use. You must use the value AES256.
x-goog-copy-source-encryption-key string An RFC 4648 Base64-encoded string of the source object's AES-256 encryption key.
x-goog-copy-source-encryption-key-sha256 string An RFC 4648 Base64-encoded string of the SHA256 hash of the source object's encryption key.
Response

JSON

When using the JSON API, the metadata for a customer-supplied encryption key is returned in the response body, which includes the following properties:

Property nameValueDescription
customerEncryptionobjectInformation about the encryption used for the request.
customerEncryption.encryptionAlgorithmstringThe encryption algorithm that was used. Always contains the value AES256.
customerEncryption.keySha256stringAn RFC 4648 Base64-encoded string of the SHA256 hash of your encryption key. You can use this SHA256 hash to uniquely identify the AES-256 encryption key required to decrypt the object, which you must store securely.

XML

When using the XML API, the response includes the following headers:

Header nameValueDescription
x-goog-encryption-algorithmstringThe encryption algorithm that was used. Always contains the value AES256.
x-goog-encryption-key-sha256stringAn RFC 4648 Base64-encoded string of the SHA256 hash of your encryption key. You can use this SHA256 hash to uniquely identify the AES-256 encryption key required to decrypt the object, which you must store securely.

You receive an HTTP 400 error in the following cases:

  • You upload an object using a customer-supplied encryption key, and you attempt to perform another operation on the object (other than requesting or updating most metadata or deleting the object) without providing the key.
  • You upload an object using a customer-supplied encryption key, and you attempt to perform another operation on the object with an incorrect key.
  • You upload an object without providing a customer-supplied encryption key, and you attempt to perform another operation on the object with a customer-supplied encryption key.
  • You specify an encryption algorithm, key, or SHA256 hash that is not valid.

Encryption keys with gsutil

To use a customer-supplied encryption key with gsutil, add the following option to the [GSUtil] section of your boto configuration file:

Option name Value Description
encryption_key string An RFC 4648 Base64-encoded string of your AES-256 encryption key.

You can optionally specify one or more decryption keys. While the encryption_key option is used by gsutil as both an encryption and decryption key, any decryption_key options you specify are used only to decrypt objects. For details, see the gsutil documentation.

As long as you have encryption or decryption keys in your boto configuration file, they are used for all gsutil commands. When decrypting, gsutil calculates the SHA256 hash of the supplied encryption and decryption keys and selects the correct key to use for a particular object by matching the SHA256 hash in the object's metadata.

You receive an error if you upload an object using a customer-supplied encryption key and then attempt to perform another operation on the object (other than requesting or updating metadata or deleting the object) without providing the key.

Encryption key rotation

If an object is encrypted using a customer-supplied encryption key, you can rotate the object's key by rewriting the object. Rewrites are supported through the JSON API, but not the XML API. See Rotating an encryption key for examples of key rotation.

Client-side encryption

Server-side encryption can be used in combination with client-side encryption. With client-side encryption, you manage your own client-side encryption keys and encrypt data before writing it to Cloud Storage. In this case, your data is encrypted twice: once with your client-side key and once with a server-side key. When you read an object, you must decrypt the object on the client side.

Send feedback about...

Cloud Storage Documentation