Managing Data Encryption

Google Cloud Storage always encrypts your data on the server side, before it is written to disk, at no additional charge. By default, Cloud Storage uses its own server-side encryption keys to encrypt data. Alternatively, you can provide your own encryption keys.

In addition, you can choose to encrypt data on the client side before you write it to Cloud Storage.

This page explains how to manage each type of data encryption.

Server-side encryption

This section explains how server-side encryption works with Cloud Storage.

Default behavior

By default, Google Cloud Storage manages server-side encryption keys on your behalf using the same hardened key management systems that we use for our own encrypted data, including strict key access controls and auditing. Cloud Storage encrypts user data at rest using AES-256, and each encryption key is itself encrypted with a regularly rotated set of master keys. There is no setup or configuration required, no need to modify the way you access the service, and no visible performance impact. Data is automatically and transparently decrypted when read by an authorized user.

To protect your data as it travels over the Internet during read and write operations, use Transport Layer Security, commonly known as TLS or HTTPS.

Customer-supplied encryption keys

You can also choose to provide your own AES-256 key for server-side encryption. This key is known as a customer-supplied encryption key. If you provide an encryption key, Cloud Storage does not permanently store your key on Google's servers or otherwise manage your key. Instead, you provide your key for each Cloud Storage operation, and your key is purged from Google's servers after the operation is complete. Cloud Storage stores only a cryptographic hash of the key so that future requests can be validated against the hash. Your key cannot be recovered from this hash, and the hash cannot be used to decrypt your data.

When an object is encrypted with a customer-supplied encryption key, you cannot retrieve the object's CRC32C checksum and MD5 hash unless you supply your key. In addition, the object's ETag contains a different value than the MD5 hash.

If you are performing a compose operation on objects encrypted by customer-supplied encryption keys, the component objects must be encrypted by the same key, and you need to provide the key with the compose request. The resulting composite object is encrypted by the same key.

To protect your data as it travels over the Internet during read and write operations, use Transport Layer Security, commonly known as TLS or HTTPS. TLS is required when you provide an encryption key.

Generating a customer-supplied encryption key

You can use the following Python command to generate a Base64-encoded AES-256 key, as well as the Base64-encoded SHA256 hash of the key:

python -c 'import base64; import hashlib; import os; key = os.urandom(32); \
    print "Key: %sSHA256 hash: %s" % (base64.encodestring(key), \
    base64.encodestring(hashlib.sha256(key).digest()))'

Using customer-supplied encryption keys with the JSON API

You can use your own customer-supplied encryption key for any operation that is supported by Cloud Storage.

Request

When you use a customer-supplied encryption key, you are required to provide both the AES-256 key and a SHA256 hash of the key. You should store both the AES-256 key and the SHA256 hash of the key securely. Google will store the SHA256 hash of your key in the object's metadata, where you can retrieve it later. This SHA256 hash cannot be used by Google (or anyone else) to decrypt your data. It is stored as a way for you to uniquely identify the AES-256 key that was used to encrypt a particular object.

To use a customer-supplied encryption key, include the following HTTP headers in your request:

Header name Value Description
x-goog-encryption-algorithm string The encryption algorithm to use. You must use the value AES256.
x-goog-encryption-key string An RFC 4648 Base64-encoded string of your AES-256 encryption key.
x-goog-encryption-key-sha256 string An RFC 4648 Base64-encoded string of the SHA256 hash of your encryption key.

If you are performing a rewrite operation, you must also include the following HTTP headers in your request. The headers listed above will be used for the destination object, and the following headers will be used for the source object.

Header name Value Description
x-goog-copy-source-encryption-algorithm string The encryption algorithm to use. You must use the value AES256.
x-goog-copy-source-encryption-key string An RFC 4648 Base64-encoded string of the object's original AES-256 encryption key.
x-goog-copy-source-encryption-key-sha256 string An RFC 4648 Base64-encoded string of the SHA256 hash of the object's original encryption key.

Cloud Storage will use your encryption key to encrypt or decrypt the object as necessary for the request.

Response

The metadata for the customer-supplied encryption key is returned in the response, which includes the following additional properties:

Property name Value Description
customerEncryption object Information about the encryption used for the request.
customerEncryption.encryptionAlgorithm string The encryption algorithm that was used. Always contains the value AES256.
customerEncryption.keySha256 string An RFC 4648 Base64-encoded string of the SHA256 hash of your encryption key. You can use this SHA256 hash to uniquely identify the AES-256 encryption key required to decrypt the object, which you must store securely.

You will receive an HTTP 400 error in the following cases:

  • You upload an object using a customer-supplied encryption key, and you attempt to perform another operation on the object (other than requesting or updating metadata) without providing the key.
  • You upload an object using a customer-supplied encryption key, and you attempt to perform another operation on the object with an incorrect key.
  • You upload an object without providing a customer-supplied encryption key, and you attempt to perform another operation on the object with a customer-supplied encryption key.
  • You specify an encryption algorithm, key, or SHA256 hash that is not valid.
Sample code

The following sample code illustrates how to use customer-supplied encryption keys to upload an object:

Node.js

For more on installing and creating a Cloud Storage client, refer to Cloud Storage Client Libraries.

function uploadEncryptedFile (bucketName, srcFileName, destFileName, key) {
  // Instantiates a client
  const storageClient = Storage();

  // References an existing bucket, e.g. "my-bucket"
  const bucket = storageClient.bucket(bucketName);

  const config = {
    // The path to which the file should be uploaded, e.g. "file_encrypted.txt"
    destination: destFileName,
    // Encrypt the file with a customer-supplied key, e.g. "my-secret-key"
    encryptionKey: new Buffer(key, 'base64')
  };

  // Encrypts and uploads a local file, e.g. "./local/path/to/file.txt".
  // The file will only be retrievable using the key used to upload it.
  return bucket.upload(srcFileName, config)
    .then((results) => {
      const file = results[0];

      console.log(`File ${srcFileName} uploaded to ${file.name}.`);

      return file;
    });
}

PHP

For more on installing and creating a Cloud Storage client, refer to Cloud Storage Client Libraries.

use Google\Cloud\Storage\StorageClient;

/**
 * Upload an encrypted file.
 *
 * @param string $bucketName the name of your Google Cloud bucket.
 * @param string $objectName the name of your Google Cloud object.
 * @param resource $source the path to the file to upload.
 * @param string $base64EncryptionKey the base64 encoded encryption key.
 *
 * @return void
 */
function upload_encrypted_object($bucketName, $objectName, $source, $base64EncryptionKey)
{
    $storage = new StorageClient();
    $file = fopen($source, 'r');
    $bucket = $storage->bucket($bucketName);
    $object = $bucket->upload($file, [
        'name' => $objectName,
        'encryptionKey' => $base64EncryptionKey,
    ]);
    printf('Uploaded encrypted %s to gs://%s/%s' . PHP_EOL,
        basename($source), $bucketName, $objectName);
}

Python

For more on installing and creating a Cloud Storage client, refer to Cloud Storage Client Libraries.

def upload_encrypted_blob(bucket_name, source_file_name,
                          destination_blob_name, base64_encryption_key):
    """Uploads a file to a Google Cloud Storage bucket using a custom
    encryption key.

    The file will be encrypted by Google Cloud Storage and only
    retrievable using the provided encryption key.
    """
    storage_client = storage.Client()
    bucket = storage_client.get_bucket(bucket_name)
    # Encryption key must be an AES256 key represented as a bytestring with
    # 32 bytes. Since it's passed in as a base64 encoded string, it needs
    # to be decoded.
    encryption_key = base64.b64decode(base64_encryption_key)
    blob = Blob(destination_blob_name, bucket, encryption_key=encryption_key)

    blob.upload_from_filename(source_file_name)

    print('File {} uploaded to {}.'.format(
        source_file_name,
        destination_blob_name))

Using customer-supplied encryption keys with the XML API

You can use your own customer-supplied encryption key for any operation that is supported by Cloud Storage.

Request

When you use a customer-supplied encryption key, you are required to provide both the AES-256 key and a SHA256 hash of the key. You should store both the AES-256 key and the SHA256 hash of the key securely. Google will store the SHA256 hash of your key in the object's metadata, where you can retrieve it later. This SHA256 hash cannot be used by Google (or anyone else) to decrypt your data. It is stored as a way for you to uniquely identify the AES-256 key that was used to encrypt a particular object.

To use a customer-supplied encryption key, include the following HTTP headers in your request:

Header name Value Description
x-goog-encryption-algorithm string The encryption algorithm to use. You must use the value AES256.
x-goog-encryption-key string An RFC 4648 Base64-encoded string of your AES-256 encryption key.
x-goog-encryption-key-sha256 string An RFC 4648 Base64-encoded string of the SHA256 hash of your encryption key.

Cloud Storage will use your encryption key to encrypt or decrypt the object as necessary for the request.

Response

The response includes the following headers:

Header name Value Description
x-goog-encryption-algorithm string The encryption algorithm that was used. Always contains the value AES256.
x-goog-encryption-key-sha256 string An RFC 4648 Base64-encoded string of the SHA256 hash of your encryption key. You can use this SHA256 hash to uniquely identify the AES-256 encryption key required to decrypt the object, which you must store securely.

You will receive an HTTP 400 error in the following cases:

  • You upload an object using a customer-supplied encryption key, and you attempt to perform another operation on the object (other than requesting or updating metadata) without providing the key.
  • You upload an object using a customer-supplied encryption key, and you attempt to perform another operation on the object with an incorrect key.
  • You upload an object without providing a customer-supplied encryption key, and you attempt to perform another operation on the object with a customer-supplied encryption key.
  • You specify an encryption algorithm, key, or SHA256 hash that is not valid.

Using customer-supplied encryption keys with gsutil

You can use your own customer-supplied encryption key for any operation that is supported by gsutil.

Installing a current version of gsutil

To use a customer-supplied encryption key with gsutil, you need to install gsutil 4.18 or later.

Updating the boto configuration file

To use a customer-supplied encryption key, add the following option to your boto configuration file:

Option name Value Description
encryption_key string An RFC 4648 Base64-encoded string of your AES-256 encryption key.

You can also specify multiple decryption keys, which will only be used to decrypt objects. (The encryption key is also used as a decryption key.) For details, see the gsutil documentation. After you specify encryption or decryption keys in your boto configuration file, they will be used for all subsequent gsutil commands.

You will receive an error if you upload an object using a customer-supplied encryption key, and you attempt to perform another operation on the object (other than requesting or updating metadata) without providing the key.

Rotating customer-supplied encryption keys

If an object is encrypted using a customer-supplied encryption key, you can rotate the object's key by rewriting the object. Rewrites are supported only by the JSON API, not by the XML API.

If you accidentally use your encryption key over an unencrypted (HTTP) connection, it is possible for an attacker to intercept your key. When this happens, the Cloud Storage API returns an error message warning you that your key may be compromised. You should immediately rotate your keys.

Restrictions on customer-supplied encryption keys

Customer-supplied encryption keys are available only in the following countries:

Australia, Belgium, Canada, Columbia, Denmark, Finland, France, Germany, Ireland, Israel, Italy, Japan, Mexico, Netherlands, New Zealand, Norway, Sweden, Taiwan, United Kingdom (UK), United States (US)

If your country is not on this list, you can request to add your country.

Client-side encryption

Server-side encryption can be used in combination with client-side encryption. With client-side encryption, you manage your own client-side encryption keys and encrypt data before writing it to Cloud Storage. In this case, your data is encrypted twice: once with your client-side key and once with a server-side key. When you read an object, you must decrypt the object on the client side.

Send feedback about...

Cloud Storage Documentation