Application-level encryption: Memorystore for Redis

This document gives Redis users on Google Cloud an overview of encryption methods that help to mitigate malicious attacks without sacrificing the speed and flexibility of Redis. In particular, this document describes how to help secure sensitive data by implementing envelope encryption using Memorystore and Cloud Key Management Service (Cloud KMS)—an approach to security that offers minimal impact on performance and provides flexibility with key rotation.

In the tutorial that's associated with this document, you apply this approach by creating an application that communicates with the Cloud Key Management Service API in order to encrypt content stored in a Memorystore for Redis store.

This document is intended for application developers and security professionals, and it assumes that you have basic knowledge of Java programming and security concepts. All the code snippets in this document are written in Java.

Memorystore for Redis

Memorystore for Redis provides a fully managed service that's powered by Redis's in-memory datastore to build application caches for sub-millisecond data access.

Redis's core security model relies on keeping its instances within an environment that only trusted clients can access directly. This security model implements only basic security concepts, such as locking down access to network interfaces, requiring basic authentication, and obfuscating command names. Private IPs and Identity and Access Management (IAM) role-based access control provide further security and protection for Redis instances. Standard high availability instances are always replicated across zones and provide 99.9%-availability service level agreements (SLAs).

Redis use cases for sub-millisecond data access

Redis has many applications for modern security architectures. For many enterprises, a basic security architecture is sufficient to meet most security requirements. However, some industries have higher security requirements.

For example, consider a machine learning application in the financial services industry. Financial transactions contain account details that need to be cached for low-latency inferencing. Because of the low-latency requirements, the data can't be tokenized, masked, or anonymized. Therefore, it's important to create additional layers of security to help protect data and reduce the risk of exposure even from unlikely attack vectors.

Considerations for using envelope encryption

The general concept of envelope encryption is to combine encryption in layers. In this process, you encrypt data with one or more data encryption keys (DEKs). You then encrypt the DEKs (not the raw data) with a key encryption key (KEK). The following diagram illustrates this process.

Architecture of envelope encryption.

Envelope encryption simplifies the process of managing data encryption at scale. It also reduces the overhead of key rotation, because you only rotate the KEK, not the DEK. In this approach, you encrypt the raw data only once, using the DEK. Re-encrypting the DEK presents a much lower processing load than re-encrypting the entire contents of the Redis database. If you refresh the DEK key during a rotation cycle, then you need to reprocess the entire encrypted data set. However, with envelope encryption, you use the best practices of key rotation while reducing the overhead of reprocessing an entire corpus of encrypted data.

The following sections discuss (with code snippets) how you can implement envelope encryption using Cloud KMS, the Tink library, and Memorystore. The code samples in this document are from the Git repository for the tinkCryptoHelper application.

Key management with Tink and Redis

For encryption at the application layer, Google provides Cloud Key Management Service (Cloud KMS), which lets you create, import, and manage cryptographic keys and perform cryptographic operations in a centralized cloud service. You can use these keys to perform encryption operations by using Cloud KMS directly, Cloud HSM, or Cloud External Key Manager (Cloud EKM), or by using customer-managed encryption key (CMEK) integrations within other Google Cloud services.

Cloud KMS is designed around a hierarchical layer consisting of key rings that hold keys. Each key can have multiple versions, one of which has the primary attribute. Key rings are assigned to a region and are part of a project from which they inherit permissions.

A key is addressed through a URI that reflects this hierarchy—for example, in the following pattern:

gcp-kms://projects/PROJECT_ID/locations/LOCATION/keyRings/KEYRING_ID/cryptoKeys/KEY_ID

The variables in this pattern are defined as follows:

If you have additional compliance requirements, Google Cloud also offers Cloud HSM, a cloud-hosted Hardware Security Module (HSM) service—for example, if you need to perform cryptographic operations within a hardware environment certified by FIPS 140-2 Level 3.

By default, Google Cloud encrypts customer content that's stored at rest (with some minor exceptions) using envelope encryption along with an internal key management service (KMS).

The Tink library

The approach described in this document uses the Tink library and Cloud KMS to manage encryption. Tink is a multi-language, cross-platform library that provides cryptographic APIs that are written by a group of cryptographers and security engineers at Google. The aim of Tink is to provide a secure API that's straightforward to use and reduces common cryptographic pitfalls. For more details about Tink's design considerations, you can listen to this Real World Crypto talk and view the accompanying slides.

The key management lifecycle

Before you start to encrypt and decrypt working data, you must first configure components of the key management lifecycle for the Redis client application. The following diagram provides an overview of the relationship between the keys and the data.

Components of the KMS lifecycle include keys and IAM
roles.

In this lifecycle, all data that's stored in the Redis database is encrypted with a single DEK. The first step in the process is to generate a DEK. Tink organizes collections of keys into keysets, which are then wrapped by a KeysetHandle object, as the following code sample shows:

KeysetHandle k = KeysetHandle.generateNew
    (AeadKeyTemplates.createAesGcmKeyTemplate(256 / 8));

All keys in a keyset correspond to a single primitive. As a best practice, this document and the associated tutorial use AES in Galois/Counter Mode (GCM).

To enable access to the DEK, you must securely store the DEK. For this document, you use Cloud KMS to encrypt the DEK with a KEK. In the Tink library, the KEK that Cloud KMS uses is a key that has a specific version, belongs to a specific key ring, resides in a specific location, and is associated with a specific project, as illustrated by the URI earlier in this document.

It's good practice to create the Cloud KMS KEK in a project that's isolated from other Google Cloud resources. By following this practice, you can ensure that only users or service accounts with the Owner IAM role on that project can access the keys in Cloud KMS and decrypt data. For best practices on managing permissions and access, see Separation of duties.

To provide access to the KEK in Cloud KMS, you use a service account that has permissions to perform encrypt and decrypt operations on the KEK. For more information, see Choosing the right IAM roles. You can obtain the credentials for the service account either through the Google Cloud Console or through Cloud Shell by using the gcloud iam service-accounts keys create command. To access the KMS using Tink, the service account credentials (kmsCredentialsFilename) are provided to instantiate a Cloud KMS client, as the following code sample shows:

KmsClient kmsClient = new
    GcpKmsClient().withCredentials(kmsCredentialsFilename);

In KMS, the keyset that has the DEK encrypted by the KEK is written to a file (in this case, to a JSON file on the local file system):

k.write(JsonKeysetWriter.withFile(new File(keysetFilename)),
    kmsClient.getAead(keyResourceIdUri));

Encrypting and decrypting Redis data

After you initiate the key management lifecycle and securely store the DEK, the Redis client application can use the DEK to encrypt and decrypt Redis values, as the following diagram shows.

Memorystore for Redis uses the DEK to perform cryptographic processes on
data.

Using Tink to perform encryption and decryption operations with the stored DEK works as follows:

  1. The Aead primitive is retrieved from the KeysetHandle object.
  2. The encrypt (or decrypt) method is executed, providing data to be encrypted.

The following snippet shows the code that performs encryption and decryption functions:

Aead aead = k.getPrimitive(Aead.class);
byte[] ciphertext = aead.encrypt(clearText.getBytes("UTF-8"), aa);
String decryptedText = new String(aead.decrypt(ciphertext, aa), "UTF-8");

The associated data (aa) that's passed to the Aead.encrypt method (authenticated encryption with associated data) binds encryption to a context (such as the ID of a record). Associated data can be considered the same as the salt argument that's used in password encryption. This approach helps ensure the authenticity and integrity of the data.

For this Redis client application, the associated data might be a constant defined in the module that performs encryption and decryption. When the Redis client application starts, the DEK needs to be loaded and decrypted with the KEK from Cloud KMS in order to encrypt the data to be stored in Redis.

Using Tink, you load the DEK keyset, which was created and stored during the initialization process. Tink handles the decryption by using the KEK that's provided through the key URI from Cloud KMS, as the following code snippet shows:

File keyset = new File(keysetFilename);
if (keyset.exists()) {
    KeysetHandle k =
        KeysetHandle.read(JsonKeysetReader.withFile(keyset),
            kmsClient.getAead(keyResourceIdUri));

    Aead aead = k.getPrimitive(Aead.class);
    String decText = new String(aead.decrypt(ciphertext, aa));
}

Blending encryption with Redis

The previous section describes using Tink to encrypt and decrypt data without reference to the Redis database. Integrating the encryption with Redis is straightforward. In the tutorial associated with this document, you use Jedis as the client Java library. You can pass and encrypt key-value pairs by using only a few lines of code. You can apply this code to any data storage. The following code sample demonstrates how to apply this code in Redis:

void set(Iterator<Map.Entry<String, String>> kvs) throws
             NullPointerException, GeneralSecurityException, IOException {
    try (Pipeline p = jedisPool.getResource().pipelined()) {
      while (kvs.hasNext()) {
        Map.Entry<String, String> r = kvs.next();
        p.set(r.getKey(), cryptoHelper.encrypt(r.getValue()));
      }
      p.sync();
    }
}

List<String> get(String... keys) throws UnsupportedEncodingException, NullPointerException,
GeneralSecurityException, IOException {
    Jedis j = jedisPool.getResource();
    List<String> values = j.mget(keys);
    List<String> l = new ArrayList<String>();
    for (String v : values) {
      l.add(cryptoHelper.decrypt(v));
    }
    return l;
}

The keys remain in clear text and only the values are encrypted with the DEK. Consequently, it's important not to embed any sensitive information into the Redis keys. One drawback with encrypting the values is that a nested value structure can't be queried through native Redis functions. You need to account for this characteristic when you design a data access layer.

What's next