Verifying end-to-end data integrity

This page discusses using fields in the Cloud Key Management Service API to detect and prevent unintended changes to data as it moves between client systems and Cloud KMS. These guidelines supplement the ways that Google Cloud automatically protects your data at rest and in transit.

Automatic data protection

All data on Google Cloud is automatically encrypted in transit and at rest within Google Cloud.

Cloud KMS takes extra steps to protect encryption keys at rest by encrypting each encryption key using another cryptographic key called a key encryption key (KEK). To learn more about this technique, refer to envelope encryption.

Each Cloud KMS cryptographic operation includes automatic checks for data corruption. If corruption is detected, the operation is aborted and a detailed error is logged.

These automatic protections are important, but they don't prevent client-side data corruption. For example, data corruption during encryption can lead to data that can't be decrypted.

This topic discusses ways to detect and prevent client-side data corruption to minimize the risk of data loss.

Calculating and verifying checksums

Each cryptographic operation's request and response include CRC32C checksum fields, such as plaintext_crc32c. You can calculate the checksum and compare the calculated and returned values.

On the server, the Cloud Key Management Service API sets boolean fields, such as verified_plaintext_crc32c, to true to indicate that it received the relevant checksum, and returns an INVALID_ARGUMENT error if the checksum doesn't match the value calculated by the server.

Keep the following guidelines in mind when calculating and comparing checksums:

  • Calculate checksums, using a binary encoding, as soon as data is sent or received.
  • Do not store unencrypted plaintext checksums to or from the Cloud Key Management Service API. For example, do not store the unencrypted contents of the EncryptRequest.plaintext_crc32c or DecryptResponse.plaintext_crc32c fields. To keep a record of a checksum, serialize it along with the relevant data and pass the compound object to Cloud KMS for encryption. When you decrypt data, you can verify the checksum against your calculated expected value.
  • If a decryption operation results in mismatched checksums, design your application to retry the operation a limited number of times, in case of transient problems.
  • If an encryption operation results in mismatched checksums for the encrypted data (the ciphertext), discard the encrypted result and try again.

For encryption requests:

  • If you include the checksum field in the request, the server sets a related verification field in the response to true to indicate that it received the checksum and attempted to verify its value.

  • If you include the checksum field but the response sets the verification field to false, the server did not receive the checksum field, and other information may be missing from the response. Retry the request a limited number of times in case the error is transient.

  • If the checksum was received but did not match, an INVALID_ARGUMENT error is returned. The error includes the name of the field and the checksum that didn't match. For example:

    The checksum in field plaintext_crc32c did not match the data in field plaintext.
    

    Retry the request a limited number of times in case the error is transient.

For decryption requests, you can calculate a checksum and compare it to the value of DecryptResponse.plaintext_crc32c in the response.

The following checksum fields are included in requests to and responses from the Cloud Key Management Service API.

Field in request Field in response
EncryptRequest.plaintext_crc32c EncryptResponse.verified_plaintext_crc32c
EncryptionRequest.additional_authenticated_data_crc32c EncryptionResponse.verified_additional_authenticated_data_crc32c
AsymmetricSignRequest.digest_crc32c AsymmetricSignResponse.verified_digest_crc32c
AsymmetricSignRequest.signature_crc32c AsymmetricSignResponse.verified_signature_crc32c
AsymmetricSignRequest.signature_crc32c AsymmetricSignResponse.verified_signature_crc32c
GetPublicKeyRequest.pem_crc32c GetPublicKeyResponse.verified_pem_crc32c
Not applicable DecryptResponse.plaintext_crc32c
MacSignRequest.data_crc32c MacSignResponse.verified_data_crc32c
MacVerifyRequest.data_crc32c MacVerifyResponse.verified_data_crc32c
MacVerifyRequest.mac_crc32c MacVerifyResponse.verified_mac_crc32c
Not applicable MacVerifyResponse.verified_success_integrity

Verifying resource names

Responses for Encrypt, GetPublicKey, AsymmetricSign, MacSign, and MacVerify, include a name field that contains the name of the relevant Cloud Key Management Service API object. You can compare the value of the name field to the value you expect, and discard results that do not match.

Data verification diagram

This diagram shows when to verify each type of data related to cryptographic operations and when to verify data from each type of source. You can also view a summary of the data verification fields.

Diagram showing different data verification fields and the actions they correspond to

Data verification field summary

Use this tables when designing your application, to determine which fields you can use to verify your data before and after each cryptographic operation.

Encrypt

API Server-side input verification Client-side resource verification Client-side output verification Client-side verification of server-side input
EncryptRequest plaintext_crc32c, additional_authenticated_data_crc32c
EncryptResponse name ciphertext_crc32c verified_plaintext_crc32c, verified_additional_authenticated_data_crc32c

Decrypt

API Server-side input verification Client-side resource verification Client-side output verification Client-side verification of server-side input
DecryptRequest ciphertext_crc32c, additional_authenticated_data_crc32c
DecryptResponse plaintext_crc32c

AsymmetricSign

API Server-side input verification Client-side resource verification Client-side output verification Client-side verification of server-side input
AsymmetricSignRequest digest_crc32c
AsymmetricSignResponse name signature_crc32c verified_digest_crc32c

AsymmetricDecrypt

API Server-side input verification Client-side resource verification Client-side output verification Client-side verification of server-side input
AsymmetricDecryptRequest ciphertext_crc32c
AsymmetricDecryptResponse plaintext_crc32c verified_ciphertext_crc32c

PublicKey

API Server-side input verification Client-side resource verification Client-side output verification Client-side verification of server-side input
PublicKey name pem_crc32c

MacSign

API Server-side input verification Client-side resource verification Client-side output verification Client-side verification of server-side input
MacSignRequest data_crc32c
MacSignResponse mac_crc32c verified_data_crc32c

MacVerify

API Server-side input verification Client-side resource verification Client-side output verification Client-side verification of server-side input
MacVerifyRequest data_crc32c mac_crc32c
MacVerifyResponse verified_data_crc32c verified_mac_crc32c verified_success_integrity

What's next