Use customer-managed encryption keys (CMEK) for runtimes

By default, Google Cloud automatically encrypts data when it is at rest using encryption keys managed by Google. If you have specific compliance or regulatory requirements related to the keys that protect your data, you can use customer-managed encryption keys (CMEK) for your resources.

This guide describes how to use CMEK for Colab Enterprise runtimes.

For more information about how to use CMEK for Vertex AI, see the Vertex AI CMEK page.

For more information about CMEK in general, including when and why to enable it, see the CMEK documentation.

CMEK for Colab Enterprise runtimes

The following sections describe basic information about CMEK for Colab Enterprise that you must understand before configuring CMEK for your runtimes.

When you run code in a Colab Enterprise notebook, a runtime runs the code on one or more virtual machine (VM) instances managed by Colab Enterprise. When you enable CMEK for your Colab Enterprise runtimes, the key that you designate, rather than a key managed by Google, is used to encrypt data on these VMs. The CMEK key encrypts the following kinds of data:

  • The copy of your code on the VMs.
  • Any data that gets loaded by your code.
  • Any temporary data that gets saved to the local disk by your code.

You can start, stop, and upgrade your runtime without affecting its CMEK encryption.

In general, the CMEK key does not encrypt metadata associated with your operation, like the runtime's name or your notebook's name and region. This metadata is always encrypted using Google's default encryption mechanism.

Supported keys

Colab Enterprise supports the following types of CMEK keys:

Key availability varies by key type and region. For more information about the geographical availability of CMEK keys, see Cloud KMS locations.

Restrictions and limitations

Colab Enterprise supports CMEK with the following restrictions and limitations:

  • Colab Enterprise notebook files are stored in Dataform, and using CMEK for those notebook files isn't currently supported.

  • The default quota in Vertex AI is one encryption key per project and region. If you need to register more than one key for a region in your project, contact your Google account team to request a quota increase for CMEK configurations, providing a justification for why you need more than one key.

Cloud KMS quotas and Colab Enterprise

When you use CMEK in Colab Enterprise, your projects can consume Cloud KMS cryptographic requests quotas. Encryption and decryption operations using CMEK keys affect Cloud KMS quotas only if you use hardware (Cloud HSM) or external (Cloud EKM) keys. For more information, see Cloud KMS quotas.

Configure CMEK for your runtimes

The following sections describe how to create a key ring and key in Cloud Key Management Service, grant Colab Enterprise encrypter and decrypter permissions for your key, and create a runtime template configures to use CMEK. Any runtime that Colab Enterprise generates from this runtime template uses CMEK encryption.

Before you begin

This guide assumes that you use two separate Google Cloud projects to configure CMEK for Colab Enterprise runtimes:

  • A project for managing your encryption key (referred to as the "Cloud KMS project").
  • A project for accessing your Colab Enterprise resources and interacting with any other Google Cloud products that you need (referred to as the "Notebook project").

This recommended setup supports a separation of duties.

Alternatively, you can use a single Google Cloud project for the whole guide. To do so, use the same project for all of the following tasks that refer to the Cloud KMS project and the tasks that refer to the Notebook project.

Set up the Cloud KMS project

  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

  2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  3. Make sure that billing is enabled for your Google Cloud project.

  4. Enable the Cloud KMS API.

    Enable the API

    5.

  5. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  6. Make sure that billing is enabled for your Google Cloud project.

  7. Enable the Cloud KMS API.

    Enable the API

    8.

Set up the Notebook project

  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

  2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  3. Make sure that billing is enabled for your Google Cloud project.

  4. Enable the Vertex AI API.

    Enable the API

    5.

  5. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  6. Make sure that billing is enabled for your Google Cloud project.

  7. Enable the Vertex AI API.

    Enable the API

    8.

Set up the Google Cloud CLI

The gcloud CLI is required for some steps in this guide and optional for others.

Install the Google Cloud CLI, then initialize it by running the following command: 

gcloud init

Create a key ring and key

Follow the Cloud KMS guide to creating symmetric keys to create a key ring and a key. When you create your key ring, specify a region that supports Colab Enterprise operations as the key ring's location. Colab Enterprise only supports CMEK when your runtime and key use the same region. You must not specify a dual-regional, multi-regional, or global location for your key ring.

Make sure to create your key ring and key in your Cloud KMS project.

Grant Colab Enterprise permissions

To use CMEK for your resources, you must grant Colab Enterprise permission to encrypt and decrypt data using your key. Colab Enterprise uses a Google-managed service agent to run operations using your resources. This service account is identified by an email address with the following format: service-PROJECT_NUMBER@gcp-sa-aiplatform.iam.gserviceaccount.com.

To find the appropriate service account for your Notebook project, go to the IAM page in the Google Cloud console and find the member that matches this email address format, with the project number for your Notebook project replacing the NOTEBOOK_PROJECT_NUMBER variable. The service account also has the name Vertex AI Service Agent.

Go to the IAM page

Make note of the email address for this service account, and use it in the following steps to grant it permission to encrypt and decrypt data using your key. You can grant permission by using the Google Cloud console or by using the Google Cloud CLI:

Google Cloud console

  1. In the Google Cloud console, Click Security and select Key Management. This will take you to Cryptographic Keys page and select your Cloud KMS project.

    Go to the Cryptographic Keys page

  2. Click on the name of the key ring that you created in a preceding section of this guide to go to the Key ring details page.

  3. Select the checkbox for the key that you created in a preceding section of this guide. If an info panel labeled with the name of your key is not already open, click Show info panel.

  4. In the info panel, click Add member to open the Add members to "KEY_NAME" dialog. In this dialog, do the following:

    1. In the New members box, enter the service account email address that you made a note of in the preceding section: service-NOTEBOOK_PROJECT_NUMBER@gcp-sa-aiplatform.iam.gserviceaccount.com

    2. In the Select a role drop-down list, click Cloud KMS and then select the Cloud KMS CryptoKey Encrypter/Decrypter role.

    3. Click Save.

gcloud

Run the following command:

gcloud kms keys add-iam-policy-binding KEY_NAME \
  --keyring=KEY_RING_NAME \
  --location=REGION \
  --project=KMS_PROJECT_ID \
  --member=serviceAccount:service-NOTEBOOK_PROJECT_NUMBER@gcp-sa-aiplatform.iam.gserviceaccount.com \
  --role=roles/cloudkms.cryptoKeyEncrypterDecrypter

In this command, replace the following placeholders:

  • KEY_NAME: The name of the key that you created in a preceding section of this guide.
  • KEY_RING_NAME: The key ring that you created in a preceding section of this guide.
  • REGION: The region where you created your key ring.
  • KMS_PROJECT_ID: The ID of your Cloud KMS project.
  • NOTEBOOK_PROJECT_NUMBER: The project number of your Notebook project, which you noted in the preceding section as part of a service account email address.

Configure a runtime template with the KMS key

When you create a new CMEK-supported resource you can specify your key as one of the create parameters. To create a Colab Enterprise runtime, you create a runtime template with your CMEK key specified as a parameter. Any runtime that Colab Enterprise generates from this runtime template uses CMEK encryption.

To create a runtime template by using the Google Cloud console, you specify the key in the Create new runtime template dialog. Do the following:

  1. In the Google Cloud console, go to the Colab Enterprise Runtime Templates page.

    Go to Runtime templates

  2. Click  New template.

    The Create new runtime template dialog appears.

  3. In the Configure compute section, in Encryption, select Cloud KMS key.

  4. For Key type, select Cloud KMS, and then, in the next field, select your customer-managed encryption key.

  5. Complete the rest of the instance creation dialog, and then click Create.

    Your runtime template appears in the list on the Runtime templates tab.

What's next