Using customer-managed encryption keys (CMEK)

By default, Google Cloud automatically encrypts data when it is at rest using encryption keys managed by Google. If you have specific compliance or regulatory requirements related to the keys that protect your data, you can use customer-managed encryption keys (CMEK) for your AI Platform Notebooks instances.

You can read more about the specific benefits of using CMEK with AI Platform Notebooks in the following section of this guide. For more information about CMEK in general, including when and why to enable it, see the Cloud Key Management Service documentation.

This guide describes some benefits of using CMEK for AI Platform Notebooks and describes how to configure a new AI Platform Notebooks instance to use CMEK.

Understanding CMEK for AI Platform Notebooks

The following sections describe basic information about CMEK for AI Platform Notebooks that you must understand before configuring CMEK for your AI Platform Notebooks instances.

Benefits of CMEK

In general, CMEK is most useful if you need full control over the keys used to encrypt your data. With CMEK, you can manage your keys within Cloud KMS. For example, you can rotate or disable a key or you can set up a rotation schedule using the Cloud KMS API. For more information about CMEK in general, including when and why to enable it, see the Cloud KMS documentation.

When you run an AI Platform Notebooks instance, your instance runs on a virtual machine (VM) managed by AI Platform Notebooks. When you enable CMEK for an AI Platform Notebooks instance, the key that you designate, rather than a key managed by Google, is used to encrypt data on the boot disks of the VM.

The CMEK key does not encrypt metadata associated with your AI Platform Notebooks instance, like the instance's name and region. Metadata associated with AI Platform Notebooks instances is always encrypted using Google's default encryption mechanism.

Using CMEK with other Google Cloud products

Configuring CMEK for AI Platform Notebooks does not automatically configure CMEK for other Google Cloud products that you use together with AI Platform Notebooks. To use CMEK to encrypt data in other Google Cloud products, additional configuration is required.

Limitations

Regional AI Platform Notebooks instances can be encrypted by keys in the same location or in the global location. For example, a disk in zone us-west1-a can be encrypted by a key in us-west1 or global. Global instances can be encrypted by keys in any location.

Configuring CMEK for your AI Platform Notebooks instance

The following sections describe how to create a key ring and key in Cloud Key Management Service, grant the service account encrypter and decrypter permissions for your key, and create an AI Platform Notebooks instance that uses CMEK.

Before you begin

This guide assumes that you use two separate Google Cloud projects to configure CMEK for AI Platform Notebooks:

  • A project for managing your encryption key (referred to as the "Cloud KMS project").
  • A project for accessing AI Platform Notebooks and interacting with any other Google Cloud products that you need for your use case (referred to as the "AI Platform Notebooks project").

This recommended setup supports a separation of duties.

Alternatively, you can use a single Google Cloud project for the whole guide. To do so, use the same project for all of the following tasks that refer to the Cloud KMS project and the tasks that refer to the AI Platform Notebooks project.

Setting up the Cloud KMS project

  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
  2. In the Google Cloud Console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  3. Make sure that billing is enabled for your Cloud project. Learn how to confirm that billing is enabled for your project.

  4. Enable the Cloud KMS API.

    Enable the API

Setting up the AI Platform Notebooks project

  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
  2. In the Google Cloud Console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  3. Make sure that billing is enabled for your Cloud project. Learn how to confirm that billing is enabled for your project.

  4. Enable the AI Platform Notebooks API.

    Enable the API

Setting up the gcloud command-line tool

The gcloud tool is required for some steps in this guide and optional for others. Install and initialize the Cloud SDK.

Creating a key ring and key

Follow the Cloud KMS guide to creating symmetric keys to create a key ring and a key. When you choose your key ring's location, use either global or the location where your AI Platform Notebooks instance will be. Make sure to create your key ring and key in your Cloud KMS project.

Granting AI Platform Notebooks permissions

To use CMEK for your AI Platform Notebooks instance, you must grant AI Platform Notebooks permission to encrypt and decrypt data using your key. AI Platform Notebooks uses a Google-managed service account to run your AI Platform Notebooks instance. This service account is identified by an email address with the following format:

service-AI_PLATFORM_NOTEBOOKS_PROJECT_NUMBER@compute-system.iam.gserviceaccount.com

To find the appropriate service account for your AI Platform Notebooks project, go to the IAM page in the Google Cloud Console and find the member that matches this email address format, with the project number for your AI Platform Notebooks project replacing the AI_PLATFORM_NOTEBOOKS_PROJECT_NUMBER variable. The service account also has the name Compute Engine default service account.

Go to the IAM page

Make note of the email address for this service account, and use it in the following steps to grant it permission to encrypt and decrypt data using your key. You can grant permission by using the Google Cloud Console or by using the gcloud command-line tool:

Cloud Console

  1. In the Cloud Console, go to the Cryptographic Keys page and select your Cloud KMS project.

    Go to the Cryptographic Keys page

  2. Click on the name of the key ring that you created in a preceding section of this guide to go to the Key ring details page.

  3. Select the checkbox for the key that you created in a preceding section of this guide. If an info panel labeled with the name of your key is not already open, click Show info panel.

  4. In the info panel, click Add member to open the Add members to "KEY_NAME" dialog. In this dialog, do the following:

    1. In the New members box, enter the service account email address that you made a note of in the preceding section: service-AI_PLATFORM_NOTEBOOKS_PROJECT_NUMBER@compute-system.iam.gserviceaccount.com
    2. In the Select a role drop-down list, click Cloud KMS and then select the Cloud KMS CryptoKey Encrypter/Decrypter role.

    3. Click Save.

gcloud

Run the following command:

gcloud kms keys add-iam-policy-binding KEY_NAME \
  --keyring=KEY_RING_NAME \
  --location=REGION \
  --project=KMS_PROJECT_ID \
  --member=serviceAccount:service-AI_PLATFORM_NOTEBOOKS_PROJECT_NUMBER@compute-system.iam.gserviceaccount.com \
  --role=roles/cloudkms.cryptoKeyEncrypterDecrypter

In this command, replace the following placeholders:

  • KEY_NAME: The name of the key that you created in a preceding section of this guide.
  • KEY_RING_NAME: The key ring that you created in a preceding section of this guide.
  • REGION: The region where you created your key ring.
  • KMS_PROJECT_ID: The ID of your Cloud KMS project.
  • AI_PLATFORM_NOTEBOOKS_PROJECT_NUMBER: The project number of your AI Platform Notebooks project, which you noted in the preceding section as part of a service account email address.

Creating an AI Platform Notebooks instance with CMEK

Now that you have granted AI Platform Notebooks permission to encrypt and decrypt data using your key, you can create an AI Platform Notebooks instance that encrypts data using this key.

The following examples show how to do this using the Google Cloud Console and gcloud tool.

Cloud Console

To use the Cloud Console to create an AI Platform Notebooks instance with a customer-managed encryption key, use the following steps:

  1. Go to the AI Platform Notebooks page in the Google Cloud Console.

    Go to the AI Platform Notebooks page

  2. Click New Instance, and then select Customize instance.

    Create new AI Platform Notebooks instance with options

  3. On the New notebook instance page, provide the following information for your new instance:

    • Instance name: Provide a name for your new instance.
    • Region: Enter the region that your key and key ring are in.
    • Zone: Select a zone within the region that you selected.
    • Environment: Select the environment and operating system that you want to use.
    • Machine type: Select the number of CPUs and amount of RAM for your new instance.
    • GPUs: Select the GPU type and Number of GPUs for your new instance.

      Select the option to Install NVIDIA GPU driver automatically for me.

  4. To change the default encryption settings, expand the Disk(s) section.

  5. Under Encryption, select Customer-managed key to use customer-managed encryption keys.

  6. Click Select a customer-managed key. If the customer-managed key that you want to use is in the drop-down list, select it. If not, enter the resource ID for your customer-managed key. The resource ID for your customer-managed key looks like this:

    projects/AI_PLATFORM_NOTEBOOKS_PROJECT_NUMBER/locations/global/keyRings/KEY_RING_NAME/cryptoKeys/KEY_NAME

  7. Click Create.

  8. AI Platform Notebooks creates a new instance based on your specified properties. An Open JupyterLab link becomes active when it's ready to use.

gcloud

To use the gcloud tool to create an AI Platform Notebooks instance with a customer-managed encryption key, run the following command. This example assumes that you want to create an AI Platform Notebooks instance with an n1-standard-1 machine type and a standard 100 GB standard persistent boot disk.

gcloud notebooks instances create notebook-vm-cmek \
  --location=REGION \
  --vm-image-family=IMAGE_FAMILY \
  --vm-image-project=deeplearning-platform-release \
  --machine-type="n1-standard-1" \
  --boot-disk-type="PD_STANDARD" \
  --boot-disk-size=100 \
  --kms-key=KEY_NAME \
  --kms-project=KMS_PROJECT_ID \
  --kms-location=REGION \
  --kms-keyring=KEY_RING_NAME \
  --disk-encryption=CMEK \
  --metadata='proxy-mode=project_editors'

In this command, replace the following placeholders:

  • REGION: The region where you created your key ring and where you plan to create your new AI Platform Notebooks instance.
  • IMAGE_FAMILY: The image family that you want to use to create your AI Platform Notebooks.
  • KEY_NAME: The name of the key that you created in a preceding section of this guide.
  • KMS_PROJECT_ID: The ID of your Cloud KMS project.
  • KEY_RING_NAME: The key ring that you created in a preceding section of this guide.

What's next