Dataproc Confidential Compute

You can create a Dataproc cluster that uses Compute Engine Confidential VMs to provide inline memory encryption. Confidential VMs use the N2D machine type (with AMD Secure Encrypted Virtualization (SEV)).

Create a cluster with confidential VMs

gcloud command

To create a Dataproc cluster that uses confidential VMs, use the gcloud dataproc clusters create command with the --confidential-compute, flag.

Requirements:

  • The master and worker instances must use the N2D machine type (with AMD Secure Encrypted Virtualization (SEV)).
  • The cluster must use one of the supported Ubuntu images.
  • The cluster must be created in a region and Compute Engine zone that supports the AMD EPYC Rome CPU (N2D machine type) used by confidential VMs (see the CPUs column in Available regions and zones). You can run the following command to list the CPUs supported in a Compute Engine zone: 
    gcloud compute zones describe ZONE_NAME --format="value(availableCpuPlatforms)"
      
gcloud dataproc clusters create cluster-name \  
    --confidential-compute \  
    --image-version=Ubuntu image version \
    --region=region with zone that supports the AMD EPYC Rome CPU \
    --zone=zone within the region that supports the AMD EPYC Rome CPU \
    --master-machine-type=N2D machine type \  
    --worker-machine-type=N2D machine type" \  
    other args ...

REST API

To create a Dataproc cluster that uses confidential VMs, include the ConfidentialInstanceConfig as part of a clusters.create request. Set enableConfidentialCompute to true.

Requirements:

  • masterConfig.machineTypeUri masterConfig.machineTypeUri, and, if applicable, secondaryWorkerConfig.machineTypeUri: Master and worker instances must use the N2D machine type (with AMD Secure Encrypted Virtualization (SEV)).
  • softwareConfig.imageVersion: The cluster must use one of the supported Ubuntu images.
  • gceClusterConfig.zoneUri: The cluster must be created in a Compute Engine zone that supports the N2D AMD EPYC Rome CPU used by confidential VMs (see the CPUs column in Available regions and zones). You can run the following command to list the CPUs supported in a Compute Engine zone: 
    gcloud beta compute zones describe "ZONE_NAME --format="value(availableCpuPlatforms)"