Enabling nested virtualization


This document describes how to enable nested virtualization on a virtual machine (VM) instance and how to confirm that you can create a nested VM. Enable nested virtualization on a VM by using one of the following methods:

  • Recommended. Enable nested virtualization directly on a new or existing VM by setting the enableNestedVirtualization field to true while creating the VM or by updating the VM. This is the recommended method because it doesn't require that you create a custom image or use the special license key.

  • Enable nested virtualization by using the special license key by creating a boot disk, creating a custom image with the special nested virtualization license key, and then creating a VM that uses the custom image.

Enabling nested virtualization directly on a new VM

Enable nested virtualization directly on a VM by using the following procedure.

gcloud

Create an L1 VM with nested virtualization enabled by using the following gcloud compute instances create command:

gcloud compute instances create VM_NAME \
  --enable-nested-virtualization \
  --zone=ZONE \
  --min-cpu-platform="Intel Haswell"

Replace the following:

  • VM_NAME: the name of the new L1 VM with nested virtualization enabled

  • ZONE: the zone for the new L1 VM with nested virtualization enabled

API

Create an L1 VM with nested virtualization enabled by using the following instances.insert method:

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances
{
  ...
  "name": "VM_NAME",
  ...
  "minCpuPlatform": "Intel Haswell",
  "advancedMachineFeatures": {
    "enableNestedVirtualization": true
  },
  ...
}

Replace the following:

  • PROJECT_ID: the project ID

  • ZONE: the zone for the new L1 VM with nested virtualization enabled

  • VM_NAME: the name of the new L1 VM with nested virtualization enabled

Enabling nested virtualization directly on an existing VM

Enable nested virtualization on an existing VM by using the following procedure.

gcloud

  1. Export the properties of the VM by using the following gcloud compute instances export command:

    gcloud compute instances export VM_NAME \
      --destination=YAML_FILE_PATH \
      --zone=ZONE
    

    Replace the following:

    • VM_NAME: the name of the VM from which to export properties

    • YAML_FILE_PATH: the path and file name of a .yaml file in which to save the exported configuration data

    • ZONE: the zone that contains the VM

  2. In the VM configuration file that was saved in FILE_PATH, update the value for enableNestedVirtualization. If the value is not in the file, add the following:

    advancedMachineFeatures:
      enableNestedVirtualization: true
    
  3. Update the VM with the value for enableNestedVirtualization by using the following gcloud compute instances update-from-file command:

    gcloud compute instances update-from-file VM_NAME \
      --source=FILE_PATH \
      --most-disruptive-allowed-action=RESTART \
      --zone=ZONE
    

    Replace the following:

    • VM_NAME: the name of the VM to update

    • FILE_PATH: the path to the updated VM configuration file

    • ZONE: the zone that contains the VM to update

API

Update the value for enableNestedVirtualization by using the following instances.update method:

PUT https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/VM_NAME?most_disruptive_allowed_action=RESTART

{
  ⋮
  "advanced_machine_features": {
    ⋮
    "enableNestedVirtualization": "true"
  },
  ⋮
}

Replace the following:

  • PROJECT_ID: the ID of the project

  • ZONE: the zone that contains the VM

  • VM_NAME: the name of the VM from which to export properties

Enabling nested virtualization by using the special license key

You can enable nested virtualization on VM by creating a custom image with a special license key that enables VMX on the L1 VM. The license key does not incur additional charges.

  1. Create a boot disk from a public image or from a custom image. The following example uses debian-cloud for the image project and debian-10 for the image family. If you already have a VM instance with an existing disk, you can skip this step.

    gcloud

    gcloud compute disks create DISK_NAME \
      --zone=ZONE \
      --image-project=debian-cloud \
      --image-family=debian-10
    

    Replace the following:

    • DISK_NAME: the name of the new disk

    • ZONE: the zone to create the disk in

    API

    POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/disks
    
    {
      ...
      "name": "DISK_NAME",
      "sourceImage": "projects/debian-cloud/global/images/family/debian-10",
      ...
    }
    

    Replace the following:

    • PROJECT_ID: the project ID

    • ZONE: the zone to create the disk in

    • DISK_NAME: the name of the new disk

  2. Create a custom image with the special license key that is required for nested virtualization.

    gcloud

    gcloud compute images create IMAGE_NAME \
      --source-disk DISK_NAME \
      --source-disk-zone ZONE \
      --licenses "https://www.googleapis.com/compute/v1/projects/vm-options/global/licenses/enable-vmx"
    

    Replace the following:

    • IMAGE_NAME: the name of the new image

    • DISK_NAME: the name of the previously created disk

    • ZONE: the zone to create the image in

    API

    POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/images
    
    {
      ...
      "licenses": ["projects/vm-options/global/licenses/enable-vmx"],
      "name": "IMAGE_NAME",
      "sourceDisk": "zones/ZONE/disks/DISK_NAME",
      ...
    }
    

    Replace the following:

    • PROJECT_ID: the project ID

    • IMAGE_NAME: the name of the new image

    • ZONE: the zone to create the image in

    • DISK_NAME: the name of the previously created disk

  3. Optionally delete the source disk after creating the image with the special license.

    gcloud

    gcloud compute disks delete DISK_NAME --zone=ZONE
    

    Replace the following:

    • DISK_NAME: the name of the disk to delete

    • ZONE: the zone containing the disk to delete

    API

    DELETE https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/disks/DISK_NAME
    

    Replace the following:

    • PROJECT_ID: the project ID

    • ZONE: the zone containing the disk to delete

    • DISK_NAME: the name of the disk to delete

  4. Create a VM that uses the new image with the special license. The minimum CPU platform must be "Intel Haswell".

    gcloud

    gcloud compute instances create VM_NAME \
        --zone=ZONE \
        --min-cpu-platform "Intel Haswell" \
        --image IMAGE_NAME
    

    Replace the following:

    • VM_NAME: the name of the VM

    • ZONE: the zone to create the VM in

    • IMAGE_NAME: the name of the previously created image

    API

    POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances
    
    {
      ...
      "name": "VM_NAME",
      "minCpuPlatform": "Intel Haswell",
      "disks": [
        {
          "initializeParams": {
            "sourceImage": "IMAGE_NAME"
          }
        }
      ]
      ...
    }
    
    

    Replace the following:

    • PROJECT_ID: the project ID

    • VM_NAME: the name of the VM

    • ZONE: the zone to create the VM in

    • IMAGE_NAME: the name of the previously created image

Confirming that nested virtualization is enabled on the VM

  1. Connect to the VM instance.

    gcloud compute ssh VM_NAME
    

    Replace VM_NAME with the name of the VM to connect to.

  2. Confirm that nested virtualization is enabled. A nonzero response confirms that nested virtualization is enabled.

    grep -cw vmx /proc/cpuinfo
    

What's next