Enable nested virtualization

This document describes how to enable nested virtualization on a virtual machine (VM) instance and how to confirm that you can create a nested VM. Enable nested virtualization on a VM by using one of the following methods:

  • Recommended. Enable nested virtualization directly on a new or existing VM by setting the enableNestedVirtualization field to true while creating the VM or by updating the VM. This is the recommended method because it doesn't require that you create a custom image or use the special license key.

  • Enable nested virtualization by using the special license key by creating a boot disk, creating a custom image with the special nested virtualization license key, and then creating a VM that uses the custom image.

Before you begin

  • If you haven't already, then set up authentication. Authentication is the process by which your identity is verified for access to Google Cloud services and APIs. To run code or samples from a local development environment, you can authenticate to Compute Engine by selecting one of the following options:

    Select the tab for how you plan to use the samples on this page:

    gcloud

    1. Install the Google Cloud CLI, then initialize it by running the following command: 

      gcloud init
    2. Set a default region and zone.

    REST

    To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.

      Install the Google Cloud CLI, then initialize it by running the following command: 

      gcloud init

    For more information, see Authenticate for using REST in the Google Cloud authentication documentation.

Enable nested virtualization directly on a new VM

Enable nested virtualization directly on a VM by using the following procedure.

gcloud

Create an L1 VM with nested virtualization enabled by using the following gcloud compute instances create command:

gcloud compute instances create VM_NAME \
  --enable-nested-virtualization \
  --zone=ZONE \
  --min-cpu-platform="Intel Haswell"

Replace the following:

  • VM_NAME: the name of the new L1 VM with nested virtualization enabled

  • ZONE: the zone for the new L1 VM with nested virtualization enabled

REST

Create an L1 VM with nested virtualization enabled by using the following instances.insert method:

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances
{
  ...
  "name": "VM_NAME",
  ...
  "minCpuPlatform": "Intel Haswell",
  "advancedMachineFeatures": {
    "enableNestedVirtualization": true
  },
  ...
}

Replace the following:

  • PROJECT_ID: the project ID

  • ZONE: the zone for the new L1 VM with nested virtualization enabled

  • VM_NAME: the name of the new L1 VM with nested virtualization enabled

Enable nested virtualization directly on an existing VM

Enable nested virtualization on an existing VM by using the following procedure.

gcloud

  1. Export the properties of the VM by using the following gcloud compute instances export command:

    gcloud compute instances export VM_NAME \
  --destination=YAML_FILE_PATH \
  --zone=ZONE

    Replace the following:

    • VM_NAME: the name of the VM from which to export properties

    • YAML_FILE_PATH: the path and file name of a .yaml file in which to save the exported configuration data

    • ZONE: the zone that contains the VM

  2. In the VM configuration file that was saved in FILE_PATH, update the value for enableNestedVirtualization. If the value is not in the file, add the following:

    advancedMachineFeatures:
  enableNestedVirtualization: true

  3. Update the VM with the value for enableNestedVirtualization by using the following gcloud compute instances update-from-file command:

    gcloud compute instances update-from-file VM_NAME \
  --source=FILE_PATH \
  --most-disruptive-allowed-action=RESTART \
  --zone=ZONE

    Replace the following:

    • VM_NAME: the name of the VM to update

    • FILE_PATH: the path to the updated VM configuration file

    • ZONE: the zone that contains the VM to update

REST

Update the value for enableNestedVirtualization by using the following instances.update method:

PUT https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/VM_NAME?most_disruptive_allowed_action=RESTART

{
  ⋮
  "advanced_machine_features": {
    ⋮
    "enableNestedVirtualization": "true"
  },
  ⋮
}

Replace the following:

  • PROJECT_ID: the ID of the project

  • ZONE: the zone that contains the VM

  • VM_NAME: the name of the VM from which to export properties

Enable nested virtualization by using the special license key

You can enable nested virtualization on VM by creating a custom image with a special license key that enables VMX on the L1 VM. The license key does not incur additional charges.

  1. Create a boot disk from a public image or from a custom image. The following example uses debian-cloud for the image project and debian-10 for the image family. If you already have a VM instance with an existing disk, you can skip this step.

    gcloud 

    gcloud compute disks create DISK_NAME \
  --zone=ZONE \
  --image-project=debian-cloud \
  --image-family=debian-10

    Replace the following:

    • DISK_NAME: the name of the new disk

    • ZONE: the zone to create the disk in

    REST 

    POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/disks

{
  ...
  "name": "DISK_NAME",
  "sourceImage": "projects/debian-cloud/global/images/family/debian-10",
  ...
}

    Replace the following:

    • PROJECT_ID: the project ID

    • ZONE: the zone to create the disk in

    • DISK_NAME: the name of the new disk

  2. Create a custom image with the special license key that is required for nested virtualization.

    gcloud 

    gcloud compute images create IMAGE_NAME \
  --source-disk DISK_NAME \
  --source-disk-zone ZONE \
  --licenses "https://www.googleapis.com/compute/v1/projects/vm-options/global/licenses/enable-vmx"

    Replace the following:

    • IMAGE_NAME: the name of the new image

    • DISK_NAME: the name of the previously created disk

    • ZONE: the zone to create the image in

    REST 

    POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/images

{
  ...
  "licenses": ["projects/vm-options/global/licenses/enable-vmx"],
  "name": "IMAGE_NAME",
  "sourceDisk": "zones/ZONE/disks/DISK_NAME",
  ...
}

    Replace the following:

    • PROJECT_ID: the project ID

    • IMAGE_NAME: the name of the new image

    • ZONE: the zone to create the image in

    • DISK_NAME: the name of the previously created disk

  3. Optionally delete the source disk after creating the image with the special license.

    gcloud 

    gcloud compute disks delete DISK_NAME --zone=ZONE

    Replace the following:

    • DISK_NAME: the name of the disk to delete

    • ZONE: the zone containing the disk to delete

    REST 

    DELETE https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/disks/DISK_NAME

    Replace the following:

    • PROJECT_ID: the project ID

    • ZONE: the zone containing the disk to delete

    • DISK_NAME: the name of the disk to delete

  4. Create a VM that uses the new image with the special license. The minimum CPU platform must be "Intel Haswell".

    gcloud 

    gcloud compute instances create VM_NAME \
    --zone=ZONE \
    --min-cpu-platform "Intel Haswell" \
    --image IMAGE_NAME

    Replace the following:

    • VM_NAME: the name of the VM

    • ZONE: the zone to create the VM in

    • IMAGE_NAME: the name of the previously created image

    REST 

    POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances

{
  ...
  "name": "VM_NAME",
  "minCpuPlatform": "Intel Haswell",
  "disks": [
    {
      "initializeParams": {
        "sourceImage": "IMAGE_NAME"
      }
    }
  ]
  ...
}

    Replace the following:

    • PROJECT_ID: the project ID

    • VM_NAME: the name of the VM

    • ZONE: the zone to create the VM in

    • IMAGE_NAME: the name of the previously created image

Confirm that nested virtualization is enabled on the VM

  1. Connect to the VM instance.

    gcloud compute ssh VM_NAME

    Replace VM_NAME with the name of the VM to connect to.

  2. Confirm that nested virtualization is enabled. Any response other than 0 confirms that nested virtualization is enabled.

    grep -cw vmx /proc/cpuinfo

What's next