Create a VM with attached GPUs

Compute Engine provides graphics processing units (GPUs) that you can add to your virtual machines (VMs). You can use these GPUs to accelerate specific workloads on your VMs such as machine learning and data processing.

This page explains how to create a VM with attached GPUs. If you want to add GPUs to existing VMs, see Add or remove GPUs.

For more information about what you can do with GPUs and what types of GPU hardware are available, read GPU platforms.

Before you begin

  • If you want to use the command-line examples in this guide, do the following:
    1. Install or update to the latest version of the Google Cloud CLI.
    2. Set a default region and zone.
  • If you want to use the API examples in this guide, set up API access.
  • Read about GPU pricing on Compute Engine to understand the cost to use GPUs on your VMs.
  • Read about restrictions for VMs with GPUs.
  • Check your GPU quota.
  • Choose an operating system image:
    • If you are using GPUs for machine learning, you can use a Deep Learning VM image for your VM. The Deep Learning VM images have GPU drivers pre-installed and include packages, such as TensorFlow and PyTorch. You can also use the Deep Learning VM images for general GPU workloads. For information about the images available and the packages installed on the images, see Choosing an image.
    • You can also use any public image or custom image, but some images might require a unique driver or install process that is not covered in this document. You must identify which drivers are appropriate for your images. For steps to install drivers, see installing GPU drivers.

Checking GPU quota

To protect Compute Engine systems and users, new projects have a global GPU quota, which limits the total number of GPUs you can create in any supported zone.

Use the regions describe command to ensure that you have sufficient GPU quota in the region where you want to create VMs with GPUs.

gcloud compute regions describe REGION

Replace REGION with the region that you want to check for GPU quota.

If you need additional GPU quota, request a quota increase. When you request a GPU quota, you must request a quota for the GPU types that you want to create in each region and an additional global quota for the total number of GPUs of all types in all zones.

If your project has an established billing history, it will receive quota automatically after you submit the request.

Limitations

The following limitations apply for Windows VMs with attached GPUs (A100):

  • a2-megagpu-16g machine types are not supported on Windows operating system. When using Windows operating systems, choose a different machine type. For a list of machine types, see NVIDIA® A100 GPUs.
  • For Windows VMs that use A2 machine types, you cannot do a quick format of the attached local SSDs. To format these local SSDs, you must do a full format by using the diskpart utility and specifying format fs=ntfs label=tmpfs.

Create VMs with attached GPUs

To create a VM with attached GPUs, complete the following steps:

  1. Create the VM. The method used to create a VM depends on the GPU model.

  2. For the VM to use the GPU, you need to install the GPU driver on your VM.

  3. If you enabled an NVIDIA RTX virtual workstation(formerly known as NVIDIA GRID), install a driver for virtual workstation.

Create a VM with attached GPUs (A100 GPUs)

This section covers how to create VMs with attached NVIDIA A100 GPUs. For other GPU types, see Create a VM with attached GPUs (other GPU types).

If you're creating a Windows VMs with GPUs, see limitations in this document.

Console

  1. In the Google Cloud console, go to the Create an instance page.

    Go to Create an instance

  2. Specify a Name for your VM. See Resource naming convention.

  3. Select a region and zone where GPUs are available. See the list of available A100 GPU zones.

  4. In the Machine configuration section, select the GPU machine family, and then do the following:

    1. In the GPU Type list, select NVIDIA Tesla A100. The Series of the machine type is set based on the GPU type.

      When you select NVIDIA Tesla A100, the Series of the machine type is set to A2 and the Machine type is set to an A2 machine type.

    2. In the Number of GPUs list, select the number of GPUs.

  5. In the Boot disk section, click Change. This opens the Boot disk configuration page.

  6. On the Boot disk configuration page, do the following:

    1. On the Public images tab, choose a supported Compute Engine image or Deep Learning VM image.
    2. Specify a boot disk size of at least 40 GB.
    3. To confirm your boot disk options, click Save.

  7. Configure any other VM settings that you require. For example, you can change the Preemptibility settings to configure your VM as a preemptible instance. This reduces the cost of your VM and the attached GPUs. For more information, see GPUs on preemptible instances.

  8. To create and start the VM, click Create.

gcloud

To create and start a VM use the gcloud compute instances create command with the following flags. VMs with GPUs cannot live migrate, make sure that you set the --maintenance-policy TERMINATE flag.

The --preemptible is an optional flag that configures your VM as a preemptible instance. This reduces the cost of your VM and the attached GPUs. For more information, see GPUs on preemptible instances.

gcloud compute instances create VM_NAME \
    --machine-type MACHINE_TYPE \
    --zone ZONE \
    --boot-disk-size DISK_SIZE \
    [--image IMAGE | --image-family IMAGE_FAMILY] \
    --image-project IMAGE_PROJECT \
    --maintenance-policy TERMINATE --restart-on-failure \
    [--preemptible]

Replace the following:

  • VM_NAME: the name for the new VM.
  • MACHINE_TYPE: the A2 machine type that you selected for the VM.
  • ZONE: the zone for the VM. This zone must support A100 GPUs.
  • DISK_SIZE: the size of your boot disk in GB. Specify a boot disk size of at least 40 GB.

  • IMAGE or IMAGE_FAMILY that supports GPUs. Specify one of the following:

    • IMAGE: the required version of a public image. For example, --image debian-10-buster-v20200309.
    • IMAGE_FAMILY: an image family. This creates the VM from the most recent, non-deprecated OS image. For example, if you specify --image-family debian-10, Compute Engine creates a VM from the latest version of the OS image in the Debian 10 image family.

    You can also specify a custom image or Deep Learning VM image.

  • IMAGE_PROJECT: the Compute Engine image project that the image family belongs to. If using a custom image or Deep Learning VM image, specify the project that those images belong to.

API

In the API, create a POST request to the instances.insert method. VMs with GPUs cannot live migrate, make sure you set the onHostMaintenance parameter to TERMINATE.

The "preemptible": true is an optional parameter that configures your VM as a preemptible instance. This reduces the cost of your VM and the attached GPUs. For more information, see GPUs on preemptible instances.

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances
{
  "machineType": "projects/PROJECT_ID/zones/ZONE/machineTypes/MACHINE_TYPE",
  "disks":
  [
    {
      "type": "PERSISTENT",
      "initializeParams":
      {
        "diskSizeGb": "DISK_SIZE",
        "sourceImage": "projects/IMAGE_PROJECT/global/images/family/IMAGE_FAMILY"
      },
      "boot": true
    }
  ],
  "name": "VM_NAME",
  "networkInterfaces":
  [
    {
      "network": "projects/PROJECT_ID/global/networks/NETWORK"
    }
  ],
  "scheduling":
  {
    "onHostMaintenance": "terminate",
    "automaticRestart": true,
    ["preemptible": true]
  },
}

Replace the following:

  • VM_NAME: the name of the VM.
  • PROJECT_ID: your project ID.
  • ZONE: the zone for the VM. This zone must support A100 GPUs.
  • MACHINE_TYPE: a A2 machine type that you selected for the VM.

  • IMAGE or IMAGE_FAMILY: specify one of the following:

    • IMAGE: the required version of a public image. For example, "sourceImage": "projects/debian-cloud/global/images/debian-10-buster-v20200309"
    • IMAGE_FAMILY: an image family. This creates the VM from the most recent, non-deprecated OS image. For example, if you specify "sourceImage": "projects/debian-cloud/global/images/family/debian-10", Compute Engine creates a VM from the latest version of the OS image in the Debian 10 image family.

    You can also specify a custom image or Deep Learning VM image.

  • IMAGE_PROJECT: the Compute Engine image project that the image family belongs to. If using a custom image or Deep Learning VM image, specify the project that those images belong to.

  • DISK_SIZE: the size of your boot disk in GB. Specify a boot disk size of at least 40 GB.

  • NETWORK: the VPC network that you want to use for the VM. You can specify default to use your default network.

Next: Install the GPU driver on your VM, so that your system can use the device. For A100 GPUs, CUDA 11 is required.

Examples (A100 GPUs)

In these examples, VMs are created with attached NVIDIA A100 GPUs by using the Google Cloud CLI. However, you can also use either the Google Cloud console or the Compute Engine API to create these VMs.

The following examples show how to create VMs using the following images:

DLVM image

Using DLVM images is the easiest way to get started because these images already have the NVIDIA drivers and CUDA libraries pre-installed.

These images also provide performance optimizations.

The following DLVM images are supported for NVIDIA A100:

  • common-cu110: NVIDIA driver and CUDA pre-installed
  • tf-ent-1-15-cu110: NVIDIA driver, CUDA, TensorFlow Enterprise 1.15.3 pre-installed
  • tf2-ent-2-1-cu110: NVIDIA driver, CUDA, TensorFlow Enterprise 2.1.1 pre-installed
  • tf2-ent-2-3-cu110: NVIDIA driver, CUDA, TensorFlow Enterprise 2.3.1 pre-installed
  • pytorch-1-6-cu110: NVIDIA driver, CUDA, Pytorch 1.6

For more information about the DLVM images that are available, and the packages installed on the images, see the Deep Learning VM documentation.

  1. Create a VM using the tf2-ent-2-3-cu110 image and the a2-highgpu-1g machine type. In this example, optional flags such as boot disk size and scope are specified.

    gcloud compute instances create VM_NAME \
   --project PROJECT_ID \
   --zone us-central1-c \
   --machine-type a2-highgpu-1g \
   --maintenance-policy TERMINATE --restart-on-failure \
   --image-family tf2-ent-2-3-cu110 \
   --image-project deeplearning-platform-release \
   --boot-disk-size 200GB \
   --metadata "install-nvidia-driver=True,proxy-mode=project_editors" \
   --scopes https://www.googleapis.com/auth/cloud-platform

    Replace the following:

    • VM_NAME: the name of your VM
    • PROJECT_ID : your project ID.

  2. The preceding example command also generates a AI Platform Notebook for the VM. To access the notebook, in the Google Cloud console, go to the AI Platform page.

    Go to the AI Platform page

Public or custom image

You can create VMs with attached GPUs that use either a public image that is available on Compute Engine or a custom image.

To create a VM using the most recent, non-deprecated image from the CentOS 7 image family and the a2-highgpu-1g machine type, complete the following steps:

  1. Create the VM. In this example, optional flags such as boot disk type and size are also specified.

    gcloud compute instances create VM_NAME \
    --project PROJECT_ID \
    --zone us-central1-c \
    --machine-type a2-highgpu-1g \
    --maintenance-policy TERMINATE --restart-on-failure \
    --image-family centos-7 \
    --image-project centos-cloud \
    --boot-disk-size 200GB \
    --boot-disk-type pd-ssd

    Replace the following:

    • VM_NAME: the name of your VM
    • PROJECT_ID : your project ID.

  2. Install NVIDIA driver and CUDA. For NVIDIA A100 GPUs, CUDA version 11 or higher is required.

COS

You can create VMs with attached GPUs that use Container-optimized (COS) images.

To create a VM using the cos-85-lts image and the a2-highgpu-1g machine type, complete the following steps from your local client. The following examples can be run on a Mac or Linux client:

  1. If one doesn't already exist, create a /tmp directory.

    mkdir /tmp

  2. Add a configuration file /cloud-init.yaml to the /tmp directory.

    This information is required to set up your Container-optimized VM and also installs NVIDIA driver and CUDA when the VM boots up.

    cat <<'EOF' > /tmp/cloud-init.yaml
#cloud-config

write_files:
  - path: /etc/systemd/system/cos-gpu-installer.service
    permissions: 0755
    owner: root
    content: |
      [Unit]
      Description=Run the GPU driver installer container
      Requires=network-online.target gcr-online.target
      After=network-online.target gcr-online.target

      [Service]
      User=root
      Type=oneshot
      RemainAfterExit=true
      Environment=INSTALL_DIR=/var/lib/nvidia
      ExecStartPre=/bin/mkdir -p ${INSTALL_DIR}
      ExecStartPre=/bin/mount --bind ${INSTALL_DIR} ${INSTALL_DIR}
      ExecStartPre=/bin/mount -o remount,exec ${INSTALL_DIR}
      ExecStart=/usr/bin/docker run --privileged \
                                    --net=host  \
                                    --pid=host \
                                    --volume ${INSTALL_DIR}:/usr/local/nvidia \
                                    --volume /dev:/dev \
                                    --volume /:/root \
                                    --env NVIDIA_DRIVER_VERSION=450.80.02 \
                                    gcr.io/cos-cloud/cos-gpu-installer:v20200701
      StandardOutput=journal+console
      StandardError=journal+console

runcmd:
  - systemctl daemon-reload
  - systemctl enable cos-gpu-installer.service
  - systemctl start cos-gpu-installer.service

EOF

  3. Create a Container-optimized VM using the cos-85-lts image family and the a2-highgpu-1g.

    You need to provide the configuration file by using the -metadata-from-file user-data flag.

    In this example, the optional flag boot disk size is also specified.

    gcloud compute instances create VM_NAME \
   --project PROJECT_ID \
   --zone us-central1-a \
   --machine-type a2-highgpu-1g \
   --maintenance-policy TERMINATE --restart-on-failure \
   --image-family cos-85-lts \
   --image-project cos-cloud \
   --boot-disk-size 200GB \
   --metadata-from-file user-data=/tmp/cloud-init.yaml

    Replace the following:

    • VM_NAME: the name of your VM
    • PROJECT_ID : your project ID.

    After the VM is created, log in to the VM and run the following command to verify that the NVIDIA driver is installed.

    /var/lib/nvidia/bin/nvidia-smi

    It takes approximately 5 minutes for the driver to be installed.

Multi-Instance GPUs (A100)

A Multi-Instance GPU partitions a single NVIDIA A100 GPU within the same VM into as many as seven independent GPU instances. They run simultaneously, each with its own memory, cache and streaming multiprocessors. This setup enables the A100 GPU to deliver guaranteed quality-of-service (QoS) at up to 7x higher utilization compared to earlier GPU models.

For more information about using Multi-Instance GPUs, see NVIDIA Multi-Instance GPU User Guide.

To create Multi-Instance GPUs, complete the following steps:

  1. Create a VM with attached A100 GPUs.

  2. Enable NVIDIA GPU drivers.

  3. Enable Multi-Instance GPUs and reboot the VM.

    sudo nvidia-smi -mig 1
sudo reboot

  4. Review the Multi-Instance GPU shapes that are available.

    sudo nvidia-smi mig --list-gpu-instance-profiles

    The output is similar to the following:

    +--------------------------------------------------------------------------+
| GPU instance profiles:                                                   |
| GPU   Name          ID    Instances   Memory     P2P    SM    DEC   ENC  |
|                           Free/Total   GiB              CE    JPEG  OFA  |
|==========================================================================|
|   0  MIG 1g.5gb     19     7/7        4.75       No     14     0     0   |
|                                                          1     0     0   |
+--------------------------------------------------------------------------+
|   0  MIG 2g.10gb    14     3/3        9.75       No     28     1     0   |
|                                                          2     0     0   |
+--------------------------------------------------------------------------+
|   0  MIG 3g.20gb     9     2/2        19.62      No     42     2     0   |
|                                                          3     0     0   |
+--------------------------------------------------------------------------+
|   0  MIG 4g.20gb     5     1/1        19.62      No     56     2     0   |
|                                                          4     0     0   |
+--------------------------------------------------------------------------+
|   0  MIG 7g.40gb     0     1/1        39.50      No     98     5     0   |
|                                                          7     1     1   |
+--------------------------------------------------------------------------+

  5. Create the Multi-Instance GPU (GI) and associated compute instances (CI) that you want. You can create these instances by specifying either the full or shortened profile name, profile ID, or a combination of both. For more information, see Creating GPU Instances.

    The following example creates two MIG 3g.20gb GPU instances by using a combination of the shortened profile name (3g.20gb), and the profile ID (9).

    The -C flag is also specified which creates the associated compute instances for the required profile.

    sudo nvidia-smi mig -cgi 9,3g.20gb -C

    The output is similar to the following:

    Successfully created GPU instance ID  2 on GPU  0 using profile MIG 3g.20gb (ID  9)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  2 using profile MIG 3g.20gb (ID  2)
Successfully created GPU instance ID  1 on GPU  0 using profile MIG 3g.20gb (ID  9)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  1 using profile MIG 3g.20gb (ID  2)

  6. Check that the two Multi-Instance GPUs are created:

    sudo nvidia-smi mig -lgi

    The output is similar to the following:

    +----------------------------------------------------+
| GPU instances:                                     |
| GPU   Name          Profile  Instance   Placement  |
|                       ID       ID       Start:Size |
|====================================================|
|   0  MIG 3g.20gb       9        1          0:4     |
+----------------------------------------------------+
|   0  MIG 3g.20gb       9        2          4:4     |
+----------------------------------------------------+

  7. Check that both the GIs and corresponding CIs are created.

    sudo nvidia-smi

    The output is similar to the following:

    Tue May 18 18:32:22 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.80.02    Driver Version: 450.80.02    CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  A100-SXM4-40GB      Off  | 00000000:00:04.0 Off |                   On |
| N/A   43C    P0    52W / 350W |     22MiB / 40537MiB |     N/A      Default |
|                               |                      |              Enabled |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| MIG devices:                                                                |
+------------------+----------------------+-----------+-----------------------+
| GPU  GI  CI  MIG |         Memory-Usage |        Vol|         Shared        |
|      ID  ID  Dev |           BAR1-Usage | SM     Unc| CE  ENC  DEC  OFA  JPG|
|                  |                      |        ECC|                       |
|==================+======================+===========+=======================|
|  0    1   0   0  |     11MiB / 20096MiB | 42      0 |  3   0    2    0    0 |
|                  |      0MiB / 32767MiB |           |                       |
+------------------+----------------------+-----------+-----------------------+
|  0    2   0   1  |     11MiB / 20096MiB | 42      0 |  3   0    2    0    0 |
|                  |      0MiB / 32767MiB |           |                       |
+------------------+----------------------+-----------+-----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Create a VM with attached GPUs (other GPU types)

This section describes how to create VMs using the following GPU types.

NVIDIA GPUs:

  • NVIDIA T4: nvidia-tesla-t4
  • NVIDIA P4: nvidia-tesla-p4
  • NVIDIA P100: nvidia-tesla-p100
  • NVIDIA V100: nvidia-tesla-v100
  • NVIDIA K80: nvidia-tesla-k80

NVIDIA RTX (formerly known as NVIDIA GRID) virtual workstation GPUs:

  • NVIDIA T4 Virtual Workstation: nvidia-tesla-t4-vws
  • NVIDIA P4 Virtual Workstation: nvidia-tesla-p4-vws

  • NVIDIA P100 Virtual Workstation: nvidia-tesla-p100-vws

    For these virtual workstations, an NVIDIA RTX Virtual Workstation license is automatically added to your VM.

Console

  1. In the Google Cloud console, go to the Create an instance page.

    Go to Create an instance

  2. Specify a Name for your VM. For more information, see Resource naming convention.

  3. Select a region and zone that supports the GPU model.

    To learn about GPU models supported by different regions and zones, see GPU regions and zones availability.

  4. From the Machine configuration section, complete the following steps:

    1. In Series, select N1.
    2. In Machine type, select a N1 machine type. Alternatively, you can specify custom machine type settings.

    3. Expand the CPU platform and GPU section.

      1. Click Add GPU.
      2. Specify the GPU type and number of GPUs.

      3. If your GPU model supports virtual workstations, and you plan on running graphics-intensive workloads on this VM, select Enable Virtual Workstation (NVIDIA GRID).

        For information about NVIDIA RTX virtual workstations, see NVIDIA RTX virtual workstations for graphics workloads (formerly known as NVIDIA GRID).

  5. To select your operating system, in the Boot disk section, click Change. This opens the Boot disk configuration page.

  6. On the Boot disk configuration page, do the following:

    1. On the Public images tab, choose a supported Compute Engine image or Deep Learning VM image.
    2. Specify a boot disk size of at least 40 GB.
    3. To confirm your boot disk options, click Save.

  7. Configure other VM settings that you require. For example, you can change the Preemptibility settings to configure your VM as a preemptible VM. This reduces the cost of your VM and the attached GPUs. For more information, see GPUs on preemptible VMs.

  8. To create and start the VM, click Create.

gcloud

To create and start a VM use the gcloud compute instances create command with the following flags.

The --preemptible is an optional flag that configures your VM as a preemptible instance. This reduces the cost of your VM and the attached GPUs. For more information, see GPUs on preemptible instances.

gcloud compute instances create VM_NAME \
    --machine-type MACHINE_TYPE \
    --zone ZONE \
    --boot-disk-size DISK_SIZE \
    --accelerator type=ACCELERATOR_TYPE,count=ACCELERATOR_COUNT \
    [--image IMAGE | --image-family IMAGE_FAMILY] \
    --image-project IMAGE_PROJECT \
    --maintenance-policy TERMINATE --restart-on-failure \
    [--preemptible]

Replace the following:

  • VM_NAME: the name for the new VM.
  • MACHINE_TYPE: the machine type that you selected for your VM.
  • ZONE: the zone for the VM. This zone must support the GPU type.
  • DISK_SIZE: the size of your boot disk in GB. Specify a boot disk size of at least 40 GB.

  • IMAGE or IMAGE_FAMILY that supports GPUs. Specify one of the following:

    • IMAGE: the required version of a public image. For example, --image debian-10-buster-v20200309.
    • IMAGE_FAMILY: an image family. This creates the VM from the most recent, non-deprecated OS image. For example, if you specify --image-family debian-10, Compute Engine creates a VM from the latest version of the OS image in the Debian 10 image family.

    You can also specify a custom image or Deep Learning VM image.

  • IMAGE_PROJECT: the Compute Engine image project that the image family belongs to. If using a custom image or Deep Learning VM image, specify the project that those images belong to.

  • ACCELERATOR_COUNT: the number of GPUs that you want to add to your VM. See GPUs on Compute Engine for a list of GPU limits based on the machine type of your VM.

  • ACCELERATOR_TYPE: the GPU model that you want to use. If you plan on running graphics-intensive workloads on this VM, use one of the virtual workstation models.

    Choose one of the following values:

    • NVIDIA GPUs:

      • NVIDIA T4: nvidia-tesla-t4
      • NVIDIA P4: nvidia-tesla-p4
      • NVIDIA P100: nvidia-tesla-p100
      • NVIDIA V100: nvidia-tesla-v100
      • NVIDIA K80: nvidia-tesla-k80

    • NVIDIA RTX (formerly known as NVIDIA GRID) virtual workstation GPUs:

      • NVIDIA T4 Virtual Workstation: nvidia-tesla-t4-vws
      • NVIDIA P4 Virtual Workstation: nvidia-tesla-p4-vws

      • NVIDIA P100 Virtual Workstation: nvidia-tesla-p100-vws

        For these virtual workstations, an NVIDIA RTX Virtual Workstation license is automatically added to your VM.

Example

For example, you can use the following gcloud command to start an Ubuntu 16.04 VM with 1 NVIDIA K80 GPU and 2 vCPUs in the us-east1-d zone.

gcloud compute instances create gpu-instance-1 \
    --machine-type n1-standard-2 \
    --zone us-east1-d \
    --boot-disk-size 40GB \
    --accelerator type=nvidia-tesla-k80,count=1 \
    --image-family ubuntu-1604-lts \
    --image-project ubuntu-os-cloud \
    --maintenance-policy TERMINATE --restart-on-failure

API

Identify the GPU type that you want to add to your VM. Submit a GET request to list the GPU types that are available to your project in a specific zone.

The "preemptible": true is an optional parameter that configures your VM as a preemptible instance. This reduces the cost of your VM and the attached GPUs. For more information, see GPUs on preemptible instances.

GET https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/acceleratorTypes

Replace the following:

  • PROJECT_ID: project ID.
  • ZONE: zone from which you want to list the available GPU types.

In the API, create a POST request to the instances.insert method. Include the acceleratorType parameter to specify which GPU type you want to use, and include the acceleratorCount parameter to specify how many GPUs you want to add. Also set the onHostMaintenance parameter to TERMINATE.

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances
{
  "machineType": "projects/PROJECT_ID/zones/ZONE/machineTypes/MACHINE_TYPE",
  "disks":
  [
    {
      "type": "PERSISTENT",
      "initializeParams":
      {
        "diskSizeGb": "DISK_SIZE",
        "sourceImage": "projects/IMAGE_PROJECT/global/images/family/IMAGE_FAMILY"
      },
      "boot": true
    }
  ],
  "name": "VM_NAME",
  "networkInterfaces":
  [
    {
      "network": "projects/PROJECT_ID/global/networks/NETWORK"
    }
  ],
  "guestAccelerators":
  [
    {
      "acceleratorCount": ACCELERATOR_COUNT,
      "acceleratorType": "projects/PROJECT_ID/zones/ZONE/acceleratorTypes/ACCELERATOR_TYPE"
    }
  ],
  "scheduling":
  {
    "onHostMaintenance": "terminate",
    "automaticRestart": true,
    ["preemptible": true]
  },
}

Replace the following:

  • VM_NAME: the name of the VM.
  • PROJECT_ID: your project ID.
  • ZONE: the zone for the VM. This zone must support the GPU type.
  • MACHINE_TYPE: the machine type that you selected for the VM. See GPUs on Compute Engine to see what machine types are available based on your desired GPU count.

  • IMAGE or IMAGE_FAMILY: specify one of the following:

    • IMAGE: the required version of a public image. For example, "sourceImage": "projects/debian-cloud/global/images/debian-10-buster-v20200309"
    • IMAGE_FAMILY: an image family. This creates the VM from the most recent, non-deprecated OS image. For example, if you specify "sourceImage": "projects/debian-cloud/global/images/family/debian-10", Compute Engine creates a VM from the latest version of the OS image in the Debian 10 image family.

    You can also specify a custom image or Deep Learning VM image.

  • IMAGE_PROJECT: the Compute Engine image project that the image family belongs to. If using a custom image or Deep Learning VM image, specify the project that those images belong to.

  • DISK_SIZE: the size of your boot disk in GB. Specify a boot disk size of at least 40 GB.

  • NETWORK: the VPCnetwork that you want to use for the VM. You can specify default to use your default network.

  • ACCELERATOR_COUNT: the number of GPUs that you want to add to your VM. See GPUs on Compute Engine for a list of GPU limits based on the machine type of your VM.

  • ACCELERATOR_TYPE: the GPU model that you want to use. If you plan on running graphics-intensive workloads on this VM, use one of the virtual workstation models.

    Choose one of the following values:

    • NVIDIA GPUs:

      • NVIDIA T4: nvidia-tesla-t4
      • NVIDIA P4: nvidia-tesla-p4
      • NVIDIA P100: nvidia-tesla-p100
      • NVIDIA V100: nvidia-tesla-v100
      • NVIDIA K80: nvidia-tesla-k80

    • NVIDIA RTX (formerly known as NVIDIA GRID) virtual workstation GPUs:

      • NVIDIA T4 Virtual Workstation: nvidia-tesla-t4-vws
      • NVIDIA P4 Virtual Workstation: nvidia-tesla-p4-vws

      • NVIDIA P100 Virtual Workstation: nvidia-tesla-p100-vws

        For these virtual workstations, an NVIDIA RTX Virtual Workstation license is automatically added to your VM.

Next: To ensure that your system can use the GPUs, complete the following steps:

What's next?