Overview of creating an instance with attached GPUs

Compute Engine provides graphics processing units (GPUs) that you can add to your virtual machines (VMs). You can use these GPUs to accelerate specific workloads on your VMs such as machine learning and data processing.

You can also use some GPU machine types on AI Hypercomputer. AI Hypercomputer is a supercomputing system that is optimized to support your artificial intelligence (AI) and machine learning (ML) workloads. This option is recommended for creating a densely allocated, performance-optimized infrastructure that has integrations for Google Kubernetes Engine (GKE) and Slurm schedulers.

This document provides an overview of the steps required for creating a VM with attached GPUs.

For more information about GPUs on Compute Engine, see About GPUs.

Select the GPU model

For a list of GPU models that are available, see GPU platforms. Also make a note of the machine type that is supported for the selected GPU model.

For each model, it might also be helpful to review the following:

Limitations

In addition to the restrictions for all VMs with GPUs, each machine series with attached GPUs has the following limitations:

A4 VMs

  • You can only request capacity by using the supported provisioning options for an A4 machine type.
  • You don't receive sustained use discounts and flexible committed use discounts for VMs that use an A4 machine type.
  • You can only use an A4 machine type in certain regions and zones.
  • You can't use Persistent Disk (regional or zonal) on a VM that uses an A4 machine type.
  • The A4 machine type is only available on the Emerald Rapids CPU platform.
  • You can't change the machine type of an existing VM to an A4 machine type. You can only create new A4 VMs. After creating a VM using an A4 machine type, you can't change the machine type.
  • A4 machine types don't support sole-tenancy.
  • You can't run Windows operating systems on an A4 machine type.

A3 Ultra VMs

  • You can only request capacity by using the supported provisioning options for an A3 Ultra machine type.
  • You don't receive sustained use discounts and flexible committed use discounts for VMs that use an A3 Ultra machine type.
  • You can only use an A3 Ultra machine type in certain regions and zones.
  • You can't use Persistent Disk (regional or zonal) on a VM that uses an A3 Ultra machine type.
  • The A3 Ultra machine type is only available on the Emerald Rapids CPU platform.
  • You can't change the machine type of an existing VM to an A3 Ultra machine type. You can only create new A3-ultra VMs. After creating a VM using an A3 Ultra machine type, you can't change the machine type.
  • A3 Ultra machine types don't support sole-tenancy.
  • You can't run Windows operating systems on an A3 Ultra machine type.

A3 Mega VMs

A3 High VMs

  • You can only request capacity by using the supported provisioning options for an A3 High machine type.
  • You don't receive sustained use discounts and flexible committed use discounts for VMs that use an A3 High machine type.
  • You can only use an A3 High machine type in certain regions and zones.
  • You can't use regional Persistent Disk on a VM that uses an A3 High machine type.
  • The A3 High machine type is only available on the Sapphire Rapids CPU platform.
  • You can't change the machine type of an existing VM to an A3 High machine type. You can only create new A3-high VMs. After creating a VM using an A3 High machine type, you can't change the machine type.
  • A3 High machine types don't support sole-tenancy.
  • You can't run Windows operating systems on an A3 High machine type.
  • For a3-highgpu-1g, a3-highgpu-2g, anda3-highgpu-4g machine types, the following limitations apply:
    • For these machine types, you must create VMs using Spot VMs or a feature that uses the Dynamic Workload Scheduler (DWS), such as resize requests in a MIG. For detailed instructions on either of these options, review the following:
    • You can't use Hyperdisk Balanced with these machine types.
    • You can't create reservations.
    If you try to create a VM by using standard provisioning or try to create a reservation for these machine types, you'll receive an error message.

A3 Edge VMs

A2 Standard VMs

  • You can only request capacity by using the supported provisioning options for an A2 Standard machine type.
  • You don't receive sustained use discounts and flexible committed use discounts for VMs that use an A2 Standard machine type.
  • You can only use an A2 Standard machine type in certain regions and zones.
  • The A2 Standard machine type is only available on the Cascade Lake platform.
  • If your VM uses an A2 Standard machine type, you can only switch from one A2 Standard machine type to another A2 Standard machine type. You can't change to any other machine type. For more information, see Modify accelerator-optmized VMs.
  • You can't use the Windows operating system with A2 Standard machine types. When using Windows operating systems, choose a different A2 Standard machine type.
  • You can't do a quick format of the attached Local SSDs on Windows VMs that use A2 Standard machine types. To format these Local SSDs, you must do a full format by using the diskpart utility and specifying format fs=ntfs label=tmpfs.
  • A2 Standard machine types don't support sole-tenancy.

A2 Ultra VMs

  • You can only request capacity by using the supported provisioning options for an A2 Ultra machine type.
  • You don't receive sustained use discounts and flexible committed use discounts for VMs that use an A2 Ultra machine type.
  • You can only use an A2 Ultra machine type in certain regions and zones.
  • The A2 Ultra machine type is only available on the Cascade Lake platform.
  • If your VM uses an A2 Ultra machine type, you can't change the machine type. If you need to use a different A2 Ultra machine type, or any other machine type, you must create a new VM.
  • You can't change any other machine type to an A2 Ultra machine type. If you need a VM that uses an A2 Ultra machine type, you must create a new VM.
  • You can't do a quick format of the attached Local SSDs on Windows VMs that use A2 Ultra machine types. To format these Local SSDs, you must do a full format by using the diskpart utility and specifying format fs=ntfs label=tmpfs.

G2 VMs

  • You can only request capacity by using the supported provisioning options for an G2 machine type.
  • You don't receive sustained use discounts and flexible committed use discounts for VMs that use an G2 machine type.
  • You can only use an G2 machine type in certain regions and zones.
  • The G2 machine type is only available on the Cascade Lake platform.
  • Standard Persistent Disk (pd-standard) is not supported on VMs that use G2 standard machine types. For supported disk types, see Supported disk types for G2.
  • You can't create Multi-Instance GPUs on G2 machine types.
  • If you need to change the machine type of a G2 VM, review Modify accelerator-optmized VMs.
  • You can't use Deep Learning VM Images as boot disks for your VMs that use G2 machine types.
  • The current default driver for Container-Optimized OS doesn't support L4 GPUs running on G2 machine types. Container-Optimized OS also only support a select set of drivers. If you want to use Container-Optimized OS on G2 machine types, review the following notes:
    • Use a Container-Optimized OS version that supports the minimum recommended NVIDIA driver version 525.60.13 or later. For more information, review the Container-Optimized OS release notes.
    • When you install the driver, specify the latest available version that works for the L4 GPUs. For example, sudo cos-extensions install gpu -- -version=525.60.13.
  • You must use the Google Cloud CLI or REST to create G2 VMs for the following scenarios:
    • You want to specify custom memory values.
    • You want to customize the number of visible CPU cores.

N1+GPU VMs

To learn about the limitations for N1 instances with GPUs, see features for the N1 machine series and GPUs for the N1 machine series.

Choose an operating system

If you are using GPUs for machine learning, use one of the following operating systems:

  • Images optimized for AI workloads. You can use Ubuntu and Rocky images, which are available in accelerator-optimized versions with drivers and CUDA toolkit pre-installed. See OS images in the AI Hypercomputer documentation.
  • Deep Learning VM Images. Each Deep Learning VM has a GPU driver installation tool and includes packages such as TensorFlow and PyTorch. You can also use a Deep Learning VM for general GPU workloads. To learn more about available images and packages installed on these images, see Choosing an image.

Alternatively, you can use any public image or custom image. Note that some images might require a unique driver or have an install process that is out of the scope of the Compute Engine documentation. To help identify which drivers are appropriate for your OS image, see installing GPU drivers.

Check GPU quota

To protect Compute Engine systems and users, new projects have a global GPU quota, which limits the total number of GPUs you can create in any supported zone. To review GPU quota, see GPU quota.

If you need additional GPU quota, request a quota increase. When you request GPU quota, you must request quota for the GPU types that you want to create in each region and an additional global quota for the total number of GPUs of all types in all zones.

If your project has an established billing history, it will receive quota automatically after you submit the request.

GPU VMs and preemptible allocation quotas

VMs that use the standard provisioning model typically can't use preemptible allocation quotas. Preemptible quotas are for temporary workloads and are usually more available. If your project doesn't have preemptible quota, and you have never requested it, then all VMs in your project consume standard allocation quotas.

If you request preemptible allocation quota, then VMs that use the standard provisioning model must meet all of the following criteria to consume preemptible allocation quota:

When you consume preemptible allocation for time-bound GPU workloads, you can benefit from both uninterrupted run time and the high obtainability of preemptible allocation quota. For more information, see Preemptible quotas.

Create a VM that has attached GPUs

To create a VM that has attached GPUs, complete the following steps:

  1. Create the VM. The method used to create a VM depends on the GPU model selected.

  2. For the VM to use the GPU, you need to install the GPU driver on your VM. If you enabled an NVIDIA RTX virtual workstation (formerly known as NVIDIA GRID), install a driver for virtual workstation.

What's next?