Google Cloud is focused on delivering world-class artificial intelligence (AI) infrastructure to power your most demanding GPU-accelerated workloads across a wide range of segments. You can use GPUs on Google Cloud to run AI, machine learning (ML), scientific, analytics, engineering, consumer, and enterprise applications.
Through our partnership with NVIDIA, Google Cloud delivers the latest GPUs while optimizing the software stack with a wide array of storage and networking options. For a full list of GPUs available, see GPU platforms.
The following sections outline the benefits of GPUs on Google Cloud.
GPU-accelerated VMs
On Google Cloud, you can access and provision GPUs in the way that best suits your needs. A specialized accelerator-optimized machine family is available, with pre-attached GPUs and networking capabilities that are ideal for maximizing performance. These are available in the A3, A2, and G2 machine series.
Multiple provisioning options
You can provision clusters by using the accelerator-optimized machine family with any of the following open-source or Google Cloud products.
Vertex AI
Vertex AI is a fully-managed machine learning (ML) platform that you can use to train and deploy ML models and AI applications. In Vertex AI applications, you can use GPU-accelerated VMs to improve performance in the following ways:
- Use GPU-enabled VMs in custom training GKE worker pools.
- Use open source LLM models from the Vertex AI Model Garden.
- Reduce prediction latency.
- Improve performance of Vertex AI Workbench notebook code.
- Improve performance of a Colab Enterprise runtime.
GKE and Slurm
Large-scale orchestration platforms, such as GKE, are ideal for provisioning large clusters that can be used for training and fine-tuning large-scale ML models. Large-scale ML models are those that use vast quantities of data.
The following orchestration platforms are available on Google Cloud.
Google Kubernetes Engine (GKE): is a service that you can use to deploy and operate containerized applications at scale using Google's infrastructure.
Slurm: is an open-source cluster management and job scheduling tool. On Google Cloud you can deploy Slurm clusters by using Cluster Toolkit.
Run large-scale model training and fine-tuning
For training or fine-tuning large-scale models, we recommend using a cluster of
A3 Mega (a3-megagpu-8g) machines and deploying with a scheduler such as
GKE or Slurm.
Deployment option |
Deployment guides |
Slurm |
|
GKE |
Run mainstream model training and fine-tuning
For training and fine-tuning of mainstream models, we recommend using the
A3 High with 8 GPUs (a3-highgpu-8g) and deploying with a scheduler such
as GKE or Slurm. You can also use an A2 or G2 machine type.
Deployment option |
Deployment guides |
Workloads |
GKE |
Inference: Serve models on GKE Training: Train a model on GKE |
|
Slurm |
||
Single VMs |
||
Compute Engine
You can also create and manage single VMs or smaller clusters of VMs with attached GPUs on Compute Engine. This method is mostly used for running graphics-intensive workloads, simulation workloads, or small-scale training. For these workloads we recommend G2, small A3 High (those with 1, 2, or 4 GPUs attached), and N1 machine types with T4, P4, P100 and V100 GPUs.
Deployment option |
Deployment guides |
Create a VM for serving and single node workloads |
|
Create managed instance groups (MIGs) |
|
Create VMs in bulk |
|
Create a single VM |
Create a single GPU VM (Standard or Spot VMs) |
Create virtual workstations |
Cloud Run
You can configure GPUs for your Cloud Run service. GPUs are ideal for running AI inference workloads using large language models on Cloud Run.
On Cloud Run, consult these resources for running AI workloads on GPUs:
- Configure GPUs for a Cloud Run service
- Load large ML models on Cloud Run with GPUs
- Tutorial: Run LLM inference on Cloud Run GPUs with Ollama