Network bandwidths and GPUs

Higher network bandwidths can improve the performance of distributed workloads. On Compute Engine, network bandwidth depends on machine type and the number of CPUs. For virtual machine (VM) instances that have attached GPUs, the configuration of your GPU count, CPU, and memory impacts the network bandwidth as well. For more information about GPUs, see GPUs on Compute Engine.

To achieve the 50-100 Gbps rates, your VM must use Google Virtual NIC (gVNIC). For more information about creating GPU VMs that use gVNIC, see Creating VMs that use higher bandwidths.

The maximum available bandwidth per attached GPUs on Compute Engine is as follows:

  • For VMs that have P100, P4, and K80 GPUs attached, a maximum bandwidth of 32 Gbps is available. This is similar to the maximum rate available to VMs that do not have GPUs attached. For more information about network bandwidths, see maximum egress data rate.
  • For VMs that have V100, T4, and A100 GPUs attached, you can now get a maximum bandwidth of up to 50 or 100 Gbps, based on the GPU count. To create VMs instances with V100, T4, and A100 GPUs that use up to 100 Gbps, see using network bandwidths of up to 100 Gbps.

VM configurations

To test or verify the bandwidth speed for any configuration, you can use the benchmarking test. For more information, see Checking network bandwidth.

The following tables summarize the maximum available network bandwidth for different VM configurations of T4, and V100 GPU types as well as the maximum available network bandwidth for the fixed VM configurations of A100 GPU types.

V100 VM configuration

GPU count vCPUs Memory Network bandwidth
1 1 - 12 1 - 78 GB 24 Gbps
2 1 - 24 1 - 156 GB 32 Gbps
4 1 - 48 1 - 312 GB 50 Gbps
8 1 - 96 1 - 624 GB 100 Gbps

T4 VM configuration

GPU count vCPUs Memory Network bandwidth
1 1 - 24 1 - 156 GB 32 Gbps
25 - 48 * 1 - 312 GB 32 Gbps
2 1 - 48 1 - 312 GB 50 Gbps
4 1 - 96 1 - 624 GB 100 Gbps

*Preview: Attaching 25 vCPUs or more to a single T4 GPU VM, is currently only available in preview. All other VM shapes are generally available.

A100 VM configuration

Machine type (GPU count) vCPUs Memory Network bandwidth
a2-highgpu-1g (1) 12 85 GB 24 Gbps
a2-highgpu-2g (2) 24 170 GB 32 Gbps
a2-highgpu-4g (4) 48 340 GB 50 Gbps
a2-highgpu-8g (8) 96 680 GB 100 Gbps
a2-megagpu-16 (16) 96 1360 GB 100 Gbps

What's next?