Configuring a VM with higher bandwidth

Compute Engine lets you select a high-bandwidth Tier 1 configuration for certain general-purpose and compute-optimized virtual machine (VM) instances. VMs with high-bandwidth configurations are especially useful for large, distributed compute workloads with lots of heavy internode communications such as high performance computing (HPC), machine learning (ML), and deep learning (DL). You can expect the best network performance for traffic within the same zone, using internal IP addresses.

Combining these high throughput VMs with high-performance local SSD is beneficial for I/O-intensive, flash-optimized databases.

High-bandwidth configurations require:

VMs in the N2, N2D, C2, or C2D series.
Google virtual NIC (gVNIC) network driver.
gVNIC-compatible OS or custom image.

Before you begin

If you want to use the command-line examples in this guide, do the following:
1. Install or update to the latest version of the Google Cloud CLI.
2. Set a default region and zone.
If you want to use the API examples in this guide, set up API access.

Limitations

Compute Engine is the only product area supporting this feature.
This feature is excluded from committed use discounts.
Tier 1 bandwidth configuration is only supported on N2, N2D EPYC Milan, C2 and C2D VMs.
Tier 1 bandwidth configuration is only compatible with VMs that are running the gVNIC virtual network driver.

Bandwidth tiers

Total egress bandwidth is the available outbound bandwidth from a VM, regardless of whether the traffic is going to destinations that have an internal IP address or external IP address. You can think of this bandwidth as the physical NIC speed for a given VM.

Default bandwidth ranges from 10 Gbps to 32 Gbps depending on the machine family and VM size.
Tier 1 bandwidth increases the maximum egress bandwidth for VMs, and ranges from 50 Gbps to 100 Gbps depending on the size of your N2, N2D, C2 or C2D VM.

General-purpose N2 VMs

The following table describes the egress bandwidth configurations for N2 VMs.

vCPUs	Default	Tier 1
2	10 Gbps	Not applicable (N/A)
4	10 Gbps	N/A
8	16 Gbps	N/A
16	32 Gbps	N/A
32	32 Gbps	50 Gbps
48	32 Gbps	50 Gbps
64	32 Gbps	75 Gbps
80	32 Gbps	100 Gbps
96	32 Gbps	100 Gbps
128	32 Gbps	100 Gbps

General-purpose N2 (custom size shapes) VMs

The following table describes the egress bandwidth configurations for custom-sized N2 VMs.

vCPUs	Default	Tier 1
32-62	32 Gbps	50 Gbps
64-78	32 Gbps	75 Gbps
80-128	32 Gbps	100 Gbps

General-purpose N2D VMs

The following table describes the egress bandwidth configurations for N2D VMs.

vCPUs	Default	Tier 1
2	10 Gbps	Not applicable (N/A)
4	10 Gbps	N/A
8	16 Gbps	N/A
16	32 Gbps	N/A
32	32 Gbps	N/A
48	32 Gbps	50 Gbps
64	32 Gbps	50 Gbps
80	32 Gbps	50 Gbps
96	32 Gbps	100 Gbps
128	32 Gbps	100 Gbps
224	32 Gbps	100 Gbps

General-purpose N2D (custom size shapes) VMs

The following table describes the egress bandwidth configurations for custom-sized N2D VMs.

vCPUs	Default	Tier 1
48-94	32 Gbps	50 Gbps
96	32 Gbps	100 Gbps

Compute-optimized C2 VMs

The following table describes the egress bandwidth configurations for C2 VMs.

vCPUs	Default	Tier 1
4	10 Gbps	Not applicable (N/A)
8	16 Gbps	N/A
16	32 Gbps	N/A
30	32 Gbps	50 Gbps
60	32 Gbps	100 Gbps

Compute-optimized C2D VMs

The following table describes the egress bandwidth configurations for C2D VMs.

vCPUs	Default	Tier 1
2	10 Gbps	Not applicable (N/A)
4	10 Gbps	N/A
8	16 Gbps	N/A
16	32 Gbps	N/A
32	32 Gbps	50 Gbps
56	32 Gbps	50 Gbps
112	32 Gbps	100 Gbps

Creating a VM with high-bandwidth configuration

You can enable a high-bandwidth configuration during VM creation. However, you can also edit a VM to add or remove a high-bandwidth configuration at any time. Your VM must be in a STOPPED state before you can edit its configuration. RESTART your VM to apply the updated higher configuration.

Creating VMs and containers with high-bandwidth configuration

Use the Google Cloud console, the Google Cloud CLI or the Compute Engine API to add a high-bandwidth network configuration to a new VM or to create a VM running container images.

Console

In the Google Cloud console, go to the VM instances page.

Go to VM instances
Select your project and click Continue.
Click Create instance.
Specify a Name for your VM. For more information, see Resource naming convention.
Select a region and zone that supports a high-bandwidth configuration.
Select a Machine configuration for your VM. To create a a high-bandwidth VM, you must select an N2, N2D, C2, or C2D VM from the Series drop-down menu. Your machine type must align with the bandwidth tier size requirement.
In the Boot disk section, click Change, and then select a gVNIC-compatible or custom image.
Specify your firewall rules.
Open the NETWORKING, DISKS, SECURITY, MANAGEMENT, SOLE-TENANCY menu to change your networking properties.
Click on Networking and then do the following:
1. Under the Network interface card drop-down menu, select the gVNIC network interface card.
2. Under Network bandwidth check the Increase total egress bandwidth box.
3. Specify your Network interfaces.
Click Create.

gcloud

Use the gcloud compute instances create command to create a VM with a gVNIC virtual network driver. Use the --network-performance-configs flag and the --network-interface flag to configure a network performance setting for a VM. If you do not specify these flags the VM is created with the default network performance configuration.

To create a VM running container images, use the gcloud compute instances create-with-container command.

gcloud compute instances create VM_NAME \
    --image=MACHINE_IMAGE \
    --machine-type=MACHINE_TYPE \
    --network-performance-configs=total-egress-bandwidth-tier=TIER_1 \
    --network-interface=nic-type=GVNIC

Replace the following:

VM_NAME: the name of the VM
MACHINE_IMAGE: use a gVNIC image
MACHINE_TYPE: a machine type that supports a high-bandwidth configuration

For example:

gcloud compute instances create instance-1 \
    --network-performance-configs=total-egress-bandwidth-tier=TIER_1 \
    --network-interface=nic-type=GVNIC \
    --image=projects/ubuntu-os-cloud/global/images/ubuntu-2110-impish-v20211014 \
    --machine-type=n2-standard-32

API

Call the Compute Engine API instances.insert method to create a VM with a high-bandwidth network configuration. Within the request body, set the networkPerformanceConfig parameters to, totalEgressBandwidthTier and TIER_1. Set the networkInterface parameters to nicType and GVNIC.

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances

{
  "name": VM_NAME,
    "description": string,
    ...
    "networkPerformanceConfig": {
        "totalEgressBandwidthTier": TIER_1
  },
    "networkInterfaces": [
      {
        "nicType": "GVNIC"
    },
    ...
  }

Replace the following:

PROJECT_ID: your project ID
ZONE: the zone where you want to create the VM
VM_NAME: the name of the VM

Updating a VM to include high-bandwidth configuration

Refer to the Updating instance properties documentation to ensure you are meeting all the requirements to successfully update your VM. Use the Google Cloud console, the Google Cloud CLI or the Compute Engine API to update a VM to include a Tier 1 high-bandwidth network configuration.

Console

You can only edit a VM to change the VM size or to change the network configuration to include or exclude a high-bandwidth configuration. Your VM must already have a gVNIC card associated with it. It is not possible to edit your VM to add a network card.

In the Google Cloud console, go to the VM instances page.

Go to VM instances
Select your project and click Continue.
Select the VM you wish to change.
Stop the VM.
Select Edit VM.
Update the VM size or edit your VM's high-bandwidth network configuration. You can only make this change if your VM was originally configured with a gVNIC card.
Save your changes.
Restart your VM.

gcloud

To edit a VM, use the gcloud compute instances export command to export your VM's information to a .yaml file.

 gcloud compute instances export VM_NAME \
     --zone=ZONE --destination=PATH_TO_FILE

Replace the following:

VM_NAME: the name of the VM
ZONE: the name of the zone where the VM is located
PATH_TO_FILE: a file name with a .yaml extension.

For example:

 gcloud compute instances export instance-1 \
     --zone=europe-west1-a --destination=test-file.yaml

Open the gcloud text editor, or the editor or your choice.
Open the .yaml file you created.
Scroll to the end of the file and add the following information:

  networkPerformanceConfig:
    totalEgressBandwidthTier: TIER_1

Setting totalEgressBandwidthTier to TIER_1 adds a high-bandwidth configuration. Setting it to DEFAULT removes the configuration.

Next, use the gcloud compute instance update-from-file command to update the VM with the file changes.

gcloud compute instances update-from-file VM_NAME \
    --source=PATH_TO_FILE \
    --most-disruptive-allowed-action=RESTART

Replace the following:

VM_NAME: the name of the VM
PATH_TO_FILE: your .yaml file name

The --most-disruptive-allowed-action=RESTART flag setting automatically restarts your VM with the updated configuration.

API

Call the Compute Engine API instances.update method to modify the network configuration.

PUT https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/RESOURCE_ID?most_disruptive_allowed_action=RESTART

{
  "networkPerformanceConfig":{
    "totalEgressBandwidthTier": "TIER_1"
    },
    ...
 }

Replace the following:

PROJECT_ID: your project ID
ZONE: the zone where your VM resides
RESOURCE_ID: the name of your VM

Verifying high-bandwidth configuration in a VM

Use the Google Cloud console, the Google Cloud CLI or the Compute Engine API to generate a description of an existing VM or an existing VM running container images to verify the VM's bandwidth tier.

Console

In the Google Cloud console, go to the VM instances page.

Go to VM instances
Select your project and click Continue.
Click on the instance to see its configuration details.

gcloud

Use the gcloud compute instances describe command to check if your VM has a high-bandwidth network configuration. This output might be missing if you did not set a network performance configuration during VM creation.

For example:

gcloud compute instances describe VM_NAME
    --format="text(VM_NAME, networkPerformanceConfig)"

The output is similar to the following:

  name: instance-1
  network-performance-configs.totalEgressBandwidthTier:TIER_1

API

Call the Compute Engine API instances.get method to view the network configuration. If the high-bandwidth is not set at VM creation, this output might be missing from the output.

GET https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/RESOURCE_ID/

{
  "name": RESOURCE_ID,
    "description": string,
    ...
    "networkPerformanceConfig": {
        "totalEgressBandwidthTier": "TIER_1"
    },
    ...
  }

Replace the following:

PROJECT_ID: your project name
ZONE: the zone where your VM resides
RESOURCE_ID: the name of your VM

Creating an instance template with high-bandwidth configuration

Use the Google Cloud console, the Google Cloud CLI or the Compute Engine API to create an instance template with a high-bandwidth configuration. Refer to the Creating an instance template documentation to ensure you are meeting all the requirements to create your VM instance template.

Console

In the Cloud console, go to the Instance templates page.

Go to Instance templates
Click Create instance template.
Enter values for the following fields, or accept the default values.
Specify a Name for your instance template. For more information, see Resource naming convention.
Select a region and zone that supports a high-bandwidth configuration.
Select a Machine configuration for your VM. To create a a high-bandwidth VM, you must select an N2, N2D, C2, or C2D VM from the Series drop-down menu. Your machine type must align with the vCPU bandwidth tier size requirement.
In the Boot disk section, click Change, and then select a gVNIC-compatible or custom image.
Specify your firewall rules.
Open the NETWORKING, DISKS, SECURITY, MANAGEMENT, SOLE-TENANCY menu to change your networking properties.
Click on Networking and then do the following:
1. Under the Network interface card drop-down menu, select the gVNIC network interface card.
2. Under Network bandwidth check the Increase total egress bandwidth box.
3. Specify your Network interfaces.
Click Create.

gcloud

Use the gcloud compute instance-templates create command with the --network-performance-configs, and the --network-interface flags.

gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME \
    --image=MACHINE_IMAGE \
    --machine-type=MACHINE_TYPE \
    --network-performance-configs=total-egress-bandwidth-tier=TIER_1 \
    --network-interface=nic-type=GVNIC

Replace the following:

INSTANCE_TEMPLATE_NAME: the name of your instance template
MACHINE_IMAGE: use a gVNIC image
MACHINE_TYPE: a machine type that supports a high-bandwidth configuration.

For example:

gcloud compute instance-templates create instance-template-1 \
    --image=projects/ubuntu-os-cloud/global/images/ubuntu-2110-impish-v20211014 \
    --network-performance-configs=total-egress-bandwidth-tier=TIER_1 \
    --machine-type=n2-standard-32 \
    --network-interface=nic-type=GVNIC

API

Call the Compute Engine API instanceTemplates.insert method. Within the request body, set the networkPerformanceConfig parameter to, totalEgressBandwidthTier and TIER_1. Set the networkInterfaces parameter to nicType and GVNIC.

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/instancesTemplates

{
  "name": "INSTANCE_TEMPLATE_NAME",
  "properties": {
    "machineType": "zones/ZONE/machineTypes/MACHINE_TYPE",
    ...
    "networkPerformanceConfig": {
        "totalEgressBandwidthTier": "TIER_1"
  },
    "networkInterfaces": [
      {
        "nicType": "GVNIC"
    },
    ...
  }

Replace the following:

PROJECT_ID: your project name
INSTANCE_TEMPLATE_NAME: your instance template name
ZONE: the zone where your VM is located
MACHINE_TYPE: the machine type of the VM
RESOURCE_ID: the name of your VM

Benchmarking a higher bandwidth configuration

You can run a benchmark test to check your VM's performance with a higher bandwidth configuration. Be sure to remove the benchmarking resources you created during testing to avoid unexpected resource charges.

What's next

High-bandwidth pricing