Configuring a VM with higher bandwidth

Compute Engine lets you select a high-bandwidth Tier 1 configuration for N2, N2D, and C2 virtual machine (VM) instances. VMs with high-bandwidth configurations are especially useful for large, distributed compute workloads with lots of heavy internode communication such as high performance computing (HPC), machine learning (ML), and deep learning (DL). You can expect the best network performance for traffic within the same zone, using internal IP addresses.

Combining these high throughput VMs with high-performance local SSD is beneficial for I/O-intensive, flash-optimized databases.

Before you begin

• If you want to use the command-line examples in this guide, do the following:
  1. Install or update to the latest version of the gcloud command-line tool.
  2. Set a default region and zone.
• If you want to use the API examples in this guide, set up API access.

Limitations

• Configure this feature using the gcloud tool or the Compute Engine API.
• Compute Engine is the only product area supporting this feature.
• This feature is excluded from committed use discounts.
• Tier 1 bandwidth configuration is only supported on N2, N2D EPYC Milan, and C2 VMs.
• Tier 1 bandwidth configuration is only compatible with VMs that are running the gVNIC virtual network driver.

Bandwidth tiers

Total egress bandwidth is the available outbound bandwidth from a VM, regardless of whether the traffic is going to destinations that have an internal IP address or external IP address. You can think of this bandwidth as the physical NIC speed for a given VM.

• Default bandwidth ranges from 10 Gbps to 32 Gbps depending on the machine family and VM size.
• Tier 1 bandwidth increases the maximum egress bandwidth for VMs, and ranges from 50 Gbps to 100 Gbps depending on your N2, N2D, or C2 instance.

General-purpose N2 VMs

The following table describes the egress bandwidth configurations for N2 VMs.

vCPUs	Default	Tier 1
2	10 Gbps	Not applicable (N/A)
4	10 Gbps	N/A
8	16 Gbps	N/A
16	32 Gbps	N/A
32	32 Gbps	50 Gbps
48	32 Gbps	50 Gbps
64	32 Gbps	75 Gbps
80	32 Gbps	100 Gbps
96	32 Gbps	100 Gbps
128	32 Gbps	100 Gbps

General-purpose N2 (custom size shapes) VMs

The following table describes the egress bandwidth configurations for custom-sized N2 VMs.

vCPUs	Default	Tier 1
16-30	32 Gbps	N/A
32-62	32 Gbps	50 Gbps
64-78	32 Gbps	75 Gbps
80-128	32 Gbps	100 Gbps

General-purpose N2D VMs

The following table describes the egress bandwidth configurations for N2D VMs.

vCPUs	Default	Tier 1
2	10 Gbps	Not applicable (N/A)
4	10 Gbps	N/A
8	16 Gbps	N/A
16	32 Gbps	N/A
32	32 Gbps	N/A
48	32 Gbps	50 Gbps
64	32 Gbps	50 Gbps
80	32 Gbps	50 Gbps
96	32 Gbps	100 Gbps
128	32 Gbps	100 Gbps
224	32 Gbps	100 Gbps

General-purpose N2D (custom size shapes) VMs

The following table describes the egress bandwidth configurations for custom-sized N2D VMs.

vCPUs	Default	Tier 1
48-94	32 Gbps	50 Gbps
96	32 Gbps	100 Gbps

Compute-optimized C2 VMs

The following table describes the egress bandwidth configurations for C2 VMs.

vCPUs	Default	Tier 1
4	10 Gbps	N/A
8	16 Gbps	N/A
16	32 Gbps	N/A
30	32 Gbps	50 Gbps
60	32 Gbps	100 Gbps

Creating a VM with high-bandwidth configuration

You can enable a high-bandwidth configuration during VM creation. However, you can also edit a VM to add or remove a high-bandwidth configuration at any time. Your VM must be in a STOPPED state before you can edit its configuration. RESTART your VM to apply the updated higher configuration.

Creating VMs and containers with high-bandwidth configuration

Note: Permissions for this feature are the same as the permissions required for creating or updating a VM or a container.

Use the gcloud command-line tool or Compute Engine API to add a high-bandwidth network configuration to a new VM or to create a VM running container images.

Use the gcloud beta compute instances create command to create a VM with a gVNIC virtual network driver. Use the --network-performance-configs flag and the --network-interface flag to configure a network performance setting for a VM. If you do not specify these flags the VM is created with the default network performance configuration.

gcloud

To create a VM, use the gcloud beta compute instances create command. To create a VM running container images, use the gcloud beta compute instances create-with-container command.

gcloud beta compute instances create VM_NAME \
    --image=MACHINE_IMAGE \
    --machine-type=MACHINE_TYPE \
    --network-performance-configs=PROPERTY=VALUE \
    --network-interface=PROPERTY=VALUE

Replace the following:

• VM_NAME: the name of your VM
• MACHINE_IMAGE: use a gVNIC image
• MACHINE_TYPE: a machine type that supports a high-bandwidth configuration
• PROPERTY: enter total-egress-bandwidth-tier
• VALUE: enter TIER_1
• PROPERTY: enter nic-type
• VALUE: enter GVNIC

For example:

gcloud beta compute instances create instance-1 \
    --network-performance-configs=total-egress-bandwidth-tier=TIER_1 \
    --network-interface=nic-type=GVNIC \
    --image=projects/ubuntu-os-cloud/global/images/ubuntu-2110-impish-v20211014 \
    --machine-type=n2-standard-32

API

Call the Compute Engine API instances.insert method to create a VM with a high-bandwidth network configuration. Within the request body, set the networkPerformanceConfigs parameters to, totalEgressBandwidthTier and TIER_1. Set the networkInterface parameters to nicType and GVNIC.

POST https://compute.googleapis.com/compute/beta/projects/PROJECT_ID/zones/ZONE/instances

{
  "name": VM_NAME,
    "description": string,
    ...
    "networkPerformanceConfigs": {
        "totalEgressBandwidthTier": TIER_1
  },
    "networkInterfaces": [
      {
        "nicType": "GVNIC"
    },
    ...
  }

Replace the following:

• PROJECT_ID: your project ID
• ZONE: the zone where you want to create the VM
• VM_NAME: the name of your VM

Updating a VM to include high-bandwidth configuration

Refer to the Updating instance properties documentation to ensure you are meeting all the requirements to successfully update your VM. Use the gcloud command-line tool or the Compute Engine API to update a VM to include Tier 1 high-bandwidth network configuration.

gcloud

To edit a VM, use the gcloud beta compute instances export command to export your VM's information to a .yaml file.

 gcloud beta compute instances export VM_NAME \
     --destination=PATH_TO_FILE

Replace the following:

• VM_NAME: the name of your VM
• PATH_TO_FILE: a file name with a .yaml extension.

For example:

 gcloud beta compute instances export instance-1 \
     --destination=test-file.yaml

1. Open the gcloud text editor, or the editor or your choice.
2. Open the .yaml file you created.
3. Scroll to the end of the file and add the following information:

  networkPerformanceConfig:
    totalEgressBandwidthTier: TIER_1

Setting totalEgressBandwidthTier to TIER_1 adds a high-bandwidth configuration. Setting it to DEFAULT removes the configuration.

Next, use the gcloud beta compute instance update-from-file command to update the instance with the file changes.

gcloud beta compute instances update-from-file VM_NAME \
    --source=PATH_TO_FILE \
    --most-disruptive-allowed-action=RESTART

Replace the following:

• VM_NAME: the name of your VM
• PATH_TO_FILE: your .yaml file name

The --most-disruptive-allowed-action=RESTART flag setting automatically restarts your VM with the updated configuration.

API

Call the Compute Engine API instances.update method to modify the network configuration.

PUT https://compute.googleapis.com/compute/beta/projects/PROJECT_ID/zones/ZONE/instances/RESOURCE_ID?most_disruptive_allowed_action=RESTART

{
  "networkPerformanceConfig":{
    "totalEgressBandwidthTier": "TIER_1"
    },
    ...
 }

Replace the following:

• PROJECT_ID: your project ID
• ZONE: the zone where your VM resides
• RESOURCE_ID: the name of your VM

Verifying high-bandwidth configuration in a VM

Generate a description of an existing VM or an existing VM running container images to verify the VM's bandwidth tier.

gcloud

Use the gcloud beta compute instances describe command to check if your VM has a high-bandwidth network configuration. This output might be missing if you did not set a network performance configuration during VM creation.

For example:

gcloud beta compute instances describe VM_NAME
    --format="text(VM_NAME, networkPerformanceConfig)"

The output is similar to the following:

  name: instance-1
  network-performance-configs.totalEgressBandwidthTier:TIER_1
 

API

Call the Compute Engine API instances.get method to view the network configuration. If the high-bandwidth is not set at VM creation, this output might be missing from the output.

GET https://compute.googleapis.com/compute/beta/projects/PROJECT_ID/zones/ZONE/instances/RESOURCE_ID/

{
  "name": RESOURCE_ID,
    "description": string,
    ...
    "networkPerformanceConfigs": {
        "totalEgressBandwidthTier": "TIER_1"
    },
    ...
  }

Replace the following:

• PROJECT_ID: your project name
• ZONE: the zone where your VM resides
• RESOURCE_ID: the name of your VM

Creating an instance template with high-bandwidth configuration

Refer to the Creating an instance template documentation to ensure you are meeting all the requirements to successfully create your VM instance template.

gcloud

Use the gcloud beta compute instance-templates create command with the --network-performance-configs, and the --network-interface flags.

gcloud beta compute instance-templates create INSTANCE_TEMPLATE_NAME \
    --image=MACHINE_IMAGE \
    --machine-type=MACHINE_TYPE \
    --network-performance-configs=PROPERTY=VALUE \
    --network-interface=PROPERTY=VALUE

Replace the following:

• INSTANCE_TEMPLATE_NAME: the name of your instance template
• MACHINE_IMAGE: use a gVNIC image
• MACHINE_TYPE: a machine type that supports a high-bandwidth configuration.
• PROPERTY: enter total-egress-bandwidth-tier
• VALUE: enter TIER_1
• PROPERTY: enter nic-type
• VALUE: enter GVNIC

For example:

gcloud beta compute instance-templates create instance-template-1 \
    --image=projects/ubuntu-os-cloud/global/images/ubuntu-2110-impish-v20211014 \
    --network-performance-configs=total-egress-bandwidth-tier=TIER_1 \
    --machine-type=n2-standard-32 \
    --network-interface=nic-type=GVNIC

API

Call the Compute Engine API instanceTemplates.insert method. Within the request body, set the networkPerformanceConfigs parameter to, totalEgressBandwidthTier and TIER_1. Set the networkInterfaces parameter to nicType and GVNIC.

POST https://compute.googleapis.com/compute/beta/projects/PROJECT_ID/global/instancesTemplates

{
  "name": "INSTANCE_TEMPLATE_NAME",
  "properties": {
    "machineType": "zones/ZONE/machineTypes/MACHINE_TYPE",
    ...
    "networkPerformanceConfigs": {
        "totalEgressBandwidthTier": "TIER_1"
  },
    "networkInterfaces": [
      {
        "nicType": "GVNIC"
    },
    ...
  }

Replace the following:

• PROJECT_ID: your project name
• INSTANCE_TEMPLATE_NAME: your instance template name
• ZONE: the zone where your VM is located
• MACHINE_TYPE: the machine type of the VM
• RESOURCE_ID: the name of your VM

Benchmarking a higher bandwidth configuration

You can run a benchmark test to check your VM's performance with a higher bandwidth configuration. Be sure to remove the benchmarking resources you created during testing to avoid unexpected resource charges.

What's next

• High-bandwidth pricing