Compute Engine lets you select a high-bandwidth Tier 1 configuration for certain general-purpose and compute-optimized virtual machine (VM) instances. VMs with high-bandwidth configurations are especially useful for large, distributed compute workloads with lots of heavy internode communications such as high performance computing (HPC), machine learning (ML), and deep learning (DL). You can expect the best network performance for traffic within the same zone, using internal IP addresses.
Combining these high throughput VMs with high-performance local SSD is beneficial for I/O-intensive, flash-optimized databases.
High-bandwidth configurations require:
- VMs in the N2, N2D, C2, or C2D series.
- Google virtual NIC (gVNIC) network driver.
- gVNIC-compatible OS or custom image.
Before you begin
- If you want to use the command-line examples in this guide, do the following:
- Install or update to the latest version of the Google Cloud CLI.
- Set a default region and zone.
- If you want to use the API examples in this guide, set up API access.
Limitations
- Compute Engine is the only product area supporting this feature.
- This feature is excluded from committed use discounts.
- Tier 1 bandwidth configuration is only supported on N2, N2D EPYC Milan, C2 and C2D VMs.
- Tier 1 bandwidth configuration is only compatible with VMs that are running the gVNIC virtual network driver.
Bandwidth tiers
Total egress bandwidth is the available outbound bandwidth from a VM, regardless of whether the traffic is going to destinations that have an internal IP address or external IP address. You can think of this bandwidth as the physical NIC speed for a given VM.
- Default bandwidth ranges from 10 Gbps to 32 Gbps depending on the machine family and VM size.
- Tier 1 bandwidth increases the maximum egress bandwidth for VMs, and ranges from 50 Gbps to 100 Gbps depending on the size of your N2, N2D, C2 or C2D VM.
General-purpose N2 VMs
The following table describes the egress bandwidth configurations for N2 VMs.
vCPUs | Default | Tier 1 |
---|---|---|
2 | 10 Gbps | Not applicable (N/A) |
4 | 10 Gbps | N/A |
8 | 16 Gbps | N/A |
16 | 32 Gbps | N/A |
32 | 32 Gbps | 50 Gbps |
48 | 32 Gbps | 50 Gbps |
64 | 32 Gbps | 75 Gbps |
80 | 32 Gbps | 100 Gbps |
96 | 32 Gbps | 100 Gbps |
128 | 32 Gbps | 100 Gbps |
General-purpose N2 (custom size shapes) VMs
The following table describes the egress bandwidth configurations for custom-sized N2 VMs.
vCPUs | Default | Tier 1 |
---|---|---|
32-62 | 32 Gbps | 50 Gbps |
64-78 | 32 Gbps | 75 Gbps |
80-128 | 32 Gbps | 100 Gbps |
General-purpose N2D VMs
The following table describes the egress bandwidth configurations for N2D VMs.
vCPUs | Default | Tier 1 |
---|---|---|
2 | 10 Gbps | Not applicable (N/A) |
4 | 10 Gbps | N/A |
8 | 16 Gbps | N/A |
16 | 32 Gbps | N/A |
32 | 32 Gbps | N/A |
48 | 32 Gbps | 50 Gbps |
64 | 32 Gbps | 50 Gbps |
80 | 32 Gbps | 50 Gbps |
96 | 32 Gbps | 100 Gbps |
128 | 32 Gbps | 100 Gbps |
224 | 32 Gbps | 100 Gbps |
General-purpose N2D (custom size shapes) VMs
The following table describes the egress bandwidth configurations for custom-sized N2D VMs.
vCPUs | Default | Tier 1 |
---|---|---|
48-94 | 32 Gbps | 50 Gbps |
96 | 32 Gbps | 100 Gbps |
Compute-optimized C2 VMs
The following table describes the egress bandwidth configurations for C2 VMs.
vCPUs | Default | Tier 1 |
---|---|---|
4 | 10 Gbps | Not applicable (N/A) |
8 | 16 Gbps | N/A |
16 | 32 Gbps | N/A |
30 | 32 Gbps | 50 Gbps |
60 | 32 Gbps | 100 Gbps |
Compute-optimized C2D VMs
The following table describes the egress bandwidth configurations for C2D VMs.
vCPUs | Default | Tier 1 |
---|---|---|
2 | 10 Gbps | Not applicable (N/A) |
4 | 10 Gbps | N/A |
8 | 16 Gbps | N/A |
16 | 32 Gbps | N/A |
32 | 32 Gbps | 50 Gbps |
56 | 32 Gbps | 50 Gbps |
112 | 32 Gbps | 100 Gbps |
Creating a VM with high-bandwidth configuration
You can enable a high-bandwidth configuration during VM creation.
However, you can also edit a VM to add or remove a high-bandwidth configuration
at any time. Your VM must be in a STOPPED
state before you can edit its
configuration. RESTART
your VM to apply the updated higher configuration.
Creating VMs and containers with high-bandwidth configuration
Use the Google Cloud console, the Google Cloud CLI or the Compute Engine API to add a high-bandwidth network configuration to a new VM or to create a VM running container images.
Console
In the Google Cloud console, go to the VM instances page.
Select your project and click Continue.
Click Create instance.
Specify a Name for your VM. For more information, see Resource naming convention.
Select a region and zone that supports a high-bandwidth configuration.
Select a Machine configuration for your VM. To create a a high-bandwidth VM, you must select an N2, N2D, C2, or C2D VM from the Series drop-down menu. Your machine type must align with the bandwidth tier size requirement.
In the Boot disk section, click Change, and then select a gVNIC-compatible or custom image.
Specify your firewall rules.
Open the NETWORKING, DISKS, SECURITY, MANAGEMENT, SOLE-TENANCY menu to change your networking properties.
Click on Networking and then do the following:
- Under the Network interface card drop-down menu, select the gVNIC network interface card.
- Under Network bandwidth check the Increase total egress bandwidth box.
- Specify your Network interfaces.
Click Create.
gcloud
Use the
gcloud compute instances create command
to create a VM with a gVNIC virtual network driver.
Use the --network-performance-configs
flag and the --network-interface
flag
to configure a network performance setting for a VM. If you do not specify
these flags the VM is created with the default network performance configuration.
To create a VM running container images, use the gcloud compute instances create-with-container command.
gcloud compute instances create VM_NAME \ --image=MACHINE_IMAGE \ --machine-type=MACHINE_TYPE \ --network-performance-configs=total-egress-bandwidth-tier=TIER_1 \ --network-interface=nic-type=GVNIC
Replace the following:
- VM_NAME: the name of the VM
- MACHINE_IMAGE: use a gVNIC image
- MACHINE_TYPE: a machine type that supports a high-bandwidth configuration
For example:
gcloud compute instances create instance-1 \ --network-performance-configs=total-egress-bandwidth-tier=TIER_1 \ --network-interface=nic-type=GVNIC \ --image=projects/ubuntu-os-cloud/global/images/ubuntu-2110-impish-v20211014 \ --machine-type=n2-standard-32
API
Call the Compute Engine API
instances.insert method
to create a VM with a high-bandwidth network configuration. Within the
request body, set the
networkPerformanceConfig
parameters to, totalEgressBandwidthTier
and
TIER_1
. Set the networkInterface
parameters to nicType
and GVNIC
.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances { "name": VM_NAME, "description": string, ... "networkPerformanceConfig": { "totalEgressBandwidthTier": TIER_1 }, "networkInterfaces": [ { "nicType": "GVNIC" }, ... }
Replace the following:
- PROJECT_ID: your project ID
- ZONE: the zone where you want to create the VM
- VM_NAME: the name of the VM
Updating a VM to include high-bandwidth configuration
Refer to the Updating instance properties documentation to ensure you are meeting all the requirements to successfully update your VM. Use the Google Cloud console, the Google Cloud CLI or the Compute Engine API to update a VM to include a Tier 1 high-bandwidth network configuration.
Console
You can only edit a VM to change the VM size or to change the network configuration to include or exclude a high-bandwidth configuration. Your VM must already have a gVNIC card associated with it. It is not possible to edit your VM to add a network card.
In the Google Cloud console, go to the VM instances page.
Select your project and click Continue.
Select the VM you wish to change.
Stop the VM.
Select Edit VM.
Update the VM size or edit your VM's high-bandwidth network configuration. You can only make this change if your VM was originally configured with a gVNIC card.
Save your changes.
Restart your VM.
gcloud
To edit a VM, use the gcloud compute instances export command to export your VM's information to a .yaml file.
gcloud compute instances export VM_NAME \ --zone=ZONE --destination=PATH_TO_FILE
Replace the following:
- VM_NAME: the name of the VM
- ZONE: the name of the zone where the VM is located
- PATH_TO_FILE: a file name with a
.yaml
extension.
For example:
gcloud compute instances export instance-1 \ --zone=europe-west1-a --destination=test-file.yaml
- Open the gcloud text editor, or the editor or your choice.
- Open the
.yaml
file you created. - Scroll to the end of the file and add the following information:
networkPerformanceConfig: totalEgressBandwidthTier: TIER_1
Setting totalEgressBandwidthTier
to TIER_1
adds a high-bandwidth
configuration. Setting it to DEFAULT
removes the configuration.
Next, use the gcloud compute instance update-from-file command to update the VM with the file changes.
gcloud compute instances update-from-file VM_NAME \ --source=PATH_TO_FILE \ --most-disruptive-allowed-action=RESTART
Replace the following:
- VM_NAME: the name of the VM
- PATH_TO_FILE: your
.yaml
file name
The --most-disruptive-allowed-action=RESTART
flag setting automatically
restarts your VM with the updated configuration.
API
Call the Compute Engine API instances.update method to modify the network configuration.
PUT https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/RESOURCE_ID?most_disruptive_allowed_action=RESTART { "networkPerformanceConfig":{ "totalEgressBandwidthTier": "TIER_1" }, ... }
Replace the following:
- PROJECT_ID: your project ID
- ZONE: the zone where your VM resides
- RESOURCE_ID: the name of your VM
Verifying high-bandwidth configuration in a VM
Use the Google Cloud console, the Google Cloud CLI or the Compute Engine API to generate a description of an existing VM or an existing VM running container images to verify the VM's bandwidth tier.
Console
In the Google Cloud console, go to the VM instances page.
Select your project and click Continue.
Click on the instance to see its configuration details.
gcloud
Use the gcloud compute instances describe command to check if your VM has a high-bandwidth network configuration. This output might be missing if you did not set a network performance configuration during VM creation.
For example:
gcloud compute instances describe VM_NAME --format="text(VM_NAME, networkPerformanceConfig)"
The output is similar to the following:
name: instance-1 network-performance-configs.totalEgressBandwidthTier:TIER_1
API
Call the Compute Engine API instances.get method to view the network configuration. If the high-bandwidth is not set at VM creation, this output might be missing from the output.
GET https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/RESOURCE_ID/ { "name": RESOURCE_ID, "description": string, ... "networkPerformanceConfig": { "totalEgressBandwidthTier": "TIER_1" }, ... }
Replace the following:
- PROJECT_ID: your project name
- ZONE: the zone where your VM resides
- RESOURCE_ID: the name of your VM
Creating an instance template with high-bandwidth configuration
Use the Google Cloud console, the Google Cloud CLI or the Compute Engine API to create an instance template with a high-bandwidth configuration. Refer to the Creating an instance template documentation to ensure you are meeting all the requirements to create your VM instance template.
Console
In the Cloud console, go to the Instance templates page.
Click Create instance template.
Enter values for the following fields, or accept the default values.
Specify a Name for your instance template. For more information, see Resource naming convention.
Select a region and zone that supports a high-bandwidth configuration.
Select a Machine configuration for your VM. To create a a high-bandwidth VM, you must select an N2, N2D, C2, or C2D VM from the Series drop-down menu. Your machine type must align with the vCPU bandwidth tier size requirement.
In the Boot disk section, click Change, and then select a gVNIC-compatible or custom image.
Specify your firewall rules.
Open the NETWORKING, DISKS, SECURITY, MANAGEMENT, SOLE-TENANCY menu to change your networking properties.
Click on Networking and then do the following:
- Under the Network interface card drop-down menu, select the gVNIC network interface card.
- Under Network bandwidth check the Increase total egress bandwidth box.
- Specify your Network interfaces.
Click Create.
gcloud
Use the gcloud compute instance-templates create command
with the --network-performance-configs
, and the --network-interface
flags.
gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME \ --image=MACHINE_IMAGE \ --machine-type=MACHINE_TYPE \ --network-performance-configs=total-egress-bandwidth-tier=TIER_1 \ --network-interface=nic-type=GVNIC
Replace the following:
- INSTANCE_TEMPLATE_NAME: the name of your instance template
- MACHINE_IMAGE: use a gVNIC image
- MACHINE_TYPE: a machine type that supports a high-bandwidth configuration.
For example:
gcloud compute instance-templates create instance-template-1 \ --image=projects/ubuntu-os-cloud/global/images/ubuntu-2110-impish-v20211014 \ --network-performance-configs=total-egress-bandwidth-tier=TIER_1 \ --machine-type=n2-standard-32 \ --network-interface=nic-type=GVNIC
API
Call the Compute Engine API
instanceTemplates.insert method.
Within the request body, set the networkPerformanceConfig
parameter
to, totalEgressBandwidthTier
and TIER_1
. Set the networkInterfaces
parameter to nicType
and GVNIC
.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/instancesTemplates { "name": "INSTANCE_TEMPLATE_NAME", "properties": { "machineType": "zones/ZONE/machineTypes/MACHINE_TYPE", ... "networkPerformanceConfig": { "totalEgressBandwidthTier": "TIER_1" }, "networkInterfaces": [ { "nicType": "GVNIC" }, ... }
Replace the following:
- PROJECT_ID: your project name
- INSTANCE_TEMPLATE_NAME: your instance template name
- ZONE: the zone where your VM is located
- MACHINE_TYPE: the machine type of the VM
- RESOURCE_ID: the name of your VM
Benchmarking a higher bandwidth configuration
You can run a benchmark test to check your VM's performance with a higher bandwidth configuration. Be sure to remove the benchmarking resources you created during testing to avoid unexpected resource charges.