Compute Engine lets you select a high-bandwidth per VM Tier_1 networking performance configuration for certain general-purpose and compute-optimized virtual machine (VM) instances. VMs with Tier_1 networking configurations are especially useful for large, distributed compute workloads with lots of heavy internode communications, such as high performance computing (HPC), machine learning (ML), and deep learning (DL).
Combining these high throughput VMs with high-performance local SSD storage is beneficial for I/O-intensive, flash-optimized databases.
Tier_1 networking configurations require:
- VMs in the N2, N2D, C2, or C2D series that have at least 30 vCPUs.
- Google virtual NIC (gVNIC) network driver.
- gVNIC-compatible OS or custom image.
Before you begin
- Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
- Install the Google Cloud CLI.
-
To initialize the gcloud CLI, run the following command:
gcloud init
-
Create or select a Google Cloud project.
-
Create a Cloud project:
gcloud projects create PROJECT_ID
-
Select the Cloud project that you created:
gcloud config set project PROJECT_ID
-
-
Make sure that billing is enabled for your Cloud project. Learn how to check if billing is enabled on a project.
-
Enable the Compute Engine API:
gcloud services enable compute.googleapis.com
- Install the Google Cloud CLI.
-
To initialize the gcloud CLI, run the following command:
gcloud init
-
Create or select a Google Cloud project.
-
Create a Cloud project:
gcloud projects create PROJECT_ID
-
Select the Cloud project that you created:
gcloud config set project PROJECT_ID
-
-
Make sure that billing is enabled for your Cloud project. Learn how to check if billing is enabled on a project.
-
Enable the Compute Engine API:
gcloud services enable compute.googleapis.com
Limitations
- Compute Engine is the only product area supporting Tier_1 networking.
- Purchasable stock keeping units (SKUs) for Tier_1 networking are excluded from committed use discounts.
- Tier_1 networking is only supported on N2, N2D, C2, and C2D VMs that have at least 30 vCPUs.
- Tier_1 networking requires VMs that use the gVNIC virtual network driver.
Bandwidth tiers
The egress bandwidth limit represents the maximum possible amount of data per unit of time (for example, gigabits per second, Gbps) that Google Cloud allows a VM to emit from its network interfaces (NICs). The egress bandwidth includes data transferred to all persistent disks attached to the VM.
- The default bandwidth limit ranges from 10 Gbps to 32 Gbps, depending on the machine type and VM size.
- Tier_1 networking increases the maximum egress bandwidth limit for VMs. The maximum egress bandwidth limit ranges from 50 Gbps to 100 Gbps, depending on the size of your VM.
- The actual egress bandwidth is always less than or equal to the egress bandwidth limit.
To achieve the highest possible egress bandwidth, all of the following must be true:
- The sending and receiving VMs must be in the same zone.
- The VMs must have NICs in the same VPC network or in VPC networks connected by VPC Network Peering.
- Packets sent between the VMs must use internal IP address destinations.
For a complete discussion about egress and ingress bandwidth limits, see Network bandwidth.
General-purpose N2 VMs
The following table describes the egress bandwidth limits for N2 VMs.
vCPUs | Internal IP | Tier_1 Internal IP | External IP | Tier_1 External IP |
---|---|---|---|---|
2 | 10 Gbps | Not applicable (N/A) | 7 Gbps | N/A |
4 | 10 Gbps | N/A | 7 Gbps | N/A |
8 | 16 Gbps | N/A | 7 Gbps | N/A |
16 | 32 Gbps | N/A | 7 Gbps | N/A |
32 | 32 Gbps | 50 Gbps | 7 Gbps | 25 Gbps |
48 | 32 Gbps | 50 Gbps | 7 Gbps | 25 Gbps |
64 | 32 Gbps | 75 Gbps | 7 Gbps | 25 Gbps |
80 | 32 Gbps | 100 Gbps | 7 Gbps | 25 Gbps |
96 | 32 Gbps | 100 Gbps | 7 Gbps | 25 Gbps |
128 | 32 Gbps | 100 Gbps | 7 Gbps | 25 Gbps |
General-purpose N2 (custom size shapes) VMs
The following table describes the egress bandwidth limits for custom-sized N2 VMs.
vCPUs | Internal IP | Tier_1 Internal IP | External IP | Tier_1 External IP |
---|---|---|---|---|
32-62 | 32 Gbps | 50 Gbps | 7 Gbps | 25 Gbps |
64-78 | 32 Gbps | 75 Gbps | 7 Gbps | 25 Gbps |
80-128 | 32 Gbps | 100 Gbps | 7 Gbps | 25 Gbps |
General-purpose N2D VMs
The following table describes the egress bandwidth limits for N2D VMs.
vCPUs | Internal IP | Tier_1 Internal IP | External IP | Tier_1 External IP |
---|---|---|---|---|
2 | 10 Gbps | Not applicable (N/A) | 7 Gbps | N/A |
4 | 10 Gbps | N/A | 7 Gbps | N/A |
8 | 16 Gbps | N/A | 7 Gbps | N/A |
16 | 32 Gbps | N/A | 7 Gbps | N/A |
32 | 32 Gbps | N/A | 7 Gbps | N/A |
48 | 32 Gbps | 50 Gbps | 7 Gbps | 25 Gbps |
64 | 32 Gbps | 50 Gbps | 7 Gbps | 25 Gbps |
80 | 32 Gbps | 50 Gbps | 7 Gbps | 25 Gbps |
96 | 32 Gbps | 100 Gbps | 7 Gbps | 25 Gbps |
128 | 32 Gbps | 100 Gbps | 7 Gbps | 25 Gbps |
224 | 32 Gbps | 100 Gbps | 7 Gbps | 25 Gbps |
General-purpose N2D (custom size shapes) VMs
The following table describes the egress bandwidth limits for custom-sized N2D VMs.
vCPUs | Internal IP | Tier_1 Internal IP | External IP | Tier_1 External IP |
---|---|---|---|---|
48-94 | 32 Gbps | 50 Gbps | 7 Gbps | 25 Gbps |
96 | 32 Gbps | 100 Gbps | 7 Gbps | 25 Gbps |
Compute-optimized C2 VMs
The following table describes the egress bandwidth limits for C2 VMs.
vCPUs | Internal IP | Tier_1 Internal IP | External IP | Tier_1 External IP |
---|---|---|---|---|
4 | 10 Gbps | Not applicable (N/A) | 7 Gbps | N/A |
8 | 16 Gbps | N/A | 7 Gbps | N/A |
16 | 32 Gbps | N/A | 7 Gbps | N/A |
30 | 32 Gbps | 50 Gbps | 7 Gbps | 7 Gbps |
60 | 32 Gbps | 100 Gbps | 7 Gbps | 7 Gbps |
Compute-optimized C2D VMs
The following table describes the egress bandwidth limits for C2D VMs.
vCPUs | Internal IP | Tier_1 Internal IP | External IP | Tier_1 External IP |
---|---|---|---|---|
2 | 10 Gbps | Not applicable (N/A) | 7 Gbps | N/A |
4 | 10 Gbps | N/A | 7 Gbps | N/A |
8 | 16 Gbps | N/A | 7 Gbps | N/A |
16 | 32 Gbps | N/A | 7 Gbps | N/A |
32 | 32 Gbps | 50 Gbps | 7 Gbps | 7 Gbps |
56 | 32 Gbps | 50 Gbps | 7 Gbps | 7 Gbps |
112 | 32 Gbps | 100 Gbps | 7 Gbps | 7 Gbps |
Creating a VM with Tier_1 networking
You can enable Tier_1 networking during VM creation, if the VM uses the gVNIC network interface. You can also edit a VM to add or remove Tier_1 networking, provided the VM was created with the gVNIC interface.
To add or remove Tier_1 networking on an existing VM:
- Stop the VM.
- Make the network configuration changes.
- Restart the VM.
Creating VMs and containers that use Tier_1 networking
Permissions for this feature are the same as the permissions required for creating or updating a VM or a container.
Use the Google Cloud console, the Google Cloud CLI or the Compute Engine API to add a Tier_1 networking to a new VM or to create a VM running container images.
Console
In the Google Cloud console, go to the VM instances page.
Select your project and click Continue.
Click Create instance.
Specify a Name for your VM. For more information, see Resource naming convention.
Select a region and zone that supports Tier_1 networking.
Select a Machine configuration for your VM. To create a VM with Tier_1 networking, you must select an N2, N2D, C2, or C2D VM from the Series drop-down menu. Your machine type must align with the bandwidth tier size requirement.
In the Boot disk section, click Change, and then select a gVNIC-compatible or custom image.
Specify your firewall rules.
Open the NETWORKING, DISKS, SECURITY, MANAGEMENT, SOLE-TENANCY menu to change your networking properties.
Click on Networking and then do the following:
- Under the Network interface card drop-down menu, select the gVNIC network interface card.
- Under Network bandwidth, select the Enable per VM Tier_1 networking performance checkbox.
- Specify your Network interfaces.
Click Create.
gcloud
Use the
gcloud compute instances create command
to create a VM with a gVNIC virtual network driver.
Use the ‑‑network‑performance‑configs
flag and the ‑‑network‑interface
flag
to configure a network performance setting for a VM. If you do not specify
these flags the VM is created with the default network performance configuration.
To create a VM running container images, use the gcloud compute instances create‑with‑container command.
gcloud compute instances create VM_NAME \ --image=OS_IMAGE \ --machine-type=MACHINE_TYPE \ --network-performance-configs=total-egress-bandwidth-tier=TIER_1 \ --network-interface=nic-type=GVNIC
Replace the following:
- VM_NAME: the name of the VM
- OS_IMAGE: an image that supports gVNIC image
- MACHINE_TYPE: a machine type that supports a high-bandwidth configuration
For example:
gcloud compute instances create instance-1 \ --network-performance-configs=total-egress-bandwidth-tier=TIER_1 \ --network-interface=nic-type=GVNIC \ --image-family=rocky-linux-8-optimized-gcp \ --image-project=rocky-linux-cloud \ --machine-type=n2-standard-32
API
Call the Compute Engine API
instances.insert method
to create a VM with a high-bandwidth network configuration. Within the
request body, set the
networkPerformanceConfig
parameters to, totalEgressBandwidthTier
and
TIER_1
. Set the networkInterface
parameters to nicType
and GVNIC
.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances { "name": VM_NAME, "description": string, ... "networkPerformanceConfig": { "totalEgressBandwidthTier": TIER_1 }, "networkInterfaces": [ { "nicType": "GVNIC" }, ... }
Replace the following:
- PROJECT_ID: your project ID
- ZONE: the zone where you want to create the VM
- VM_NAME: the name of the VM
Updating a VM to include Tier_1 networking
Refer to the Updating instance properties documentation to ensure you are meeting all the requirements to successfully update your VM. Use the Google Cloud console, the Google Cloud CLI or the Compute Engine API to update a VM.
You can modify a VM to change the network configuration to include or exclude per VM Tier_1 networking performance. Your VM must already have a gVNIC interface associated with it. You can't edit your VM to add a network interface.
Console
In the Google Cloud console, go to the VM instances page.
Select your project.
Click the name of the VM that you wish to modify.
Stop the VM.
Select Edit.
If your VM was originally configured with a gVNIC card, select the Enable per VM Tier_1 networking performance checkbox to add per VM Tier_1 networking performance, or deselect the checkbox to remove this feature from your VM.
Save your changes.
Restart your VM.
gcloud
Export your VM's information to a
.yaml
file using thegcloud compute instances export
command.gcloud compute instances export VM_NAME \ --zone=ZONE --destination=PATH_TO_FILE
Replace the following:
- VM_NAME: the name of the VM
- ZONE: the name of the zone where the VM is located
- PATH_TO_FILE: a file name with a
.yaml
extension.
For example:
gcloud compute instances export instance-1 \ --zone=europe-west1-c --destination=test-file.yaml
Open the gcloud text editor, or the editor or your choice.
Open the
.yaml
file you created.Scroll to the end of the file and add the following information:
networkPerformanceConfig: totalEgressBandwidthTier: TIER_1
Setting
totalEgressBandwidthTier
toTIER_1
adds Tier_1 networking. Setting it toDEFAULT
removes the configuration.Use the
gcloud compute instance update‑from‑file
command to update the VM with the file changes.gcloud compute instances update-from-file VM_NAME \ --source=PATH_TO_FILE \ --most-disruptive-allowed-action=RESTART
Replace the following:
- VM_NAME: the name of the VM
- PATH_TO_FILE: your
.yaml
file name
The
‑‑most‑disruptive‑allowed‑action=RESTART
flag setting automatically restarts your VM with the updated configuration.
API
Call the Compute Engine API instances.update method to modify the network configuration.
PUT https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/RESOURCE_ID?most_disruptive_allowed_action=RESTART { "networkPerformanceConfig":{ "totalEgressBandwidthTier": "TIER_1" }, ... }
Setting totalEgressBandwidthTier
to TIER_1
adds
Tier_1 networking. Setting it to DEFAULT
removes the configuration.
Replace the following:
- PROJECT_ID: your project ID
- ZONE: the zone where your VM resides
- RESOURCE_ID: the name of your VM
Verifying high-bandwidth configuration in a VM
Use the Google Cloud console, the Google Cloud CLI or the Compute Engine API to generate a description of an existing VM or an existing VM running container images to verify the VM's bandwidth tier.
Console
In the Google Cloud console, go to the VM instances page.
Select your project and click Continue.
Click on the instance to see its configuration details and see if your VM uses per VM Tier_1 networking performance.
gcloud
Use the
gcloud compute instances describe
command
to check if your VM uses per VM Tier_1 networking performance.
For example:
gcloud compute instances describe VM_NAME \ --format="text(name, networkPerformanceConfig)"
The output is similar to the following:
name: instance-1 networkPerformanceConfig.totalEgressBandwidthTier:TIER_1
If the output shows the value DEFAULT
, then Tier_1 networking is not
enabled.
API
Call the Compute Engine API
instances.get
method
to view the network configuration.
GET https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/RESOURCE_ID/
Replace the following:
- PROJECT_ID: your project name
- ZONE: the zone where your VM resides
- RESOURCE_ID: the name of your VM
The output should contain the following lines:
{ "name": RESOURCE_ID, "description": string, ... "networkPerformanceConfig": { "totalEgressBandwidthTier": "TIER_1" }, ... }
If the output shows the value DEFAULT
, then Tier_1 networking is not
enabled.
Creating an instance template with Tier_1 networking
Use the Google Cloud console, the Google Cloud CLI or the Compute Engine API to create an instance template with per VM Tier_1 networking performance. Refer to the Creating an instance template documentation to ensure you are meeting all the requirements to create your VM instance template.
Console
In the Google Cloud console, go to the Instance templates page.
Click Create instance template.
Enter values for the following fields, or accept the default values.
Specify a Name for your instance template. For more information, see Resource naming convention.
Select a region and zone that supports Tier_1 networking.
Select a Machine configuration for your VM. To create a a VM with Tier_1 networking, you must select an N2, N2D, C2, or C2D VM from the Series drop-down menu. Your machine type must align with the vCPU bandwidth tier size requirement.
In the Boot disk section, click Change, and then select a gVNIC-compatible or custom image.
Specify your firewall rules.
Open the NETWORKING, DISKS, SECURITY, MANAGEMENT, SOLE-TENANCY menu to change your networking properties.
Click on Networking and then do the following:
- Under the Network interface card drop-down menu, select the gVNIC network interface card.
- Under Network bandwidth, select the Enable per VM Tier_1 networking performance checkbox.
- Specify your Network interfaces.
Click Create.
gcloud
Use the gcloud compute instance‑templates create
command
with the ‑‑network‑performance‑configs
, and the ‑‑network‑interface
flags.
gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME \ --image=OS_IMAGE \ --machine-type=MACHINE_TYPE \ --network-performance-configs=total-egress-bandwidth-tier=TIER_1 \ --network-interface=nic-type=GVNIC
Replace the following:
- INSTANCE_TEMPLATE_NAME: the name of your instance template
- OS_IMAGE: an image that supports gVNIC image
- MACHINE_TYPE: a machine type that supports Tier_1 networking, as described in Bandwidth tiers.
For example:
gcloud compute instance-templates create instance-template-1 \ --image-family=rocky-linux-8-optimized-gcp \ --image-project=rocky-linux-cloud \ --network-performance-configs=total-egress-bandwidth-tier=TIER_1 \ --machine-type=n2-standard-32 \ --network-interface=nic-type=GVNIC
API
Call the Compute Engine API
instanceTemplates.insert
method.
Within the request body, set the networkPerformanceConfig
parameter
to totalEgressBandwidthTier
and TIER_1
. Set the networkInterfaces
parameter to nicType
and GVNIC
.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/instancesTemplates { "name": "INSTANCE_TEMPLATE_NAME", "properties": { "machineType": "zones/ZONE/machineTypes/MACHINE_TYPE", ... "networkPerformanceConfig": { "totalEgressBandwidthTier": "TIER_1" }, "networkInterfaces": [ { "nicType": "GVNIC" }, ... }
Replace the following:
- PROJECT_ID: your project name
- INSTANCE_TEMPLATE_NAME: your instance template name
- ZONE: the zone where your VM is located
- MACHINE_TYPE: the machine type of the VM
- RESOURCE_ID: the name of your VM
Benchmarking a higher bandwidth configuration
You can run a benchmark test to check your VM's performance with per VM Tier_1 networking performance. Be sure to remove the benchmarking resources you created during testing to avoid unexpected resource charges.