Configure per VM Tier_1 networking performance

---

Compute Engine lets you select a high-bandwidth per VM Tier_1 networking performance configuration for certain general-purpose and compute-optimized virtual machine (VM) instances. VMs with Tier_1 networking configurations are especially useful for large, distributed compute workloads with lots of heavy internode communications, such as high performance computing (HPC), machine learning (ML), and deep learning (DL).

Combining these high throughput VMs with high-performance local SSD storage is beneficial for I/O-intensive, flash-optimized databases.

Tier_1 networking configurations require:

• VMs in the N2, N2D, C2, or C2D series that have at least 30 vCPUs.
• Google virtual NIC (gVNIC) network driver.
• gVNIC-compatible OS or custom image.

Before you begin

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
2. Install the Google Cloud CLI.

3. To initialize the gcloud CLI, run the following command:

   gcloud init

4. Create or select a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   • Create a Cloud project:

     gcloud projects create PROJECT_ID

   • Select the Cloud project that you created:

     gcloud config set project PROJECT_ID

5. Make sure that billing is enabled for your Cloud project. Learn how to check if billing is enabled on a project.

6. Enable the Compute Engine API:

   gcloud services enable compute.googleapis.com

7. Install the Google Cloud CLI.

8. To initialize the gcloud CLI, run the following command:

   gcloud init

9. Create or select a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   • Create a Cloud project:

     gcloud projects create PROJECT_ID

   • Select the Cloud project that you created:

     gcloud config set project PROJECT_ID

10. Make sure that billing is enabled for your Cloud project. Learn how to check if billing is enabled on a project.

11. Enable the Compute Engine API:

    gcloud services enable compute.googleapis.com

Limitations

• Compute Engine is the only product area supporting Tier_1 networking.
• Purchasable stock keeping units (SKUs) for Tier_1 networking are excluded from committed use discounts.
• Tier_1 networking is only supported on N2, N2D, C2, and C2D VMs that have at least 30 vCPUs.
• Tier_1 networking requires VMs that use the gVNIC virtual network driver.

Bandwidth tiers

The egress bandwidth limit represents the maximum possible amount of data per unit of time (for example, gigabits per second, Gbps) that Google Cloud allows a VM to emit from its network interfaces (NICs). The egress bandwidth includes data transferred to all persistent disks attached to the VM.

• The default bandwidth limit ranges from 10 Gbps to 32 Gbps, depending on the machine type and VM size.
• Tier_1 networking increases the maximum egress bandwidth limit for VMs. The maximum egress bandwidth limit ranges from 50 Gbps to 100 Gbps, depending on the size of your VM.
• The actual egress bandwidth is always less than or equal to the egress bandwidth limit.

To achieve the highest possible egress bandwidth, all of the following must be true:

• The sending and receiving VMs must be in the same zone.
• The VMs must have NICs in the same VPC network or in VPC networks connected by VPC Network Peering.
• Packets sent between the VMs must use internal IP address destinations.

For a complete discussion about egress and ingress bandwidth limits, see Network bandwidth.

General-purpose N2 VMs

The following table describes the egress bandwidth limits for N2 VMs.

vCPUs	Internal IP	Tier_1 Internal IP	External IP	Tier_1 External IP
2	10 Gbps	Not applicable (N/A)	7 Gbps	N/A
4	10 Gbps	N/A	7 Gbps	N/A
8	16 Gbps	N/A	7 Gbps	N/A
16	32 Gbps	N/A	7 Gbps	N/A
32	32 Gbps	50 Gbps	7 Gbps	25 Gbps
48	32 Gbps	50 Gbps	7 Gbps	25 Gbps
64	32 Gbps	75 Gbps	7 Gbps	25 Gbps
80	32 Gbps	100 Gbps	7 Gbps	25 Gbps
96	32 Gbps	100 Gbps	7 Gbps	25 Gbps
128	32 Gbps	100 Gbps	7 Gbps	25 Gbps

General-purpose N2 (custom size shapes) VMs

The following table describes the egress bandwidth limits for custom-sized N2 VMs.

vCPUs	Internal IP	Tier_1 Internal IP	External IP	Tier_1 External IP
32-62	32 Gbps	50 Gbps	7 Gbps	25 Gbps
64-78	32 Gbps	75 Gbps	7 Gbps	25 Gbps
80-128	32 Gbps	100 Gbps	7 Gbps	25 Gbps

General-purpose N2D VMs

The following table describes the egress bandwidth limits for N2D VMs.

vCPUs	Internal IP	Tier_1 Internal IP	External IP	Tier_1 External IP
2	10 Gbps	Not applicable (N/A)	7 Gbps	N/A
4	10 Gbps	N/A	7 Gbps	N/A
8	16 Gbps	N/A	7 Gbps	N/A
16	32 Gbps	N/A	7 Gbps	N/A
32	32 Gbps	N/A	7 Gbps	N/A
48	32 Gbps	50 Gbps	7 Gbps	25 Gbps
64	32 Gbps	50 Gbps	7 Gbps	25 Gbps
80	32 Gbps	50 Gbps	7 Gbps	25 Gbps
96	32 Gbps	100 Gbps	7 Gbps	25 Gbps
128	32 Gbps	100 Gbps	7 Gbps	25 Gbps
224	32 Gbps	100 Gbps	7 Gbps	25 Gbps

General-purpose N2D (custom size shapes) VMs

The following table describes the egress bandwidth limits for custom-sized N2D VMs.

vCPUs	Internal IP	Tier_1 Internal IP	External IP	Tier_1 External IP
48-94	32 Gbps	50 Gbps	7 Gbps	25 Gbps
96	32 Gbps	100 Gbps	7 Gbps	25 Gbps

Compute-optimized C2 VMs

The following table describes the egress bandwidth limits for C2 VMs.

vCPUs	Internal IP	Tier_1 Internal IP	External IP	Tier_1 External IP
4	10 Gbps	Not applicable (N/A)	7 Gbps	N/A
8	16 Gbps	N/A	7 Gbps	N/A
16	32 Gbps	N/A	7 Gbps	N/A
30	32 Gbps	50 Gbps	7 Gbps	7 Gbps
60	32 Gbps	100 Gbps	7 Gbps	7 Gbps

Compute-optimized C2D VMs

The following table describes the egress bandwidth limits for C2D VMs.

vCPUs	Internal IP	Tier_1 Internal IP	External IP	Tier_1 External IP
2	10 Gbps	Not applicable (N/A)	7 Gbps	N/A
4	10 Gbps	N/A	7 Gbps	N/A
8	16 Gbps	N/A	7 Gbps	N/A
16	32 Gbps	N/A	7 Gbps	N/A
32	32 Gbps	50 Gbps	7 Gbps	7 Gbps
56	32 Gbps	50 Gbps	7 Gbps	7 Gbps
112	32 Gbps	100 Gbps	7 Gbps	7 Gbps

Creating a VM with Tier_1 networking

You can enable Tier_1 networking during VM creation, if the VM uses the gVNIC network interface. You can also edit a VM to add or remove Tier_1 networking, provided the VM was created with the gVNIC interface.

To add or remove Tier_1 networking on an existing VM:

1. Stop the VM.
2. Make the network configuration changes.
3. Restart the VM.

Creating VMs and containers that use Tier_1 networking

Permissions for this feature are the same as the permissions required for creating or updating a VM or a container.

Note: The value "-" in the Network interface card drop down menu indicates that the NIC type can be either gVNIC or VirtioNet(VirtIO) depending on the machine family type. For some machine family types, only gVNIC or VirtIO is available for the network interface card. If both gVNIC and VirtIO are available for a VM, the network interface for the VM defaults to VirtIO.

Use the Google Cloud console, the Google Cloud CLI or the Compute Engine API to add a Tier_1 networking to a new VM or to create a VM running container images.

Console

1. In the Google Cloud console, go to the VM instances page.

   Go to VM instances

2. Select your project and click Continue.

3. Click Create instance.

4. Specify a Name for your VM. For more information, see Resource naming convention.

5. Select a region and zone that supports Tier_1 networking.

6. Select a Machine configuration for your VM. To create a VM with Tier_1 networking, you must select an N2, N2D, C2, or C2D VM from the Series drop-down menu. Your machine type must align with the bandwidth tier size requirement.

7. In the Boot disk section, click Change, and then select a gVNIC-compatible or custom image.

8. Specify your firewall rules.

9. Open the NETWORKING, DISKS, SECURITY, MANAGEMENT, SOLE-TENANCY menu to change your networking properties.

10. Click on Networking and then do the following:

    1. Under the Network interface card drop-down menu, select the gVNIC network interface card.
    2. Under Network bandwidth, select the Enable per VM Tier_1 networking performance checkbox.
    3. Specify your Network interfaces.

11. Click Create.

gcloud

Use the gcloud compute instances create command to create a VM with a gVNIC virtual network driver. Use the ‑‑network‑performance‑configs flag and the ‑‑network‑interface flag to configure a network performance setting for a VM. If you do not specify these flags the VM is created with the default network performance configuration.

To create a VM running container images, use the gcloud compute instances create‑with‑container command.

gcloud compute instances create VM_NAME \
    --image=OS_IMAGE \
    --machine-type=MACHINE_TYPE \
    --network-performance-configs=total-egress-bandwidth-tier=TIER_1 \
    --network-interface=nic-type=GVNIC

Replace the following:

• VM_NAME: the name of the VM
• OS_IMAGE: an image that supports gVNIC image
• MACHINE_TYPE: a machine type that supports a high-bandwidth configuration

For example:

gcloud compute instances create instance-1 \
    --network-performance-configs=total-egress-bandwidth-tier=TIER_1 \
    --network-interface=nic-type=GVNIC \
    --image-family=rocky-linux-8-optimized-gcp \
    --image-project=rocky-linux-cloud \
    --machine-type=n2-standard-32

API

Call the Compute Engine API instances.insert method to create a VM with a high-bandwidth network configuration. Within the request body, set the networkPerformanceConfig parameters to, totalEgressBandwidthTier and TIER_1. Set the networkInterface parameters to nicType and GVNIC.

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances

{
  "name": VM_NAME,
    "description": string,
    ...
    "networkPerformanceConfig": {
        "totalEgressBandwidthTier": TIER_1
  },
    "networkInterfaces": [
      {
        "nicType": "GVNIC"
    },
    ...
  }

Replace the following:

• PROJECT_ID: your project ID
• ZONE: the zone where you want to create the VM
• VM_NAME: the name of the VM

Updating a VM to include Tier_1 networking

Refer to the Updating instance properties documentation to ensure you are meeting all the requirements to successfully update your VM. Use the Google Cloud console, the Google Cloud CLI or the Compute Engine API to update a VM.

You can modify a VM to change the network configuration to include or exclude per VM Tier_1 networking performance. Your VM must already have a gVNIC interface associated with it. You can't edit your VM to add a network interface.

Console

1. In the Google Cloud console, go to the VM instances page.

   Go to VM instances

2. Select your project.

3. Click the name of the VM that you wish to modify.

4. Stop the VM.

5. Select Edit.

6. If your VM was originally configured with a gVNIC card, select the Enable per VM Tier_1 networking performance checkbox to add per VM Tier_1 networking performance, or deselect the checkbox to remove this feature from your VM.

7. Save your changes.

8. Restart your VM.

gcloud

1. Export your VM's information to a .yaml file using the gcloud compute instances export command.

   gcloud compute instances export VM_NAME \
    --zone=ZONE --destination=PATH_TO_FILE
   

   Replace the following:

   • VM_NAME: the name of the VM
   • ZONE: the name of the zone where the VM is located
   • PATH_TO_FILE: a file name with a .yaml extension.

   For example:

    gcloud compute instances export instance-1 \
     --zone=europe-west1-c --destination=test-file.yaml
   

2. Open the gcloud text editor, or the editor or your choice.

3. Open the .yaml file you created.

4. Scroll to the end of the file and add the following information:

   networkPerformanceConfig:
   totalEgressBandwidthTier: TIER_1
   

   Setting totalEgressBandwidthTier to TIER_1 adds Tier_1 networking. Setting it to DEFAULT removes the configuration.

5. Use the gcloud compute instance update‑from‑file command to update the VM with the file changes.

   gcloud compute instances update-from-file VM_NAME \
     --source=PATH_TO_FILE \
     --most-disruptive-allowed-action=RESTART
   

   Replace the following:

   • VM_NAME: the name of the VM
   • PATH_TO_FILE: your .yaml file name

   The ‑‑most‑disruptive‑allowed‑action=RESTART flag setting automatically restarts your VM with the updated configuration.

API

Call the Compute Engine API instances.update method to modify the network configuration.

PUT https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/RESOURCE_ID?most_disruptive_allowed_action=RESTART

{
  "networkPerformanceConfig":{
    "totalEgressBandwidthTier": "TIER_1"
    },
    ...
 }

Setting totalEgressBandwidthTier to TIER_1 adds Tier_1 networking. Setting it to DEFAULT removes the configuration.

Replace the following:

• PROJECT_ID: your project ID
• ZONE: the zone where your VM resides
• RESOURCE_ID: the name of your VM

Verifying high-bandwidth configuration in a VM

Use the Google Cloud console, the Google Cloud CLI or the Compute Engine API to generate a description of an existing VM or an existing VM running container images to verify the VM's bandwidth tier.

Console

1. In the Google Cloud console, go to the VM instances page.

   Go to VM instances

2. Select your project and click Continue.

3. Click on the instance to see its configuration details and see if your VM uses per VM Tier_1 networking performance.

gcloud

Use the gcloud compute instances describe command to check if your VM uses per VM Tier_1 networking performance.

For example:

gcloud compute instances describe VM_NAME  \
    --format="text(name, networkPerformanceConfig)"

The output is similar to the following:

  name: instance-1
  networkPerformanceConfig.totalEgressBandwidthTier:TIER_1
 

If the output shows the value DEFAULT, then Tier_1 networking is not enabled.

API

Call the Compute Engine API instances.get method to view the network configuration.

GET https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/RESOURCE_ID/

Replace the following:

• PROJECT_ID: your project name
• ZONE: the zone where your VM resides
• RESOURCE_ID: the name of your VM

The output should contain the following lines:

{
  "name": RESOURCE_ID,
    "description": string,
    ...
    "networkPerformanceConfig": {
        "totalEgressBandwidthTier": "TIER_1"
    },
    ...
  }

If the output shows the value DEFAULT, then Tier_1 networking is not enabled.

Creating an instance template with Tier_1 networking

Use the Google Cloud console, the Google Cloud CLI or the Compute Engine API to create an instance template with per VM Tier_1 networking performance. Refer to the Creating an instance template documentation to ensure you are meeting all the requirements to create your VM instance template.

Console

1. In the Google Cloud console, go to the Instance templates page.

   Go to Instance templates

2. Click Create instance template.

3. Enter values for the following fields, or accept the default values.

4. Specify a Name for your instance template. For more information, see Resource naming convention.

5. Select a region and zone that supports Tier_1 networking.

6. Select a Machine configuration for your VM. To create a a VM with Tier_1 networking, you must select an N2, N2D, C2, or C2D VM from the Series drop-down menu. Your machine type must align with the vCPU bandwidth tier size requirement.

7. In the Boot disk section, click Change, and then select a gVNIC-compatible or custom image.

8. Specify your firewall rules.

9. Open the NETWORKING, DISKS, SECURITY, MANAGEMENT, SOLE-TENANCY menu to change your networking properties.

10. Click on Networking and then do the following:

    1. Under the Network interface card drop-down menu, select the gVNIC network interface card.
    2. Under Network bandwidth, select the Enable per VM Tier_1 networking performance checkbox.
    3. Specify your Network interfaces.

11. Click Create.

gcloud

Use the gcloud compute instance‑templates create command with the ‑‑network‑performance‑configs, and the ‑‑network‑interface flags.

gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME \
    --image=OS_IMAGE \
    --machine-type=MACHINE_TYPE \
    --network-performance-configs=total-egress-bandwidth-tier=TIER_1 \
    --network-interface=nic-type=GVNIC

Replace the following:

• INSTANCE_TEMPLATE_NAME: the name of your instance template
• OS_IMAGE: an image that supports gVNIC image
• MACHINE_TYPE: a machine type that supports Tier_1 networking, as described in Bandwidth tiers.

For example:

gcloud compute instance-templates create instance-template-1 \
    --image-family=rocky-linux-8-optimized-gcp \
    --image-project=rocky-linux-cloud \
    --network-performance-configs=total-egress-bandwidth-tier=TIER_1 \
    --machine-type=n2-standard-32 \
    --network-interface=nic-type=GVNIC

API

Call the Compute Engine API instanceTemplates.insert method. Within the request body, set the networkPerformanceConfig parameter to totalEgressBandwidthTier and TIER_1. Set the networkInterfaces parameter to nicType and GVNIC.

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/instancesTemplates

{
  "name": "INSTANCE_TEMPLATE_NAME",
  "properties": {
    "machineType": "zones/ZONE/machineTypes/MACHINE_TYPE",
    ...
    "networkPerformanceConfig": {
        "totalEgressBandwidthTier": "TIER_1"
  },
    "networkInterfaces": [
      {
        "nicType": "GVNIC"
    },
    ...
  }

Replace the following:

• PROJECT_ID: your project name
• INSTANCE_TEMPLATE_NAME: your instance template name
• ZONE: the zone where your VM is located
• MACHINE_TYPE: the machine type of the VM
• RESOURCE_ID: the name of your VM

Benchmarking a higher bandwidth configuration

You can run a benchmark test to check your VM's performance with per VM Tier_1 networking performance. Be sure to remove the benchmarking resources you created during testing to avoid unexpected resource charges.

What's next

• High-bandwidth pricing