Multiple network interfaces

This page provides an overview of multiple network interfaces in a virtual machine (VM) instance, including how they work and sample configurations. For information about creating configurations that use multiple interfaces, see Create VMs with multiple network interfaces.

VMs with multiple network interface controllers are referred to as multi-NIC VMs.

Note: You can't create multiple network interfaces for a bare metal instance.

Google Cloud Virtual Private Cloud (VPC) networks are by default isolated private networking domains. Networks have a global scope and contain regional subnets. VM instances within a VPC network can communicate among themselves by using internal IP addresses as long as firewall rules permit. However, no internal IP address communication is allowed between networks, unless you set up mechanisms such as VPC Network Peering or Cloud VPN.

Every instance in a VPC network has a default network interface. When you configure a network interface, you select a VPC network and a subnet within that VPC network to connect the interface to. You can create additional network interfaces attached to your VMs, but each interface must attach to a different VPC network. Multiple network interfaces let you create configurations in which an instance connects directly to several VPC networks.

Each instance can have up to eight interfaces, depending on the instance's type. For more information, see Maximum number of network interfaces.

The stack type of an interface determines the IP addresses that it can have and the types of subnets to which it can connect. The stack type of an interface can be IPv4-only, dual-stack (IPv4 and IPv6), or IPv6-only (Preview). For more information, see Stack types.

Typically, you might require multiple interfaces if you want to configure an instance as a network appliance that does load balancing, Intrusion Detection and Prevention (IDS/IPS), Web Application Firewall (WAF), or WAN optimization between networks. Multiple network interfaces are also useful when applications running in an instance require traffic separation, such as separation of data plane traffic from management plane traffic.

Each interface on a VM is affected by the MTU of the attached network. For more information about interface MTU, see Maximum transmission unit.

Use cases

Use multiple network interfaces when an individual instance needs access to more than one VPC network, but you don't want to connect both networks directly.

• Network and security function: Multiple network interfaces enable virtualized network appliance functions such as load balancers, network address translation (NAT) servers, and proxy servers that are configured with multiple network interfaces. For more details, see Example 1: Networking and security virtual appliances.

• Perimeter isolation (also known as DMZ isolation): An important best practice in tiered networking architectures is to isolate public-facing services from an internal network and its services. Use multiple network interfaces to create configurations where there are separate network interfaces on the instance, one of them accepting public-facing traffic and another handling backend private traffic that has more restrictive access controls.

  Any resources that can be reached from the internet should be separated from your internal network and its services. This drastically limits the scope and damage that a security breach can cause. For example, you can place a second network interface on each web server that connects to a mid-tier network where an application server resides. The application server can also be dual-homed to a backend network where the database server resides. Each dual-homed instance receives and processes requests on the frontend, initiates a connection to the backend, and then sends requests to the servers on the backend network.

  By configuring separate interfaces, one public-facing and another private- facing, you can apply separate firewall rules and access controls to each interface separately and enforce security functions in communications from the public to private domain. For more information, see Example 2: Using third-party appliances in a Shared VPC network scenario.

Configuration examples

This section examines several common examples of how to use multiple network interfaces.

Example 1: Networking and security virtual appliances

Networking and security virtual appliances, such as web application firewalls (WAF), security application-level firewalls, and WAN accelerators, are usually configured with multiple virtual interfaces. Each of the multiple interfaces is configured with its own internal IP address and, optionally, with its own external IP address.

Figure 1 describes an example configuration of an application-level firewall that controls traffic from the internet to a VPC network. The application-level firewall is implemented in Compute Engine VMs.

In this example, the default route of the appliance VM has been configured to use nic1.

Figure 1. An instance with a VM appliance has three network interfaces. Each interface is connected to a subnet that is in a different VPC network (click to enlarge).

Provision and configure instances for example 1

The following assumes that subnet0, subnet1, and subnet2 already exist, with non-overlapping ranges.

To create the VM and network interfaces in this example, use the following command:

gcloud compute instances create vm-appliance \
    --network-interface subnet=subnet0,no-address \
    --network-interface subnet=subnet1 \
    --network-interface subnet=subnet2,no-address \
    --machine-type n1-standard-4

This command creates an instance with three network interfaces:

• nic0 is attached to subnet0 and has no external IP address.
• nic1 is attached to subnet1 and has an ephemeral external IP address.
• nic2 is attached to subnet2 and has no external IP address.

Example 2: Using third-party appliances in a Shared VPC network scenario

This setup is useful when you want to share a single set of centralized third-party appliances for workloads or applications that are hosted in different projects. In figure 2, there are four distinct applications—App1, App2, App3, and App4—that are hosted in different service projects. You need them to be protected for all internet ingress, and you need egress traffic to be inspected and filtered in a third-party appliance that is centrally located in the Shared VPC host project.

Figure 2. An instance in a Shared VPC host project hosts a VM appliance. The instance has a network interface for each of four service projects and another interface for the network-perimeter VPC network (click to enlarge).

Provision and configure the VM and network interfaces for example 2

To create the VM and network interfaces in this example, use the following command:

gcloud compute instances create VM-appliance \
    --network-interface subnet=subnet-perimeter,address='reserved-address' \
    --network-interface subnet=subnet-1,no-address \
    --network-interface subnet=subnet-2,no-address \
    --network-interface subnet=subnet-3,no-address \
    --network-interface subnet=subnet-4,no-address \
    --machine-type=n1-standard-4

This creates an instance with five network interfaces:

• nic0 is attached to subnet-perimeter, which is part of network-perimeter, with a static address reserved-address.
• nic1 is attached to subnet-1, which is part of network-1, with no external IP address.
• nic2 is attached to subnet-2, which is part of network-2, with no external IP address.
• nic3 is attached to subnet-3, which is part of network-3, with no external IP address.
• nic4 is attached to subnet-4, which is part of network-4, with no external IP address.

Additional operational details

Multiple network interfaces in a Shared VPC environment

Shared VPC enables you to share VPC networks across projects in your Google Cloud organization.

Shared VPC lets you create instances associated with a Shared VPC network that is hosted in a centralized Shared VPC host project. For information about configuring Shared VPC networks, see Provisioning Shared VPC.

Note: Connecting to a Shared VPC network using interfaces other than nic0 is supported for the following configurations:

• VM instances: supported in the Google Cloud console, the gcloud CLI, and the API.
• Instance templates: supported in the gcloud CLI and the API.
• Managed instance groups: supported in the gcloud CLI and the API.

To create instances with one or more interfaces associated with Shared VPC networks, you must have the Compute Network User role (roles/compute.networkUser) in the Shared VPC host project.

DNS resolution with multiple network interfaces

When an internal DNS query is made with the instance hostname, it resolves to the primary interface (nic0) of the instance. If the nic0 interface of the instance is attached to a subnet in a VPC network that is different from the VPC network of the instance issuing the internal DNS query, the query fails.

Private Compute Engine DNS records are not generated per interface.

DHCP behavior with multiple network interfaces

In a default multiple interface configuration, the OS is configured to use DHCP. The DHCP and ARP behavior of each of the multiple interfaces is the same as the DHCP and ARP in an instance with a single interface.

In a multiple interface instance that uses DHCP, every interface gets a route for the subnet that it is in. In addition, the instance gets a single default route that is associated with the primary interface eth0. Unless manually configured otherwise, any traffic leaving an instance for any destination other than a directly connected subnet will leave the instance using the default route on eth0.

The behavior is the same for interfaces with IPv6 addresses. The interface gets a route for the IPv6 subnet range that it is in, as well as a single IPv6 default route.

In this sample, the primary interface eth0 gets the default route (default via 10.138.0.1 dev eth0), and both interfaces eth0 and eth1 get routes for their respective subnets.

instance-1:~$ ip route
default via 10.138.0.1 dev eth0
10.137.0.0/20 via 10.137.0.1 dev eth1
10.137.0.1 dev eth1 scope link
10.138.0.0/20 via 10.138.0.1 dev eth0
10.138.0.1 dev eth0 scope link

For more information, see the following tutorial: Configure routing for an additional interface.

Custom static routes and multiple network interfaces

When a VM instance has multiple interfaces and a network tag, the network tag might not impact all of the VM's interfaces. A VM's network tag impacts an interface if the interface is in a VPC network that contains a static route with a matching tag.

For example:

1. A VM has two interfaces: nic0 and nic1. The nic0 interface is in vpc-net-a. The nic1 interface is in vpc-net-b. The VM has a network tag called vpn-ok. The tag is an attribute on the instance, not on a specific interface.
2. The vpc-net-a network has a custom static route with a tag called vpn-ok.
3. The vpc-net-b network has a custom static route with a tag called vpn-123.

These numbered steps correspond to figure 3:

Figure 3. The custom static route in vpc-net-a affects nic0 because they have a tag in common, while the custom static route in vpc-net-b doesn't affect nic1 (click to enlarge).

In the case of the vpc-net-a network, because it has a route with a tag in common with the VM, the VM's vpn-ok tag applies to the VM's nic0 interface in vpc-net-a. In contrast, because the vpc-net-b network doesn't have a static route with the vpn-ok tag, the VM's vpn-ok network tag is ignored on the VM's nic1 interface.

Tags in routes in instances with multiple network interfaces

If you choose to use tags with routes, note that tags are applied at the instance level and, as such, tags apply to all interfaces of a virtual machine instance. If this is not desirable, make sure that the tags applied to the routes are unique to each VPC network.

Load balancers and multiple network interfaces

Except for Internal TCP/UDP Load Balancing, all Google Cloud load balancers only distribute traffic to the first interface (nic0) of a backend instance.

Firewall rules and multiple network interfaces

Each VPC network has its own set of firewall rules. If an instance's interface is in a particular VPC network, that network's firewall rules apply to that interface.

For example, suppose a VM instance has two interfaces:

• nic0 in VPC network network-1
• nic1 in VPC network network-2

Firewall rules that you create for the network-1 network apply to nic0. Firewall rules that you create for the network-2 network apply to nic1.

For more information, see VPC firewall rules.

Firewalls in instances with multiple network interfaces

• Ingress firewall rules can use either network tags or service accounts to identify sources, targets (destinations), or both.

• Egress firewall rules can use either network tags or service accounts to identify targets (sources).

For more information, see source and target filtering by service account.

Network tags and service accounts identify instances, not specific interfaces. Keep in mind that firewall rules are associated with a single VPC network, and each interface of a multi-NIC instance must be in a subnet that is in a unique VPC network.

The following example demonstrates how you can effectively use source tags for ingress allow firewall rules. The vm1 instance has two network interfaces:

• nic0 in network-1
• nic1 in network-2

Suppose you need to allow the following traffic from vm1:

• SSH traffic from vm1 to any instance in network-1
• HTTP and HTTPS traffic from vm1 to any instance in network-2

To accomplish this, you can do the following:

1. Assign two network tags to vm1: vm1-network1 and vm1-network2

2. Create an ingress allow firewall rule in network-1 with the following components to allow SSH traffic from vm1 to all VMs in network-1:

   • Action: allow
   • Direction: ingress
   • Sources: VMs with tag vm1-network1
   • Targets: All instances in the VPC network
   • Protocols and ports: tcp:22

3. Create an ingress allow firewall rule in network-2 with the following components to allow HTTP and HTTPS traffic from vm1 to all VMs in network-2:

   • Action: allow
   • Direction: ingress
   • Sources: VMs with tag vm1-network2
   • Targets: All instances in the VPC network
   • Protocols and ports: tcp:80,443

Figure 4 shows this firewall configuration example:

Figure 4. Firewall rule 1 and firewall rule 2 each have a source tag that is associated with VM1. Firewall rule 1, which is in network-1, only affects nic0 of VM1 because they are both in network-1. Firewall rule 2 only affects nic1 of VM1 because they also share a network (click to enlarge).

What's next

• Read Create VMs with multiple network interfaces.