Multiple Network Interfaces Overview and Examples

This page provides an overview of multiple network interfaces in a virtual machine (VM) instance, including how they work and sample configurations. For information on creating configurations that use multiple interfaces, see Creating Multiple Network Interfaces.

Overview

Google Cloud Platform VPC networks are by default isolated private networking domains. Networks have a global scope and contain regional subnets. VM instances within a VPC network can communicate among themselves via internal IP addresses as long as firewall rules permit. However, no internal IP address communication is allowed between networks, unless you set up mechanisms such as VPC peering or VPN.

Every instance in a VPC network has a default network interface. You can create additional network interfaces attached to your VMs. Multiple network interfaces enable you to create configurations in which an instance connects directly to several VPC networks. Each of the interfaces must have an internal IP address, and each interface can also have an external IP address. Each instance can have up to 8 interfaces, depending on the instance's type. For more information, see Maximum number of interfaces.

Typically, you might require multiple interfaces if you wish to configure an instance as a network appliance that does load balancing, Intrusion Detection and Prevention (IDS/IPS), Web Application Firewall (WAF), or WAN optimization between networks. Multiple network interfaces are also useful when applications running in an instance require traffic separation, such as separation of data plane traffic from management plane traffic.

Use Cases

Use multiple network interfaces when an individual instance needs access to more than one VPC network, but you don't want to connect both networks directly.

  • Network and security function: Multiple network interfaces enable virtualized network appliance functions such as load balancers, network address translation (NAT) servers, and proxy servers that are configured with multiple network interfaces. See Example 1: Networking and security virtual appliances for more details.

  • Perimeter and DMZ isolation: An important best practice in tiered networking architectures is to isolate public-facing services from an internal network and its services. Use multiple network interfaces to create configurations where there are separate network interfaces on the instance, one of them accepting public-facing traffic and another handling back-end private traffic that has more restrictive access controls.

    Any resources that can be reached from the Internet should be separated from your internal network and its services. This drastically limits the scope and damage that a security breach can cause. For example, you can place a second network interface on each web server that connects to a mid-tier network where an application server resides. The application server can also be dual-homed to a back-end network where the database server resides. Each dual-homed instance receives and processes requests on the front end, initiates a connection to the back end, and then sends requests to the servers on the back-end network.

    By configuring separate interfaces, one public-facing and another private- facing, you can apply separate firewall rules and access controls to each interface separately and enforce security functions in communications from the public to private domain. For more information, see Example 2: Using third-party appliances in a Shared VPC Network scenario

  • Bandwidth isolation across separate interfaces: This includes avoiding head-of-line issues between the control plane and the data plane. You can separate management, control, storage, and data plane networks using network interfaces. In some applications, control (for example, heartbeat signals) is highly sensitive. In such applications, it is desirable to isolate control from the datapath interfaces, to guarantee that minimum bandwidth will be available in case of a traffic peak or traffic congestion. In this scenario, use dedicated virtual interfaces to separate control traffic from other traffic. Each interface has a virtual queue. The virtual queue prevents bandwidth spikes and DDoS attacks on one VPC network from affecting other VPC networks. These per-interface virtual queues also prevent head-of-line blocking and provide a fair share of the instance's CPU for each I/O interface. See Example 3: Separation of management and dataplane interfaces in SaaS architectures for more details.

Configuration examples

This section examines several common examples of how to use multiple network interfaces.

Example 1: Networking and security virtual appliances

Networking and security virtual appliances, such as WAF, security application- level firewalls, and WAN accelerators, are usually configured with multiple virtual interfaces. Each of the multiple interfaces is configured with its own private IP address and optionally with its own public IP address.

The figure below describes a typical setup. In this specific case, you configure a virtual network appliance on the path from public to private connectivity. In this way, traffic can only reach a private VPC network from a public external client, through an application level virtualized firewall enforcement point. This application-level firewall is enforced on top of virtual machines.

Use case 1: Provisioning and configuring instances with multiple interfaces (click to
    enlarge)
Use case 1: Provisioning and configuring instances with multiple interfaces (click to enlarge)

Provisioning and configuring instances for example 1

The following assumes that subnet0, subnet1, and subnet2 already exist, with non-overlapping ranges. To configure VM-appliance in this example, use the following command.

gcloud compute instances create vm-appliance \
    --network-interface subnet=subnet0,no-address \
    --network-interface subnet=subnet1 \
    --network-interface subnet=subnet2,no-address \
    --machine-type n1-standard-4

This command creates an instance with three network interfaces:

  • nic0 is attached to subnet0 and has no public IP address
  • nic1 is attached to subnet1 and has an ephemeral public IP address
  • nic2 is attached to subnet2 and has no public IP address

Example 2: Using third-party appliances in a Shared VPC Network scenario

This setup is useful when you want to share a single set of centralized third-party appliances for workloads or applications that are hosted in different projects. In the example shown below, there are four distinct applications, App1, App2, App3 and App4, that are hosted in different service projects. You need them to be protected for all Internet ingress and you need egress traffic to be inspected and filtered in a third-party appliance that is centrally located in the Shared VPC host project.

Use case 2: Shared VPC example with a third-party appliance (click to enlarge)
Use case 2: Shared VPC example with a third-party appliance (click to enlarge)

Provisioning and configuring the VM and network interfaces for example 2

To create the VM and network interfaces in this example, use the following commands.

To create VM-appliance:

gcloud compute instances create VM-appliance \
    --network-interface subnet=subnet-dmz,address='reserved-address' \
    --network-interface subnet=subnet-1,no-address \
    --network-interface subnet=subnet-2,no-address
    --network-interface subnet=subnet-3,no-address \
    --network-interface subnet=subnet-4,no-address

This creates an instance with five network interfaces:

  • nic0 is attached to subnet-dmz, which is part of network-dmz, with a static address 'reserved-address'
  • nic1 is attached to subnet-1, which is part of network-1, with no public IP
  • nic2 is attached to subnet-2, which is part of network-2, with no public IP
  • nic3 is attached to subnet-3, which is part of network-3, with no public IP
  • nic4 is attached to subnet-4, which is part of network-4, with no public IP

Example 3: Separation of management and dataplane interfaces in SaaS architectures

If you operate a service in GCP that you offer to different customers (tenants), you can have a separate management interface to manage your service. You can use the management interface to perform software upgrades, maintenance procedures, and troubleshooting independent of the interfaces used by your tenants.

The following diagram describes a service that is hosted in the cloud. As a service producer, you isolate the service provided to each of your tenants in separate VPC networks. That is, you create a VPC network for each of your tenants and create specific instances to service each customer. For example, you create instances to serve Tenant A and isolate these instances in Network A so that the service is accessible by Tenant A's on-premises machines, but the rest of the tenants cannot access that service. You then create the same configuration for Tenant B and Tenant C.

However, you must have access to each of the tenant's GCP-hosted services to perform software updates, to provide support, and for troubleshooting purposes. You can achieve this by adding a management interface on each of the instances and associating it with your management network, which is connected privately to your on-premises facilities.

Use case 3: SaaS example (click to enlarge)
Use case 3: SaaS example (click to enlarge)

Tenant A's instance has two interfaces: one interface belongs to Tenant A's network and the second interface belongs to a management network of the producer (SaaS provider). Similarly, Tenant B's instance has two interfaces. One interface belongs to Tenant B's network and the second interface belongs to a management network of the producer (SaaS provider).

Provisioning and configuring instances for example 3

To configure the tenant instances in this example, use the following example commands.

To create VM-A:

gcloud compute instances create vm-a \
    --network-interface subnet=subnet-a,no-address \
    --network-interface subnet=subnet-mgmt,private-network-ip=198.168.0.2, \
    no-address

This creates an instance with two network interfaces:

  • nic0 is attached to subnet-a, which is part of network-a
  • nic1 is attached to subnet-mgmt, which is part of network-mgmt

To create VM-B:

gcloud compute instances create vm-b\
    --network-interface subnet=subnet-b,no-address \
    --network-interface subnet=subnet-mgmt,private-network-ip=198.168.0.3, \
    no-address

This creates an instance with two network interfaces:

  • nic0 is attached to subnet-b, which is part of network-b
  • nic1 is attached to subnet-mgmt, which is part of network-mgmt

To create VM-C:

gcloud compute instances create vm-c\
    --network-interface subnet=subnet-c,no-address \
    --network-interface subnet=subnet-mgmt,private-network-ip=198.168.0.4, \
    no-address

This creates an instance with two network interfaces:

  • nic0 is attached to subnet-c, which is part of network-c
  • nic1 is attached to subnet-mgmt, which is part of network-mgmt

Additional operational details

Multiple network interfaces in a Shared VPC environment

Shared VPC enables you to share VPC networks across projects in your Cloud Organization.

Shared VPC allows you to create instances associated with a Shared VPC network that is hosted in a centralized Shared VPC host project. See Provisioning Shared VPC for full information on configuring Shared VPC networks.

When you create instances with multiple network interfaces, your instances or instance templates can have certain interfaces attached to subnets local to the project, while other interfaces can be attached to Shared VPC networks.

To create instances with one or more interfaces associated with Shared VPC networks, you must have the compute.networkUser role in the Shared VPC host project.

DNS resolution with multiple network interfaces

When an internal DNS query is made with the instance hostname, it resolves to the primary interface (nic0) of the instance. If the nic0 interface of the instance belongs to a VPC network different from the VPC network of the instance issuing the internal DNS query, the query will fail.

Private Compute Engine DNS records will not be generated per interface.

DHCP behavior with multiple network interfaces

In a default multiple interface configuration, the OS is configured to use DHCP. The DHCP and ARP behavior of each of the multiple interfaces is the same as the DHCP and ARP in an instance with a single interface.

In a multiple interface instance that uses DHCP, every interface gets a route for the subnet that it is in. In addition, the instance gets a single default route that is associated with the primary interface eth0. Unless manually configured otherwise, any traffic leaving an instance for any destination other than a directly connected subnet will leave the instance via the default route on eth0.

In this sample, the primary interface eth0 gets the default route (default via 10.138.0.1 dev eth0), and both interfaces eth0 and eth1 get routes for their respective subnets.

instance-1:~$ ip route
default via 10.138.0.1 dev eth0
10.137.0.0/20 via 10.137.0.1 dev eth1
10.137.0.1 dev eth1 scope link
10.138.0.0/20 via 10.138.0.1 dev eth0
10.138.0.1 dev eth0 scope link

For more information, read Configuring Policy Routing.

Advanced routes with multiple network interfaces

Advanced Routes are applied per VPC network, or to specific instances within a VPC network that match a configured target-tag.

Each VPC network can have a different set of routes. When you configure an instance with multiple interfaces, each interface belongs to a different VPC network. Accordingly, the traffic associated with each interface is subject to the routes of the VPC network that the interface belongs to.

Using tags in routes in instances with multiple network interfaces

If you choose to use tags with routes, note that tags are applied at the instance level and, as such, tags apply to all interfaces of a virtual machine instance. If this is not desireable, you set up your configuration so that only certain tags are used in routes in a given VPC network, effectively ensuring those tags only apply to the interfaces associated with the specific VPC network.

Load balancers with multiple network interfaces

When a load-balanced backend, such as an instance group or a target pool, has multiple network interfaces, the load balancer sends traffic only to the default interface of the backend instance.

This is true for all types of load balancers supported in GCP.

Firewall rules with multiple network interfaces

You can configure firewall rules to only allow specific traffic defined by a combination of the following:

  • Source IP range, a list of permitted IP address blocks, or source tags, a set of permitted instances
  • Destination protocol and/or port.

Firewall rules are applied per VPC network, or to specific instances within a VPC network that match a configured target tag.

Each VPC network can have a different set of firewall rules. When you configure an instance with multiple network interfaces, each interface belongs to a different VPC network. Each interface is restricted by the firewall rules applied to the VPC network it belongs to.

Using tags in firewalls in instances with multiple network interfaces

If you decide to use tags to implement firewall rules, keep in mind that tags are applied at the instance level and apply to all interfaces of an instance. If this is undesirable, set up your configuration so that only certain tags are used in routes and firewall rules in a given VPC network, ensuring that those tags only apply to the interfaces associated with the specific VPC network.

For example, if you use the name of the VPC network in a tag, a user can clearly identify which VPC network those tags apply to and subsequently, which interface as well.

After choosing tag names to identify specific VPC networks, use only the tags for firewall rules that apply to that particular VPC network.

As an example, consider the following scenario: You have two VPC networks, network1 and network2. An instance, vm-1, is attached to both VPC networks through two different interfaces: nic0 is associated with network1 and nic1 is associated with network2.

The figure below describes this setup.

Using tags in firewalls (click to enlarge)
Using tags in firewalls (click to enlarge)

In this scenario, apply the following firewall rules:

  1. All instances in network1 accept SSH connections only from VM1.
  2. All instances in network2 accept HTTP/HTTPS connections only from VM1

To set up these rules:

  1. All the instances in network1 accept SSH connections only from VM1.

    1. Apply a tag in VM1 that is only intended to be used in Network1. In this example, the tag is vm1-network1-foo:

      gcloud compute instances add-tags vm1 \
          --tags vm1-network1-foo
      
    2. Configure the firewall rules in network1 to allow ssh (tcp, port 22) using the tag vm1-network1-foo as a source tag:

      gcloud compute firewall-rules create allow-ssh-from-vm1 \
          --allow tcp:22 \
          --network network1 \
          --source-tags vm1-network1-foo
      
  2. All the instances in network2 accept HTTP/HTTPS connections only from VM1.

    1. Apply a tag in VM1 that is only intended to be used in Network2. In this example, vm1-network2-foo:

      gcloud compute instances add-tags vm1 \
          --tags vm1-network2-foo
      
    2. Configure firewall rules in Network2 to allow http and https (tcp, port 80,443) using such tag vm1-network2-foo as a source tag:

      gcloud compute firewall-rules create allow-http-https-from-vm1 \
          --allow tcp:80,443 \
          --network network2\
          --source-tags vm1-network2-foo
      
Firewall rules (click to enlarge)
Firewall rules (click to enlarge)

Next steps

Read Creating Instances with Multiple Network Interfaces.

Monitor your resources on the go

Get the Google Cloud Console app to help you manage your projects.

Send feedback about...

Compute Engine Documentation