Advanced VPC Concepts

This page goes into additional detail on Virtual Private Cloud (VPC) networks. Please review the main VPC Concepts if you have not already.

Before you begin

Advanced VPC networking details

This section provides some low-level VPC networking details. You do not need to read this for typical usage, but it provides more insight about how VPC networking works. The following diagram describes these low-level details, with more information in the corresponding sections.

A more detailed diagram of the Compute Engine

Who Handles What

Under the hood, different VPC networking features are handled by different parts of the Compute Engine system. Some of these are standard networking features that are well documented, and some of them are specific to VPC networks. Some features you can configure, and some you cannot. VPC networks use Linux's VIRTIO network module to model Ethernet card and router functionality, but higher levels of the networking stack, such as ARP lookups, are handled using standard networking software.

ARP lookup
The instance kernel issues ARP requests and the VPC network issues ARP replies. The mapping between MAC addresses and IP addresses is handled by the instance kernel.
MAC lookup table, IP lookup table, active connection table
These tables are hosted on the underlying VPC network and cannot be inspected or configured.
DNS server
Each instance's metadata server acts as a DNS server. It stores the DNS entries for all VPC network IP addresses in the local VPC network and calls Google's public DNS server for entries outside the VPC network. You cannot configure this DNS server, but you can set up your own DNS server if you like and configure your instances to use that server instead by editing the /etc/resolv.conf file.
Packet handling between the VPC network and the outside
Packets coming into or out of the VPC network are handled by network code that examines the packet against firewall rules, against the external IP lookup table, and against the active connections table. The VPC network also performs NAT on packets coming into and out of the VPC network.
Packets received by an instance
These packets are received and turned into a stream by the instance kernel in the standard fashion.
Packets sent by an instance
Packets are sent by the instance kernel in the standard way. Interface and network functionality are modeled using the VIRTIO network module.

Detailed connection walkthroughs

Here are more details about what happens when an instance makes a VPC network call.

An instance makes a call:

  1. If the target address is an instance name or a URL such as, the instance calls the DNS service on its metadata server and gets back the matching IP address. You can configure your instance to consult another DNS service, although then you will not be able to resolve instance names.
  2. The destination IP address is examined against the subnet's IP address range, which every instance knows.

    1. If the IP address is outside the VPC network:

      1. The instance sends the packet to the subnet's gateway MAC address with the destination set to the packet's final destination. The instance might need to make an ARP request to resolve the gateway's MAC address.

      2. The VPC network rewrites the IP header to declare the instance's external IP address as the source. If the instance has no external IP address, the call is not allowed, and the VPC network drops the packet without informing the sender.

      3. The VPC network records the outgoing packet and adds the source and destination to the active connections table.

      4. The VPC network sends the packet on to its destination.

      5. The destination gets the packet and responds if it chooses.

      6. The VPC network receives the response, consults the active connections table, notes that this is an active connection, and allows it. The VPC network consults its network/external IP lookup table and replaces the instance's external IP address with the matching network address and sends the packet to the source instance.

      7. The instance receives the packet.

    2. If the destination IP address is within the VPC network:

      1. The instance is configured with an IP with mask, so the instance sends the packet to the subnet's gateway MAC address. The instance first might need to make an ARP request to resolve the gateway's MAC address.

      2. The VPC network, using Proxy ARP, responds with the MAC address of the destination instance.

      3. The gateway receives the packet and routes the packet to the destination IP within the VPC network.

      4. The target instance receives the packet. The target instance checks ingress firewall to determine if the packet is allowed. If not, the packet is dropped silently. Otherwise, the instance processes the packet.

An external instance or computer calls an instance:

  1. The external caller sends a packet to an instance's external IP address, which is owned by the VPC network.

  2. The VPC network compares the packet against the active connections table to see whether this is an existing connection:

    1. If it is not an existing connection, then the VPC network looks for a firewall rule to allow the connection.
    2. If there is no firewall rule, the VPC network drops the packet without informing the sender.
  3. If there is an existing connection or valid firewall rule, the VPC network examines its lookup table and replaces the external IP with the corresponding internal IP in the packet, logs the incoming packet in the active connections table, and sends the packet to the target instance.

  4. The instance receives the packet and responds as described in If the IP address is outside the VPC network IP range when sending a packet outside the network range.

  5. The VPC network receives the reply, finds the matching incoming request in the active connections table, and allows the packet through. Before sending, it modifies the source IP address by replacing the instance's internal IP with the corresponding external IP from its lookup table.

Measuring VPC network throughput

Outbound or egress traffic from a virtual machine is subject to maximum VPC network egress throughput caps. These caps are dependent on the number of vCPUs that a virtual machine instance has. Each vCPU is subject to a 2 Gbits/second (Gbps) cap for peak performance. This cap normally only applies to VM instances in the same zone communicating via internal IP addresses. Traffic for other destinations and traffic using external IP addresses is normally slower. Learn more about egress throughput caps.

To measure an instance's performance in relation to these caps, use the PerfKitBenchMarker to measure the egress throughput performance of your instances.

For example, run the following commands while you are on a local computer. The command will create an instance and measure its performance, where:

  • [MACHINE_TYPE] is the machine type you want to test (for example, n1-standard-32).
  • [ZONE] is the zone to create the instance in.
  • [NUMBER_OF_VCPUS] is the number of vCPUs of the instance (for example, 32 for n1-standard-32 machine type).

To measure single stream performance:

./ --cloud=GCP --machine_type=[MACHINE_TYPE] --benchmarks=iperf --ip_addresses=INTERNAL --zones=[ZONE]

To measure multistream performance:

./ --cloud=GCP --machine_type=[MACHINE_TYPE] --benchmarks=iperf --ip_addresses=INTERNAL --zones=[ZONE] --iperf_sending_thread_count=[NUMBER_OF_VCPUS]

What's next

  • See the VPC Overview for information on GCP VPC networks.
  • See Using VPC for instructions on creating and modifying VPC networks.

Monitor your resources on the go

Get the Google Cloud Console app to help you manage your projects.

Send feedback about...

Compute Engine Documentation