Using Networks and Firewalls

Your Google Cloud Platform (GCP) networks connect your instances to each other and to the Internet. You can segment your networks, create firewall rules to grant access to specific ports on your instances, and create static routes to forward traffic to specific destinations.

Before you begin

Contents

Networks

Every virtual machine instance is created as a member of a network. In a legacy network, it is a member of a single global IP range. In a subnet network, it is a member of a single subnetwork which is a member of a network. Networks and subnetworks handle communication between instances and serve as a gateway between instances and other networks. A network is constrained to a single project; it cannot span projects. However, a project can have multiple networks.

Each instance has an internal IP address drawn from the IP range of the subnetwork or network it is created in. Each instance can also have an external IP address, which is routable over the Internet. The external IP address can be ephemeral or static. Ephemeral addresses cannot move between instances and last only until you delete the instance that is associated with the address. Static addresses exist as an independent system resource that you can assign to any instance and move around when you like. You can keep static addresses as long as you like, even if it is not assigned to an instance. Reserved IP addresses cost money to keep. Any communication between instances in different networks, even within the same project, must be through external IP addresses.

Network types

Legacy (non-subnet) network

With the release of Subnetworks for Google Cloud, most networks will be subnet networks. However, you can still create legacy networks if needed.

A legacy Compute Engine network has a single network IPv4 prefix range and a single gateway IP address for the whole network. The network is global in scope and spans all cloud regions.

In a legacy network, instance IP addresses are not grouped by region or zone. One IP address can appear in one region, and the following IP address can be in a different region. Any given range of IPs can be spread across all regions, and the IP addresses of instances created within a region\ are not necessarily contiguous.

The figure below shows a Legacy (non-subnet) network. Traffic from the Internet passes through a global switching function in the network (shown in the diagram as a virtual switch), then down to individual instances.

Instances in a region can have IP addresses that are not grouped in any way. As shown in the example, instances from 10.240.0.0/16 are spread unpredictably across regions 1 and 2. For example, 10.240.1.4 is in region 2, 10.240.1.5 is in region 1, and 10.240.1.6 is in region 2.

Diagram of a legacy network (click to enlarge)
Diagram of a legacy network (click to enlarge)

Subnet network

A subnet network divides your global network in to regional subnets, each with its own IPv4 prefix. Subnetwork networks are regional in scope, unlike legacy networks which are global in scope. Subnets in a network do not have to be contiguous. For example, one subnet in a network can have a range of 10.240.0.0/16 and another can have a range of 192.168.0.0/16. As such, there is no overall network IP range or gateway address in a subnet network. Instead, each subnetwork has an IP range and gateway address. Subnetworks belonging to a single network must have non-overlapping RFC1918 ranges. A subnetwork belongs to only one network and each instance can only belong to one subnetwork.

A subnet network can be of auto mode or custom mode. An auto mode network has one subnet per region, each with a predetermined IP range and gateway. These subnets are created automatically when you create the auto mode network, and each subnet has the same name as the overall network. A custom mode network has no subnets at creation. In order to create an instance in a custom mode network, you must first create a subnetwork in that region and specify its IP range. A custom mode network can have zero, one, or many subnets per region.

The default network for a new project is an auto subnet network. You can create up to four additional networks in a project. Additional networks can be auto subnet networks, custom subnet networks, or legacy networks.

Each instance created within a subnetwork is assigned an IPv4 address from that subnetwork's range.

The figure below shows a subnet network. A new instance takes its IP address from the given subnetwork's prefix. Subnet1 is in region 1 and has all of its private IPs allocated from 10.240.0.0/24 (instance IPs 10.240.0.1 and 10.240.0.2), and Subnet2 in region 2 has all of its private IPs allocated from 192.168.1.0/24 (instance IPs 192.168.1.1 and 192.168.1.2). You can also see that Subnet3 crosses zones within a region. Subnetworks are not restricted by zone, only region. You can choose to create instances from one subnet in one zone and instances from another subnet in a separate zone, if you desire.

Diagram of a subnet network (click to enlarge)
Diagram of a subnet network (click to enlarge)

Network properties

Quotas and limits

A GCP network can contain a maximum of 7000 virtual machine instances. This number cannot be increased.

For a list of other resources and their quotas, visit the Quotas page. Many of these quotas can be increased upon request via the Request increase button on that page.

IP ranges

Auto subnet networks have one subnetwork per region with the IP ranges and gateways shown here. Every project contains a single network called default that is an auto subnet network. Because default is an auto subnet network, it will always have these ranges.

Region IP range Default gateway
us-west1 10.138.0.0/20 10.138.0.1
us-central1 10.128.0.0/20 10.128.0.1
us-east1 10.142.0.0/20 10.142.0.1
europe-west1 10.132.0.0/20 10.132.0.1
asia-east1 10.140.0.0/20 10.140.0.1

Custom subnet networks can use any valid RFC1918 IP range. Ranges do not have to be contiguous between subnetworks. For example, some subnetworks can use ranges from 10.0.0.0/8 while others use ranges from the 192.168.0.0/16 space. Ranges must be unique and non-overlapping within a network. Custom subnetworks should have at least a /29 IPv4 range.

Legacy networks only have a single RFC1918 range, which you specify when you create the network.

Network routes

All networks have automatically created routes to the Internet (default route) and to the IP ranges in the network. The route names are automatically generated and will look different each time. The auto subnet networks, including the default network, have ranges that look like this:

gcloud compute routes list
NAME                           NETWORK   DEST_RANGE    NEXT_HOP                 PRIORITY
default-route-02a98b9a14f7edc4 default   10.128.0.0/20                          1000
default-route-081fa300345dd52a default   0.0.0.0/0     default-internet-gateway 1000
default-route-93a38d78c77eac66 default   10.132.0.0/20                          1000
default-route-999664b72dd247e7 default   10.140.0.0/20                          1000
default-route-a1f15d0858cd51e1 default   10.142.0.0/20                          1000

A custom subnet network has a default route, plus a route for each subnet that you create.

Legacy networks start with only two routes, the default route and the route to the overall IP range.

With these routes, the network knows how to send traffic to the Internet and to all instances you create. However, traffic cannot reach the instances until appropriate firewall rules are in place.

Firewall rules

Each network has its own firewall controlling access to the instances.

All traffic to instances, even from other instances, is blocked by the firewall unless firewall rules are created to allow it.

The default network has automatically created firewall rules, which are shown below. No manually created network of any type has automatically created firewall rules. For all networks except the default network, you must create any firewall rules you need.

Firewall rules are only "allow" rules. You cannot create "deny" rules. If you need to restrict traffic from reaching certain instances, create rules that allow traffic to the other instances, then remove the firewall rule that allowed traffic to all of the instances.

The firewall rules automatically created for the default network are as follows:

default-allow-internal
Allows network connections of any protocol and port between instances on the network.
default-allow-ssh
Allows SSH connections from any source to any instance on the network over TCP port 22.
default-allow-rdp
Allows RDP connections from any source to any instance on the network over TCP port 3389.
default-allow-icmp
Allows ICMP traffic from any source to any instance on the network.

Internal DNS and resolv.conf

An internal fully qualified domain name (FQDN) for an instance looks like this:

hostName.c.[PROJECT_ID].internal

You can always connect from one instance to another using this FQDN. If you want to connect to an instance using, for example, just hostName, you need information from the internal DNS resolver that is provided as part of Compute Engine. Compute Engine instances receive internal DNS resolution information as part of their DHCP leases. You can use any DNS resolver on your instances, as long as it supports the Local Subnet Routes feature documented in RFC3442.

By default, many flavors of Linux store DHCP information in resolv.conf, so this section assumes that's where your instance stores it. Compute Engine instances are configured to renew DHCP leases every 24 hours. DHCP renewal will overwrite this file, undoing any changes you might have made.

Sample resolv.conf with explanation

# Local domain name. Computed from your project name.
domain c.[PROJECT_ID].internal
# Search list for hostname lookup. Starting with entries that represent
# your project and ending with google.internal to facilitate metadata server requests.
# Note: Older projects might have a projectNumber.google.internal
# (eg. 1234.google.internal).
# Note: Compute Engine provides up to a maximum of 3 entries for the search path.
search c.[PROJECT_ID].internal google.internal.
# Address of the DNS server to resolve project specific, and global domain names.
nameserver 169.254.169.254
nameserver 10.128.0.1

Sample dhcp.lease with explanation

lease {
  # What interface we are using for the network
 interface "eth0";
 fixed-address 10.128.0.3;
 option subnet-mask 255.255.255.255;
 option routers 10.128.0.1;
 # Lease timeout, older VM instances will have this value set to infinite.
 option dhcp-lease-time 86400;
 option dhcp-message-type 5;
 option domain-name-servers 169.254.169.254,10.128.0.1;
 option dhcp-server-identifier 169.254.169.254;
 option interface-mtu 1430;
 # Search path options that are copied into the resolv.conf
 option domain-search "c.[PROJECT_ID].internal.", "google.internal.";
 option ntp-servers 169.254.169.254;
 option rfc3442-classless-static-routes 32,10,128,0,1,0,0,0,0,0,10,128,0,1;
 option host-name "vm1.c.[PROJECT_ID].internal";
 option domain-name "c.[PROJECT_ID].internal.";
 renew 4 2016/02/25 04:38:57;
 rebind 4 2016/02/25 16:00:08;
 expire 4 2016/02/25 19:00:08;
}

If you need to modify this, file please note the following:

  • Search path can only handle 6 records, 3 of which are provided by default by Compute Engine. If you add entries to the search path such that the total number of entries is greater than 6, search rules after the 6th one will not be applied by your OS. This can cause Compute Engine functionality like accessing instances via their instance names to stop working.
  • Manually editing resolv.conf will result in it being reverted to the default DHCP every time your instance's 24-hour DHCP lease expires. To, make static modifications your your instance's resolv.conf, many flavors of Linux allow items to be prepended or appended to the DHCP policy. For example, dhclient.conf provides this functionality in Debian.

User-created networks

You can create up to five networks per project, including the default network. If you do not want the default network, you can delete it and create a different network instead. If you need more than five networks for your project, you can request additional quota.

You cannot create a network with the same name as any existing subnet in the same project. You cannot create a subnet with the same name as an existing network in the same project, except that each network can have one subnet per region that has the same name as its parent network.

Note that there is no way to modify an existing network. You can add and remove the firewall rules that are associated with a network, and you can add and remove routes, but you cannot change the address range or gateway of a network after the network has been created.

Creating a new network with auto-created subnetwork ranges

  1. Create a new network in your project with automatically allocated subnetworks. The --mode flag specifies auto, which indicates that you would like the subnetworks to be automatically created with pre-set IP blocks. Because this is a subnet network, there is no network-level IPv4 range or gateway IP, so none will be displayed.

    gcloud compute networks create auto-network1 \
        --mode auto
    

    NAME          MODE IPV4_RANGE GATEWAY_IPV4
    auto-network1 auto

  2. List your new auto-created subnetworks.

    gcloud compute networks subnets list
    

    NAME          REGION          NETWORK       RANGE
    auto-network1 asia-east1      auto-network1 10.140.0.0/20
    auto-network1 us-central1     auto-network1 10.128.0.0/20
    auto-network1 europe-west1    auto-network1 10.132.0.0/20

At this point, the network has routes to the Internet and to any instances you might create. However, it has no firewall rules that allow access to instances, even from other instances. You must create firewall rules to allow access.

Creating a new network with custom subnet ranges

In manually assigning subnetwork ranges, you first create a custom subnet network, then create the subnetworks that you want within a region. You do not have to specify subnetworks for all regions right away, or even at all, but you cannot create instances in regions that have no subnetwork defined.

When you create a new subnetwork, its name must be unique in that project for that region, even across networks. The same name can appear twice in a project as long as each one is in a different region.

Because this is a subnet network, there is no network-level IPv4 range or gateway IP, so none will be displayed.

  1. Create a new custom subnet network in your project.

    gcloud compute networks create custom-network1 \
        --mode custom
    

    NAME            MODE   IPV4_RANGE GATEWAY_IPV4
    custom-network1 custom

  2. Specify the subnetwork prefix for your first region. In this example, we're assigning 192.168.1.0/24 to region us-central1.

    gcloud compute networks subnets create subnet-us-central-192 \
       --network custom-network1 \
       --region us-central1 \
       --range 192.168.1.0/24
    

    NAME                  REGION      NETWORK         RANGE
    subnet-us-central-192 us-central1 custom-network1 192.168.1.0/24

  3. Specify the subnetwork prefix for your second region. In this example, we're assigning 192.168.5.0/24 to region europe-west1.

    gcloud compute networks subnets create subnet-europe-west-192 \
         --network custom-network1 \
         --region europe-west1 \
         --range 192.168.5.0/24
    

    NAME                   REGION       NETWORK         RANGE
    subnet-europe-west-192 europe-west1 custom-network1 192.168.5.0/24

  4. Specify the subnetwork prefix for your third region. In this example, we're assigning 192.168.7.0/24 to region asia-east1.

    gcloud compute networks subnets create subnet-asia-east-192 \
       --network custom-network1 \
       --region asia-east1 \
       --range 192.168.7.0/24
    

    NAME                 REGION     NETWORK         RANGE
    subnet-asia-east-192 asia-east1 custom-network1 192.168.7.0/24

  5. List your networks. If you also created an auto subnet network in the prior section, those subnets will be listed as well.

    gcloud compute networks subnets list
    

    NAME                           REGION          NETWORK         RANGE
    subnet-europe-west-192         europe-west     custom-network1 192.168.5.0/24
    subnet-us-central-192          us-central1     custom-network1 192.168.1.0/24
    subnet-asia-east-192           asia-east1      custom-network1 192.168.7.0/24

At this point, the network has routes to the Internet and to any instances that you might create. But, it has no firewall rules allowing access to instances, even from other instances. You must create firewall rules to allow access.

Creating a legacy network

You can still create a legacy network without subnets. Legacy networks have a single global IP range. You cannot create subnetworks in a legacy network or switch from legacy to auto or custom subnet networks.

  1. Create a new legacy network in your project.

      gcloud compute networks create legacy-network1 \
          --mode legacy \
          --range 10.240.0.0/16
    

      Created
      [https://www.googleapis.com/compute/latest/projects/PROJECT_ID/global/networks/legacy-network1].
      NAME            MODE   IPV4_RANGE    GATEWAY_IPV4
      legacy-network1 legacy 10.240.0.0/16 10.240.0.1

At this point, the network has routes to the Internet and to any instances that you might create. But, it has no firewall rules allowing access to instances, even from other instances. You must create firewall rules to allow access.

Listing existing subnetworks

You can list all subnetworks in an existing project.

List general details on all subnetworks.

gcloud compute networks subnets list
NAME              REGION          NETWORK         RANGE
auto-network1     europe-west1    auto-network1   10.132.0.0/20
europe-west-192   europe-west     custom-network1 192.168.5.0/24
auto-network1     us-central1     auto-network1   10.128.0.0/20
us-central-192    us-central1     custom-network1 192.168.1.0/24
asia-east-192     asia-east1      custom-network1 192.168.7.0/24
auto-network1     asia-east1      auto-network1   10.140.0.0/20

You can tailor the list by providing additional parameters:

  • --regions REGION,[REGION,...]] Restrict the list to subnets in particular regions.
  • --network NETWORK Restrict the list to only subnets in a particular network.

Describe an existing subnetwork

The describe command provides details on the indicated subnetwork.

Get details on an existing subnetwork:

gcloud compute networks subnets describe subnet-asia-east-192 \
    --region asia-east1
creationTimestamp: '2015-10-21T15:58:35.271-07:00'
gatewayAddress: 192.168.7.1
id: '2048026325272602228'
ipCidrRange: 192.168.7.0/24
kind: compute#subnetwork
name: subnet-asia-east-192
network: https://www.googleapis.com/compute/latest/projects/PROJECT_ID/global/networks/custom-network1
region: https://www.googleapis.com/compute/latest/projects/PROJECT_ID/regions/asia-east1
selfLink: https://www.googleapis.com/compute/latest/projects/PROJECT_ID/regions/asia-east1/subnetworks/subnet-asia-east-192

Switch a network from auto to custom

You can change an auto subnet mode network to be a custom subnet mode network.

After the change, the custom subnet mode network has the same subnet names and IP ranges of the original auto mode subnet network.

After the switch, you can add new subnets to the custom subnet mode network. see step 2 of Creating a new network with custom subnet ranges. You can also delete subnets in the network, including the original auto subnets.

gcloud beta compute networks switch-mode [NETWORK_NAME] --mode custom

Expand a custom subnet

You can expand the IP range of a subnet in a custom subnet mode network. You cannot shrink it.

To set the size of the new subnet, specify the new netmask. For example, if the original IP range is 10.128.131.0/24, specifying --prefix-length 20 sets the new IP range to 10.128.128.0/20. The new network range must be larger than the original, which means the prefix length must be smaller. The new subnet must not overlap with other subnets in the same network in any region. The new subnet must stay inside the RFC1918 address spaces.

Since subnets are uniquely identified by region and name, you must specify those in the command.

Depending on the size and IP allocation of the existing network, this command may take from several seconds to several minutes to run. You may be unable to create new instances during that time, but existing instances are unaffected. During the expansion, existing instances are still accessible and will still pass traffic.

gcloud beta compute networks subnets expand-ip-range [SUBNET_NAME]
  --network [NETWORK_NAME]
  --region [REGION]
  --prefix-length [PREFIX_LENGTH]
  • --prefix-length - the new numeric prefix length for the subnetwork. Must be smaller than the existing prefix length. For example, if the current subnet is a /24, the new prefix length must be 23 or smaller.

Delete a subnetwork or network

You can delete any subnetwork in your project. You can also delete an entire network. You cannot delete a network or subnetwork that still has instances or resources.

Delete a subnetwork

You can only delete manually created subnetworks. Automatically created subnetworks cannot be deleted individually; you must delete the entire network.

You cannot delete any subnetwork that still has instances or other manually created resources using it.

gcloud compute networks subnets delete subnet-asia-east-192 \
    --region asia-east1
The following subnetworks will be deleted:
 - [subnet-asia-east-192] in [asia-east1]
Do you want to continue (Y/n)?  y
Deleted [https://www.googleapis.com/compute/latest/projects/PROJECT_ID/regions/asia-east1/subnetworks/subnet-asia-east-192].

Delete a network

For a legacy network, you can explicitly delete the network resource only if the network resource is not in use by any resources.

For an auto subnet network, you can explicitly delete the network resource only if both of the following items are true:

  • Any child subnetworks in the network are not in use by any other resource.
  • The network resource is not in use by any other resources.

Example resources that restrict the deletion for the above include firewalls, custom routes, instances, target VPN gateways, and routers.

For a custom subnet network, the network can only be deleted if all child subnetworks have been deleted.

To delete the network:

gcloud compute networks delete auto-network1
The following networks will be deleted:
 - [auto-network1]
Do you want to continue (Y/n)?  y
Deleted [https://www.googleapis.com/compute/latest/projects/PROJECT_ID/global/networks/auto-network1].

Measuring network throughput

Outbound or egress traffic from a virtual machine is subject to maximum network egress throughput caps. These caps are dependent on the number of cores that a virtual machine instance has. Each core is subject to a 2 Gbits/second (Gbps) cap for peak performance. Learn more about egress throughput caps.

To measure an instance's performance in relation to these caps, use the PerfKitBenchMarker to measure the egress throughput performance of your instances.

For example, run the following commands while you are on a local computer. The command will create an instance and measure its performance, where:

  • [MACHINE_TYPE] is the machine type you want to test (for example, n1-standard-32).
  • [ZONE] is the zone to create the instance in.
  • [NUMBER_OF_CORES] is the number of cores of the instance (for example, 32 for n1-standard-32 machine type).

To measure single stream performance:

./pkb.py --cloud=GCP --machine_type=[MACHINE_TYPE] --benchmarks=iperf --ip_addresses=INTERNAL --zones=[ZONE]

To measure multistream performance:

./pkb.py --cloud=GCP --machine_type=[MACHINE_TYPE] --benchmarks=iperf --ip_addresses=INTERNAL --zones=[ZONE] --iperf_sending_thread_count=[NUMBER_OF_CORES]

Advanced networking details

This section provides some low-level details not covered in the previous sections. You do not need to read this for typical usage, but it provides more insight about how networking works in Compute Engine. The following diagram describes these low-level details, with more information in the corresponding sections.

A more detailed diagram of the Compute Engine
    network

Network

  • Stores a lookup table that tracks every active connection. When a packet arrives that is part of an active connection, it will be sent to the destination without consulting any firewall rules.
  • Stores a lookup table that associates external IP addresses with instances. All packets to the external IP are routed to the network, which looks up the internal IP corresponding to that address, and forwards it to the instance.
  • Performs MAC address lookups (proxy ARP) for a given IP address.
  • Routes packets to and from instances on the network.
  • Routes packets externally (billing involved).

Firewalls

  • Compute Engine uses a firewall that can block packets into an instance; it cannot block packets from leaving an instance. If you want to block outgoing packets from an instance, you must configure another technology, such as iptables, on your instances.
  • Once a connection has been made, all incoming and outgoing packets with the same two IP addresses, same instance port, and same protocol will be permitted until the connection expires, which it will do after about 10 minutes of inactivity. However, if a reply is sent to a different instance port (for example if an FTP request asks for a response from another port), the response is not automatically allowed; there must be a firewall rule permitting a connection on the new port.

Instances

  • Each instance has a metadata server that also acts as a DNS resolver for that instance. DNS lookups are performed for instance names. The metadata server itself stores all DNS information for the local network and queries Google's public DNS servers for any addresses outside of the local network.
  • An instance is not aware of any external IP address assigned to it. Instead, the network stores a lookup table that matches external IP addresses with the internal IP addresses of the relevant instances.

Who Handles What

Under the hood, different networking features are handled by different parts of the Compute Engine system. Some of these are standard networking features that are well documented, and some of them are specific to Compute Engine. Some features you can configure, and some you cannot. Compute Engine uses Linux's VIRTIO network module to model Ethernet card and router functionality, but higher levels of the networking stack, such as ARP lookups, are handled using standard networking software.

ARP lookup
The instance kernel issues ARP requests and the Network issues ARP replies. The mapping between MAC addresses and IP addresses is handled by the instance kernel.
MAC lookup table, IP lookup table, active connection table
These tables are hosted on the underlying network and cannot be inspected or configured.
DNS server
Each instance's metadata server acts as a DNS server. It stores the DNS entries for all network IP addresses in the local network and calls Google's public DNS server for entries outside the network. You cannot configure this DNS server, but you can set up your own DNS server if you like and configure your instances to use that server instead by editing the /etc/resolv.conf file.
Packet handling between the network and the outside
Packets coming into or out of the network are handled by network code that examines the packet against firewall rules, against the external IP lookup table, and against the active connections table. The network also performs NAT on packets coming into and out of the network.
Packets received by an instance
These packets are received and turned into a stream by the instance kernel in the standard fashion.
Packets sent by an instance
Packets are sent by the instance kernel in the standard way. Interface and network functionality are modeled using the VIRTIO network module.

Detailed connection walkthroughs

Here are more details about what happens when an instance makes a network call.

An instance makes a call:

  1. If the target address is an instance name or a URL such as www.google.com, the instance calls the DNS service on its metadata server and gets back the matching IP address. You can configure your instance to consult another DNS service, although then you will not be able to resolve instance names.
  2. The destination IP address is examined against the subnetwork's IP address range, which every instance knows.

    1. If the IP address is outside the network:

      1. The instance sends the packet to the subnetwork's gateway MAC address with the destination set to the packet's final destination. The instance might need to make an ARP request to resolve the gateway's MAC address.

      2. The network rewrites the IP header to declare the instance's external IP address as the source. If the instance has no external IP address, the call is not allowed, and the network drops the packet without informing the sender.

      3. The network records the outgoing packet and adds the source and destination to the active connections table.

      4. The network sends the packet on to its destination.

      5. The destination gets the packet and responds if it chooses.

      6. The network receives the response, consults the active connections table, notes that this is an active connection, and allows it. The network consults its network/external IP lookup table and replaces the instance's external IP address with the matching network address and sends the packet to the source instance.

      7. The instance receives the packet.

    2. If the destination IP address is within the network:

      1. The instance is configured with an IP with 255.255.255.255 mask, so the instance sends the packet to the subnetwork's gateway MAC address. The instance first might need to make an ARP request to resolve the gateway's MAC address.

      2. The network, using Proxy ARP, responds with the MAC address of the destination instance.

      3. The gateway receives the packet and routes the packet to the destination IP within the network.

      4. The target instance receives the packet. The target instance checks ingress firewall to determine if the packet is allowed. If not, the packet is dropped silently. Otherwise, the instance processes the packet.

An external instance or computer calls an instance:

  1. The external caller sends a packet to an instance's external IP address, which is owned by the network.

  2. The network compares the packet against the active connections table to see whether this is an existing connection:

    1. If it is not an existing connection, then the network looks for a firewall rule to allow the connection.
    2. If there is no firewall rule, the network drops the packet without informing the sender.
  3. If there is an existing connection or valid firewall rule, the network examines its lookup table and replaces the external IP with the corresponding network IP in the packet, logs the incoming packet in the active connections table, and sends the packet to the target instance.

  4. The instance receives the packet and responds as described in If the IP address is outside the network IP range when sending a packet outside the network range.

  5. The network receives the reply, finds the matching incoming request in the active connections table, and allows the packet through. Before sending, it modifies the source IP address by replacing the instance's network IP with the corresponding external IP from its lookup table.

Firewalls

See Subnetworks and firewall rules for more information on how you can use firewall rules to isolate subnetworks.

Each network has a firewall that blocks all traffic to instances, but does not block outbound traffic from instances. To allow traffic to come into instances, you must create "allow" rules for the firewall.

The default network has automatically created firewall rules, but user created networks do not.

Each firewall rule specifies a permitted incoming connection request, which is defined by source, destination, ports, and protocol. When a request is sent to an instance, whether internally or from another network or the Internet, Compute Engine allows the request if any rule in the firewall permits the connection.

The firewall does not restrict an instance from sending a packet. Any instance can send packets to any other instance in its network. However, whether that packet is accepted is determined by the firewall rules associated with the target instance. To regulate outgoing requests, use another system, such as iptables.

Only instances with an external IP address can send packets outside the network, but instances in different subnetworks of the same network can reach each other without using external IP addresses.

Only instances with an external IP address and a permitting firewall rule can be addressed directly from outside the network. Alternatively, you can route packets to instances without external IP address via an externally addressable proxy server.

You cannot share firewall rules between projects or between networks in the same project.

Compute Engine uses a connection tracking table to support stateful firewall filtering. The maximum number of connections in the table depends on the instance type. Current maximum connections are as follows:

  • 130000 per instance for instances with shared-core machine types
  • 130000 per CPU for instances with 1 to 8 CPUs
  • 130000*8 (1040000) per instance for instances with >8 CPUs

A firewall rule exposes the following properties, which can also be set using the gcloud compute firewall-rules create command. Each rule is composed of several elements, most of which must be defined before a connection is allowed:

network
[Required] The network that this firewall rule is assigned to. Each firewall rule can be associated with one and only one network. You must provide a network when making an API call to create a firewall rule. However, if you use gcloud compute and do not specify a network, the rule will automatically be assigned to the default network.
sourceRanges

Identifies permitted callers by a list of IP address blocks expressed in CIDR notation. If neither sourceRanges nor sourceTags are specified, the default is 0.0.0.0/0, which means that the rule applies to all incoming connections from inside or outside the network. If both sourceRanges and sourceTags are specified, an inbound connection is allowed if either the range of the source matches sourceRanges or the tag of the source matches sourceTags.

"sourceRanges": [ "198.51.100.0/24", "203.0.113.0/25" ]
sourceTags

If the source is within the network and is associated with one of the specified tags, the connection will be accepted. If neither sourceRanges nor sourceTags are specified, sourceRanges defaults to 0.0.0.0/0, which means that the rule applies to all incoming connections from inside or outside the network. If both sourceRanges and sourceTags are specified, an inbound connection is allowed if either the range of the source matches sourceRanges or the tag of the source matches sourceTags.

"sourceTags": [ "management" ]
targetTags

[Optional] A list of instance tags that specify which instances on the network can accept requests from the specified sources. If not specified, this firewall rule applies to all instances on this network. For example:

"targetTags": [ "web", "database" ]
allowed

[Required] An array of allowed connections permitted by this firewall rule. Each contains an IP protocol and an optional range of ports (for TCP and UDP traffic) that should be allowed to reach the instances specified by targetTags.

IPProtocol
[Required] The protocols allowed over this connection. This can be the (case-sensitive) string values tcp, udp, icmp, esp, ah, sctp, or any IP protocol number.
ports
[Optional] An array of target ports allowed for this connection. This is only applicable for TCP and UDP connections. Each value is a string that is either a single port or a port range. If not specified, all ports are allowed. Example: ["80", "160", "300-500"]

For example:

"allowed": [
  {
    "IPProtocol": "tcp",
    "ports": [ "22" ],
  },
  {
    "IPProtocol": "udp",
    "ports": [ "161" ],
  }
]
name

[Required] The firewall name. The name must be unique in this project, 1-63 characters long, and match the following regular expression: [a-z]([-a-z0-9]*[a-z0-9])? The first character must be a lowercase letter and all following characters must be a dash, lowercase letter, or digit. The last character cannot be a dash.

The default network in any project starts with the following standard firewall rules. You can modify or delete these rules, and you can add, modify, and delete additional rules.

default-allow-internal
Allows network connections of any protocol and port between any two instances.
default-allow-ssh
Allows TCP connections from any external or internal source to any instance on the network over port 22. Instances without external IP addresses will still not be directly reachable from external sources.
default-allow-icmp
Allows ICMP traffic from any source to any instance on the network.
default-allow-rdp
Allows remote desktop protocol traffic to port 3389.

The default rules above do not allow HTTP or HTTPS traffic from external sources. Setting Up an External HTTP Connection has instructions for allowing such traffic.

Example

The following example describes the default-allow-ssh firewall rule. The source CIDR of 0.0.0.0/0 means that the connection can come from anywhere, inside or outside of the network.

gcloud compute firewall-rules describe default-allow-ssh
allowed:
- IPProtocol: tcp
  ports:
  - '22'
creationTimestamp: '2014-07-08T10:32:46.214-07:00'
description: Allow SSH from anywhere
id: '13715544083725015625'
kind: compute#firewall
name: default-allow-ssh
network: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/networks/default
selfLink: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/firewalls/default-allow-ssh
sourceRanges:
- 0.0.0.0/0

Useful gcloud compute commands:

Add a firewall rule

To add a firewall rule, create a Firewall rule resource describing the permitted IP ranges, target tags, target ports, source tags, protocol, and network. Any instance connected to the specified network will have that firewall applied. In gcloud compute, use the firewall-rules create command:

gcloud compute firewall-rules create [FIREWALL_RULE] \
    --allow [PROTOCOL][:[PORT][-[PORT]]],[[PROTOCOL][:[PORT][-[PORT]]],…] \
    [--network [NETWORK]; default="default"] \
    [--source-ranges [CIDR_RANGE],[[CIDR_RANGE],…]]

Example

The following example adds a firewall that supports HTTP connections over port 80 from any source to any instance in the network that exposes an external IP address.

gcloud compute firewall-rules create allow-http --description "Incoming http allowed." \
         --allow tcp:80 --format json
{
  "allowed": [
    {
      "IPProtocol": "tcp",
      "ports": [
        "80"
      ]
    }
  ],
  "creationTimestamp": "1234-56-78T09:12:34.567",
  "description": "Incoming http allowed.",
  "id": "13AAA70BBBB5639CCCC9",
  "kind": "compute#firewall",
  "name": "allow-http",
  "network": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/networks/default",
  "selfLink": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/firewalls/allowhttp",
  "sourceRanges": [
    "0.0.0.0/0"
  ]
}

Delete a firewall rule

To delete a firewall rule, run the firewall-rules delete command:

gcloud compute firewall-rules delete [FIREWALL_RULE]

Routes

Routes are global resources that are attached to a single network, but not to regional resources. User-created routes apply to all instances in a network. This means that you can add a route that forwards traffic from one instance to another instance within the same network, even across subnetworks, without requiring external IP addresses. By default, routes exists between subnetworks. However, unless you are using the default network, you must create firewall rules to allow traffic on your network.

In certain circumstances, you can create additional routes to specifically forward some packets to alternate destinations. For example, all networks have a default route that sends traffic out of the network. However, you could create a different route that specifies that packets destined for 0.0.0.0/0 should be routed to a virtual machine instance running a proxy instead.

Routes allow you to implement more advanced networking functions in your virtual machines, such as setting up many-to-one NAT and transparent proxies. If you do not need any advanced routing solutions, the default routes should be sufficient for handling most outgoing traffic.

Auto-created routes

Certain routes are created when you create a network.

For subnet networks

The following routes are created.

  • A default route for Internet traffic (0/0) is created when the network is created.
  • One route is created for each subnetwork when the subnetwork is created. These routes are for local traffic in the network, which allows VM instances in any subnetwork to send traffic to instances in any other or same subnetwork in that network. This route counts against overall route quota. The overall route quota for a project is currently 100, and each subnet created uses one route against the quota. Note that networks you create do not have firewall rules yet, so even though there are routes pointing to instances, the firewall will prevent traffic to instances until you add firewall rules.

For legacy networks

Two routes are created at network creation time.

  • A default route for Internet traffic (0/0) is created when the network is created.
  • For the destination IP range within the IPv4 range of the network, a virtual network route is defined. For the default legacy network, default firewall rules are also automatically created, so traffic between instances is allowed from the beginning. For manually created legacy networks, you have to create firewall rules that allow inbound traffic to instances.

For instructions on creating additional routes, see the gcloud compute routes create command.

Instance routing tables

Each route in the Routes collection may apply to one or more instances. A route applies to an instance if the network and instance tags match. If the network matches and there are no instance tags specified, the route applies to all instances in that network. Compute Engine then uses the Routes collection to create individual read-only routing tables for each instance.

A good way to visualize this is to imagine a massively scalable virtual router at the core of each network. Every virtual machine instance in the network is directly connected to this router, and all packets leaving a virtual machine instance are first handled at this layer before they are forwarded on to their next hop. The virtual network router selects the next hop for a packet by consulting the routing table for that instance. The diagram below describes this relationship, where the green boxes are virtual machine instances, the router is at the center, and the individual routing tables are indicated by the tan boxes.

The Routes collection for the legacy network in the diagram might look like this:

NAME                           NETWORK DEST_RANGE    NEXT_HOP                 PRIORITY
default-route-68079898SAMPLEe7 default 0.0.0.0/0     default-internet-gateway   1000
default-route-78SAMPLEd2bc5762 default 10.100.0.0/16                            1000
vpngateway                     default 172.12.0.0/16 us-central1-a/instances/vpngateway  1000

A closer look at the vpngateway route exposes the vpn tag on the route:

gcloud compute routes describe vpngateway
creationTimestamp: '2014-07-28T15:26:27.023-07:00'
destRange: 172.12.0.0/16
id: '12304245498973864442'
kind: compute#route
name: vpngateway
network: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/networks/default
nextHopGateway: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/us-central1-a/instances/vpngateway
priority: 1000
selfLink: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/routes/vpngateway
tags:
- vpn

The vpngateway route ensures that any instance with the vpn tag automatically has a routing table that contains the vpngateway route alongside the two default routes. In the diagram, both vm1 and vm2 have these routes in their routing table, so all outgoing traffic destined for the 172.12.0.0/16 external IP range is handled by the vpngateway instance.

An instance's routing table is a read-only entity. You cannot directly edit these tables. If you want to add, remove, or edit a route, you must do so through the Routes collection.

Individual routes

A single route is made up of the following:

name
[Required] The user-friendly name for this route. For example, internetroute for a route that allows access to the Internet.
network
[Required] The name of the network this route applies to. For example, the default network.
destRange
[Required] The destination IP range that this route applies to. If the destination IP of a packet falls in this range, it matches this route. For example, 0.0.0.0/0. See the Route Selection section to understand how Compute Engine uses all matching routes to select a single next hop for a packet.
instanceTags
[Required] The list of instance tags this route applies to. If this is empty, this route applies to all instances within the specified network. In the API, this is a required field. In gcloud compute, this is an optional field and gcloud compute assumes an empty list if this field is not specified.

Exactly one of the following next hop specifications is required:

nextHopInstance

The fully-qualified URL of the instance that should handle matching packets. The instance must already exist and have IP forwarding enabled. For example:

https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/[ZONE]/instances/<instance>

If a next hop instance crashes and is restarted by the system, or if you delete an instance and recreate it with the same name in the same zone, Compute Engine will continue to route matching packets to the new instance.

nextHopIp

The network IP address of an instance that should handle matching packets. The IP address must lie within the address space of the network. For example, if your network is 10.240.0.0/16, you cannot specify nextHopIp=1.1.1.1. The instance must already exist and have IP forwarding enabled. If the next hop instance crashes and is later restarted by the system with the same IP address or if the user deletes the instance and recreates it with the same IP address, Google Compute Engine continues routing matching packets to the new instance.

nextHopNetwork

[Read-Only] The URL of the local network handling matching packets. This is only available to the default local route. You cannot manually set this field.

nextHopGateway

The URL of a gateway that should handle matching packets. Currently, there is only the Internet gateway available:

/projects/[PROJECT_ID]/global/gateways/default-internet-gateway
nextHopVpnTunnel

The URL of a Compute Engine VPN tunnel that should handle matching packets.

priority

[Required] The priority of this route. Priority is used to break ties in the case where there is more than one matching route of maximum length. A lower value is higher priority; a priority of 100 has higher priority than 200. For example, the following routes are tied because they have the same prefix length and they are in the same network. The differing priority breaks the tie so that vpnroute wins.

NAME                       NETWORK     DEST_RANGE         NEXT_HOP                            PRIORITY
vpnroute                   default     192.168.0.0/16     [ZONE]/instances/vpninstance          1000
vpnroute-backup            default     192.168.0.0/16     [ZONE]/instances/vpninstance-backup   2000

Under this configuration, VPN traffic would normally be handled by vpninstance, but would fall back to vpninstance-backup if vpnroute is deleted.

In the API, this is a required field. In gcloud compute, this is an optional field and gcloud compute assumes a default priority of 1000 if the field is not specified.

Route selection

When an outgoing packet leaves a virtual machine instance, Compute Engine uses the following steps to decide which route to use and where to forward the packet:

  1. Compute Engine discards all but the most specific routes that match the packet’s destination address. For example, if destinationRange=10.1.1.1 and there is a route for 10.1.1.0/24 and a route for 10.240.0.0/16, Compute Engine selects the 10.1.1.0/24 route because it is more specific.

  2. If there are multiple routes with the same prefix length, Compute Engine discards all routes except the ones with the smallest priority value (smallest priority value indicates highest priority). There may still be more than one route left at this point.

  3. Compute Engine computes a hash value of the IP protocol field, the source and destination IP addresses, and the source and destination ports. Compute Engine uses this hash value to select a single next hop from the remaining ties.

  4. If a next hop is found, Compute Engine forwards the packet. If a next hop is not found, the packet is dropped and Compute Engine replies with an ICMP destination or network unreachable error.

It is important to note that Compute Engine does not consider network distance when selecting a next hop. The next hop instance or gateway could be in a different zone than the instance sending the packet, so you should engineer your routing tables to control locality. For example, you can use instance tags to direct packets for instances in different zones to prefer a local transparent proxy or VPN gateway. By tagging instances by zone, you can ensure that packets leaving an instance in one zone will only be sent to a next hop in the same zone.

List routes

To see a list of routes for a project using gcloud compute, run the routes list command:

gcloud compute routes list

For example:

gcloud compute routes list
NAME                           NETWORK   DEST_RANGE    NEXT_HOP                 PRIORITY
default-route-02a98b9a14f7edc4 default   10.128.0.0/20                          1000
default-route-081fa300345dd52a default   0.0.0.0/0     default-internet-gateway 1000
default-route-0fcb41f96ba1b57d legacy1   0.0.0.0/0     default-internet-gateway 1000
default-route-907f4b8dbd772dc3 legacy1   10.240.0.0/16                          1000
default-route-93a38d78c77eac66 default   10.132.0.0/20                          1000
default-route-999664b72dd247e7 default   10.140.0.0/20                          1000
default-route-a1f15d0858cd51e1 default   10.142.0.0/20                          1000

Add a route

To add a route to the routing table using gcloud compute, use the routes create command:

gcloud compute routes create ROUTE --destination-range DEST_RANGE --network NETWORK NEXT_HOP

Delete a route

To delete a route, run the routes delete command:

gcloud compute routes delete [ROUTE]

To delete a route in the API, make a HTTP DELETE request to the following URI, with an empty request body:

https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/routes/<route-name>

Consistency of route operations

When you make changes to the Routes collection, these changes are eventually consistent across all instances. This means that after you update, add, or remove a route, the changes are only guaranteed to have taken affect on all applicable instances once your operation returns a status of DONE. A PENDING or RUNNING operation means that the change may have taken effect on some instances but might not have taken effect on all instances.

If you make a sequence of changes, these changes may be applied to your instances in any order. There is no guarantee that the order in which you make your requests will be the order in which these requests are processed. Since routing changes do not take affect instantaneously, different instances may observe different changes at different times. However, your operation will move to a DONE state once the changes have been implemented on all affected instances.

Enable IP forwarding for instances

By default, Compute Engine instances are blocked from sending packets whose source IP address does not match the IP address of the instance sending the packet. Similarly, Compute Engine won't deliver a packet whose destination IP address is different than the IP address of the instance receiving the packet. However, both capabilities are required if you want to use instances to help route packets. To disable this source and destination IP check, enable the canIpForward field, which allows an instance to send and receive packets with non-matching destination or source IPs.

To set the canIpForward field in gcloud compute, use the --can-ip-forward flag when creating your instance:

gcloud compute instances create ... --can-ip-forward

In the API, set the canIpForward field to true when you construct the request body to create a new instance:

{
  "name": "myinstance",
  "description": "",
  "image": "https://www.googleapis.com/compute/v1/projects/debian-cloud/global/images/debian-8-jessie-vYYYYMMDD",
  "machineType": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/machineTypes/n1-standard-2",
  "networkInterfaces": [
      ...
  ],
  "canIpForward": true
}

You can only set this field at instance creation time. After an instance is created, the field becomes read-only.

Interacting with firewall rules

Just creating a route does not ensure that your packets will be received by the specified next hop. Firewall rules still determine whether incoming traffic is allowed into a network or instance. For example, if you create a route that sends packets through multiple instances, each instance must have an associated firewall rule to accept packets from the previous instance.

For IP address matching, only the source IP address of the packet is used, which is not necessarily the IP address of the instance sending the packet. If you have a firewall rule that specifies only packets from 10.240.0.3 are accepted, all packets with that source IP address are accepted, regardless of the IP address of the instance that sent the packets.

  • If instance 10.240.0.3 has canIpForward enabled and spoofs a packet to have source IP 10.240.0.4, the firewall will reject the packet.
  • If instance 10.240.0.4 has canIpForward enabled and spoofs a packet to have source IP 10.240.0.3, the firewall will accept the packet.

Source tags are aliases for the source IPs of packets, not the IPs of the instances sending the packets. For example, if a source tag named mytag is assigned to an instance with IP 10.240.0.3, a rule that allows traffic from mytag would allow any packets with a source IP of 10.240.0.3, regardless of which instance sends the packet. This is important because an instance with IP forwarding enabled can send a packet with a source IP address different from the instance IP address. Target tags, on the other hand, are aliases for the IP of the receiving instance only, so there is no ambiguity.

In the future, Compute Engine may change the way firewall rules handle source tags.

For more information, see Firewalls.

Routing packets to the Internet

Currently, any packets sent to the Internet must be sent by an instance that has an external IP address. If you create a route that sends packets to the Internet from a particular instance, that instance must also have an external IP. If you create a route that sends packets to the Internet gateway, but the source instance doesn't have an external IP address, the packet will be dropped.

Special Configurations

Configure a static internal IP address

Although Compute Engine does not allow you to set a static internal IP address that persists beyond the life of the instance, you can use a combination of routes and an instance's --can-ip-forward ability to add an IP address as a static internal IP address that can be used as a target for your desired virtual machine instance by other instances in the same network. You can also modify the instance itself so it sends traffic from that IP address.

For example, if you want to assign 10.1.1.1 specifically as an internal IP address to a virtual machine instance, you can create a static route that sends traffic from 10.1.1.1 to your instance, even if the instance's internal IP address is not 10.1.1.1. You can also configure the instance to send traffic from 10.1.1.1.

Use the following instructions to:

Note that the following instructions assume that your project uses the default network.

Before you configure a static internal IP address, keep in mind that:

  • You can assign a specific internal IP address to an instance during instance creation, without using a route. For more information, read the Specifying an internal IP address at instance creation documentation.
  • It is also possible to communicate with an instance using the instance name. Even if the instance's internal IP might change, the instance name remains the same, so you can configure your applications to send packets to the instance name instead. Using an instance name is more robust than configuring a static internal IP address using routes.

Set a static target internal IP address using routes

This section only sets up the instance and network to receive packets sent to the IP (target IP). To configure the instance to send packets with this IP address (source IP), see Set a static source IP address

Debian 8


  1. Choose an internal IP address that doesn't belong to any network in your project.

    Use an address from a private range, like 10.x.x.x or 192.168.x.x, but not from a range that is in use in the project. This example uses 10.1.1.1.

  2. Create a new virtual machine instance and enable IP forwarding.

    By default, Compute Engine won't deliver packets whose destination IP address is different than the IP address of the instance receiving the packet. To disable this destination IP check, you can enable IP forwarding for the instance.

    gcloud compute instances create my-configurable-instance --can-ip-forward
    
  3. Create a static network route to direct packets destined for 10.1.1.1 to your instance.

    The target of the static route must be an IP address that is not in any network of the project.

    gcloud compute routes create ip-10-1-1-1 \
        --next-hop-instance my-configurable-instance \
        --next-hop-instance-zone us-central1-f \
        --destination-range 10.1.1.1/32
    
  4. Create a firewall rule to allow traffic on this IP.

    In this case, the firewall rule allows all traffic through the 10.0.0.0/8 IP address range. You can add additional IP ranges or limit access as needed for your use case.

    gcloud compute firewall-rules create allow-internal \
      --allow icmp,udp,tcp --source-range 10.0.0.0/8
    
  5. Add a new virtual interface to your instance.

    1. ssh into your instance.

      gcloud compute ssh my-configurable-instance
      
    2. Append the following lines to the /etc/network/interfaces file.

      # Change to root first
      user@my-configurable-instance:~$ sudo su -
      

      # Append the following lines
      root@my-configurable-instance:~# cat <<EOF >>/etc/network/interfaces
      auto eth0:0
      iface eth0:0 inet static
      address 10.1.1.1
      netmask 255.255.255.255
      EOF
      

    3. Restart the instance.

      root@my-configurable-instance:~# sudo reboot
      

  6. Check that your virtual machine instance interface is up by pinging 10.1.1.1 from inside your instance.

    user@my-configurable-instance:~$ ping 10.1.1.1
    

    You can also try pinging the interface from another instance in your project.

Ubuntu 14.04


  1. Choose an internal IP address that doesn't belong to any network in your project.

    Use an address from a private range, like 10.x.x.x or 192.168.x.x, but not from a range that is in use in the project. This example uses 10.1.1.1.

  2. Create a new virtual machine instance and enable IP forwarding.

    By default, Compute Engine won't deliver packets whose destination IP address is different than the IP address of the instance receiving the packet. To disable this destination IP check, you can enable IP forwarding for the instance.

    gcloud compute instances create my-configurable-instance --can-ip-forward
    
  3. Create a static network route to direct packets destined for 10.1.1.1 to your instance.

    The target of the static route must be an IP address that is not in any network of the project.

    gcloud compute routes create ip-10-1-1-1 \
        --next-hop-instance my-configurable-instance \
        --next-hop-instance-zone us-central1-f \
        --destination-range 10.1.1.1/32
    
  4. Create a firewall rule to allow traffic on this IP.

    In this case, the firewall rule allows all traffic through the 10.0.0.0/8 IP address range. You can add additional IP ranges or limit access as needed for your use case.

    gcloud compute firewall-rules create allow-internal \
        --allow icmp,udp,tcp --source-range 10.0.0.0/8
    
  5. Add a new virtual interface to your instance.

    1. ssh into your instance.

      gcloud compute ssh my-configurable-instance
      
    2. Append the following lines to the /etc/network/interfaces file.

      # Change to root first
      user@my-configurable-instance:~$ sudo su -
      

      # Append the following lines
      root@my-configurable-instance:~# cat <<EOF >>/etc/network/interfaces
      auto eth0:0
      iface eth0:0 inet static
      address 10.1.1.1
      netmask 255.255.255.255
      EOF
      

    3. Restart the instance.

      root@my-configurable-instance:~# sudo reboot
      

  6. Check that your virtual machine instance interface is up by pinging 10.1.1.1 from inside your instance.

    user@my-configurable-instance:~$ ping 10.1.1.1
    

    You can also try pinging the interface from another instance in your project.

CentOS 6


  1. Choose an internal IP address that doesn't belong to any network in your project.

    Use an address from a private range, like 10.x.x.x or 192.168.x.x, but not from a range that is in use in the project. This example uses 10.1.1.1.

  2. Create a new virtual machine instance and enable IP forwarding.

    By default, Compute Engine won't deliver packets whose destination IP address is different than the IP address of the instance receiving the packet. To disable this destination IP check, you can enable IP forwarding for the instance.

    gcloud compute instances create my-configurable-instance --can-ip-forward
    
  3. Create a static network route to direct packets destined for 10.1.1.1 to your instance.

    The target of the static route must be an IP address that is not in any network of the project.

    gcloud compute routes create ip-10-1-1-1 \
        --next-hop-instance my-configurable-instance \
        --next-hop-instance-zone us-central1-f \
        --destination-range 10.1.1.1/32
    
  4. Create a firewall rule to allow traffic on this IP.

    In this case, the firewall rule allows all traffic through the 10.0.0.0/8 IP address range. You can add additional IP ranges or limit access as needed for your use case.

    gcloud compute firewall-rules create allow-internal \
      --allow icmp,udp,tcp --source-range 10.0.0.0/8
    
  5. Add a new virtual interface to your instance.

    1. ssh into your instance.

      gcloud compute ssh my-configurable-instance
      
    2. Add the following lines to a new file named /etc/sysconfig/network-scripts/ifcfg-eth0:0:

      # Change to root first
      user@my-configurable-instance:~$ sudo su -
      

      # Add the following lines
      root@my-configurable-instance:~# cat <<EOF >>/etc/sysconfig/network-scripts/ifcfg-eth0:0
      DEVICE="eth0:0"
      BOOTPROTO="static"
      IPADDR=10.1.1.1
      NETMASK=255.255.255.255
      ONBOOT="yes"
      EOF
      

    3. Bring up your new interface.

      root@my-configurable-instance:~# ifup eth0:0
      

  6. Check that your virtual machine instance interface is up by pinging 10.1.1.1 from inside your instance.

    user@my-configurable-instance:~$ ping 10.1.1.1
    

    You can also try pinging the interface from another instance in your project.

Set a static source IP address

In addition to setting a static target internal IP for a virtual machine, you can also set up the instance to use the static internal IP as its source IP when communicating with other virtual machines.

This section only sets up the instance to use the IP as a source address (source IP). To have the network route packets with this destination IP to the instance, also set a static target IP address.

To set the source IP as the static internal IP on a Linux instance:

# Change to root first
user@my-configurable-instance:~$ sudo su -
# Get the next hop route of your packets
root@my-configurable-instance:~# route -n
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
10.240.0.1      0.0.0.0         255.255.255.255 UH    0      0        0 eth0
# Add a route, where 10.0.0.0/16 is the address range the route applies;
# replace this range with an address range of your choice.
# This means that all instance in this range will see the source IP of your
# instance as 10.1.1.1
root@my-configurable-instance:~# ip route add 10.0.0.0/16 dev eth0:0 via 10.240.0.1 src 10.1.1.1

To test this out, try using the instance to ping an IP address in the address range 10.0.0.0/16 on the same network. For example, the following example demonstrates the tcpdump of a request made from an instance with a static internal IP of 10.1.1.1 to 10.0.0.2:

root@my-configurable-instance-2:~# tcpdump -npi eth0 -vv icmp or arp
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
17:59:05.374093 IP (tos 0x0, ttl 64, id 59993, offset 0, flags [DF], proto ICMP (1), length 84)
    10.1.1.1 > 10.0.0.2: ICMP echo request, id 4319, seq 1, length 64
17:59:05.374126 IP (tos 0x0, ttl 64, id 55671, offset 0, flags [none], proto ICMP (1), length 84)
    10.0.0.2 > 10.1.1.1: ICMP echo reply, id 4319, seq 1, length 64
17:59:06.375432 IP (tos 0x0, ttl 64, id 60166, offset 0, flags [DF], proto ICMP (1), length 84)
    10.1.1.1 > 10.0.0.2: ICMP echo request, id 4319, seq 2, length 64

Set up a network proxy

You can design your network so that only one instance has external access, and all other instances in the network use that instance as a proxy server to the outside world. This is useful if you want to control access into or out of your network, or reduce the cost of paying for multiple external IP addresses.

This particular example discusses how to set up a network proxy on Compute Engine instances that use a Debian image. It uses a gateway instance as a Squid proxy server but this is only one way of setting up a proxy server.

To set up a Squid proxy server:

  1. Set up one instance with an external (static or ephemeral) IP address. For this example, name your instance gateway-instance.
  2. Set up one or more instances without external IP addresses by specifying gcloud compute instances create ... --no-address. For this example, call this instance hidden-instance.
  3. Learn how to connect from one instance to another because you will not be able to connect directly into your internal-only instances.
  4. Add a firewall to allow tcp traffic on port 3128:

    gcloud compute firewall-rules create FIREWALL_RULE --network NETWORK --allow tcp:3128
    
  5. Install Squid on gateway-instance, and configure it to allow access from any machines on the shared internal network (RFC1918, RFC4193, and RFC4291 IP spaces). This assumes that gateway-instance and hidden-instance are both connected to the same network, which enables them to connect to each other.

    user@gateway-instance:~$ sudo apt-get install squid3
    

    Enable any machine on the local network to use the Squid3 server. The following sed commands uncomment and enable the acl localnet src entries in the Squid config files for local networks and machines.

    user@gateway-instance:~$ sudo sed -i 's:#\(http_access allow localnet\):\1:' /etc/squid3/squid.conf
    

    user@gateway-instance:~$ sudo sed -i 's:#\(http_access deny to_localhost\):\1:' /etc/squid3/squid.conf
    

    user@gateway-instance:~$ sudo sed -i 's:#\(acl localnet src 10.0.0.0/8.*\):\1:' /etc/squid3/squid.conf
    

    user@gateway-instance:~$ sudo sed -i 's:#\(acl localnet src 172.16.0.0/12.*\):\1:' /etc/squid3/squid.conf
    

    user@gateway-instance:~$ sudo sed -i 's:#\(acl localnet src 192.168.0.0/16.*\):\1:' /etc/squid3/squid.conf
    

    user@gateway-instance:~$ sudo sed -i 's:#\(acl localnet src fc00\:\:/7.*\):\1:' /etc/squid3/squid.conf
    

    user@gateway-instance:~$ sudo sed -i 's:#\(acl localnet src fe80\:\:/10.*\):\1:' /etc/squid3/squid.conf
    

    # Prevent proxy access to metadata server
    user@gateway-instance:~$ sudo cat <<EOF >>/etc/squid3/squid.conf
    acl to_metadata dst 169.254.169.254
    http_access deny to_metadata
    EOF
    

    # Start Squid
    user@gateway:~$ sudo service squid3 start
    

  6. Configure hidden-instance to use gateway-instance as its proxy. Use ssh to connect into hidden-instance and define its proxy URL addresses to point to gateway-instance on port 3128 (the default Squid configuration) as shown here:

    user@gateway-instance:~$ ssh hidden-instance
    

    user@hidden-instance:~$ sudo -s
    

    root@hidden-instance:~# echo "export http_proxy=\"http://gateway-instance.$(dnsdomainname):3128\"" >> /etc/profile.d/proxy.sh
    

    root@hidden-instance:~# echo "export https_proxy=\"http://gateway-instance.$(dnsdomainname):3128\"" >> /etc/profile.d/proxy.sh
    

    root@hidden-instance:~# echo "export ftp_proxy=\"http://gateway-instance.$(dnsdomainname):3128\"" >> /etc/profile.d/proxy.sh
    

    root@hidden-instance:~# echo "export no_proxy=169.254.169.254,metadata,metadata.google.internal" >> /etc/profile.d/proxy.sh
    

    Update sudoers to pass these env variables through.

    root@hidden-instance:~# cp /etc/sudoers /tmp/sudoers.new
    

    root@hidden-instance:~# chmod 640 /tmp/sudoers.new
    

    root@hidden-instance:~# echo "Defaults env_keep += \"ftp_proxy http_proxy https_proxy no_proxy"\" >>/tmp/sudoers.new
    

    root@hidden-instance:~# chmod 440 /tmp/sudoers.new
    

    root@hidden-instance:~# visudo -c -f /tmp/sudoers.new && cp /tmp/sudoers.new /etc/sudoers
    

  7. Exit sudo, load the variables, and run apt-get on hidden-instance. It should now work using gateway as a proxy. If gateway were not serving as a proxy, apt-get would not work because hidden-instance has no direct connection to the Internet.

    root@hidden-instance:~# exit
    

    user@hidden-instance:~$ source ~/.profile
    

    user@hidden-instance:~$ sudo apt-get update
    

Set up an external HTTP connection

The default firewall rules do not allow HTTP or HTTPS connections to your instances. However, it is fairly simple to add a rule that does allow them. Note that an instance must have an external IP address before it can receive traffic from outside its network.

You can add a firewall rule to allow HTTP or HTTPS connections through gcloud compute or the Google Cloud Platform Console. You can also add a firewall rule through the API.

Console

You can use the Cloud Platform Console to create an overall firewall rule for all instances on the network, or you can allow individual instances access to HTTP and HTTPS connections by selecting the respective option when you create that instance. The latter option is described first, because it provides more control over individual instances.

  1. In the Cloud Platform Console, go to the VM Instances page.

    Go to the VM Instances page

  2. Click the Create instance button.
  3. In the Firewall section, select Allow HTTP traffic and Allow HTTPS traffic.
  4. Click the Create button to create the instance.

By selecting these checkboxes, Compute Engine automatically creates a default-http or default-https rule that applies to all instances with either the http-server or https-server tags. Your new instance is also tagged with the appropriate tag depending your checkbox selection.

If you already have existing default-http and default-https firewall rules, you can apply the firewall rule to existing instances by enabling the Allow HTTP or Allow HTTPS options on the instance's details page.

  1. Go to the VM instances page.
  2. Click the name of the desired instance.
  3. Click Edit button at the top of the page.
  4. Scroll down to the Firewalls section.
  5. Check the Allow HTTP or Allow HTTPS options under your desired network.
  6. Click Save.

In a similar manner, you can also disable external HTTP or HTTPS access for an instance by unchecking one or both checkboxes.

By allowing specific instances to be tagged for HTTP and HTTPS traffic rather than creating an overall firewall rule that applies to all instances, Compute Engine limits the possible security implications of allowing external traffic to all virtual machines in a project. However, if you would like to create a firewall rule that allows HTTP or HTTPS traffic to all virtual machine instances, you can create your own firewall rule:

  1. Go to the Networks page.
  2. Select the network where you would to apply the firewall rule.
  3. Click Create new next to Firewall rules.
  4. Name your firewall rule, and add tcp:80 in the Protocols & Ports box, or tcp:443 for HTTPS traffic.
  5. Click Create.
gcloud

If you want to allow HTTP and HTTPS traffic to all virtual machines in a project, the following command creates a firewall that allows incoming HTTP and HTTPS requests from anywhere to any instance connected to this network.

gcloud compute firewall-rules create FIREWALL_RULE --allow tcp:80,tcp:443

**Example**

gcloud compute firewall-rules create sample-http \
 --description "Incoming http and https allowed." \
 --allow tcp:80,tcp:443
gcloud compute firewall-rules describe sample-http
allowed:
- IPProtocol: tcp
  ports:
  - '80'
  - '443'
creationTimestamp: '2014-06-13T13:27:12.206-07:00'
id: '5057780722612413546'
kind: compute#firewall
name: sample-http
network: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/networks/default
selfLink: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/firewalls/samplehttp
sourceRanges:
- 0.0.0.0/0

Configure a NAT gateway

This example shows setting up the gateway in a legacy network. Adjust the network ranges shown if you're using a subnetwork.

You can create more complicated networking scenarios by making changes to the Routes collection. This section describes how you can set up a network address translation (NAT) gateway instance that can route traffic from internal-only virtual machine instances to the Internet. This allows you to use one external IP address to send traffic from multiple virtual machine instances but only expose a single virtual machine to the Internet.

  1. To start, create a GCP network to host your virtual machine instances for this scenario. In this example, the legacy network range used is 10.240.0.0/16 with a gateway of 10.240.0.1. However, you can select your own IPv4 range and gateway address as well. You can also create a subnet network instead.

    gcloud compute networks create gce-network \
        --mode legacy \
        --range 10.240.0.0/16
    
  2. Create firewall rules to allow ssh connections in the new network you just created.

    gcloud compute firewall-rules create gce-network-allow-ssh --allow tcp:22 --network gce-network
    gcloud compute firewall-rules create gce-network-allow-internal --allow tcp:1-65535,udp:1-65535,icmp \
        --source-ranges 10.240.0.0/16 --network gce-network
    
  3. Create a virtual machine to act as a NAT gateway on gce-network or the default network.

    gcloud compute instances create nat-gateway --network gce-network --can-ip-forward \
        --zone us-central1-a \
        --image-family debian-8 \
        --image-project debian-cloud \
        --tags nat
    
  4. Tag any virtual machine instances without an external IP address that will use the gateway instance with the tag no-ip, or create a new virtual machine without an external IP address and tag the instance with the no-ip tag.

    • Add tags to an existing instance.

      gcloud compute instances add-tags existing-instance --tags no-ip
      
    • Alternatively, create a new virtual machine without an external IP address.

      gcloud compute instances create example-instance --network gce-network --no-address \
          --zone us-central1-a \
          --image-family debian-8 \
          --image-project debian-cloud \
          --tags no-ip
      
  5. Create a route to send traffic destined to the Internet through your gateway instance.

    gcloud compute routes create no-ip-internet-route --network gce-network \
        --destination-range 0.0.0.0/0 \
        --next-hop-instance nat-gateway \
        --next-hop-instance-zone us-central1-a \
        --tags no-ip --priority 800
    

    Setting the priority of this route ensures that this route wins if there are any other conflicting routes. 1000 is the default priority and a value lower than 1000 will take precedent.

  6. Next, log onto your gateway instance and configure iptables to NAT internal traffic to the Internet.

    gcloud compute ssh nat-gateway --zone us-central1-a
    

    On your instance, configure iptables:

    $ sudo sysctl -w net.ipv4.ip_forward=1
    

    $ sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
    

    The first sudo command tells the kernel that you want to allow IP forwarding. The second sudo command masquerades packets received from internal instances as if they were sent from the NAT gateway instance.

Set up a VPN gateway

This section describes installing Strongswan VPN software on one of your instances. It assumes a legacy network. For Compute Engine VPN, see Cloud VPN.

This section describes an example scenario that connects a VPN gateway instance in Compute Engine to a Debian-based VPN gateway in your non-Cloud network. This scenario is just an example and may or may not be suitable for your network. Consult with your network administrator for more information.

  1. To start, create a Compute Engine network to connect via VPN.

    gcloud compute networks create gce-network --range 10.120.0.0/16
    
  2. Create a VPN gateway virtual machine on gce-network.

    gcloud compute instances create vpn-gateway --can-ip-forward \
        --network gce-network \
        --zone us-central1-a \
        --image debian-7 \
        --tags vpn
    
  3. Make note of your newly created virtual machine's internal IP address by running the following command:

    gcloud compute instances describe --zone us-central1-a vpn-gateway
    

    Note the address next to networkIP (not natIP). It should begin with 10.120.x.x.

  4. Create a "plain old" non-VPN gateway virtual machine to talk to your local network.

    gcloud compute instances create povm-1 --network gce-network --image debian-7 --tags vpn --zone us-central1-a
    
  5. Create a route in gce-network to route traffic through vpn-gateway if it is destined for your local network.

    gcloud compute routes create gce-network-via-gateway --destination-range LOCAL_NETWORK_ADDRESS_SPACE \
        --next-hop-address VPN_GATEWAY_NETWORK_IP \
        --network gce-network \
        --tags vpn
    

    The VPN_GATEWAY_NETWORK_IP value should be the networkIp that you noted from step 3.

  6. Add the following Compute Engine firewall rules for your Compute Engine network to accept incoming traffic.

    gcloud compute firewall-rules create ssh --source-ranges 0.0.0.0/0 --allow tcp:22 --network gce-network
    
    gcloud compute firewall-rules create  allow-internal --source-ranges 10.120.0.0/16 --allow tcp:1-65535,udp:1-65535,icmp \
        --network gce-network --target-tags vpn
    
    gcloud compute firewall-rules create allow-ipsec-nat --source-ranges IP_LOCAL_VPN_GATEWAY/32 \
        --allow udp:4500,udp:500 --network gce-network --target-tags vpn
    
    gcloud compute firewall-rules create allow-all-peer --source-ranges LOCAL_NETWORK_ADDRESS_SPACE \
        --allow tcp:1-65535,udp:1-65535,icmp --network gce-network --target-tags vpn
    

    You might also need to set up firewall settings in your local network to accept incoming traffic from the Compute Engine network. Depending on your network, this process can vary.

  7. Install VPN software and configure gateway guest OS. You will need your VPN gateway machine's external IP address. Run the following command:

    gcloud compute instances describe vpn-gateway
    

    Copy the external IP address and create a file named ipsec.conf on your virtual machine gateway instance. Populate it with the following contents:

    conn myconn
      authby=psk
      auto=start
      dpdaction=hold
      esp=aes128-sha1!
      forceencaps=yes
      ike=aes128-sha1-modp2048!
      keyexchange=ikev2
      mobike=no
      type=tunnel
      left=%any
      leftid=<vpn-vm-gateway-external-address>
      leftsubnet=<internal-ip-subnet>
      leftauth=psk
      leftikeport=4500
      right=<public-ip-of-your-local-vpn-gateway-machine>
      rightsubnet=<your-local-network-address-space>
      rightauth=psk
      rightikeport=4500
    

    Your <internal-ip-subnet> value should be either 10.120.0.0/16 or 10.240.0.0/16, based on the internal-ip value that you noted in step 3.

    Then, run the following commands, replacing <secret-key> with a secret key you choose:

    $ sudo apt-get update
    

    $ sudo apt-get install strongswan -y
    

    $ echo "%any : PSK \"<secret-key>\"" | sudo tee /etc/ipsec.secrets > /dev/null
    

    $ sudo sysctl -w net.ipv4.ip_forward=1
    

    $ sudo cp ipsec.conf /etc
    

    $ sudo ipsec restart
    

    $ sudo ipsec up myconn
    

    Assuming your local gateway machine is running a Debian-based operating system, you can use the same steps to install VPN on your local machine. Make a copy of your ipsec.conf file with the following changes on your local gateway machine:

    conn myconn
      authby=psk
      auto=start
      dpdaction=hold
      esp=aes128-sha1!
      forceencaps=yes
      ike=aes128-sha1-modp2048!
      keyexchange=ikev2
      mobike=no
      type=tunnel
      left=%any
      leftid=<public-ip-of-local-VPN-gateway-machine>
      leftsubnet=<your-local-network-address-space>
      leftauth=psk
      leftikeport=4500
      rightid=<vpn-vm-gateway-external-address>
      rightsubnet=10.120.0.0/16
      rightauth=psk
      rightikeport=4500
    

    Run the same commands described above on your local VPN gateway machine.

  8. Try it out:

    gcloud compute ssh povm-1 --command 'ping -c 3 LOCAL_NETWORK_EXTERNAL_ADDRESS'
    

Troubleshooting

If you are experiencing issues with your VPN setup based on the instructions above, try these tips to troubleshoot your setup:

  1. Determine whether the two VPN endpoints are able to communicate at all.

    Use netcat to send VPN-like traffic (UDP, port 4500). Run the following command on your local VPN endpoint:

    $ echo | nc -u  4500
    

    Run tcpdump on the receiving end to determine that your Compute Engine instance can receive the packet on port 4500:

    $ tcpdump -nn -n host  -i any
    

  2. Turn on more verbose logging by adding the following lines to your ipsec.conf files:

    config setup
      charondebug="ike 3, mgr 3, chd 3, net 3"
    
    conn myconn
      authby=psk
      auto=start
      ...
    

    Next, retry your connection. Although the connection should still fail, you can check the log for errors. The log file should be located at /var/log/charon.log on your Compute Engine instance.

Send feedback about...

Compute Engine Documentation