Using Subnetworks

Subnetworks segments your Cloud network IP space into subnetworks. Subnetwork prefixes can be automatically allocated, or you can create a custom topology.

Before you begin

Subnetworks

The key benefits of Subnetworks are as follows:

  • Subnetworks allow you to regionally segment the network IP space into prefixes (subnets) and control which prefix an instance's internal IP address is allocated from.
  • When using VPN, subnetworks allow you to target VPN tunnels to a particular region. This allows additional control over how you configure your VPN routes. This also results in lower latency in cases where the VPN would previously have to assume the IP range spanned across all regions.
  • Subnetworks in a network do not all have to come from the same larger prefix. Some subnetworks can use ranges from 10.0.0.0/8 while others use ranges from 192.168.0.0/16. This provides you the granularity and flexibility to configure unused ranges in your private IP space without having to rearrange existing allocated IP ranges.
  • The overall network IP space does not have to be determined when you first create the project. You can add new subnetworks as needed.
  • Subnetworks provide a mechanism for aggregating IP addresses based on functional groups or organizational departments, while allowing them to communicate internally through the private RFC1918 spaces. Now that you have finer control over your cloud network address space, this can make the management of your on-premises network and your cloud network easier.
  • When using two or more subnetworks as the target of a VPN gateway, peer routers will optimize the routing of the traffic to the correct subnetwork, instead of routing it across the whole network as it does with the legacy network.

This document describes how to set up and use subnetworks, including changes to other networking features such as VPN, instance groups, and load balancing, and describes how they interact with subnetworks.

This document describes the behavior of the system in the following modes:

  • Legacy (non-subnetwork) mode is the original approach for networks, where IP address allocation occurs at the global network level. This means the network address space spans across all regions. You can still create a legacy network, but subnetworks are the preferred approach and default behavior going forward.
  • Subnet mode is the new form of networks in which your network is subdivided into regional subnetworks. Each subnetwork controls the IP address range used for instances that are allocated to that subnetwork. The IP ranges of the different subnetworks in a network might be non-contiguous. There are two options for using subnetworks:
    • Auto subnet network automatically assigns a subnetwork IP prefix range to each region in your network. The instances created in a zone in a specific region in your network get assigned an IP allocated from the regional subnetwork range. The default network for a new project is an auto subnet network.
    • Custom subnet network allows you to manually define subnetwork prefixes for each region in your network. There can be zero, one, or several subnetwork prefixes created per region for a network. In order to create an instance in a zone, you must have previously created at least one subnetwork in that region. At instance creation time, you will need to specify the subnetwork in the region that the instance IP should be allocated from.

Restrictions

  • You cannot change the type of a network after you create it. For example, you cannot change an auto subnet network to a custom subnet network and vice versa.
  • You cannot create a legacy (non-subnetwork) network in the Cloud Platform Console. Use gcloud to create a legacy network if necessary.
  • You cannot change the IP prefix of a subnetwork after it is created.
  • For VPNs, the --local-traffic-selector field does not support IP ranges that do not belong to the IPv4 range of the VPN gateway.
  • The following restrictions exist when using subnetworks with other products:
  • Quotas:
    • 100 subnetworks per project. There are no restrictions on how these subnetworks are allocated per network or per region within a project.
    • The maximum number of CIDR ranges when creating VPN Tunnel, specified in the --local-traffic-selector field is 128.

Legacy (non-subnet) network

In a traditional Compute Engine network, you define a single network IPv4 prefix range for all the instances attached to that network, and that network spans all Google Cloud Platform regions.

Each instance within a network is assigned an IPv4 address from a global network IPv4 range. Instance IP addresses are not grouped by region or zone. One IP address might appear in one region, and its neighbor might be in a different region. Any given range of IPs can be spread across all regions, and the IP addresses of instances created within a region are not necessarily contiguous. This is not ideal because:

  • You can't group IP spaces by region.
  • If you want to group your instances into separate IP spaces, you have to use the network as the grouping mechanism. This is problematic if you want the groups of instances to talk to each other because the communication across networks is only possible through external public IP space. With legacy networks, the network is the only available IP grouping mechanism. There is no mechanism for grouping IPs within the network.

The figure below shows a traditional network, which is now called a Legacy (non-subnet) network. Traffic from the Internet passes through a global switching function in the network (shown in the diagram as a virtual switch), then down to individual instances.

Instances in a region can have IP addresses that are not grouped in any way. As shown in the example, instances from 10.240.0.0/16 are spread unpredictably across regions 1 and 2. For example, 10.240.1.4 is in region 2, 10.240.1.5 is in region 1, and 10.240.1.6 is in region 2.

Diagram of a legacy network (click to enlarge)
Diagram of a legacy network (click to enlarge)

Subnet network

A subnet network does not require a global network IPv4 prefix range. Instead, a network can contain multiple subnetworks, each with its own IPv4 prefix. Subnetworks belonging to a single network must have non-overlapping RFC1918 ranges. Subnetworks belonging to a single network can have non-contiguous IPv4 address prefixes (e.g., 10.240.0.0/16 and 192.168.0.0/16). A subnetwork belongs to only one network and each instance can only belong to one subnetwork.

The default network for a new project is an auto subnet network. You can create up to four additional networks in a project. Additional networks can be auto subnet networks, custom subnet networks, or legacy networks.

Creating subnetworks for a project helps you group instances by functionality or organizational department. Instances in different subnetworks can communicate with each other using their internal IPs, as long as they belong to the same network. Instances belonging to two different networks can only communicate using external IPs. For example, if you want to keep instances in your testing and production systems from talking to one another except through external IPs, you can put the instances in different networks. Instances in different networks are completely isolated and can have overlapping address ranges. So the communication across networks is only possible through external public IP space. Communication over external IP space warrants stronger security and may incur additional costs. However, if you want all the instances running one product to be able to talk to all the instances running a second product, but you want to chunk them by IP address, you can use two different subnetworks in the same network.

Each instance created within a subnetwork gets assigned an IPv4 address from that subnetwork's range. Instances within a subnetwork are within the same IPv4 range.

The figure below shows a subnet network. A new instance takes its IP address from the given subnetwork's prefix. Subnet1 is in region 1 and has all of its private IPs allocated from 10.240.0.0/24 (instance IPs 10.240.0.1 and 10.240.0.2), and Subnet2 in region 2 has all of its private IPs allocated from 192.168.1.0/24 (instance IPs 192.168.1.1 and 192.168.1.2). You can also see that Subnet3 crosses zones within a region. Subnetworks are not restricted by zone. Subnetworks are restricted only by region, but you can choose to create instances from one subnet in one zone and instances from another subnet in a separate zone if you desire.

Diagram of a subnet network (click to enlarge)
Diagram of a subnet network (click to enlarge)

Subnetworks features

Subnetworks partition your network into predictable IP address ranges. Subnetworks have the following key properties:

  • Each subnetwork has a known CIDR range. All instances created in that subnetwork have internal IP addresses assigned from a CIDR range.
  • Subnetwork ranges can be automatically allocated (auto subnet network) or specifically chosen by you (custom subnet network).
  • Each subnetwork is restricted to a single region, but you can choose to confine instances to a zone if you desire..
  • In the case where the subnetwork ranges are specifically chosen by you (custom subnet network), the network can have zero, one, or many subnetwork CIDR ranges in a given region.

Subnetwork IP ranges

Subnetwork ranges can be automatically or manually configured. This section describes both kinds of ranges.

Each subnet has the following reserved addresses:

  • Network address
  • Broadcast address
  • Gateway IP address (first address in the CIDR range)
  • Reserved address (second last address in the CIDR range)

The minimum subnetwork size that can be used is /29.

Auto subnet IP ranges

When using an auto subnet network, a subnetwork is created automatically for that network in every region. The following table shows the automatically configured IP ranges and default gateway for each region. Every auto subnet network uses these ranges.

Region IP range Default gateway
us-west1 10.138.0.0/20 10.138.0.1
us-central1 10.128.0.0/20 10.128.0.1
us-east1 10.142.0.0/20 10.142.0.1
europe-west1 10.132.0.0/20 10.132.0.1
asia-east1 10.140.0.0/20 10.140.0.1
asia-northeast1 10.146.0.0/20 10.146.0.1

Custom subnet IP ranges

Manually created subnetworks can use any valid RFC1918 IP range. Ranges do not have to be contiguous between subnetworks. For example, some subnetworks can use ranges from 10.0.0.0/8 while others use ranges from the 192.168.0.0/16 space. Ranges must be unique and non-overlapping within a network.

Subnetworks resource management

This section shows how to create subnetworks.

The gcloud Tool Guide has more information about the commands used in these procedures.

Networks and subnetworks

When you create a new subnet network, you can either accept the auto-created subnet allocations or specify your own with a custom subnet network.

You cannot create a network with the same name as any existing subnet in the same project. You cannot create a subnet with the same name as an existing network in the same project, except that each network can have one subnet per region that has the same name as its parent network.

Creating a new network with auto-created subnetwork ranges

  1. Create a new network in your project with automatically allocated subnetworks. The --mode specifies auto, indicating you would like the subnetworks to be automatically created with pre-set IP blocks. Because this is a subnet network, there is no network-level IPv4 range or gateway IP, so none will be displayed.

    gcloud compute networks create auto-network1 \
        --mode auto
    

    NAME          MODE IPV4_RANGE GATEWAY_IPV4
    auto-network1 auto

  2. List your new auto-created subnetworks.

    gcloud compute networks subnets list
    

    NAME          REGION          NETWORK       RANGE
    auto-network1 asia-east1      auto-network1 10.140.0.0/20
    auto-network1 us-central1     auto-network1 10.128.0.0/20
    auto-network1 europe-west1    auto-network1 10.132.0.0/20

Creating a new network with custom subnet ranges

To manually assign subnetwork ranges, you must first create a custom subnet network, then create the subnetworks you want within a region. You do not have to specify subnetworks for all regions right away, or even at all, but you cannot create instances in regions that have no subnetwork defined.

When creating a new subnetwork, the subnetwork name has to be unique in that project for that region, even across networks. The same name can appear twice in a project, as long as each one is in a different region.

Because this is a subnet network, there is no network-level IPv4 range or gateway IP, so none will be displayed.

  1. Create a new custom subnet network in your project.

    gcloud compute networks create custom-network1 \
        --mode custom
    

    NAME            MODE   IPV4_RANGE GATEWAY_IPV4
    custom-network1 custom

  2. Specify the subnetwork prefix for your first region. In this example, we're assigning 192.168.1.0/24 to region us-central1.

    gcloud compute networks subnets create subnet-us-central-192 \
        --network custom-network1 \
        --region us-central1 \
        --range 192.168.1.0/24
    

    NAME                  REGION      NETWORK         RANGE
    subnet-us-central-192 us-central1 custom-network1 192.168.1.0/24

  3. Specify the subnetwork prefix for your second region. In this example, we're assigning 192.168.5.0/24 to region europe-west1.

    gcloud compute networks subnets create subnet-europe-west-192 \
        --network custom-network1 \
        --region europe-west1 \
        --range 192.168.5.0/24
    

    NAME                   REGION       NETWORK         RANGE
    subnet-europe-west-192 europe-west1 custom-network1 192.168.5.0/24

  4. Specify the subnetwork prefix for your third region. In this example, we're assigning 10.1.1.0/24 to region asia-east1.

    gcloud compute networks subnets create subnet-asia-east-192 \
       --network custom-network1 \
       --region asia-east1 \
       --range 10.1.1.0/24
    

    NAME                 REGION     NETWORK         RANGE
    subnet-asia-east-192 asia-east1 custom-network1 10.1.1.0/24

  5. List your networks. If you also created an auto subnet network in the prior section, those subnets will be listed as well.

    gcloud compute networks subnets list
    

    NAME                           REGION          NETWORK         RANGE
    subnet-europe-west-192         europe-west     custom-network1 192.168.5.0/24
    subnet-us-central-192          us-central1     custom-network1 192.168.1.0/24
    subnet-asia-east-192           asia-east1      custom-network1 10.1.1.0/24

Creating a legacy network

You can still create a legacy (non-subnet) network. Legacy networks have a single global IP range. You cannot create subnetworks in a legacy network or switch from legacy to auto or custom subnet networks.

To create a new legacy network in your project:

gcloud compute networks create legacy-network1 \
    --mode legacy \
    --range 10.240.0.0/16
Created
[https://www.googleapis.com/compute/latest/projects/[PROJECT_ID]/global/networks/legacy-network1].
NAME            MODE   IPV4_RANGE    GATEWAY_IPV4
legacy-network1 legacy 10.240.0.0/16 10.240.0.1

Listing existing subnetworks

You can list all subnetworks in an existing project.

List general details on all subnetworks.

gcloud compute networks subnets list
NAME                      REGION          NETWORK         RANGE
auto-network1             europe-west1    auto-network1   10.132.0.0/20
subnet-europe-west-192    europe-west     custom-network1 192.168.5.0/24
auto-network1             us-central1     auto-network1   10.128.0.0/20
subnet-us-central-192     us-central1     custom-network1 192.168.1.0/24
subnet-asia-east-192      asia-east1      custom-network1 10.1.1.0/24
auto-network1             asia-east1      auto-network1   10.140.0.0/20

You can tailor the list by providing additional parameters:

  • --regions REGION,[REGION,...] Restrict the list to subnets in particular regions.
  • --network NETWORK Restrict the list to only subnets in a particular network.

Describe an existing subnetwork

The describe command provides details on the indicated subnetwork.

gcloud compute networks subnets describe subnet-asia-east-192 \
  --region asia-east1
creationTimestamp: '2015-10-21T15:58:35.271-07:00'
gatewayAddress: 192.168.7.1
id: '2048026325272602228'
ipCidrRange: 192.168.7.0/24
kind: compute#subnetwork
name: subnet-asia-east-192
network: https://www.googleapis.com/compute/latest/projects/[PROJECT_ID]/global/networks/custom-network1
region: https://www.googleapis.com/compute/latest/projects/[PROJECT_ID]/regions/asia-east1
selfLink: https://www.googleapis.com/compute/latest/projects/[PROJECT_ID]/regions/asia-east1/subnetworks/subnet-asia-east-192

Switch a network from auto to custom

You can change an auto subnet mode network to be a custom subnet mode network.

After the change, the custom subnet mode network has the same subnet names and IP ranges of the original auto mode subnet network.

After the switch, you can add new subnets to the custom subnet mode network. See step 2 of Creating a new network with custom subnet ranges. You can also delete subnets in the network, including the original auto subnets.

gcloud compute networks switch-mode NETWORK_NAME --mode custom

Expand a custom subnet

You can expand the IP range of a subnet in a custom subnet mode network. You cannot shrink it.

To set the size of the new subnet, specify the new netmask. For example, if the original IP range is 10.128.131.0/24, specifying --prefix-length 20 sets the new IP range to 10.128.128.0/20. The new network range must be larger than the original, which means the prefix length value must be a smaller number. The new subnet must not overlap with other subnets in the same network in any region. The new subnet must stay inside the RFC1918 address spaces.

Since subnets are uniquely identified by region and name, you must specify those in the command.

Depending on the size and IP allocation of the existing network, this command may take from several seconds to several minutes to run. You may be unable to create new instances during that time, but existing instances are unaffected. During the expansion, existing instances are still accessible and will still pass traffic.

gcloud compute networks subnets expand-ip-range [SUBNET_NAME]
  --network [NETWORK_NAME]
  --region [REGION]
  --prefix-length [PREFIX_LENGTH]
  • --prefix-length - the new numeric prefix length for the subnetwork. Must be smaller than the existing prefix length. For example, if the current subnet is a /24, the new prefix length must be 23 or smaller.

Delete a subnetwork or network

You can delete any subnetwork in your project. You can also delete an entire network. You cannot delete a network or subnetwork that still has instances or resources.

Delete a subnetwork

You can only delete manually created subnetworks. Automatically created subnetworks can not be deleted individually; you have to delete the entire network.

You cannot delete any subnetwork that still has instances, manually created routes, or other manually created resources using it.

gcloud compute networks subnets delete subnet-asia-east-192 \
    --region asia-east1
The following subnetworks will be deleted:
 - [subnet-asia-east-192] in [asia-east1]
Do you want to continue (Y/n)?  y
Deleted [https://www.googleapis.com/compute/latest/projects/[PROJECT_ID]/regions/asia-east1/subnetworks/subnet-asia-east-192].

Delete a network

For a legacy network, you can explicitly delete the network resource only if the network resource is not in use by any resources.

For an auto subnet network, you can explicitly delete the network resource only if

  • Any child subnetworks it has are not in use by any other resource AND
  • The network resource is not in use by any other resources.

Example resources in use that restrict the deletion for the above include firewalls, custom routes, instances, target VPN gateways, and routers.

For a custom subnet network, the network can only be deleted if all child subnetworks have been deleted.

To delete the network:

gcloud compute networks delete auto-network1
The following networks will be deleted:
 - [auto-network1]
Do you want to continue (Y/n)?  y
Deleted [https://www.googleapis.com/compute/latest/projects/[PROJECT_ID]/global/networks/auto-network1].

Subnetworks and other resources

This section describes how subnetworks relate to other Google Cloud Platform resources that depend upon subnetworks.

Subnetworks and instances

A subnetwork spans all of the zones within the region it is created in. However, when you create an instance, you specify which zone it is in and which subnetwork it is in. So, you can create one set of instances in subnetwork1 and in zone1 of region1 while another set of instances is in subnetwork2 and in zone2 of region1.

Create an instance in a subnetwork

This command creates an instance in the specified subnetwork and zone. Note that you cannot specify both the --network and --subnet parameters for this command. For other, non-subnetwork-related instance parameters, see gcloud compute instances create in the Cloud SDK documentation.

gcloud compute instances create instance1 \
   --zone us-central1-a \
   --subnet subnet-us-central-192
Created [https://www.googleapis.com/compute/latest/projects/[PROJECT_ID]/zones/us-central1-a/instances/instance1].
NAME      ZONE          MACHINE_TYPE  PREEMPTIBLE INTERNAL_IP EXTERNAL_IP   STATUS
instance1 us-central1-a n1-standard-1             192.168.1.4 70.32.154.181 RUNNING

Subnetworks and load balancing

Instance groups that are the targets of load balancing must be contained within a single subnetwork.

When you associate an unmanaged instance group with a backend service, the system verifies that all the instances are in the same subnetwork. If they are not all in the same subnetwork, the operation returns an error.

When you create a managed instance group for HTTP(S) load balancing, you can specify a subnetwork in the Instance Template used by the managed instance group. If the network is auto-subnet mode, the subnetwork is optional. If the network is custom mode, the subnetwork is required.

If you create an instance group that spans subnetworks, you cannot use that instance group in a backend service.

Additionally, managed instance groups cannot be updated to use an instance template that has a different network or subnetwork than the one originally defined.

Create an instance template that specifies a subnetwork

Instance template commands now have optional --subnet and --region flags that you can use so that new instances created will be placed into the subnetwork of your choice. Note, the subnet cannot be changed in the instance template after the instance template is created. The --subnet flag requires the --region flag. The subnetwork must exist before you can include it in an instance template.

This example creates a template called template-qa that only creates instances in the subnet-us-qa subnet.

gcloud compute instance-templates create template-qa \
    --region us-central1 \
    --subnet subnet-us-qa
Created [https://www.googleapis.com/compute/latest/projects/[PROJECT_ID]/global/instanceTemplates/template-qa].
NAME        MACHINE_TYPE  PREEMPTIBLE CREATION_TIMESTAMP
template-qa n1-standard-1             2015-10-23T20:34:00.791-07:00

Using this template to create instances for a managed instance group (with or without autoscaling) automatically creates the instance in the specified region and subnetwork. This lets you control the subnetwork of new instances created for load balancing.

Subnetworks and firewall rules

Firewalls are global resources attached to a single network, not regional resources. Firewall rules apply to all instances in a network. This means you can add a firewall rule that allows traffic between instances within the same network, even across subnetworks.

The default firewall rules applied to the default network remain the same. Also, any existing firewall rules that you created remain unchanged. Any networks you create subsequently in a project have no default firewall rules. In the absence of firewall rules, all inbound communication is blocked by default. Firewall rules have to be created as required to allow traffic into instances in non-default networks.

User-created networks don't contain default firewall rules. You have to create specific firewall rules allowing traffic to flow between the subnetworks that are created.

For two subnetworks to exchange traffic, “allow” permissions have to be configured in both directions.

Refer to Managing Firewalls for detailed documentation.

Isolation of subnetworks

In the default network, by default, all virtual machine instances within a network can communicate with each other, because of the default firewall rules described above. On the other hand, user defined networks do not have default firewall rules, so instances cannot be reached, even by other instances, until you define appropriate firewall rules.

Normally, all firewall rules apply to all instances in the network. However, you can add tags to certain instances and use those tags in firewall rules, thus creating rules that only apply to certain instances. Tags can be applied to instances either individually or through instance templates. You can use these tags to create additional isolation between subnetworks by selectively allowing only certain instances to communicate. If you arrange for all instances in a subnetwork to share the same tag, you can specify that tag in firewall rules to simulate a per-subnetwork firewall. For example, you can tag all instances in subnet-a with the tag tag-subnet-a and use the that tag in firewall rule target-tags. You can also use source-ranges in firewall rules.

Assume that you have a network my-network with 3 subnetworks:

  • subnet-a with IP prefix 192.168.2.0/24
  • subnet-b with IP prefix 10.128.0.0/16
  • subnet-c with IP prefix 10.240.0.0/16
Diagram demonstrating isolation of subnetworks
Diagram demonstrating isolation of subnetworks

Communication is allowed between subnet-a and any other subnet, but must be blocked between subnet-b and subnet-c:

These are the configuration steps required:

  1. Tag the instances in the subnetworks. Configure all your instances in a subnetwork with a common tag. For example, tag-subnet-a for subnetwork subnet-a:

    gcloud compute instances add-tags example-instance-subnet-a \
        --tags tag-subnet-a --zone us-central1-b
    

    Updated [https://www.googleapis.com/compute/latest/projects/[PROJECT_ID]/zones/us-central1-b/instances/example-instance-subnet-a].

    Tag example-instance-subnet-b:

    gcloud compute instances add-tags example-instance-subnet-b \
       --tags tag-subnet-b
    

    Updated [https://www.googleapis.com/compute/latest/projects/[PROJECT_ID]/zones/us-central1-b/instances/example-instance-subnet-b].

    Tag example-instance-subnet-c:

    gcloud compute instances add-tags example-instance-subnet-c \
       --tags tag-subnet-c
    

    Updated [https://www.googleapis.com/compute/latest/projects/[PROJECT_ID]/zones/us-central1-b/instances/example-instance-subnet-c].

    Note that you can tag your instances at creation time (gcloud compute instances create example-instance-subnet-a .... --tag tag-subnet-a)

    If all your instances in the subnetwork are of the same type, you can group them into a managed instance group and use an instance template to give them all the same tag automatically.

  2. Create the firewall rules to enable communication between subnet-a and subnet-b and between subnet-a and subnet-c.

    Allow traffic sourced in subnet-a CIDR (192.168.2.0/24) to all the instances belonging to subnet-b (tagged with tag-subnet-b) and to subnet-c (tagged with tag-subnet-c):

    gcloud compute firewall-rules create allow-subnet-a \
         --allow tcp,udp,icmp \
         --source-ranges 192.168.2.0/24 \
         --target-tags tag-subnet-b,tag-subnet-c
    

    Created [https://www.googleapis.com/compute/latest/projects/[PROJECT_ID]/global/firewalls/allow-subnet-a].
    NAME           NETWORK    SRC_RANGES    RULES        SRC_TAGS TARGET_TAGS
    allow-subnet-a my-network 19.168.2.0/24 tcp,udp,icmp          tag-subnet-a,tag-subnet-c

  3. Allow traffic sourced in subnet-b CIDR (10.128.0.0/16) to all the instances belonging to only subnet-a (tagged with tag-subnet-a):

    gcloud compute firewall-rules create allow-subnet-b \
       --allow tcp,udp,icmp \
       --source-ranges 10.128.0.0/16 \
       --target-tags tag-subnet-a
    

    Created [https://www.googleapis.com/compute/latest/projects/[PROJECT_ID]/global/firewalls/allow-subnet-b].
    NAME           NETWORK    SRC_RANGES    RULES        SRC_TAGS TARGET_TAGS
    allow-subnet-b my-network 10.128.0.0/16 tcp,udp,icmp          tag-subnet-a

  4. Allow traffic sourced in subnet-c CIDR (10.240.0.0/16) to all the instances belonging to only subnet-a (tagged with tag-subnet-a):

    gcloud compute firewall-rules create allow-subnet-c \
       --allow tcp,udp,icmp \
       --source-ranges 10.240.0.0/16 \
    

    Created [https://www.googleapis.com/compute/latest/projects/[PROJECT_ID]/global/firewalls/allow-subnet-c].
    NAME           NETWORK    SRC_RANGES    RULES        SRC_TAGS TARGET_TAGS
    allow-subnet-c my-network 10.240.0.0/16 tcp,udp,icmp          tag-subnet-a

Isolation of subnetworks B and C is achieved by not explicitly allowing B and C to exchange traffic.

Subnetworks and routes

Routes are global resources attached to a single network, not regional resources. User-created routes apply to all instances in a network. This means you can add a route that forwards traffic from instance to instance within the same network, even across subnetworks, without requiring external IP addresses. By default, routes exists between subnetworks.

Default routes

Certain routes are created when you create a subnetwork or if you create a legacy network.

For subnet networks

The following routes are created.

  • A default route for Internet traffic (0/0) is created when the network is created.
  • A route is created for each subnetwork when the subnetwork is created. These routes are for local traffic in the network, allowing instances in any subnetwork to send traffic to instances in any other or same subnetwork in that network. This route counts against overall route quota. The overall route quota for a project is currently 100, and each subnet created uses one route against the quota. Note that the networks you create do not have firewall rules yet, so even though there are routes pointing to instances, the firewall will prevent traffic from traveling to instances until you add firewall rules.
For legacy networks

Two routes are created at network creation time.

  • A default route for Internet traffic (0/0) is created when the network is created.
  • For the destination IP range within the IPv4 range of the network, a virtual network route is defined. For the default legacy network, default firewall rules are also automatically created, so traffic between instances is allowed from the beginning. For manually created legacy networks, you have to create firewall rules that allow inbound traffic to instances.

For instructions on creating additional routes, see the gcloud compute routes create command.

Routes and VPN tunnels

If you want to apply routes to a subnetwork only, you must apply the routes specifically to the instances that are part of the subnetwork.

Consider the following scenario, in which you have a network with two custom subnet networks: subnet-a and subnet-b. You want to route traffic from subnet-a to VPN tunnel to reach the peer network, but you do not require subnet-b to be able to route traffic through VPN tunnel-a.

Diagram of routing traffic from a single subnetwork to a VPN tunnel
>Diagram of routing traffic from a single subnetwork to a VPN tunnel

The steps to apply routes to subnet-a only in the above scenario are:

  1. Configure all your instances in a subnet-a with a common tag (e.g., tag-subnet-a):

    gcloud compute instances add-tags example-instance-subnet-a \
        --tags tag-subnet-a
    

    Updated [https://www.googleapis.com/compute/latest/projects/[PROJECT_ID]/zones/us-central1-b/instances/example-instance-subnet-a].

    Note that you can tag your instances at creation time (gcloud compute instances create example-instance-subnet-a .... --tag tag-subnet-a)

  2. Add a static route to all the instances in subnet-a. This route has as its destination the peer prefix 192.168.1.0/24 and a next hop of tunnel-a.

    gcloud compute routes create route-via-tunnel \
        --destination-range 192.168.1.0/24 \
        --network my-network \
        --tags tag-subnet-a \
        --next-hop-vpn-tunnel tunnel-a
    

Subnetworks and VPN

Subnetworks provides a way for instances to be grouped by a common IP prefix. Such a grouping is useful in configuring policy (static routes) based VPN. Subnetworks enhances Cloud VPN functionality by providing control on what IP prefixes need to be announced on policy based VPNs by enabling on-premise VPN device to set up routing policy based on subnetwork prefixes, likely reducing latency between on-premise and cloud networks.

Consider the following configuration. Without subnetworks, if you run instances in two regions you'll need the following topology:

Diagram of a VPN without subnetworks (click to enlarge)
>Diagram of a VPN without subnetworks (click to enlarge)

With a legacy network there is no control over the IP allocations, the IP prefix is fragmented between the US East and Europe regions. When your on-site VPN gateway has to make a routing decision it has no choice but to ECMP packets across both links.

When a host in the on-premise networks sends packets to instances in Europe, roughly 50% of these packets will go via the US East region, potentially adding to the VPN latency.

With subnetworks, if you define the two subnetworks such that 10.240.1.0/24 is in US East and 10.240.2.0/24 is in Europe, then they will have the following topology shown in the following diagram.

Diagram of a VPN with subnetworks (click to enlarge)
>Diagram of a VPN with subnetworks (click to enlarge)

With this approach, your peer VPN gateway has a routing table that allows it to direct packets to the correct region based on where the instance is located, thereby reducing average latency.

Subnetworks lets instances to be grouped regionally with a common IP prefix. This regionally grouped IP prefix can be statically announced during VPN tunnel creation. Users have flexibility to choose which subnetwork prefixes have to be announced to peer VPN endpoint at the VPN tunnel creation time.

Creating a Compute Engine VPN gateway

Compute Engine VPN gateway creation does not change with subnetwork. See the gcloud compute target-vpn-gateways create Cloud SDK documentation.

Creating a VPN tunnel

To create a VPN tunnel, specify the IP ranges for the tunnel and route that traffic to the tunnel.

Traffic selector is an agreement between IKE peers to permit traffic through a tunnel if the traffic matches the specified addresses. With subnetworks, you must specify which Google Cloud Platform CIDR ranges are valid for a VPN tunnel. These ranges are specified in the Local IP ranges field in the Cloud Platform Console and the --local_traffic_selector field in gcloud. These ranges are configured when the tunnel is created and may not be changed afterwards because they are used during the IKE handshake on connect.

The maximum number of CIDR ranges specified in the traffic selector is 128.

In some VPN configurations, gateways allow traffic to pass through that was not specified in the traffic selector during the IKE handshake. For consistent and predictable VPN behavior, make sure the routes destined for the tunnel match the prefixes specified during tunnel creation.

Traffic selector and auto subnet networks

If the network of the VPN gateway is an auto subnet network, the CIDR range of the subnetwork in the same region as the VPN gateway is automatically announced to the peer VPN. If you only want that subnetwork to use the tunnel, you do not have to manually specify the traffic selector. You do have to specify a route as shown in the steps below.

If you have an auto subnet network and wish subnets other than the subnet containing the gateway to use the tunnel, you must specify these ranges and create routes for them. If you use this field, you must even specify the subnet local to the gateway.

If you are using an auto subnet network and you don't specify the traffic selector, you get the default behavior, which is that the subnet local to the Cloud VPN gateway is used to create the tunnel.

Traffic selector and custom subnet networks

If the network of the VPN gateway is a custom subnet network, no IP prefix is announced to the peer VPN by default. You must use the traffic selector to specify the IP prefixes for the subnetworks that will be routed through the tunnel. You must also specify routes for traffic to reach the tunnel.

Traffic selector and legacy networks

If your network is a legacy network, then entire network is announced by default to the tunnel. If you wish to restrict the announcement to a smaller range of instances, you can use the traffic selector field to do this. However, since the IP addresses for a legacy network are not allocated predictably across regions, this can result in inefficient performance for your VPN due to ECMP if you are using multiple regions as described in the above example.

Configuration of VPN in the different types of networks

Auto subnet networks
Configuring VPN use to subnetwork in the region where VPN gateway exists

In an auto subnet network, one subnetwork is automatically created with a pre-defined CIDR range for every region in the network. For a VPN tunnel associated to an auto subnet network, if --local-traffic-selector parameter is omitted, only the IP prefix of the subnetwork in the same region as the VPN gateway is announced to the remote peer.

Consider the following scenario in an auto subnet network, in which only subnetwork-b, located in the same region as the VPN, is allowed to use the tunnel and not other subnetworks in other regions:

Diagram for configuring a VPN to use a subnetwork in a region where a VPN gateway exists (click to enlarge)
Diagram for configuring a VPN to use a subnetwork in a region where
  a VPN gateway exists (click to enlarge)

To create a VPN tunnel with an auto-created subnetwork, use the normal VPN instructions, but change the following steps as shown:

Step 2: Create a VPN Gateway in network my-autosubnet-network and region region-b.

gcloud compute target-vpn-gateways create target-vpn-gateway-b \
    --network my-autosubnet-network \
    --region region-b

Step 7: Create the VPN tunnel tunnel-b:

    gcloud compute vpn-tunnels create tunnel-b \
       --peer-address [PEER_IP_ADDRESS] \
       --region region-b \
       --shared-secret [SHARED_SECRET] \
       --target-vpn-gateway target-vpn-gateway-b

Since --local-traffic-selector is omitted, only subnetwork-b CIDR 10.130.0.0/16 is announced during tunnel creation. In other words, only traffic from instances in subnetwork-b will be allowed through VPN tunnel-b.

Step 8: Add a static route to all the instances in subnetwork-b. This route has as its destination the peer prefix 192.168.0.0/16 and a next hop of tunnel-b.

gcloud compute routes create route-via-tunnel \
    --destination-range 192.168.0.0/16 \
    --network my-autosubnet-network \
    --next-hop-vpn-tunnel-region region-b \
    --next-hop-vpn-tunnel tunnel-b
Configuring VPN use by subnetworks in other regions

In an auto subnet network, you can specify the IP prefixes that are announced during handshake. This allows you to include prefixes other than the local one, which then allows the subnets using those prefixes to use the tunnel as well.

For example, to allow the subnetwork in region-a to send and receive traffic through the tunnel in region-b, configure that the range 10.128.0.0/16 to be used in the IKE exchange of tunnel-b.

Consider the following scenario in which there is a VPN tunnel in region-b that can be used by subnetwork-a in region-a:

Diagram for configuring a VPN for use in any subnetwork in any region (click to enlarge)
Diagram for configuring a VPN for use in any subnetwork in any region
    (click to enlarge)

To create a VPN tunnel with an auto-created subnetwork and specifying --local-traffic-selector, use the normal VPN instructions, but change the following steps as shown. Note that because you are using the --local-traffic-selector parameter, all necessary networks, including the network local to the tunnel, must be specified.

Step 2: Create a VPN Gateway in network my-autosubnet-network and region region-b.

gcloud compute target-vpn-gateways create target-vpn-gateway-b \
     --network my-autosubnet-network \
     --region region-b

Step 7: Create a VPN tunnel tunnel-b :

gcloud compute vpn-tunnels create tunnel-b \
     --peer-address PEER_IP_ADDRESS \
     --region region-b \
     --shared-secret SHARED_SECRET \
     --local-traffic-selector 10.128.0.0/16,10.130.0.0/16 \
     --target-vpn-gateway target-vpn-gateway-b

By adding --local-traffic-selector with CIDRs from subnetwork-a (10.128.0.0/16) and subnetwork-b (10.128.0.0/16), traffic from any instances in those subnetworks will be allowed through the tunnel.

Step 8: Add a common tag to all your instances in subnetwork-a and subnetwork-b:

gcloud compute instances add-tags example-instance-subnet-a \
    --tags tag-subnet-a
gcloud compute instances add-tags example-instance-subnet-b \
    --tags tag-subnet-b

Step 9: Add a static route to all of the instances in subnetwork-a and subnetwork-b. This route has as destination peer prefix 192.168.0.0/16 and a next hop of tunnel-b.

gcloud compute routes create route-via-tunnel \
    --destination-range 192.168.0.0/16 \
    --network my-autosubnet-network \
    --next-hop-vpn-tunnel-region region-b \
    --next-hop-vpn-tunnel tunnel-b \
    --tags tag-subnet-a,tag-subnet-b
Custom subnet networks
Configuring VPN use by a given subnetwork

In custom subnet networks, there could be one or more subnetworks for a network in the region, and the IP ranges associated to those subnetworks are configured by you. Therefore, the IP prefixes to be announced must always be configured in --local-traffic-selector. The absence of --local-traffic-selector configured in the VPN tunnel in a custom subnet network will return an error.

Consider the following scenario in which there is a custom subnet network with a subnetwork-a in region-a with CIDR range 172.16.0.0/16 and a subnetwork-b in region-b with CIDR range 10.21.0.0/16. With the following configuration, VPN in region-b can ONLY be used by subnetwork-b:

Diagram for configuring a VPN to use a custom subnetwork in any region (click to enlarge)
>Diagram for configuring a VPN to use a custom subnetwork in any
  region (click to enlarge)

To create a VPN tunnel with a custom subnet, use the normal VPN instructions, but change the following steps as shown:

Step 2: Create a VPN Gateway in network my-custom-network and region region-b.

gcloud compute target-vpn-gateways create target-vpn-gateway-b \
    --network my-custom-network \
    --region region-b

Step 7: Create a VPN tunnel tunnel-b:

    gcloud compute vpn-tunnels create tunnel-b \
       --peer-address [PEER_IP_ADDRESS] \
       --region region-b \
       --shared-secret [SHARED_SECRET] \
       --local-traffic-selector 10.21.0.0/16 \
       --target-vpn-gateway target-vpn-gateway-b

You MUST add --local-traffic-selector with the CIDR of subnetwork-b. An error is returned if there is no --local-traffic-selector configured in the VPN tunnel in a custom subnet network.

Step 8: Add a common tag to all your instances in subnetwork-b:

gcloud compute instances add-tags example-instance-subnet-b \
    --tags tag-subnet-b

Step 9: Add a static route to all the instances in subnetwork-b. This route has as destination peer prefix 192.168.0.0/16 and as the next hop tunnel-b.

gcloud compute routes create route-via-tunnel \
    --destination-range 192.168.0.0/16 \
    --network my-custom-network \
    --tags tag-subnet-b \
    --next-hop-vpn-tunnel-region region-b \
    --next-hop-vpn-tunnel tunnel-b

FAQ

Addressing

Q: Can I change a legacy network to a subnet network?

A: Legacy network cannot be converted to subnet network.

Q: What are the valid IP prefixes for subnetworks?

A: Private Address space as defined by RFC 1918 are the valid IP prefixes for subnetworks.

Q: Can subnetworks have overlapping IP prefixes?

A: Within a network, two subnetworks cannot have overlapping IP prefixes. Two subnetworks in two different networks might have duplicate or overlapping IP prefixes.

Q: Should IP prefixes of subnetworks be contiguous in a single Network?

A: Subnetworks within a single network need not be contiguous and can have discrete IP prefixes. For example: lab-subnetwork (10.240.1.0/24) and prod-subnetwork (192.168.1.0/24) is a valid subnetwork configuration.

Q: I want to limit all my instances to a single zone and to a single subnetwork. How can I achieve the zonal scope ?

A: When creating instances, always select the same zone and subnetwork.

Q: Do auto-created subnetworks always have the same IP ranges in each region.

A: Yes, see Automatically configured IP ranges.

Routes and Firewalls

Q: Can instances in subnetworks in the same network communicate with each other by default?

A: Yes. All instances in the same network can communicate with each other, even across subnetworks, using internal IP addresses. However, for user-created networks, firewall rules have to be created to allow it.

Q: Can I prevent two subnetworks within a network from communicating?

A: Yes, subnetworks with a network can be blocked from exchanging traffic by using firewall rules. See Isolation of subnetworks.

VPN

Q: I have created a VPN for an auto subnet network and I can only use it from the subnet that is co-located in the same region by default. Can I use the VPN globally for auto subnet networks, like I did with legacy networks?

A: See VPN configuration in auto subnet network

Scenarios

Q: In what scenarios are auto subnet networks recommended?

A: Auto subnet networks are a quick starter configuration for users who want to focus on deploying applications rather than defining network topology and IP addressing.

Q: How to deploy a multi-tier (Web, API, DB) application in a single network, but with ability to isolate each tier as desired?

A: During network creation, create a custom subnet network. Create 3 subnetworks, one for each tier. E.g., 10.240.10.0/24 (Web), 10.240.30.0/24 (API), 10.240.50.0/24 (DB). See Isolation of subnetworks to isolate Web from DB subnetworks and vice versa.

Q: How to deploy a Dev, Test, and Prod environment for my application in a single network, and still isolate each environment from one another?

A: During network creation, create a custom subnet network. Create 3 subnetworks, one for each environment. E.g. 10.240.10.0/24 (Dev), 192.168.10.0/24 (Test) and 172.16.10.0/24 (Prod). See Isolation of subnetworks to isolate the environments from each other.

Q: How to deploy and isolate instances of multi-tenant application?

A: Prior to subnetworks, users created one network per tenant of their multi-tenant application and one network for shared services/applications (e.g. License, Billing). The shared service application were required to have external IPs to communicate with the tenant application in another network.

Subnetworks simplifies the multi-tenant application deployment model. Users can now create one custom subnetwork per tenant. Tenant applications in each subnetwork can be isolated from one another. Shared services/applications can be in another subnetwork, with ability to reach all tenants subnetworks.

Send feedback about...

Compute Engine Documentation