Google Cloud Platform (GCP) routes define the paths network traffic takes from a VM instance to other destinations. These destinations can be inside your VPC network (for example, in another VM) or outside of it.
Every route consists of a destination and a next hop. Traffic whose destination IP is within the destination range is sent to the next hop for delivery. This page provides an overview of how routes work in GCP.
Routing in GCP
Every VPC network uses a scalable, distributed virtual routing mechanism. Even though some routes can be applied selectively, the routing table for a VPC network is defined at the VPC network level.
Each VM instance has a controller that is kept informed of all applicable routes from the network's routing table. Each packet leaving a VM is delivered to the appropriate next hop of an applicable route based on a routing order. When you add or delete a route, the set of changes is propagated to the VM controllers using an eventually consistent design.
GCP has four different types of routes in two categories. System-generated routes are automatically created when you create a network, add a subnet, or modify the secondary IP range of a subnet. Custom routes are those that you create and maintain, either directly or by virtue of using a Cloud Router. The following table summarizes the different types of routes:
|Type||Category||Destination||Next Hop||Removable||Applies To|
||Yes||All instances in the network|
|subnet route||system-generated||Primary and secondary
subnet IP ranges
||VPC network, which forwards packets to VMs in its subnets||Only when the subnet is deleted or when you change the subnet's secondary IP ranges||All instances in the network|
• IP range that does not partially or exactly overlap with any subnet IP
• IP range broader than a subnet IP range
• An instance by name
• An instance by its IP address
• A Cloud VPN tunnel
|Yes||All instances in network unless restricted to specific instances by network tag|
• IP range that does not partially or exactly overlap with any subnet IP
• IP range broader than a subnet IP range
|IP address of the Cloud Router's BGP peer||Only by a Cloud Router if it no longer receives the route from its BGP peer||• Instances in the same region as the Cloud Router
if the VPC network is in regional dynamic routing mode
• All instances if the VPC network is in global dynamic routing mode
When you create a VPC network, GCP creates a system-generated default route. This route serves two purposes:
It defines the path out of the VPC network, including the path to the Internet. In addition to having this route, instances must meet additional requirements if they need Internet access.
It provides the standard path for Private Google Access.
The system-generated default route has a priority of
1000. Because its
destination is the broadest possible (
0.0.0.0/0), GCP will
only use it if a route with a more specific destination does not apply
to a packet. Refer to routing order for details about how
destination specificity and route priority are used to select a route.
You can delete the default route if you want to completely isolate your network from the Internet or if you need to replace it with a custom route:
You can replace the default route with a custom static or dynamic route if you want to route “Internet traffic” to a different next hop. For example, you could replace it with a custom static route whose next hop is a Cloud VPN tunnel or another instance, such as a proxy server.
If you remove the default route and do not replace it, packets destined to IP ranges not covered by other routes will be dropped.
Subnet routes are system-generated routes that define paths to each subnet in the VPC network.
Each subnet has at least one subnet route whose destination matches the primary IP range of the subnet. If the subnet has secondary IP ranges, GCP creates a subnet route with a corresponding destination for each secondary range. See networks and subnets for details about subnet IP ranges.
No other route can have a destination that matches or is more specific than the destination of a subnet route. You can create a custom route that has a broader destination range that contains the subnet route's destination range. In cases of IP range overlap: Because GCP uses destination specificity as the first criteria for routing order, the subnet route will always be the preferred next hop for packets whose destinations fit in its destination range. This is true even if another route whose destination “contains” the destination of the subnet route has a higher route priority.
The following points describe how subnet routes are created and removed:
When a subnet is created, a corresponding subnet route for the subnet's primary IP range is also created.
- If you add a secondary IP range to a subnet, a corresponding subnet route for that secondary range is created.
Auto mode VPC networks create a subnet route for the primary IP ranges of each of their automatically created subnets. You can delete these subnets only if you convert the auto mode network to custom mode. See types of VPC networks for additional details.
You cannot delete a subnet route unless you modify or delete the subnet itself:
When you remove a secondary range from a subnet, the subnet route for that secondary range is deleted automatically.
When you delete a subnet, all subnet routes for both primary and secondary ranges are deleted automatically. You cannot delete the subnet route for the subnet's primary range in any other way.
When networks are connected using VPC Network Peering, subnet routes from one network are imported into the other network, and vice versa. Thus, all primary and secondary subnet IP ranges must be unique.
- Subnet routes for subnets in peered networks cannot be removed unless you break the peering relationship. When you break the peering relationship, all imported subnet routes from the other network are automatically removed.
Firewall rules can block communication among instances. For details about instance-to-instance communication, refer to communication within the network.
Custom routes are either static routes you create manually or dynamic routes maintained automatically by one or more of your Cloud Routers.
Destinations for custom routes cannot match or be more specific than any subnet route in the network.
If you are using an auto mode VPC network, don't use
destinations that fall into the
10.128.0.0/9 CIDR block because that block
defines the current and future address space for subnet routes. Refer to the
auto mode IP ranges for more information. Static
routes with destinations inside
10.128.0.0/9 might be disabled at any time
in an auto mode VPC network. This can happen if a new
GCP region becomes available and a new subnet is created
automatically (along with its subnet route). Carefully review the
considerations for auto mode networks
for more information.
Static routes can use any of the static route next hops. You create static routes in one of two ways:
You can create them manually.
If you use the GCP Console to create a Cloud VPN tunnel that uses policy based routing or one that is a route based VPN, static routes for the remote traffic selectors are created for you. Refer to the Cloud VPN documentation for networks and tunnel routing for details.
See static route parameters for more information.
Dynamic routes are managed by one or more Cloud Routers. Their destinations always represent IP ranges outside of your VPC network, and their next hops are always BGP peer addresses. A Cloud Router can manage dynamic routes for:
Applicability and order
Routes apply to instances according to the following rules:
System-generated routes apply to all instances in a VPC network. The scope of instances to which subnet routes apply cannot be altered; however, you can replace the default route.
Custom static routes can apply to all instances or specific instances, depending on the tag attribute of the route. Static routes with a tag attribute are applicable to instances that have a corresponding network tag. If no tag attribute is specified, the static route applies to all instances in the network.
Dynamic routes apply to instances based on the dynamic routing mode of the VPC network. If the VPC network is in regional dynamic routing mode, all Cloud Routers apply routes they learn within their respective regions. If the network is in global dynamic routing mode, all Cloud Routers apply routes they learn in the whole network.
GCP uses the following procedure to select the next hop for a packet from the pool of applicable routes:
Subnet routes are considered first because GCP requires that subnet routes have the most specific destinations matching the IP address ranges of their respective subnets. Subnet routes are routes for the primary and secondary subnet IP address ranges of each subnet in your VPC network and the primary and secondary subnet IP address ranges of subnets in peered networks. If the destination for a packet fits inside the destination of a subnet route, it is delivered to that GCP subnet. You cannot override a subnet route with any other type of route.
GCP does not allow you to create a custom static route having an equal or more specific destination than any subnet route.
For custom dynamic routes, Cloud Router ignores any route to other networks received by a Cloud Router if the received route has an equal or more specific destination than any subnet route.
When you connect VPC networks using VPC Network Peering, the networks share all subnet routes. GCP prioritizes subnet routes in peer networks similar to the local network's own subnet routes: Subnet routes in peer networks must have the most specific destinations.
If the packet does not fit in the destination for a subnet route, GCP looks for a custom route with the most specific destination.
- Suppose the destination for a packet leaving a VM is
10.240.1.4and there are two routes with different destinations:
10.240.0.0/16. Because the most specific destination for
10.240.1.0/24, the route whose destination is
10.240.1.0/24defines the next hop for the packet.
- Suppose the destination for a packet leaving a VM is
If more than one custom route has the same most specific destination, GCP uses the following process to select a route from route candidates. Route candidates are custom routes that have the same most specific destination.
If your VPC network is connected to other networks through VPC Network Peering, GCP first narrows the route candidates that are all from a single VPC network:
If at least one route candidate is defined in your local VPC network, GCP uses the local route candidate and disregards all candidates from peer networks.
If the route candidates are from multiple peer networks, GCP selects one of the peer networks where at least one candidate route is defined and disregards candidate routes from other peer networks. GCP uses an internal process to select the single peer network. If you add or remove networks from the peering group, GCP might choose candidate routes from a different peer network.
GCP moves on to select a route from a single network's candidate routes.
If a single route with the highest priority is available, GCP sends the packet to its next hop.
GCP selects a route based on whether it's a static or dynamic route and the value of its next hop. GCP uses the following order:
GCP selects a custom static route whose next hop is Next Hop Instance, Next Hop IP, or Next Hop VPN Tunnel.
GCP selects a custom dynamic route that is being advertised by a Cloud Router.
GCP selects a custom static route with Next Hop Gateway.
If no single route can be selected, GCP distributes traffic among the next hops of the route candidates by using a five-tuple hash for affinity, implementing an ECMP routing design. The hash is calculated from the protocol, the source and destination IP addresses, and the source and destination ports of the packet being sent. This calculation is done for each packet sent based on the number of route candidates that are available. If the number of available route candidates changes, the has might direct the packet to a different next hop.
If no applicable destination is found, GCP drops the packet, replying with an ICMP destination or network unreachable error.
Static route parameters
Each static route consists of the following components:
Name and Description: These fields identify the route. A name is required, but a description is optional. Every route in your project must have a unique name.
Network: Each route must be associated with exactly one VPC network.
Destination range: The destination range is a single IPv4 CIDR block containing the IP addresses of systems that receive incoming packets. GCP does not support IPv6 destination ranges. Destinations must be expressed in CIDR notation, and the broadest destination possible is
Priority: Priority is used to determine which route should be used if multiple routes have identical destinations. Lower numbers indicate higher priorities; for example, a route with a priority value of
100has higher priority than one with a priority value of
Next hop: Static routes can have next hops that point to the default Internet gateway, a GCP instance, or a Cloud VPN tunnel. Refer to static route next hops for more information.
Tags: You can specify a list of network tags so that the route will only apply to instances that have at least one of the listed tags. If you don't specify tags, GCP applies the route to all instances in the network.
Static route next hops
The following are valid next hops for static routes. Refer to the
reference documentation for
more details about each type.
Next Hop Gateway (
next-hop-gateway): You can specify a default Internet gateway to define a path to public IP addresses.
Next Hop Instance (
next-hop-instance): You can direct traffic to an existing instance in GCP by specifying the name and zone of the instance. Traffic is directed to the primary internal IP address in the network.
If the internal IP address for the instance's primary network interface changes, GCP will update routing tables automatically so that traffic continues to be sent to that instance at its new IP address.
If the instance is replaced by a new instance with the same name in the same zone, GCP will update routing tables automatically so that traffic is directed to the replacement instance. It does not matter whether the instance was replaced automatically or manually.
Next Hop IP (
next-hop-address): You can reference an existing instance in GCP using the internal IP address of its primary network interface.
If the instance is replaced by a new instance, the replacement must use the same internal IP address. GCP will direct traffic to whatever instance currently has the specified internal IP address for the next hop.
The next hop IP address must be an existing instance in your VPC network. Public IP addresses and private IP addresses in networks connected to your VPC network are not valid next hops.
Next Hop VPN Tunnel (
next-hop-vpn-tunnel): For Cloud VPN tunnels using policy based routing and route based VPNs, you can direct traffic to the VPN tunnel by creating routes whose next hops refer to the tunnel by its name and region.
Instances as next hops
When creating a static routes whose next hop is an instance, either by virtue of using next hop instance or next hop IP, the instance acting as the next hop must be configured to receive incoming traffic from other instances:
Instances acting as next hops must be configured to allow IP forwarding. You can enable IP forwarding on a per-VM basis only when you create the VM. If you need to enable IP forwarding for VMs created automatically as part of a managed instance group, you must enable IP forwarding on the instance template used by the instance group.
Instances acting as next hops must have appropriately configured firewall rules. Configuring a route and specifying an instance as a next hop does not adjust firewall rules automatically. The implied deny ingress rule blocks incoming traffic unless you have firewall rules created to allow it. See the firewall rules overview for details.
Firewall rules apply to the sources and destinations of the packet, not the next hop instance. A next hop instance should be configured to leave packet sources and destinations unchanged. For example, if the source for an ingress firewall rule includes the IP range
10.240.0.3/32, packets with that source routed through a properly configured NAT instance would be allowed by the firewall rule even though the NAT instance's IP address is not
When you use an instance as a next hop, remember that the instance is a zonal resource. Selecting a zone implies you have selected a region. GCP does not consider regional distance for routes, so it is possible to create a route that sends traffic to a next hop instance in a different region. This will add egress costs and introduce network latency.
GCP considers a route whose next hop is an instance to be valid as long as the instance is running. If software within the instance, such as its operating system or the process responsible for routing packets, hangs or crashes, packets will be dropped. Consider using managed instance groups and autohealing to instruct GCP to re-create next hop instances when they fail configurable health checks.
Disabling a network interface by configuring the guest operating system of the instance does not instruct GCP to cease considering it as a route's next hop.