Google Cloud Platform (GCP) routes define the paths network traffic takes from a VM instance to other destinations. These destinations can be inside your VPC network (for example, in another VM) or outside of it.
In a GCP VPC network, a route consists of a single destination (CIDR) and a single next hop. When an instance in a VPC network sends a packet, GCP delivers the packet to the route's next hop if the packet's destination address is within the route's destination range. This page provides an overview of how routes work in GCP.
Routing in GCP
Every VPC network uses a scalable, distributed virtual routing mechanism. Even though some routes can be applied selectively, the routing table for a VPC network is defined at the VPC network level.
Each VM instance has a controller that is kept informed of all applicable routes from the network's routing table. Each packet leaving a VM is delivered to the appropriate next hop of an applicable route based on a routing order. When you add or delete a route, the set of changes is propagated to the VM controllers using an eventually consistent design.
VPC networks can include system-generated routes or custom routes.
The following table summarizes system-generated routes, which are added or updated automatically when you create a VPC network, add a subnet, expand a subnet's primary IP range, or edit a subnet's secondary IP range. They apply to all instances in the VPC network.
|subnet route||Primary and secondary
subnet IP ranges
||VPC network, which forwards packets to VMs in its subnets||Only when the subnet is deleted or when you change the subnet's secondary IP ranges|
The following table summarizes custom routes, which you create and maintain directly or by using Cloud Router. For more information, see Custom routes.
|Type||Destination||Next Hop||Removable||Applies To|
• IP range that does not partially or exactly overlap with any subnet IP
• IP range broader than a subnet IP range
• Instance by name
• Instance by IP address
• Cloud VPN tunnel
• All instances in network
• Specific instances in the network, identified by network tag, if the route can be scoped by network tag
• IP range that does not partially or exactly overlap with any subnet IP
• IP range broader than a subnet IP range
|IP address of the Cloud Router's BGP peer||Only by a Cloud Router if it no longer receives the route from its BGP peer||• Instances in the same region as the Cloud Router
if the VPC network is in regional dynamic routing mode
• All instances if the VPC network is in global dynamic routing mode
When you create a VPC network, GCP creates a system-generated default route. This route serves two purposes:
It defines the path out of the VPC network, including the path to the Internet. In addition to having this route, instances must meet additional requirements if they need Internet access.
It provides the standard path for Private Google Access.
The system-generated default route has a priority of
1000. Because its
destination is the broadest possible (
0.0.0.0/0), GCP will
only use it if a route with a more specific destination does not apply
to a packet. Refer to routing order for details about how
destination specificity and route priority are used to select a route.
You can delete the default route if you want to completely isolate your network from the Internet or if you need to replace it with a custom route:
You can replace the default route with a custom static or dynamic route if you want to route “Internet traffic” to a different next hop. For example, you could replace it with a custom static route whose next hop is a Cloud VPN tunnel or another instance, such as a proxy server.
If you remove the default route and do not replace it, packets destined to IP ranges not covered by other routes will be dropped.
Subnet routes are system-generated routes that define paths to each subnet in the VPC network.
Each subnet has at least one subnet route whose destination matches the primary IP range of the subnet. If the subnet has secondary IP ranges, GCP creates a subnet route with a corresponding destination for each secondary range. See networks and subnets for details about subnet IP ranges.
No other route can have a destination that matches or is more specific than the destination of a subnet route. You can create a custom route that has a broader destination range that contains the subnet route's destination range. In cases of IP range overlap: Because GCP uses destination specificity as the first criteria for routing order, the subnet route will always be the preferred next hop for packets whose destinations fit in its destination range. This is true even if another route whose destination “contains” the destination of the subnet route has a higher route priority.
The following points describe how subnet routes are created and removed:
When a subnet is created, a corresponding subnet route for the subnet's primary IP range is also created.
- If you add a secondary IP range to a subnet, a corresponding subnet route for that secondary range is created.
Auto mode VPC networks create a subnet route for the primary IP ranges of each of their automatically created subnets. You can delete these subnets only if you convert the auto mode network to custom mode. See types of VPC networks for additional details.
You cannot delete a subnet route unless you modify or delete the subnet itself:
When you remove a secondary range from a subnet, the subnet route for that secondary range is deleted automatically.
When you delete a subnet, all subnet routes for both primary and secondary ranges are deleted automatically. You cannot delete the subnet route for the subnet's primary range in any other way.
When networks are connected using VPC Network Peering, subnet routes from one network are imported into the other network, and vice versa. Thus, all primary and secondary subnet IP ranges must be unique.
- Subnet routes for subnets in peered networks cannot be removed unless you break the peering relationship. When you break the peering relationship, all imported subnet routes from the other network are automatically removed.
Firewall rules can block communication among instances. For details about instance-to-instance communication, refer to communication within the network.
Custom routes are either static routes you create manually or dynamic routes maintained automatically by one or more of your Cloud Routers.
Destinations for custom routes cannot match or be more specific than any subnet route in the network.
If you are using an auto mode VPC network, don't use
destinations that fall into the
10.128.0.0/9 CIDR block because that block
defines the current and future address space for subnet routes. Refer to the
auto mode IP ranges for more information. Static
routes with destinations inside
10.128.0.0/9 might be disabled at any time
in an auto mode VPC network. This can happen if a new
GCP region becomes available and a new subnet is created
automatically (along with its subnet route). Carefully review the
considerations for auto mode networks
for more information.
Static routes can use any of the static route next hops. You create static routes in one of two ways:
You can create them manually.
If you use the GCP Console to create a Cloud VPN tunnel that uses policy based routing or one that is a route based VPN, static routes for the remote traffic selectors are created for you. Refer to the Cloud VPN documentation for networks and tunnel routing for details.
See static route parameters for more information.
Dynamic routes are managed by one or more Cloud Routers. Their destinations always represent IP ranges outside of your VPC network, and their next hops are always BGP peer addresses. A Cloud Router can manage dynamic routes for:
Applicability and order
Each instance has a set of applicable routes, which are a subset of all routes in the VPC network. Applicable routes are the possible egress paths that a packet can take when sent from the instance.
System-generated routes apply to all instances in a VPC network. The scope of instances to which subnet routes apply cannot be altered; however, you can replace the default route.
Custom static routes can apply to all instances or specific instances. Static routes with a tag attribute apply to instances that have a corresponding network tag. If the route doesn't have a network tag, the route applies to all instances in the network.
When a custom static route has a VM instance next hop, the route is always valid, even if the next-hop VM is deleted, powered off, or malfunctioning. Refer to instances as next hops for more information.
When a custom static route has a Cloud VPN tunnel next hop, the route is always valid if the tunnel is up. Refer to when tunnels are down in the Cloud VPN documentation for how GCP handles routes for tunnels when they are down.
Dynamic routes apply to instances based on the dynamic routing mode of the VPC network. If the VPC network is in regional dynamic routing mode, all Cloud Routers apply routes they learn within their respective regions. If the network is in global dynamic routing mode, all Cloud Routers apply routes they learn in the whole network.
- Cloud Router automatically discards learned custom dynamic routes that correspond to inaccessible next hops (Cloud VPN tunnels using dynamic routing or Cloud Interconnect). Depending on your network, Cloud Router can take up to 40 seconds of processing time to remove a dynamic route with a next hop that's down.
For some load-balanced traffic, the applicable routes originate outside of your VPC network. Refer to load balancer return paths for an explanation.
When an instance sends a packet, GCP attempts to select one route from the set of applicable routes according to the following routing order.
Subnet routes are considered first because GCP requires that subnet routes have the most specific destinations matching the IP address ranges of their respective subnets. Subnet routes are routes for the primary and secondary subnet IP address ranges of each subnet in your VPC network and the primary and secondary subnet IP address ranges of subnets in peered networks. If the destination for a packet fits inside the destination of a subnet route, it is delivered to that GCP subnet. You cannot override a subnet route with any other type of route.
GCP does not allow you to create a custom static route having an equal or more specific destination than any subnet route.
For custom dynamic routes, Cloud Router ignores any route to other networks received by a Cloud Router if the received route has an equal or more specific destination than any subnet route.
When you connect VPC networks using VPC Network Peering, the networks share all subnet routes. GCP prioritizes subnet routes in peer networks similar to the local network's own subnet routes: Subnet routes in peer networks must have the most specific destinations.
If the packet does not fit in the destination for a subnet route, GCP looks for a custom route with the most specific destination.
- Suppose the destination for a packet leaving a VM is
10.240.1.4and there are two routes with different destinations:
10.240.0.0/16. Because the most specific destination for
10.240.1.0/24, the route whose destination is
10.240.1.0/24defines the next hop for the packet.
- Suppose the destination for a packet leaving a VM is
If more than one custom route has the same most specific destination, GCP uses the following process to select a route from route candidates. Route candidates are custom routes (static or dynamic routes) that have the same most specific destination.
If your VPC network is connected to other networks through VPC Network Peering, GCP first narrows the route candidates that are all from a single VPC network:
If at least one route candidate is defined in your local VPC network, GCP uses the local route candidate and disregards all candidates from peer networks.
If the route candidates are from multiple peer networks, GCP selects one of the peer networks where at least one candidate route is defined and disregards candidate routes from other peer networks. GCP uses an internal process to select the single peer network. If you add or remove networks from the peering group, GCP might choose candidate routes from a different peer network.
GCP moves on to select a route from a single network's candidate routes.
If a route with the highest priority is available, GCP sends the packet to its next hop.
If multiple routes have the highest priority, GCP selects a route based on whether it's a static or dynamic route and the value of its next hop. GCP uses the following order:
GCP selects a custom static route whose next hop is Next Hop Instance, Next Hop IP, or Next Hop VPN Tunnel.
GCP selects a custom dynamic route that is being advertised by a Cloud Router.
GCP selects a custom static route with Next Hop Default Internet Gateway.
If no single route can be selected, GCP distributes traffic among the next hops of the route candidates by using a five-tuple hash for affinity, implementing an ECMP routing design. The hash is calculated from the protocol, the source and destination IP addresses, and the source and destination ports of the packet being sent. This calculation is done for each packet sent based on the number of route candidates that are available. If the number of available route candidates changes, the hash might direct the packet to a different next hop.
If no applicable destination is found, GCP drops the packet, replying with an ICMP destination or network unreachable error.
Load balancer return paths
If you use one of the following GCP load balancers, GCP has special routes for the load balancers and their associated health checks, which are described in more detail in the following sections. These routes are defined outside of your VPC network, in Google's production network. They don't appear in your VPC network's routing table. You cannot remove or override them, even if you delete or replace a default route in your VPC network.
Return paths for HTTP(S), SSL proxy, and TCP proxy load balancers
For these load balancer types, Google Front End (GFE) systems serve as proxies. When a client sends a request to the load balancer, a proxy terminates the TCP session and opens a new TCP session with your backend VM. Routes defined outside of your VPC network facilitate communication from GFE proxies to your backend VMs and from your backend VMs to GFE proxies.
For more information, refer to the following pages:
Return paths for network load balancers
When a client on the Internet sends a TCP or UDP request through a network load balancer to a backend VM, GCP uses routes defined outside of your VPC network to facilitate communication from the client to your backend VM and from your backend VM back to the client.
For more information, refer to Network load balancer.
Health checks for all load balancer types
Packets sent from GCP health check probe systems have sources as described in Probe IP ranges. Routes that facilitate communication between GCP health check probe systems and your backend VMs exist outside of your VPC network, and cannot be removed. However, your VPC network must have ingress allow firewall rules to permit traffic from these systems.
Static route parameters
Each static route consists of the following components:
Name and Description: These fields identify the route. A name is required, but a description is optional. Every route in your project must have a unique name.
Network: Each route must be associated with exactly one VPC network.
Destination range: The destination range is a single IPv4 CIDR block containing the IP addresses of systems that receive incoming packets. GCP does not support IPv6 destination ranges. Destinations must be expressed in CIDR notation, and the broadest destination possible is
Priority: Priority is used to determine which route should be used if multiple routes have identical destinations. Lower numbers indicate higher priorities; for example, a route with a priority value of
100has higher priority than one with a priority value of
Next hop: Static routes can have next hops that point to the default Internet gateway, a GCP instance, or a Cloud VPN tunnel. Refer to static route next hops for more information.
Tags: You can specify a list of network tags so that the route will only apply to instances that have at least one of the listed tags. If you don't specify tags, GCP applies the route to all instances in the network.
Static route next hops
The following are valid next hops for static routes. Refer to the
reference documentation for
more details about each type.
Next Hop Gateway (
next-hop-gateway): You can specify a default Internet gateway to define a path to public IP addresses.
Next Hop Instance (
next-hop-instance): You can direct traffic to an existing instance in GCP by specifying its name and zone. Traffic is directed to the primary internal IP address of the VM's network interface in the VPC network where you define the route.
If the primary internal IP address for the VM instance's network interface in the VPC network changes, GCP will update the VPC network's routing table automatically so that traffic continues to be sent to the VM at its new IP address.
If the VM is replaced by a new VM with the same name in the same zone, GCP will update the VPC network's routing table automatically so that traffic is directed to the replacement VM.
GCP only validates that a next hop VM exists when you create the route. It does not validate that the VM is able to process or forward packets. If the VM is deleted but not replaced, the route still applies, which might result in dropped packets. Refer to instances as next hops for more information.
Next Hop Internal TCP/UDP Load Balancer (
next-hop-ilb): For Internal TCP/UDP Load Balancing, you can use a load balancer's IP address as a next hop that distributes traffic among healthy backend instances. For example, you can use a custom static route whose next hop is an internal TCP/UDP load balancer to distribute traffic among backend VMs. Custom static routes that use this next hop cannot be scoped to specific instances by network tag.
Next Hop IP (
next-hop-address): You can provide an IP address as a next hop if that address fits within either the primary or secondary IP range of an existing subnet in your VPC network. You cannot use a public IP address or an internal IP address in an on-premise network.
When you use next hop address, GCP passes traffic to any VM instance assigned to that IP address. If you replace a VM instance with another one that uses the same IP address, traffic goes to the replacement, even if it has a different name. To define a next hop by VM name, use next hop instance instead.
GCP only checks that the IP address is a valid member of a subnet's primary or secondary IP range when you create the route. It does not validate that an instance exists at the next hop IP address. If the next hop IP address is not assigned to any instance, the route still applies, which might result in dropped packets. Refer to instances as next hops for more information.
Next Hop VPN Tunnel (
next-hop-vpn-tunnel): For Cloud VPN tunnels using policy based routing and route based VPNs, you can direct traffic to the VPN tunnel by creating routes whose next hops refer to the tunnel by its name and region. GCP ignores routes whose next hops are Cloud VPN tunnels that are down. For more examples about how routes and Cloud VPN tunnels interact, refer to the Cloud VPN routing examples in the Cloud VPN documentation.
Considerations for instance-based routing
Instance-based routing refers to a static route with a next hop pointing
to an internal TCP/UDP load balancer or an instance (
When you configure instance-based routing, consider the following guidelines:
You must configure the backend VMs or the next-hop instance to forward packets from other sources (
can-ip-forward). You can enable IP forwarding on a per-VM basis when you create the VM. To enable IP forwarding for VMs created automatically as part of a managed instance group, you must enable IP forwarding in the instance template used by the instance group. You must make this configuration change in addition to any operating system configuration necessary to route packets.
Software running on the backend VM or next-hop instance must be configured appropriately. For example, third-party appliance VMs that act as routers or firewalls must be configured according to the manufacturer's instructions.
Backend VMs or next-hop instances must have appropriate firewall rules. You must configure firewall rules that apply to the packets being routed. Keep the following in mind:
- Ingress firewall rules applicable to instances that perform routing functions must include the IP addresses of routed packet sources. The implied deny ingress rule blocks all incoming packets, so you must at least create custom ingress allow firewall rules.
- Egress firewall rules applicable to instances that perform routing functions must include the IP addresses of routed packet destinations. The implied allow egress rule will permit this unless you've created a specific egress deny rule to override it.
- Take into account whether or not the backend VM or next hop instance is performing Network Address Translation (NAT) when creating your firewall rules.
Refer to the Firewall rules overview for additional guidance.
Considerations specific to next-hop instances
When you use an instance as a next hop (
next-hop-address), remember that the instance is a zonal resource. Selecting a zone implies you have selected a region. GCP does not consider regional distance for routes that use a next-hop instance, so it is possible to create a route that sends traffic to a next hop in a different region. This adds egress costs and introduces network latency.
GCP performs one of the following basic validations only when you create the route:
- If you specify next-hop instance, the instance must already exist.
- If you specify next-hop address, the IP address must be in an existing subnet's primary or secondary IP range.
If you, for example, remove an instance or delete a subnet later, GCP still considers any route that uses that instance as a next hop when it evaluates applicable routes. This might cause some or all packets for a given destination to be dropped, depending on the other routes in your network.
If software in a VM instance (such as the VM's operating system or the VM's process that routes packets) fails, packets destined for that instance are dropped. To enhance reliability, consider using an internal TCP/UDP load balancer as a next hop instead. An internal TCP/UDP load balancer requires a configured health check, and this health check determines how new connections are routed.
Disabling a network interface by configuring the guest operating system of the instance does not cause GCP to stop considering the interface to be a route's next hop.
For more information about internal TCP/UDP load balancer next hops, refer to using an internal TCP/UDP load balancer as a next hop for a route.