Google Cloud routes define the paths that network traffic takes from a virtual machine (VM) instance to other destinations. These destinations can be inside your Google Cloud Virtual Private Cloud (VPC) network (for example, in another VM) or outside it.
In a VPC network, a route consists of a single destination prefix in CIDR format and a single next hop. When an instance in a VPC network sends a packet, Google Cloud delivers the packet to the route's next hop if the packet's destination address is within the route's destination range.
This page provides an overview of how routes work in Google Cloud.
Routing in Google Cloud
Every VPC network uses a scalable, distributed virtual routing mechanism. There is no physical device that's assigned to the network. Some routes can be applied selectively, but the routing table for a VPC network is defined at the VPC network level.
Each VM instance has a controller that is kept informed of all applicable routes from the network's routing table. Each packet leaving a VM is delivered to the appropriate next hop of an applicable route based on a routing order. When you add or delete a route, the set of changes is propagated to the VM controllers by using an eventually consistent design.
The following tables summarize how Google Cloud categorizes routes in VPC networks.
|Type and destination||Next hop||Notes|
|System-generated default routes
||Applies to the whole VPC network
Can be removed or replaced
Created automatically for each subnet IP address range
Forwards packets to VMs and internal load balancers
|Applies to the whole VPC network
Created, updated, and removed automatically by Google Cloud when you create, modify, or delete a subnet or secondary IP address range of a subnet.
Supports various destinations
|Forwards packets to a static route next hop||For details about each static route next hop, see considerations for:|
Destinations that don't conflict with subnet routes or static routes
|Peer of a BGP session on a Cloud Router||Routes are added and removed automatically based on
from Cloud Routers in your VPC network.
Routes apply to VMs according to the VPC network's dynamic routing mode.
|Peering subnet route
Represents a subnet IP address range in a network connected using VPC Network Peering
|Peer VPC network
Forwards packets to VMs and internal load balancers in the peer network
|Applies to the whole VPC network
Created, updated, and removed automatically by Google Cloud when subnets are created, modified, or deleted in the peer network.
|Peering custom route
Custom static or custom dynamic route in a network connected using VPC Network Peering
|Next hop in the peer VPC network||Peering custom routes backed by static routes in the peer network apply
Matches traffic by a combination of protocol, source IP range, and destination IP range.
|An internal TCP/UDP load balancer||
Policy-based routes are evaluated before other routes are evaluated. Policy-based routes can apply to all VMs in the network, to certain VMs selected by network tag, or to traffic entering the VPC network by way of VLAN attachments that you identify.
Policy-based routes are not exchanged through VPC Network Peering.
System-generated default routes
When you create a VPC
network, it includes a
system-generated IPv4 default
0.0.0.0/0). When you create a dual-stack
subnet with an external
IPv6 address range in a VPC network, a system-generated IPv6
default route (
::/0) is added to that network if the route doesn't already
exist. Default routes serve these purposes:
The IPv4 and IPv6 default routes defines a path out of the VPC network to external IP addresses on the internet.
If you access Google APIs and services without using a Private Service Connect endpoint, the default route can serve as the path to Google APIs and services. For more information, see Configure Private Google Access and Access APIs from VMs with external IP addresses.
Google Cloud only uses a default route if a route with a more specific destination does not apply to a packet. For information about how destination specificity and route priority are used to select a route, see routing order.
If you want to completely isolate your network from the internet or if you need to replace the default route with a custom route, you can delete the default route:
IPv4 only: If you want to route internet traffic to a different next hop, you can replace the default route with a custom static or dynamic route. For example, you could replace it with a custom static route whose next hop is a proxy VM.
IPv4 and IPv6: If you delete the default route and do not replace it, packets destined to IP ranges that are not covered by other routes are dropped.
Subnet routes define paths to resources like VMs and internal load balancers in a VPC network.
Each subnet has at least one subnet route whose destination matches the primary IPv4 range of the subnet. If the subnet has secondary IP ranges, there's a corresponding subnet route for each of its secondary IP address ranges. If the subnet has an IPv6 range, there's a corresponding subnet route for the IPv6 address range. For more information about subnet IP ranges, see Subnets.
Subnet routes always have the most specific destinations. They cannot be
overridden by other routes, even if another route has a higher priority. This is
because Google Cloud considers destination specificity before priority
when selecting a route. Google Cloud uses
0 as the priority for all
Interactions with custom static routes
Local subnet routes and imported peering subnet routes interact with custom static routes in the following ways:
Google Cloud does not allow you to create a custom static route that has an equal or narrower destination than any subnet route or peering subnet route. For example, if a local subnet route or peering subnet route exists for the
10.70.1.0/24destination, you cannot create a new custom static route for
10.70.1.128/25, or any other destinations fit within
Conversely, Google Cloud does not allow you to create a new subnet or peering subnet route whose destination exactly matches or is broader than (would contain) an existing custom static route. For example, if your VPC network has a custom static route for the
10.70.1.128/25destination, Google Cloud prohibits the creation of any subnet or peering subnet route with a primary or secondary subnet IP address range of
10.70.1.0/24, or any other range that contains all the IP addresses in
Interactions with custom dynamic routes
Local subnet routes and imported peering subnet routes interact with Cloud Routers in the following ways:
When Cloud Routers learn a prefix that exactly matches the destination of an existing subnet or peering subnet route, Google Cloud does not create any custom dynamic route for the conflicting prefix. For example, when a subnet or peering subnet route with destination
10.70.1.0/24exists, if Cloud Routers in the VPC network receive the
10.70.1.0/24prefix through BGP, Google Cloud uses the subnet or peering subnet route, and does not create any custom dynamic route for
- When a subnet or peering subnet route whose destination exactly matches a prefix learned by Cloud Routers in the VPC network is created, Google Cloud removes the custom dynamic route for the conflicting prefix in order to make room for the subnet or peering subnet route.
When Cloud Routers learn prefixes that fit within the destination of an existing subnet or peering subnet route, Google Cloud does not create any custom dynamic routes for the conflicting prefixes. For example, when a subnet or peering subnet route with destination
10.70.1.0/24exists, if Cloud Routers in the VPC network receive the
10.70.1.128/25prefixes through BGP, Google Cloud uses the subnet or peering subnet route, and does not create any custom dynamic routes for
- When a subnet or peering subnet route whose destination contains prefixes learned by Cloud Routers in the VPC network is created, Google Cloud removes the custom dynamic routes for the conflicting prefixes in order to make room for the subnet or peering subnet route.
Lifecycle of subnet routes
Subnet routes are created, updated, and deleted when you create, modify, or delete subnets or their subnet IP address ranges:
When you add a subnet, Google Cloud creates a corresponding subnet route for the subnet's primary IP address range.
If you expand a subnet's primary IP address range, Google Cloud updates the corresponding subnet route's destination.
If you add or remove a secondary IP address range in a subnet, Google Cloud creates or destroys a corresponding subnet route for the secondary range.
Auto mode VPC networks create a subnet route for the primary IP ranges of each of their automatically created subnets. You can delete these subnets only if you convert the auto mode VPC network to custom mode.
You cannot delete a subnet route unless you modify or delete the subnet:
When you remove a secondary range from a subnet, the subnet route for that secondary range is deleted automatically.
When you delete a subnet, all subnet routes for both primary and secondary ranges are deleted automatically. You cannot delete the subnet route for the subnet's primary range in any other way.
The number of subnet routes in a VPC network is limited by the Maximum number of subnet IP ranges (primary and secondary).
Static routes are defined using static route parameters and support static route next hops. You can create static routes in one of two ways:
You can create static routes manually using the Google Cloud console,
gcloud compute routes create, or the
If you used the Google Cloud console to create a Classic VPN tunnel that uses policy-based routing or one that is a route-based VPN, static routes for the remote traffic selectors were created for you. For more information, see Cloud VPN networks and tunnel routing.
Dynamic routes are automatically created and deleted based on learned routes from Cloud Routers in the VPC network. The destinations of these routes always represent IP prefixes outside your VPC network. Learned routes might be ones that were learned from a Border Gateway Protocol (BGP) peer. They could also be custom learned routes that you created manually.
Dynamic routes are used by the following:
Classic VPN tunnels that use dynamic routing
Third-party network virtual appliances that were installed as router appliance instances in Network Connectivity Center
VPC networks ignore any destinations received by Cloud Routers when the destinations match either of these criteria:
The destination exactly matches a subnet IP address range.
The destination fits within (has a longer subnet mask than) a subnet IP address range.
However, a dynamic route's destination can contain
(can have a smaller subnet mask than) a subnet's IP range.
For example, if you have a subnet IP range of
you can have a dynamic route with a destination of
If the destination IP address of a packet does not fit within the
subnet route's destination, the packet is sent to the dynamic route's
next hop. The broadest possible destination is
Peering subnet routes
Peering subnet routes define paths to resources using subnets in another VPC network connected using VPC Network Peering. The other network is called a peer VPC network.
Peer networks share subnet routes in the following way:
Peer networks always export their subnet routes, for all primary and secondary IP address ranges.
Peer networks automatically import subnet routes as long as the route's destination (subnet IP address range) is not a privately used public IP address. You can configure a peer to import the subnet routes that use privately used public IP addresses if necessary.
Imported peering subnet routes cannot be removed unless you disable the peering. When you disable the peering, all imported peering subnet routes are automatically removed.
Google Cloud prevents subnet IP address ranges conflicts between peered networks.
The combined number of subnet routes in a VPC network and peering subnet routes imported from all peer networks is limited by the Maximum number of subnet IP ranges (primary and secondary).
Peering custom routes
You can configure peer networks to export or import custom routes. In this context, custom routes means both static and dynamic routes in a VPC network.
Even when a VPC network is configured to export its custom routes, it omits these types of routes:
- Static routes that use the
next-hop-gateway(default internet gateway next hop)
- Static routes that are scoped to instances by network tag.
The combined number of static and dynamic routes in a VPC network plus peering custom routes imported from all peer networks is limited by:
- Maximum number of static routes in a peering group
- Maximum number of dynamic routes in a peering group
Policy-based routes let you route packets to an internal TCP/UDP load balancer next hop. You can select the traffic that the route applies to based on criteria such as a packet's source IP address, destination IP address, or protocol. For more information, see policy-based routes.
Applicability and order
Each instance has a set of applicable routes, which are a subset of all routes in the VPC network. Applicable routes are the possible egress paths that a packet can take when sent from the instance.
Special return paths apply when routing back to proxy load balancers, health check systems, and other Google services. For more information, see load balancer return paths. These routes are not shown in any route table.
Policy-based routes can apply to instances, VLAN attachments for Cloud Interconnect, and Cloud VPN gateways in a VPC network. Policy-based routes only apply if a packet matches the criteria of the policy-based route.
Subnet and peering subnet routes apply to all instances. Subnet routes and peering subnet routes correspond to subnets in your VPC network and those imported from peer networks.
The system-generated default route applies to all instances. You can replace the system-generated default route with a custom static route if you prefer.
Custom static routes can apply to all instances or specific instances. Static routes with a tag attribute apply to instances that have that same network tag. If the route doesn't have a network tag, the route applies to all instances in the network.
Dynamic routes apply to instances based on the dynamic routing mode of the VPC network. If a VPC network uses regional dynamic routing mode, all of its Cloud Routers only create dynamic routes for learned prefixes in their local region. If a VPC network uses global dynamic routing mode, all of its Cloud Routers create dynamic routes for learned prefixes in all regions.
Cloud Routers automatically discard learned prefixes that correspond to inaccessible next hops (Cloud VPN tunnels or Cloud Interconnect attachments). Depending on your network, Cloud Routers can take up to 40 seconds of processing time to remove a dynamic route with a next hop that's down.
The following process models the VPC network route selection behavior. Starting with the set of applicable routes (as described in the previous section), you can model the route selection decisions that Google Cloud makes for a packet by following this process.
Special return paths: Special return paths are not shown in your VPC network's route table. Routes that you configure in a VPC network do not apply to response packets for certain Google Cloud load balancers, health check systems, Identity-Aware Proxy (IAP), and Cloud DNS proxies. For details, see Special return paths.
If a special return path is applicable, your route selection model contains only the special return path. All other routes are disregarded, and evaluation stops at this step.
Policy-based routes: Policy-based routes are evaluated after special return paths and before any other type of route. If a packet matches the criteria for at least one policy-based route, Google Cloud narrows your route selection model so that the model includes only the policy-based routes that match the largest number of packet characteristics—the most specific for the packet. For example, a policy-based route that matches a packet's source and destination is selected instead of a policy-based route that only matches a packet's destination.
Google Cloud delivers packets according to route selection models in the following ways:
- If a route selection model has been narrowed to a single policy-based route, Google Cloud delivers the packet to the next hop of that policy-based route.
- If a route selection model contains two or more policy-based routes,
Google Cloud distributes traffic as follows:
- If the policy-based routes have unique priorities, Google Cloud uses the next hop internal TCP/UDP load balancer of the policy-based route that has the highest priority.
- If the policy-based routes have equal priorities, Google Cloud still selects just one next hop internal TCP/UDP load balancer. Google Cloud uses a deterministic, internal algorithm to select a single next hop load balancer, ignoring other policy-based routes with the same priority.
Most specific destination: Google Cloud determines which of the applicable routes have the most specific destination that contains the destination IP address of the packet. Google Cloud disregards all other routes with less specific destinations. For example,
10.240.1.0/24is a more specific destination than
10.240.0.0/16. The most specific destination could be any type of route; however, subnet routes, peering subnet routes, and special return paths are the most specific by definition.
After this step, your route selection model contains no special return paths or policy-based routes. It includes only the routes with the most specific destinations.
Next hops in a single VPC network: This step is only applicable if the VPC network whose route behavior you are modeling is connected to one or more VPC networks using VPC Network Peering. With VPC Network Peering, custom routes with identical destinations might exist in more than one of the networks in the peering group. The requirement modeled at this step is that Google Cloud selects next hops that are all in a single VPC network.
If one or more of the routes in your route model have next hops inside the VPC network you're modeling, disregard all routes which have next hops in peer networks. In this situation, Google Cloud only uses next hops in the local VPC network, even if next hops for the same destination exist in one or more peer VPC networks.
If none of the routes in your route model have next hops inside the VPC network you're modeling, and all next hops exist in multiple peer networks, Google Cloud uses an internal algorithm to select a single peer network with the next hops for the most specific destinations. Route priority is not considered at this point. Further, if your VPC network peers with a new VPC network or if it disconnects from an existing peer VPC network, the VPC network selected for the next hops might change.
After this step, your route selection model contains only the routes with the most specific destinations, and the next hops for these routes are all in a single VPC network.
Disregard custom static routes whose next hops are down: This step models two situations where Google Cloud disregards next hops that it considers to be down. This step only applies to custom static routes. Custom dynamic routes are automatically added and removed using BGP advertisements instead.
Google Cloud ignores every next hop VM instance (
next-hop-address) if the next hop VM has been stopped or deleted. For more details, see Behavior when instances are stopped or deleted in the Considerations for next hop instances section. If your route model contains static routes whose next hop VMs are stopped or deleted, remove those routes from your model.
Google Cloud ignores every custom static route that uses a next hop Classic VPN tunnel if the tunnel does not have a Phase 1 (IKE) security association (SA). For more details, see Order of routes in the Classic VPN documentation. If your route model contains static routes whose next hops are Classic VPN tunnels without established IKE SAs, remove those routes from your model.
Disregard low priority routes: This step models how Google Cloud discards all routes except for those with the highest priority.
After this step, your route selection model might be empty, or it might contain one or more routes. If your model does contain routes, all of the routes have all of these characteristics:
- Identical, most-specific destinations
- Next hops all in one VPC network—the local VPC network or a single peer VPC network
- Next hops which are not known to be down
- Identical, highest priorities
Select only the most favorable preference category: Google Cloud avoids ECMP routing among different types of next hops. You can model this behavior by considering the preference category system described in the following table. This step refines your route model so that it only contains routes which are of the same route type or combination of route type and next hop type.
Preference category Category and next hop combination First choice (highest preference) Custom static routes with a running next hop instance (
next-hop-address) or IKE SA established next hop Classic VPN tunnel
Second choice Custom dynamic routes learned from any BGP session of any Cloud Router Third choice A single custom static route with an internal TCP/UDP load balancer next hop
Google Cloud uses an internal algorithm to select a single next hop internal TCP/UDP load balancer, ignoring the other internal TCP/UDP load balancer next hops with the same priority. For more details, see "Same destination and multiple next hop internal TCP/UDP load balancers" in the Considerations for internal TCP/UDP load balancer next hops section.
Fourth choice A custom static route using the
At the end of this step, there could be zero routes, one route, or two or more routes in the route model.
Send or drop packet: What happens depends on the number of routes remaining in the route model:
If the route model is empty, the packet is dropped with an ICMP destination or network unreachable error. Google Cloud never falls back to a less specific route which might have a working next hop.
If the route model contains a single route, Google Cloud sends the packet to the next hop. For VM next hops, Google Cloud doesn't validate that the VM next hop can process packets. For details, see Considerations common to instance and internal TCP/UDP load balancer next hops. If the single route is a subnet route or a peering subnet route, and there's no Google Cloud resource at the packet's destination IP address, the packet is dropped.
If the route model contains two or more routes, the routes share the same most specific destination, are located in a single VPC network, have next hops which are not known to be down, have the same highest priority, and belong to one route type and next hop combination (preference category). Google Cloud distributes packets among the next hops implementing equal-cost multipath (ECMP) using a hashing algorithm. Hash calculations are done for each packet at the time it is sent, based on the current number of next hops. Google Cloud uses a five-tuple hash if the packet contains port information; otherwise, it uses a three-tuple hash. If the route model changes as subsequent packets are sent, Google Cloud might direct those packets to a different next hop even if the hash is the same.
Special return paths
VPC networks have special routes for certain services. These routes are defined outside of your VPC network, in Google's production network. They don't appear in your VPC network's routing table. You cannot remove or override them, even if you delete or replace a default route in your VPC network. However, you can control traffic to and from these services by using firewall rules.
Load balancer return paths
If you use one of the following Google Cloud load balancers, Google Cloud has special routes for the load balancers and their associated health checks, which are described in more detail in the following sections.
Return paths for external HTTP(S) load balancers, external SSL proxy load balancers, and external TCP proxy load balancers
For these load balancer types, Google Front End (GFE) systems serve as proxies. When a client sends a request to the load balancer, a proxy terminates the TCP session and opens a new TCP session with your backend VM. Routes defined outside your VPC network facilitate communication from GFE proxies to your backend VMs and from your backend VMs to GFE proxies.
For more information, see the following pages:
Return paths for network load balancers
When a client on the internet sends a TCP or UDP request through a network load balancer to a backend VM, Google Cloud uses routes defined outside your VPC network to facilitate communication from the client to your backend VM and from your backend VM to the client.
For more information, see Network load balancer overview.
Health checks for all load balancer types
Packets sent from Google Cloud health check probe systems have sources as described in Probe IP ranges and firewall rules. Routes that facilitate communication between Google Cloud health check probe systems and your backend VMs exist outside your VPC network, and cannot be removed. However, your VPC network must have ingress allow firewall rules to permit traffic from these systems.
A route for
126.96.36.199/20 is included to support TCP forwarding using
IAP. Google's production
network does not accept routes for
188.8.131.52/20 from other sources on the
A route to
184.108.40.206/19 supports connectivity to forwarding
targets that use private routing or
alternative name servers that use private routing.
Google's production network does not accept routes for
other sources on the internet.
Serverless VPC Access
A route for
220.127.116.11/19 supports Serverless VPC Access. Google's production network does not accept routes for
18.104.22.168/19 from other sources on the internet.
Static route parameters
Each static route consists of the following components:
Name and Description. These fields identify the route. A name is required, but a description is optional. Every route in your project must have a unique name.
Network. Each route must be associated with exactly one VPC network.
Destination range. The destination range is a single IPv4 CIDR block that contains the IP addresses of systems that receive incoming packets. Destinations must be expressed in CIDR notation. Destinations cannot exactly match a subnet IP address range, nor can they fit within a subnet IP address range (they cannot have a longer subnet mask). However, a static route's destination can contain (can have a smaller subnet mask than) a subnet's IP range. For example, if you have a subnet IP range of
10.10.10.0/24, you can have a static route with a destination of
10.10.10.0/23. If the destination IP address of a packet does not fit within the subnet route's destination, the packet is sent to the static route's next hop. The broadest possible destination is
Priority. If multiple routes have identical destinations, priority is used to determine which route should be used. Lower numbers indicate higher priorities; for example, a route with a priority value of
100has higher priority than one with a priority value of
200. Highest route priority means the smallest possible non-negative integer. Because zero is the smallest possible non-negative integer, it has the highest priority.
Next hop. Static routes can have next hops that point to the default internet gateway, a Google Cloud instance, or a Cloud VPN tunnel. For more information, see static route next hops.
Tags. You can specify a list of network tags so that the route only applies to instances that have at least one of the listed tags. If you don't specify tags, Google Cloud applies the route to all instances in the network.
Static route next hops
The following are valid next hops for static routes. For more information about
each type, see the
Next Hop Gateway (
next-hop-gateway). You can specify a default internet gateway to define a path to external IP addresses.
Next Hop Instance by name (
next-hop-instance) or address (
next-hop-address). You can direct traffic to an existing instance with a network interface in the same VPC network as the route. For details, see Considerations for next hop instances.
Next Hop internal TCP/UDP load balancer (
next-hop-ilb). You can direct traffic to multiple backend instances of an existing internal TCP/UDP load balancer. For details, see Considerations for internal TCP/UDP load balancer next hops.
Next Hop VPN Tunnel (
next-hop-vpn-tunnel). For Classic VPN tunnels that use policy-based routing and route-based VPNs, you can direct traffic to the VPN tunnel by creating routes whose next hops refer to the tunnel by its name and region. For details, see Considerations for Classic VPN tunnel next hops.
Considerations common to instance and internal TCP/UDP load balancer next hops
Instance-based routing refers to a static route with a next hop that is a VM
Internal TCP/UDP load balancer as a next hop refers to a static route with a next
hop that is an internal TCP/UDP load balancer (
When you configure instance-based routing or an internal TCP/UDP load balancer as a next hop, consider the following guidelines:
You must configure the backend VMs or the next hop instance to forward packets from any source IP address. To configure forwarding, enable IP forwarding (
can-ip-forward) on a per-VM basis when you create the VM. For VMs created automatically as part of a managed instance group, enable IP forwarding in the instance template used by the instance group. You must make this configuration change in addition to any operating system configuration necessary to route packets.
Software running on the backend VM or the next hop instance must be configured appropriately. For example, third-party appliance VMs that act as routers or firewalls must be configured according to the manufacturer's instructions.
Backend VMs or the next hop instance must have appropriate firewall rules. You must configure firewall rules that apply to the packets being routed. Keep the following in mind:
- Ingress firewall rules applicable to instances that perform routing functions must include the IP addresses of routed packet sources. The implied deny ingress rule blocks all incoming packets, so you must at least create custom ingress allow firewall rules.
- Egress firewall rules applicable to instances that perform routing functions must include the IP addresses of routed packet destinations. The implied allow egress rule permits this unless you've created a specific egress deny rule to override it.
- Take into account whether the backend VM or Next Hop Instance is performing Network Address Translation (NAT) when creating your firewall rules.
For more information, see Implied firewall rules.
From the perspective of a packet's destination, the source region of the packet is the region that contains the backend VM or the next hop instance. For example, packets processed by backend VMs or next hop instances in
us-west1can be sent to destinations that are only accessible in
us-west1even if the original packet source is outside of
us-west1. Examples of destination resources that are only accessible if the source and destination resources are in the same region include the following:
- Internal TCP/UDP load balancers with global access turned off
- Regional internal HTTP(S) load balancers
- Internal regional TCP proxy load balancers
Considerations for next hop instances
Next hop by instance name (
next-hop-instance): If both the route and the next hop instance are in the same project, you can specify a next hop instance using its name and zone. Before you can create the route, Google Cloud requires that a next hop instance must exist and meet all of the following criteria:
- The instance's name must match the VM name of the route's next hop.
- The instance's zone must match the zone of the route's next hop.
- The instance must have one network interface in the route's VPC network.
Google Cloud automatically updates a route whose next hop is specified by
next-hop-instancein these situations:
- When the IPv4 address of the next hop instance changes
- When you replace the next hop instance, as long as the replacement instance meets the same criteria as the instance that was replaced
Next hop instance internal IP address (
next-hop-address): You can specify a next hop instance using an internal IPv4 address associated with a network interface in the route's VPC network. Before you can create the route, Google Cloud requires that a next hop instance must exist and meet all of the following criteria:
- The instance must have at least one network interface in the route's VPC network.
- The instance's network interface must have an internal IPv4 address that matches the route's next hop address. The internal IPv4 address can be either the network interface's primary IPv4 address or an IPv4 address from an alias IP range on the network interface.
Google Cloud automatically updates a route whose next hop is specified by
next-hop-addresswhen the IPv4 address is moved to a different VM, provided that the replacement VM meets the above criteria.
Next hops in a Shared VPC service project: If you need to create a route in a Shared VPC network whose next hop instance is located in a service project, you must specify the next hop using
next-hop-address. You cannot specify the next hop using
Region-to-region costs and latency: When you use a VM as a next hop, the next hop is located in a zone of a region. The route that uses the next hop is available to all instances in the same VPC network or to select instances with a matching network tag. Google Cloud does not consider regional distance for routes that use an instance as a next hop, so it is possible to create a route that sends traffic to a next hop VM in a different region. Sending packets from one region to another adds egress costs and introduces network latency.
No health checking, no configuration validation: Google Cloud never checks whether a next hop instance meets all requirements outlined in the Considerations common to instance and internal TCP/UDP load balancer next hops section. Disabling the VM's network interface by configuring the guest operating system of the instance does not cause Google Cloud to disregard the next hop instance. To enhance reliability, use an internal TCP/UDP load balancer as a next hop instead.
No symmetric hashing when connecting two VPC networks: Google Cloud does not offer symmetric hashing when using two or more next hop VMs that have multiple network interfaces in a configuration that meets all of the following criteria:
- The VMs have one network interface in one VPC network and another interface in a second VPC network.
- In each VPC network, there exists a set of at least two custom static routes for the same destination, using the same priority, where each route in the set references a unique next hop VM.
If you use two or more VMs with multiple interfaces to connect VPC networks, and you require that the same VM processes packets for a given connection in both directions, you must use a next hop internal TCP/UDP load balancer. The next hop internal TCP/UDP load balancer is needed because it offers symmetric hashing. For details about symmetric hashing, see Symmetric hashing in the internal TCP/UDP load balancers as next hops documentation.
Behavior when instances are stopped or deleted: Google Cloud does not prevent you from stopping or deleting a next hop instance, regardless of the way you specified it (using
next-hop-address). Google Cloud discards all stopped or deleted instances at the preference categorization step of the routing order. The following examples clarify this behavior:
Suppose you have three custom static routes for the
192.168.168.0/24destination: one with priority 10 whose next hop VM is stopped, a second with priority 20 whose next hop VM is running, and a third with priority 30 whose next hop VM is running. Google Cloud sends packets to the second VM, even though its route has a lower priority than the route whose next hop VM is stopped. If the VM with priority 10 is restarted, then Google Cloud uses this VM as the next hop instead.
Suppose you have three custom static routes as described previously, but all of the next hop VMs are either stopped or deleted. Packets are dropped even if there exist less specific routes with working next hops because Google Cloud considers destination specificity before it determines whether a next hop VM is running.
Considerations for internal TCP/UDP load balancer next hops
- Effect of global access. Custom static routes using internal TCP/UDP load balancer next hops are programmed in all regions. Whether the next hop is usable depends on the load balancer's global access setting. With global access enabled, the load balancer next hop is accessible in all regions of the VPC network. With global access disabled, the load balancer next hop is only accessible in the same region as the load balancer. With global access disabled, packets sent from another region to a route using an internal TCP/UDP load balancer next hop are dropped.
When all backends are unhealthy. When all backends of an internal TCP/UDP load balancer fail health checks, the routes using that load balancer next hop are still in effect. Packets processed by the route are sent to one of the next hop load balancer's backends according to traffic distribution.
Forwarding rules that use a common internal IP address (
--purpose=SHARED_LOADBALANCER_VIP) are not supported. Next hop internal TCP/UDP load balancer and internal TCP/UDP load balancer forwarding rules with a common IP address are mutually exclusive features. A next hop internal TCP/UDP load balancer must use an IP address that is unique to the load balancer's forwarding rule so that only one backend service (one load balancer) is unambiguously referenced. It's possible for forwarding rules that use a common internal IP address to reference different backend services (different internal TCP/UDP load balancers).
Forwarding rules that are used in other configurations are not supported. The forwarding rule must be part of an internal TCP/UDP load balancer configuration. If the forwarding rule is used in a different load balancer type, or other resource such as a Private Service Connect endpoint, it is not supported as a next hop.
Same destination and multiple next hop internal TCP/UDP load balancers. If you create two or more custom static routes with the same destination, using different internal TCP/UDP load balancer next hops, Google Cloud never distributes traffic among the load balancer next hops using ECMP. If the routes have unique priorities, Google Cloud uses the next hop internal TCP/UDP load balancer from the route with the highest priority. If the routes have equal priorities, Google Cloud still selects just one next hop internal TCP/UDP load balancer. In this latter situation, as illustrated in the diagram below, Google Cloud uses a deterministic, internal algorithm to select a single next hop forwarding rule (
forwarding-rule-a), ignoring other routes with the same priority.
Multiple destinations and the same next hop internal TCP/UDP load balancer.
With instance tags:
If you use instance tags (also called network tags), you can use the same next hop internal TCP/UDP load balancer for multiple custom static routes with the same destination and priority.
Without instance tags: Without instance tags, you cannot create multiple custom static routes having the same combination of destination, priority, and internal TCP/UDP load balancer next hop. For example,
route-zcan all be created, but
route-x-copycannot be created.
You can specify instance tags (also called network tags) so that the next-hop route only applies to client instances that have been configured with the tag. This lets you select which client instances get populated with which tagged next-hop route and which set of appliances to route your traffic to.
You don't need to segregate the different client instances into separate VPC networks, each pointing to their preferred internal TCP/UDP load balancer front-ending a set of appliances.
Multiple routes to the same destination prefix. With instance tags, you can specify multiple routes to the same destination with different internal load balancers as next-hops. You can use different instance tags or different priorities for these same destination routes.
Considerations for Classic VPN tunnel next hops
Region-to-region costs and latency: Google Cloud does not consider regional distance for routes that use a next hop Classic VPN tunnel. Sending traffic to a next hop Classic VPN tunnel in another region adds egress costs and introduces network latency. As a best practice, use an HA VPN tunnel with dynamic routing instead because that configuration considers regional distance.
Behavior when Classic VPN tunnels are down: Custom static routes whose next hops are Classic VPN tunnels are not automatically removed when the Classic VPN tunnels are down. For details about what happens when tunnels are down, see When tunnels are down in the Classic VPN documentation.
- To create and manage routes, see Use routes.
- To get an overview of Google Cloud VPC networks, see VPC networks.
- To create, modify, or delete VPC networks, see Create and manage VPC networks.