Cloud Router

Google Cloud Router enables dynamic Border Gateway Protocol (BGP) route updates between your Google Cloud Platform (GCP) network and your on-premises network. For the initial release, Cloud Router supports BGP for Cloud VPN only.

Cloud Router works with both legacy networks and Subnetworks.

This document only covers using Cloud Router and dynamic routes with VPN. See the Cloud VPN documentation for using VPN with static routes and for general information about VPN.

For guides detailing VPN setup between GCP and other vendors or providers, see the VPN Interoperability Guides.

Before you begin

Contents

Overview

Enterprises and large organizations such as yours use VPNs to securely connect their own networks to your Google Cloud Platform networks. Without Cloud Router, you are required to configure your VPNs using static routes. With Cloud Router, your enterprise-grade Cloud VPNs support dynamic routing.

Cloud Router peers with your VPN gateway or router and exchanges topology information via BGP. Network topology changes propagate automatically between your Google Cloud Platform network and your on-premises network, thereby eliminating the need to configure static routes for your Cloud VPN tunnels.

Cloud Router is a fully distributed and managed Google cloud service that is architected using the principles of Software-Defined Networking (SDN) and delivered on Google's software-defined network virtualization stack.

Cloud Router benefits are summarized below:

  • Enables dynamic routing VPNs. Dynamic routing is an essential requirement for large organizations with enterprise-grade VPNs.
  • Eliminates the need to configure static routes for VPN tunnels. In effect, dynamic routing with BGP provides an automatic, rapid, and interoperable way of discovering network topology changes.
  • Eliminates the need to restart VPN tunnels on topology changes. As a result, Cloud Router enables seamless network topology changes without traffic disruption.

Pain points of VPN without Cloud Router

Without Cloud Router, you can configure your VPN only using static routes. The disadvantages of using static routes are:

  • A network configuration change on either side of a VPN tunnel requires manually creating and deleting static routes corresponding to these changes. In addition, static routes changes are slow to converge.
  • When a VPN tunnel is created to work with static routes, the list of IP prefixes on either side of the tunnel must be specified before the tunnel is created. This means that every time the routes must change, the VPN tunnel has to be updated (deleted and recreated) with the new routes, which disrupts existing traffic.
  • There is no standard way to configure static routes. Different vendors use different commands.
Diagram of a legacy network (click to enlarge)
VPN without Cloud Router (click to enlarge)

Example Your combined network consists of your Google cloud network and 29 subnets (one per rack) on the peer network on other side of the VPN tunnel. For this example, assume your business is growing such that you need to add a new subnet of machines every week. This week, you're adding subnet 10.0.30.0/24, as shown in the diagram above.

In this scenario, static route based VPN would require the following changes:

  1. Static routes need to be added to Google Cloud Platform to reach the new peer subnet.
  2. Tearing down and re-creating the VPN tunnel to include the new peer subnet.

The above configuration changes to static routes and VPN tunnels can be avoided by deploying a Cloud Router. Cloud Router peers with your VPN gateway using BGP to exchange topology information. In effect, network topology changes in your Google Cloud Platform network propagate automatically to your own network and vice versa via BGP, thereby eliminating the need to configure static routes for your Cloud VPN tunnels.

Cloud Router for VPNs with legacy networks

Adding Cloud Router to your VPN deployments enables your Google Cloud Platform and peer (on-premises) networks to automatically discover each other via BGP.

Diagram of a legacy network (click to enlarge)
Cloud Router for VPNs with legacy networks (click to enlarge)

Example: Continuing the previous example, if you want automatic propagation of network configuration changes without needing to reconfigure static routes and restart the VPN tunnel every time the network changes, then you can configure VPN tunnel to use dynamic routing. The dynamic routing option uses a Cloud Router to create a BGP session with the peer VPN gateway, assuming it supports BGP. Most on-premises routers support VPN and BGP on the same physical router.

The BGP session is used by each router to inform the other router of local changes. To set up BGP, an additional IP address has to be assigned to each end of the VPN tunnel. These two IP addresses must be link-local IP addresses belonging to the IP address range 169.254.0.0/16. These addresses are not part of the IP address space on either side and are used exclusively for configuring BGP peers to create a BGP session.

You must configure two link-local IP addresses, both from the same subnet, and a netmask. After these changes are made on both sides of the tunnel, a BGP session gets established. At this point, the configuration looks like the one given in the figure above.

Cloud Router for VPNs with Subnetworks

Subnetworks allow you to regionally segment the network IP space into prefixes (subnets) and control which prefix a VM instance's internal IP address is allocated from. If you want to avoid statically managing these subnetworks including the burden of adding and removing related static routes for your VPN, you can do so by enabling dynamic routing for your VPNs using Cloud Router. A Cloud router is associated with a network for a specific region, and it will announce all these regional subnetworks to the peer gateway via BGP. Cloud router also learns peer routes using BGP and selects the best route to reach the associated prefixes.

This example shows Cloud Router with custom subnetworks. With auto subnetworks, Cloud Router automatically announces the region's /20 prefix.

Diagram of a legacy network (click to enlarge)
Cloud Router for VPNs with Subnetworks (click to enlarge)

Example: Similar to the previous example for legacy networks, this example describes Cloud Router with VPN functionality for a topology with subnetworks. Assume you have two subnets in your Google Cloud Platform network (Test, Prod) and 29 subnets (one per rack) on the other side of the VPN tunnel in your datacenter. You have one VPN tunnel connecting your Google Cloud Platform network with your datacenter.

You want to perform the following two changes to your configuration:

  1. Create a new subnet in Google cloud for Staging.
  2. Add a new rack of machines to handle growing traffic, and hence, a new subnet in your datacenter.

If you want automatic propagation of network configuration changes without the need to change VPN and static route configuration every time the subnetworks change, then you should configure the VPN tunnel to use dynamic routing. Dynamic routing is enabled by using Cloud Router to create a BGP session with the peer VPN gateway also supporting BGP on its end. By adding Cloud Router to your VPN, you can automatically discover your subnetworks via BGP.

Similar to legacy networks, to set up BGP, an additional IP address has to be assigned to each end of the VPN tunnel with subnetworks. These two IP addresses must be link-local IP addresses, which belong to the IP address range 169.254.0.0/16. These are not part of your IP address space, and these two addresses will be used exclusively for creating a BGP session.

Each Cloud Router belongs to a single network and Region and only handles the BGP sessions for subnetworks specified in the VPN tunnels in that specific network and region. If the network has VPN tunnels in multiple regions, a Cloud Router has to be created in each region where dynamic routing is desired. A single Cloud Router can be used for multiple VPN gateways as well as multiple tunnels for the network/region it is attached to.

In the example above with subnetworks, once you have enabled dynamic routing for Cloud VPN gateway with Cloud Router, and added the new subnet in Google Cloud for Staging as well as a new rack in the datacenter, these new subnetworks will now be seamlessly advertised to the other side and instances in these subnetworks start sending and receiving traffic immediately. Cloud Router for dynamic routing VPNs is a compelling proposition for use with subnetworks, and a must-have for topologies with large number of subnetworks typical for large organizations and enterprises.

Enabling Graceful Restart for uninterrupted forwarding

Most customer VPN gateways support a feature of BGP called Graceful Restart, though it might need to be enabled. Graceful Restart may also be configured on a separate router if the VPN gateway doesn't support BGP. Graceful Restart allows the peer BGP device to go offline and then recover within the graceful restart period, which is two minutes for Cloud Router, without disrupting traffic flow. This feature protects against disruption when BGP agents need software upgrades and other types of maintenance, or fail temporarily. You should enable Graceful Restart on your BGP device if it supports this feature. Graceful Restart is enabled by default for Cloud Router.

Diagram of a legacy network (click to enlarge)
Graceful Restart and Cloud Router (click to enlarge)

For instance, in the example shown above, if Cloud Router requires a maintenance update and graceful restart is enabled for your peer VPN gateway, then Cloud Router will come online again within the Graceful Restart period after being updated without causing any disruption to the traffic.

Redundancy for VPN gateways without graceful restart support

If the peer gateway does not have Graceful Restart, then a failure on either side of the BGP session will cause the session to fail and traffic flow to be disrupted. After the BGP timeout is exceeded, which is 60 seconds for Cloud Router, routes will be withdrawn from both sides. Dynamically routed VPN traffic will no longer enter the tunnel. If you have also configured static routes for the tunnel, those routes will continue to be serviced.

If the peer gateway does not support graceful restart, deploying two peer gateways, with one tunnel each, provides redundancy and failover. You also need two Cloud VPN gateways, one for each tunnel. This configuration enables you to take one tunnel and devices offline for software upgrades or maintenance. It also protects you in the event of failure of one of these peer gateway by keeping the routes active and traffic flowing through the tunnel on the other peer gateway.

The diagram below shows a single box as the Cloud Router, but with two IP addresses. The two addresses are separate ethernet interfaces inside the same Cloud Router task. Each interface is used for a separate BGP session with a separate peer. In this particular use case, since these VPN tunnels are created for redundancy purpose, both BGP sessions exchange exactly same set of route prefixes but with different nexthops that point to different VPN tunnels.

Diagram of a legacy network (click to enlarge)
Redundancy without Graceful Restart (click to enlarge)

Setting up VPN with Cloud Router

In this example, the network is a custom subnetworks network.

Diagram of a legacy network (click to enlarge)
Cloud Router for VPNs with Subnetworks (click to enlarge)

Create the VPN gateway, VPN tunnel, and a Cloud Router

Console


Click create a Cloud Router before the VPN to create a Cloud Router, then use it when creating a VPN. Click create a Cloud Router as part of the VPN flow to create a Cloud Router as part of the VPN workflow.

Create Cloud Router before VPN

Use this procedure if you wish to create your Cloud Router first, then create a VPN.

  1. Go to the create Cloud Router page in the Google Cloud Platform Console.
    Go to the Routers page
  2. Create a Cloud Router in the region where you want your VPN gateway and tunnels.
    • Name — The name of the Cloud Router. This name is displayed in the console and used in all gcloud commands to reference the router. Example: my-router
    • Network — The GCP network containing the instances the VPN gateway will serve. Example: my-network
    • Region — The region where you want to locate the Cloud Router and VPN gateway. Normally, this is the region that contains the instances you wish to reach. Example: asia-east1
    • Google ASN — The private ASN (64512 - 65534, 4200000000 - 4294967294) for the router you are configuring. It can be any private ASN you are not already using as a peer ASN in the same region and network. Example: 65001
  3. Your new router appears on the Cloud Router listing page. In the VPN Gateway column of your new router, click Configure.
  4. Populate the fields for the VPN gateway:
    • Name — The name of the VPN gateway. This name is displayed in the console and used in all gcloud commands to reference the gateway. Example: my-vpn
    • Network — The Google Cloud Platform network containing your Cloud Router and the instances the VPN gateway will serve. Example: my-network
    • Region — The region where you want to create the VPN gateway. Must be the same region as your Cloud Router. This is the region that contains the instances you wish to reach. Example: asia-east1
    • IP Address — Select a pre-existing static external IP address. If you don't have a static external IP address, you can create one.
  5. Populate fields for at least one tunnel:
    • Peer IP address — Public IP address of the peer gateway. This is the public IP address of the peer VPN gateway, not the one you are currently configuring.
    • IKE version — IKEv2 is preferred, but IKEv1 is supported if that is all the peer gateway can manage.
    • Shared Secret — Character string used in establishing encryption for that tunnel. You must enter the same shared secret into both VPN gateways. If the VPN gateway device on the peer side of the tunnel doesn't generate one automatically, you can make one up.
    • Routing options — Select Dynamic (BGP).
    • Cloud router — Select my-router.
    • BGP session — Click the "pencil" icon, then populate the following fields. When you are done, click Save and continue.
      • Namebgp-peer1
      • Peer ASN — You can use your public ASN or any private ASN (64512 - 65534, 4200000000 - 4294967294) that you are not already using in the peer or GCP network. Example: 65002
      • Google BGP IP address — The two BGP interface IP addresses must be link-local IP addresses belonging to the same /30 subnet in 169.254.0.0/16. Example: 169.254.1.1
      • Peer BGP IP address — See explanation for Google BGP IP address. Example: 169.254.1.2
  6. Click Create to create the gateway and initiate all tunnels, though tunnels will not connect until you've configured the peer router as well.
    This step automatically creates the necessary forwarding rules for the gateway and tunnels.

Create Cloud Router as part of VPN flow

Use this procedure if you don't already have a Cloud Router and want to create one as part of the VPN workflow.

  1. Go to the create VPN page in the Google Cloud Platform Console.
    Go to the VPN page
  2. Populate the fields for the VPN gateway:
    • Name — The name of the VPN. This name is displayed in the console and used in all gcloud commands to reference the VPN. Example: my-vpn
    • Network — The GCP network containing the instances the VPN gateway will serve. Example: my-network
    • Region — The region where you want to create the VPN gateway. Must be the same region as your Cloud Router. This is the region that contains the instances you wish to reach. Example: asia-east1
    • IP Address — Select a pre-existing static external IP address. If you don't have a static external IP address, click New static IP address to create one.
  3. Populate fields for at least one tunnel:
    • Peer IP address — Public IP address of the peer gateway. This is the public IP address of the peer VPN gateway, not the one you are currently configuring.
    • IKE version — IKEv2 is preferred, but IKEv1 is supported if that is all the peer gateway can manage.
    • Shared Secret — Character string used in establishing encryption for that tunnel. You must enter the same shared secret into both VPN gateways. If the VPN gateway device on the peer side of the tunnel doesn't generate one automatically, you can make one up.
    • Routing options — Select Dynamic (BGP).
    • Cloud router — Select Create cloud route, then populate the following fields. When you are done, click Save and continue.
      • Name — The name of the Cloud Router. This name is displayed in the console and used in all gcloud commands to reference the router. Example: my-router
      • Google ASN — The private ASN (64512 - 65534, 4200000000 - 4294967294) for the router you are configuring. It can be any private ASN you are not already using. Example: 65001
    • BGP session — Click the "pencil" icon, then populate the following fields. When you are done, click Save and continue.
      • Namebgp-peer1
      • Peer ASN — The private ASN (64512 - 65534, 4200000000 - 4294967294) for the router you are configuring. It can be any private ASN you are not already using. Example: 65002
      • Google BGP IP address — The two BGP interface IP addresses must be link-local IP addresses belonging to the same /30 subnet in 169.254.0.0/16. Example: 169.254.1.1
      • Peer BGP IP address — See explanation for Google BGP IP address. Example: 169.254.1.2
  4. Click Create to create the gateway, Cloud Router, and all tunnels, though tunnels will not connect until you've configured the peer router as well.
    This step automatically creates the necessary forwarding rules for the gateway and tunnels. Note that creating a BGP peer session using the Google Cloud Platform Console automatically creates a Cloud Router interface for that peer.

gcloud compute


  1. Choose or create a Google Cloud Platform network. In this example, you are creating custom subnetworks.

    gcloud compute networks create my-network \
      --mode custom
    
    NAME       MODE   IPV4_RANGE GATEWAY_IPV4
    my-network custom
    
  2. Specify the subnetwork prefix for your first subnetwork. In this example, you are assigning 10.21.0.0/16 to region asia-east1.

    gcloud compute networks subnets create subnet-1 \
        --network my-network \
        --region asia-east1 \
        --range 10.21.0.0/16
    
    NAME     REGION      NETWORK    RANGE
    subnet-1 asia-east1  my-network 10.21.0.0/16
    
  3. Create a VPN gateway in the desired region. Normally, this is the region that contains the instances you wish to reach. This step creates an unconfigured virtual VPN gateway named my-vpn in your Google Cloud Platform network.

    gcloud compute target-vpn-gateways create my-vpn \
        --project [PROJECT_ID] \
        --region asia-east1 \
        --network my-network
    
    Created [https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/asia-east1/targetVpnGateways/my-vpn].
    NAME   NETWORK    REGION
    my-vpn my-network asia-east1
    
  4. Reserve a static IP address in the Google Cloud Platform network and region where you created the VPN gateway. Make a note of the created address for use in future steps. The numerical address is shown as IP-ADDRESS in future steps.

    gcloud compute addresses create vpn-static-ip \
        --project [PROJECT_ID] \
        --region asia-east1
    
    Created [https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/asia-east1/addresses/address1].
    NAME          REGION       ADDRESS         STATUS
    vpn-static-ip asia-east1   IP-ADDRESS      RESERVED
    
  5. Create forwarding rules that forward ESP, UDP:500, and UDP:4500 traffic toward the Cloud VPN gateway. Use the static IP address IP-ADDRESS you reserved earlier. This step generates forwarding rules named fr-esp, fr-udp500, and fr-udp4500.

    First, create the fr-esp rule:

    gcloud compute forwarding-rules create fr-esp \
        --project [PROJECT_ID] \
        --region asia-east1 \
        --address IP-ADDRESS \
        --target-vpn-gateway my-vpn \
        --ip-protocol ESP
    
    Created [https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/asia-east1/forwardingRules/fr-esp].
    NAME   REGION       IP-ADDRESS      IP_PROTOCOL TARGET
    fr-esp asia-east1   IP-ADDRESS      ESP         asia-east1/targetVpnGateways/my-vpn
    

    Create fr-udp500:

    gcloud compute forwarding-rules create fr-udp500 \
        --project [PROJECT_ID] \
        --region asia-east1 \
        --address IP-ADDRESS \
        --target-vpn-gateway my-vpn \
        --ip-protocol UDP \
        --ports 500
    
    Created [https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/asia-east1/forwardingRules/fr-udp500].
    NAME      REGION       IP-ADDRESS      IP_PROTOCOL TARGET
    fr-udp500 asia-east1   IP-ADDRESS      UDP         asia-east1/targetVpnGateways/my-vpn
    

    Create fr-udp4500:

    gcloud compute forwarding-rules create fr-udp4500 \
         --project [PROJECT_ID] \
         --region asia-east1 \
         --address IP-ADDRESS \
         --target-vpn-gateway my-vpn \
         --ip-protocol UDP \
         --ports 4500
    
     Created [https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/asia-east1/forwardingRules/fr-udp4500].
     NAME       REGION       IP-ADDRESS      IP_PROTOCOL TARGET
     fr-udp4500 asia-east1   IP-ADDRESS      UDP         asia-east1/targetVpnGateways/my-vpn
    
  6. Create a Cloud Router in the region where you created the VPN gateway. This example uses ASN 65001 for the Cloud Router ASN, but you can use any private ASN (64512 - 65534, 4200000000 - 4294967294) that you are not already using in the peer network.

    gcloud compute --project [PROJECT_ID] routers create my-router \
      --region asia-east1 \
      --network my-network \
      --asn 65001
    
    Created [https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/asia-east1/routers/my-router].
    NAME      REGION     NETWORK
    my-router asia-east1 my-network
    
  7. Create a VPN tunnel on the Cloud VPN gateway that points to the external IP address PEER-GW-EXT-IP of the peer VPN gateway. You also need to supply the shared secret for the VPN tunnel, the name of the Cloud Router, and the IKE version. IKE version 2 is recommended, but use version 1 if the peer gateway does not support version 2.
    Once this command is executed, resources are allocated for this VPN tunnel, but it cannot yet pass traffic.

    gcloud compute --project [PROJECT_ID] vpn-tunnels create tunnel1 \
      --region asia-east1 \
      --ike-version 2 \
      --target-vpn-gateway my-vpn \
      --peer-address PEER-GW-EXT-IP \
      --shared-secret SHAREDSECRET \
      --router my-router
    
    Created [https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/asia-east1/vpnTunnels/tunnel1].
    NAME       REGION     GATEWAY      PEER_ADDRESS
    tunnel1    asia-east1 my-vpn       PEER-GW-EXT-IP
    
  8. Update the Cloud Router config to add a virtual interface (--interface) for the BGP peer. The BGP interface IP address must be a link-local IP address belonging to the IP address range 169.254.0.0/16 and it must belong to same subnet as the interface address of the peer router. The netmask length is recommended to be 30. Make sure each tunnel has a unique pair of IPs. Alternatively, you can leave --ip-address and --mask-length blank, and leave --peer-ip-address blank in the next step, and IP addresses will be automatically generated for you.

    gcloud compute --project [PROJECT_ID] routers add-interface my-router \
      --interface if-1 \
      --ip-address 169.254.1.1 \
      --mask-length 30 \
      --vpn-tunnel tunnel1 \
      --region asia-east1
    
    Updated [https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/asia-east1/routers/my-router].
    
  9. Update the Cloud Router config to add the BGP peer to the interface. This example uses ASN 65002 for the peer ASN. You can use your public ASN or private ASN (64512 - 65534, 4200000000 - 4294967294) that you are not already using in the peer network. The BGP peer interface IP address must be a link-local IP address belonging to the IP address range 169.254.0.0/16. It must belong to same subnet as the Google Cloud Platform-side interface. Make sure each tunnel has a unique pair of IPs.

    gcloud compute --project [PROJECT_ID] routers add-bgp-peer my-router \
      --peer-name bgp-peer1 \
      --interface if-1 \
      --peer-ip-address 169.254.1.2 \
      --peer-asn 65002 \
      --region asia-east1
    
    Updated [https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/asia-east1/routers/my-router].
    
  10. View details of the Cloud Router and confirm your settings.

    gcloud compute --project [PROJECT_ID] routers describe my-router \
      --region asia-east1
    
    bgp:
     asn: 65001
    bgpPeers:
    - interfaceName: if-bgp-peer1
      ipAddress: 169.254.1.1
      name: bgp-peer1
      peerAsn: 65002
      peerIpAddress: 169.254.1.2
    creationTimestamp: '2015-10-19T14:31:52.639-07:00'
    id: '4047683710114914215'
    interfaces:
    - ipRange: 169.254.1.1/30
      linkedVpnTunnel: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/asia-east1/vpnTunnels/tunnel1
      name: if-bgp-peer1
    kind: compute#router
    name: my-router
    network: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/networks/my-network
    region: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/asia-east1
    selfLink: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/asia-east1/routers/my-router
    

Configure firewall rules

You must configure your Google Cloud Platform firewall rules to allow inbound traffic from the peer network subnets, and you must configure the peer network firewall to allow inbound traffic from your Compute Engine prefixes. To configure your Google Cloud Platform firewall rules, run the following command. You might want to configure a range broad enough to cover future BGP updates.

Configure Google Cloud Platform firewall rules

Console


  1. Open the Firewalls page.
  2. Click CREATE FIREWALL RULE.
  3. Populate the following fields:
    • Name: vpnrule1
    • Network: my-network
    • Source filter: IP ranges.
    • Source IP ranges: The peer ranges to accept from the peer VPN gateway.
    • Allowed protocols and ports: tcp;udp;icmp
  4. Click Create.

If you have more than one peer network range, provide a comma-separated list in the Source IP ranges field (10.10.4.0/24,10.10.6.0/24).

gcloud compute


gcloud compute --project [PROJECT_ID] firewall-rules create vpnrule1 \
    --network my-network \
    --allow tcp,udp,icmp \
    --source-ranges PEER-SOURCE-RANGE

If you have more than one peer network range, provide a comma-separated list in the --source-ranges field (--source-ranges 10.10.4.0/24,10.10.6.0/24).

This rule allows TCP, UDP, and ICMP traffic from all ports to all machines in the Google Cloud Platform network, as long as the traffic is coming from the peer source ranges. If you wish to restrict the valid destinations for VPN traffic, see instructions on firewalls for information on creating more specific rules.

Configure Peer firewall rules

Configure the peer firewall with the ranges you expect to receive from your Cloud VPN gateway. You might want to configure a range broad enough to cover future BGP updates.

You must also configure the peer firewall to allow incoming TCP traffic with either source or destination port 179. This is the port used for the BGP announcements.

Configure peer gateway for the BGP session

Look up the ASNs and IP addresses of the Cloud and peer router connections, then configure the peer gateway.

Console


  1. Open the Cloud Router list.
  2. Click on the name of your Cloud Router.
  3. Make a note of the values for Google ASN, Peer ASN, Google BGP IP address, and Peer BGP IP address.

gcloud compute


gcloud compute --project [PROJECT_ID] routers describe my-router --region asia-east1
bgp:
 asn: 65001
bgpPeers:
- interfaceName: if-bgp-peer1
  ipAddress: 169.254.1.1
  name: bgp-peer1
  peerAsn: 65002
  peerIpAddress: 169.254.1.2
creationTimestamp: '2015-10-19T14:31:52.639-07:00'
id: '4047683710114914215'
interfaces:
- ipRange: 169.254.1.1/30
  linkedVpnTunnel: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/asia-east1/vpnTunnels/tunnel1
  name: if-bgp-peer1
kind: compute#router
name: my-router
network: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/networks/my-network
region: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/asia-east1
selfLink: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/asia-east1/routers/my-router

Make a note of the asn, ipAddress, peerAsn, and peerIpAddress.

Configure peer gateway for the tunnel

For peer VPN gateway settings, see Set up the peer VPN gateway in the VPN documentation.

Check the status of the tunnel

Console


Open the VPN list and look for a green circle with a checkmark. That means the tunnel is established. If you see an exclamation mark in a red circle, then the tunnel is still coming up or has already failed. Click the Log link to see current information on the tunnel.

gcloud compute


gcloud compute --project [PROJECT_ID] vpn-tunnels describe tunnel1 \
    --region asia-east1
creationTimestamp: '2015-10-19T14:33:45.449-07:00'
description: ''
detailedStatus: 'Initial handshake. More info: https://console.developers.google.com/[PROJECT_ID]/507356250768/logs?service=compute.googleapis.com&key1=targetVpnGateway&key2=4189032514050383796'
id: '2196766647665614678'
ikeVersion: 2
kind: compute#vpnTunnel
name: tunnel1
peerIp: PEER-GW-EXT-IP
region: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/asia-east1
router: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/asia-east1/routers/my-router
selfLink: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/asia-east1/vpnTunnels/tunnel1
sharedSecret: SHAREDSECRET
sharedSecretHash: AH6QHyVoninNYieomeYx95HBlKl8
status: FIRST_HANDSHAKE
targetVpnGateway: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/asia-east1/targetVpnGateways/my-vpn

A Status of ESTABLISHED means the tunnel is up. For a list of intermediate and error statuses, see Check the status of your tunnel in the VPN documentation.

Get the status of a Cloud Router and BGP sessions

Console


  1. Open the router list and look for a green circle with a checkmark in the BGP sessions column. That means the session is established. If you see an exclamation mark in a red circle, then the session is still coming up or has already failed. Click the Log link to see current information. If you see the red circle with the exclamation mark, wait a few moments, then reload the page. It can take a few moments for the session to come up.
  2. Open the routes list. Click Dynamic to see the dynamic routes learned by Cloud Routers in the network.

gcloud compute


gcloud compute --project [PROJECT_ID] routers get-status my-router \
   --region asia-east1
kind: compute#routerStatusResponse
Result:
  bestRoutes:
  - destRange: 10.0.1.0/24
    kind: compute#route
    nextHopIp: 169.254.1.1
    priority: 100
  - destRange: 10.0.2.0/24
    kind: compute#route
    nextHopIp: 169.254.1.1
    priority: 100
 bgpPeerStatus:
 - advertisedRoutes:
  - destRange: 10.21.0.0/16
    kind: compute#route
    nextHopIp: 169.254.1.2
    priority: 100
  - destRange: 192.168.1.0/24
    kind: compute#route
    nextHopIp: 169.254.1.2
    priority: 100
  ipAddress: 169.254.1.1
  name: bgp-peer1
  numLearnedRoutes: 0
  peerIpAddress: 169.254.1.2
  state: Established
  status: UP
  uptime: '10'
network: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/networks/my-network

Additional Cloud Router operations

List Cloud Routers

To list all Cloud Routers in a project, run the following command.

Console


Cloud Routers list

gcloud compute


gcloud compute --project [PROJECT_ID] routers list
NAME      REGION     NETWORK
my-router asia-east1 my-network

To restrict the list to only one region, specify --region.

Delete Cloud Router resources

For instructions on deleting resources other than the ones listed, see Deleting a tunnel and Deleting a gateway.

Remove a BGP peer

Console


In the Google Cloud Platform Console, deleting a tunnel automatically deletes the BGP peer and the Cloud Router interface for that peer.

gcloud compute


gcloud compute --project [PROJECT_ID] routers remove-bgp-peer my-router \
    --peer-name bgp-peer1 \
    --region asia-east1
Updated [https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/asia-east1/routers/my-router].

Remove a BGP interface

Console


If you are using the Console, the interface is removed when you remove the BGP peer. See Remove a BGP peer.

gcloud compute


gcloud compute --project [PROJECT_ID] routers remove-interface my-router \
    --interface if-1 \
    --region asia-east1
Updated [https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/asia-east1/routers/my-router].

Delete a Cloud Router

You cannot delete a router that is being used by a VPN tunnel. First, delete the tunnel, then delete the router. You can delete a router that still has BGP peers as long as no tunnels are using the router.

Console


  1. Go to the Cloud Routers list.
  2. Select the checkbox next to the Cloud Router you want to delete.
  3. Click DELETE.

gcloud compute


gcloud compute --project [PROJECT_ID] routers delete my-router \
    --region asia-east1
Deleted [https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/asia-east1/routers/my-router].

Logging

The following types of messages may appear in Google Cloud Logging.

To view logs for Cloud Routers, open the Cloud Routers list, then click View in the Logs column.

Cloud Router logs have the following format:

[Event Type]: [Log Text]

Some log entries refer to the router's "BGP router ID". The router is assigned a BGP router ID automatically based on the IP address of one of its interfaces.

Router events

Router events are events pertaining to your Google Cloud Router itself.

INFO log entries

  • Router Event: Router task activated
  • Router Event: Router task de-activated

BGP events

BGP events are events pertaining to a BGP configuration and session.

INFO log entries

  • BGP Event: Successfully added configuration for peers: [LIST_OF_PEERS]
  • BGP Event: BGP peering with [PEER] came up X seconds ago
  • BGP Event: BGP peering with [PEER] went down
  • BGP Event: BGP Router ID set to: [BGP_ROUTER_ID]

Route events

Route events are events pertaining to route announcements between the two BGP peers.

INFO log entries

  • Route Event: Advertising prefix to peers: [PREFIX]
  • Route Event: Withdrawing prefix from peers: [PREFIX]
  • Route Event: Prefix [PREFIX] Nexthops [LIST_OF_NEXTHOPS] received by cloud router
  • Route Event: Prefix [PREFIX] Nexthops [LIST_OF_NEXTHOPS] deleted from cloud router

ERROR log entries

  • Route Event: Dropping Route: [PREFIX] Maximum allowed routes MAX_ROUTES already in Datapath
    If you get this error, reduce the number of prefixes announced from the peer router to the [MAX_ROUTES] value.

Metrics

Cloud Router publishes metrics to Stackdriver monitoring. See Metrics List for a list of metrics published for Cloud Router.

Viewing metrics via the API

You can access these metrics via the API Explorer.

Viewing metrics via Stackdriver dashboards

You can also create custom dashboard in Stackdriver using these metrics.

To create a custom dashboard in Stackdriver, do the following:

  1. Go to the Create Dashboard in the Google Cloud Platform Console.
    Go to the Create Dashboard page
  2. Replace Untitled Dashboard with an appropriate name.
  3. Click Add Chart.
  4. In the Resource Type pull-down menu, select Cloud Router.
  5. Change the Title of the chart if desired.
  6. Select one or more metrics from the Metric Type pull-down menu.
    A graph for the metrics for all Cloud Routers should now be visible.
  7. Use the Filter controls to restrict the view to only certain routers or sessions.

Some metrics are for the Cloud Router and some metrics are for a BGP session on a given Cloud Router. Cloud Router metrics are shown by router-name while BGP sessions metrics are shown as router-name(bgp-name).

Cloud Router upgrade cycles

Cloud Router gets upgraded periodically. This process takes under 60 seconds, and the Cloud Router is not available during the upgrade. The BGP hold timer determines how long learned routes are preserved when the peered BGP router is unavailable. BGP hold timer is negotiated to be lower of the value on both sides. Cloud Router uses a value of 60 seconds for the BGP hold timer. We recommend that you set your peer BGP hold timer to 60 seconds or greater (the default value is 3 minutes). Then both routers will preserve their routes during these upgrades and traffic continues to flow.

During VPN gateway maintenance cycles with a single VPN gateway, the use of Cloud Router adds about 20 seconds to the tunnel recovery time because the BGP session is reset and routes have to be relearned. (VPN gateway recovery times are usually about a minute.) If there are redundant VPN gateways, traffic should be unaffected because only one VPN gateway is taken down at a time.

Troubleshooting

If you have completed the steps but traffic is not passing through your tunnel, verify the following:

  • Double-check that the settings on your peer device (on-premises BGP router) and the settings on your Cloud Router are correct.
  • Check the status of your Cloud Router. Make sure the state of your BGP session is ESTABLISHED.
  • Verify that the status of the tunnel is ESTABLISHED. If it is not, consult VPN troubleshooting to help figure out the problem.

Quotas and Limits

  • For each project, you can configure:
    • maximum of 10 Cloud Routers
    • maximum of 5 Cloud Router per region per network
    • You can configure a maximum of 64 interfaces, BGP peers, and VPN tunnels across all routers in a region in a given network. These can all be on one router or spread across several.
  • Cloud Router will only program learned routes in the region it is configured in.
  • For each VPN gateway, you can configure a maximum of one VPN tunnel with dynamic routing. You can configure additional static tunnels as needed.

Recommendations

  • You should enable graceful restart on the peer BGP device.
  • If graceful restart is not supported or enabled on your device, you should configure two peer devices with one tunnel each to provide redundancy.
  • For highest reliability, set up redundant Cloud VPN gateways and tunnels as shown in Redundancy for VPN gateways without graceful restart support even if your peer device supports graceful restart.

FAQ

  • Can I connect one Cloud Router to multiple Cloud VPN gateways in the same region?

    • Yes, this is the recommended optimal configuration when you have multiple instances of VPN gateways in the same region as the Cloud Router.
  • The documentation states that Cloud Router can only program learned routes in the region it is in. I want my on-premises VPN devices to connect to Google Cloud Platform instances in more than one region. How do I support this?

    • If you need the peer VPN devices to connect to instances in more than one region, you should create a Cloud Router and Google VPN gateway and tunnel in each region.
  • What is the recommendation for BGP graceful restart on my on-premises BGP device?

    • You should enable graceful restart on your BGP device.
    • Graceful restart is enabled by default for Cloud Router.
  • How long is the graceful restart period?

    • Cloud Router BGP has a graceful restart period of approximately 2 mins.
  • Do you recommend deploying redundant peer devices with a tunnel each if they have graceful restart enabled?

    • If you require the highest level of high availability, then you should deploy redundant peer devices with a tunnel each even if graceful restart is enabled. This will protect you in the event of non-transient failures, such as when one of the peer devices or the tunnels fails but never comes back up again.
  • Does Cloud Router work with Subnetworks?

    • Yes, Cloud Router works with Subnetworks and legacy networks.
  • Where can I find details on setting up Cloud VPN/Cloud Router with other VPN gateways?

  • What is my router's BGP router ID?

    • Some log entries refer to the router's "BGP router ID". The router is assigned a BGP router ID automatically based on the IP address of one of its interfaces.

Send feedback about...

Compute Engine Documentation