Configuring on-premises routers

This document describes how to configure on-premises routers for Dedicated Interconnect. If you are creating a Partner Interconnect connection, see Configuring on-premises routers for Partner Interconnect.

After you create a VLAN attachment, you need to configure your on-premises router to establish a BGP session with your Cloud Router. To configure your on-premises router, use the VLAN ID, interface IP address, and peering IP address provided by the VLAN attachment.

Using sample topologies

This document provides the following sample topologies and configurations that you can use as a guide when configuring your on-premises router:

  • Layer 3 only topology (recommended): A Dedicated Interconnect connection or connections terminating on an on-premises router. The router performs BGP peering with Cloud Router.
  • Layer2/Layer3 topology: A Dedicated Interconnect connection or connections terminating on an on-premises switch connected to an on-premises router. The router performs BGP peering with Cloud Router.

For values for third-party platforms that you might use for your on-premises router, see vendor-specific notes. For definite values, see your on-premises router documentation.

The sample topologies in this document use the following Google Cloud resources:

  • The project Sample Interconnect Project
  • The network my-network
  • The region us-east1

There are two Dedicated Interconnect connections, my-interconnect1 and my-interconnect2. These connections are already provisioned and have a status of ready to use.

Layer 3 only topology

In this topology, the Interconnect connections terminate on an on-premises router, which performs BGP peering with Cloud Router.

The following diagrams show both the physical and logical Layer 3 only topology.

Sample physical, Layer 3 only topology (click to enlarge).
Sample physical, on-premises Layer 3 only topology (click to enlarge)



Sample logical, Layer 3 only topology (click to enlarge).
Sample logical, on-premises Layer 3 only topology (click to enlarge)

Layer 2/Layer 3 topology

In this topology, the Interconnect connections terminate on an on-premises switch, which then connects to an on-premises router. The router performs BGP peering with Cloud Router.

The following diagrams show the physical and logical Layer 2/Layer 3 topology.

Sample physical Layer 2/Layer 3 topology (click to enlarge).
Sample physical Layer 2/Layer 3 topology (click to enlarge)



Sample logical Layer 2/Layer 3 topology (click to enlarge).
Sample logical Layer 2/Layer 3 topology (click to enlarge)

Configuring on-premises devices for testing

The following section describes how to configure on-premises devices for testing your Interconnect connection. For a Layer 2/Layer 3 configuration, this example describes configuring the test interface on one or more Google Cloud-facing switches, but not on the routers.

Before Google starts testing your new Dedicated Interconnect connection, configure your interfaces without VLAN tagging, which is sometimes referred to as access mode.

Sample configuration for testing

The following example shows how to configure a Juniper router before testing, showing the required parameters to configure for port channel ae0. This configuration uses the following settings:

  • A BGP IP address of 169.254.0.2 configured on port channel ae0.
  • LACP configured on port channel ae0.
  • VLAN tagging not configured on port channel ae0. You must configure your interfaces without VLAN tagging (access mode).

    set interfaces xe-0/0/0 description "my-interconnect2"
    set interfaces xe-0/0/0 gigether-options 802.3ad ae0
    set interfaces ae0 description "my-interconnect2"
    set interfaces ae0 aggregated-ether-options lacp active
    set interfaces ae0 aggregated-ether-options minimum-links 1
    set interfaces ae0 unit 0 family inet mtu 1440
    set interfaces ae0 unit 0 family inet address 169.254.0.2
    

After your Interconnect connection is working, continue to the next section to see a sample production configuration for each topology.

Configuring on-premises routers for production

This section describes how to configure the Layer 3 only topology and the Layer 2/Layer 3 topology for production use. Each sample configuration describes all device settings.

For information about how to configure on-premises devices for testing your Interconnect connection, see Configuring on-premises routers for testing.

Production on-premises router settings for both topologies

Based on the configuration in the sample Google Cloud project, the following table summarizes the on-premises router settings to use for the example topologies.

For the sample project name, VPC network, and region used on the Google Cloud side, see the topology reference.

The hold timer and keepalive timer values allow Google to quickly transfer traffic to redundant connections in the event of an issue. Set their values as shown in the table.

Graceful restart prevents BGP sessions from packet drops and route withdrawal during Cloud Router maintenance. If your on-premises device supports BGP graceful restart, enable it and set the graceful restart and stalepath timers as shown in the table.

For more information about BGP timer settings, see the recommended values for BGP timers in the Cloud Router documentation.

Settings my-interconnect1 my-interconnect2
VLAN number 1010 1020
VLAN interface IP address 169.254.10.2/29 169.254.20.2/29
On-premises ASN 64500 64500
Cloud Router ASN 65200 65200
Cloud Router BGP IP address For cr1-us-east1:
169.254.10.1
For cr2-us-east1:
169.254.20.1
BGP timers Keepalive: 20 sec Keepalive: 20 sec
Hold timer: 60 sec Hold timer: 60 sec
Graceful restart: 1 sec Graceful restart: 1 sec
Stalepath timer: 300 sec Stalepath timer: 300 sec
On-premises LAN subnet range 192.168.12.0/24 192.168.12.0/24

Configuring Layer 3 only topology for production

Use the following guidelines when configuring the Layer 3 only topology:

  • The on-premises router port (0/0 in the diagram) or ports facing Cloud Router must be part of a port channel, even if there is only one port.
  • The port channel must have LACP enabled in either active or passive mode.
  • The maximum transmission unit (MTU) of the router interface (0/0 in the diagram) should be either 1440 or 1500 bytes, depending on the MTU of the attachment and the MTU of the connected VPC network.
  • The EBGP neighbor must have multihop configured. The recommended value for this setting is 4.

Device configuration

VLAN 1010 Router (Cisco)

The following listing shows a Layer 3 only sample configuration for on-premises Router1 (Cisco) on VLAN 1010:

        interface E0/0
          description connected_to_google_edge_device
          channel-group 2 mode active
          no shut

        interface Po2
          description my-interconnect1
          no shut

        interface Po2.1010
          description attachment_vlan1010
          encapsulation dot1Q 1010
          ip address 169.254.10.2 255.255.255.248
          ip mtu 1440

        ip prefix-list TO_GCP seq 5 permit 192.168.12.0/24

        route-map TO_GCP_OUTBOUND permit 10
          match ip address prefix-list TO_GCP

        router bgp 64500
          bgp graceful-restart restart-time 1
           neighbor 169.254.10.1 description peering_to_cloud_router
           neighbor 169.254.10.1 remote-as 65200
           neighbor 169.254.10.1 ebgp-multihop 4
           neighbor 169.254.10.1 timers 20 60
           neighbor 169.254.10.1 update-source Po2.1010
           neighbor 169.254.10.1 route-map TO_GCP_OUTBOUND out
      

VLAN 1020 Router (Juniper)

The following listing shows a Layer 3 only sample configuration for on-premises Router2 (Juniper) on VLAN 1020:

        set interfaces xe-0/0/0 ether-options 802.3ad ae1
        set interfaces xe-0/0/0 description "connected_to_google_edge_device"

        set interfaces ae1 description my-interconnect2
        set interfaces ae1 flexible-vlan-tagging
        set interfaces ae1 aggregated-ether-options minimum-links 1
        set interfaces ae1 aggregated-ether-options lacp active
        set interfaces ae1 unit 1020 family inet mtu 1440
        set interfaces ae1 unit 1020 vlan-id 1020
        set interfaces ae1 unit 1020 family inet address 169.254.20.2/29

        set routing-options autonomous-system 64500

        set policy-options prefix-list TO_GCP 192.168.12.0/24

        set policy-options policy-statement TO_GCP_OUTBOUND term 1 from protocol direct
        set policy-options policy-statement TO_GCP_OUTBOUND term 1 from prefix-list TO_GCP
        set policy-options policy-statement TO_GCP_OUTBOUND term 1 then accept
        set policy-options policy-statement TO_GCP_OUTBOUND term 2 then reject

        set protocols bgp group config_vlan_1020 type external
        set protocols bgp group config_vlan_1020 multihop ttl 4
        set protocols bgp group config_vlan_1020 local-address 169.254.20.2
        set protocols bgp group config_vlan_1020 peer-as 65200
        set protocols bgp group config_vlan_1020 neighbor 169.254.20.1 export TO_GCP_OUTBOUND
        set protocols bgp group config_vlan_1020 neighbor 169.254.20.1 graceful-restart restart-time 1
      

Configuring Layer 2/Layer 3 topology for production

Use the following guidelines for your on-premises switch and routers when configuring the Layer 2/Layer 3 topology:

  • VLANs must be configured on the switch.
  • The switch port (1/1 as shown in the diagram) or ports facing toward Cloud Router must be part of a port channel.
    • The port channel must have LACP enabled, in either active or passive mode.
    • The port channel must be configured in 802.1Q trunk mode, and all VLAN IDs used by the Interconnect connection must be allowed.
    • The port channel must have 802.1Q VLAN tagging enabled.
  • The switch port (1/2 as shown in the diagram) facing toward the on-premises router can be a trunk port or an access port. This covers the case where a router port is dedicated to a single VLAN.
  • When enabling trunk mode on the switch side, the on-premises router must support subinterfaces with necessary encapsulation (dot1q tags).
  • The MTU of the router interface (0/0 in the diagram) should be 1440 bytes.
  • The EBGP neighbor must have multihop configured. The recommended value for this setting is 4.

Device configuration

VLAN 1010 (Cisco) switch

The following listing shows a Layer 2/Layer 3 sample configuration for on-premises Switch1 (Cisco) on VLAN 1010:

          vlan 1010
          name cloud_vlan1010

          interface E1/1
            description connected_to_google_edge_device
            Channel-group 1 mode active

          interface port-channel1
            description connected_to_google_edge_device
            Switchport trunk encapsulation dot1q
            Switchport mode trunk
            Switchport trunk allowed vlan 1,1010

          interface E1/2
            description connected_to_onprem_router
            channel-group 2 mode active

          interface port-channel2
            description connected_to_onprem_router
            Switchport trunk encapsulation dot1q
            Switchport mode trunk
            Switchport trunk allowed vlan 1,1010
        

VLAN 1010 (Cisco) router

The following listing shows a Layer 2/Layer 3 sample configuration for on-premises Router1 (Cisco) on VLAN 1010:

        interface E0/0
          description connected_to_onprem_switch
          channel-group 2 mode active
          no shut

        interface Po2
          description my-interconnect1
          no shut

        interface Po2.1010
          description attachment_vlan1010
          encapsulation dot1Q 1010
          ip address 169.254.10.2 255.255.255.248
          ip mtu 1440

        ip prefix-list TO_GCP seq 5 permit 192.168.12.0/24

        route-map TO_GCP_OUTBOUND permit 10
          match ip address prefix-list TO_GCP

        router bgp 64500
          bgp graceful-restart restart-time 1
          neighbor 169.254.10.1 description peering_to_cloud_router
          neighbor 169.254.10.1 remote-as 65200
          neighbor 169.254.10.1 ebgp-multihop 4
          neighbor 169.254.10.1 timers 20 60
          neighbor 169.254.10.1 update-source Po2.1010
          neighbor 169.254.10.1 route-map TO_GCP_OUTBOUND out
      

VLAN 1020 (Juniper) switch

The following listing shows a Layer 2/Layer 3 sample configuration for on-premises Switch2 (Juniper) on VLAN 1020:

        set vlans cloud_vlan1020 vlan-id 1020

        set interfaces xe-0/1/1 description "connected_to_google_edge_device"
        set interfaces xe-0/1/1 ether-options 802.3ad ae1

        set interfaces ae1 aggregated-ether-options lacp active
        set interfaces ae1 unit 0 description "connected_to_google_edge_device"
        set interfaces ae1 unit 0 family ethernet-switching port-mode trunk
        set interfaces ae1 unit 0 family ethernet-switching vlan member cloud_vlan1020

        set interfaces xe-0/1/2 description "connected_to_onprem_router"
        set interfaces xe-0/1/2 ether-options 802.3ad ae2

        set interfaces ae2 unit 0 description "connected_to_onprem_router"
        set interfaces ae2 unit 0 family ethernet-switching port-mode trunk
        set interfaces ae2 unit 0 family ethernet-switching vlan member cloud_vlan1020
      

VLAN 1020 (Juniper) router

The following listing shows a Layer 2/Layer 3 sample configuration for on-premises Router2 (Juniper) on VLAN 1020:


      set interfaces xe-0/0/0 ether-options 802.3ad ae1
      set interfaces xe-0/0/0 description connected_to_onprem_switch

      set interfaces ae1 description my-interconnect2
      set interfaces ae1 flexible-vlan-tagging
      set interfaces ae1 aggregated-ether-options minimum-links 1
      set interfaces ae1 aggregated-ether-options lacp active
      set interfaces ae1 unit 1020 family inet mtu 1440
      set interfaces ae1 unit 1020 vlan-id 1020
      set interfaces ae1 unit 1020 family inet address 169.254.20.2/29

      set routing-options autonomous-system 64500

      set policy-options prefix-list TO_GCP 192.168.12.0/24

      set policy-options policy-statement TO_GCP_OUTBOUND term 1 from protocol direct
      set policy-options policy-statement TO_GCP_OUTBOUND term 1 from prefix-list TO_GCP
      set policy-options policy-statement TO_GCP_OUTBOUND term 1 then accept
      set policy-options policy-statement TO_GCP_OUTBOUND term 2 then reject

      set protocols bgp group config_vlan_1020 type external
      set protocols bgp group config_vlan_1020 multihop ttl 4
      set protocols bgp group config_vlan_1020 local-address 169.254.20.2
      set protocols bgp group config_vlan_1020 peer-as 65200
      set protocols bgp group config_vlan_1020 neighbor 169.254.20.1 export TO_GCP_OUTBOUND
      set protocols bgp group config_vlan_1020 neighbor 169.254.20.1 graceful-restart restart-time 1
      

Best practices

Follow these best practices to ensure effective connectivity to Google Cloud from your on-premises devices when using Cloud Interconnect 99.9% and 99.99% topologies.

Configuring devices for active/active forwarding

  • Ensure that the same MED values are exchanged across all BGP sessions.
  • Enable equal-cost multipath (ECMP) routing in your BGP configuration.
  • Enable graceful restart or distribute VLAN attachments among multiple Cloud Routers in the same region. That is, ensure that no two Cloud Routers are restarted at the same time for code upgrades.
  • If you are configuring two on-premises devices, use any routing protocol to connect both devices to each other. If you are configuring your device to use redistribution, use either IBGP or IGP.

Configuring devices for active/passive forwarding

  • To avoid asymmetric routing, make sure that higher MED values are applied on the Cloud Router side and on the on-premises device side.
  • Enable graceful restart or distribute VLAN attachments among multiple Cloud Routers in the same region. That is, ensure that no two Cloud Routers are restarted at the same time for code upgrades.
  • If you are configuring two on-premises devices, make sure that both devices have Layer 3 connectivity to each other. If you are configuring your device to use redistribution, use either IBGP or IGP.

Check that your BGP sessions are working between your on-premises network and your Google Virtual Private Cloud (VPC) network. For more information, see Viewing Cloud Router status and routes in the Cloud Router documentation.

What's next