Build HA VPN connections between Google Cloud and AWS

This tutorial demonstrates how to create highly available VPN connections between Google Cloud and Amazon Web Services (AWS) for direct communication between VPC networks across the two cloud platforms.

This tutorial assumes that you're familiar with basic concepts of virtual private cloud (VPC) networks, Border Gateway Protocol (BGP), virtual private networks (VPNs), and IPsec tunnels.

Google Cloud provides a highly available (HA) VPN service to connect your VPC network to environments running outside of Google Cloud, such as on-premises or on AWS through an IPsec VPN connection. HA VPN provides an SLA of 99.99% service availability when configured based on Google best practices.

Architecture overview

The architecture described in this document includes the following components:

  • Cloud Router: Fully distributed and managed Google Cloud service to provide dynamic routing using BGP for your VPC networks.
  • HA VPN Gateway: Google-managed VPN gateway running on Google Cloud. Each HA VPN gateway is a regional resource that has two interfaces, each with its own external IP addresses: interface 0 and 1.
  • VPN tunnels: Connect the HA VPN gateway to the peer VPN gateway and serve as virtual tunnels through which encrypted traffic passes.
  • Peer VPN gateway: Two AWS Site-to-Site VPN endpoints, which can be from an AWS virtual private gateway or AWS transit gateway.

Each of the peer VPN gateway connections comes with two tunnels that are pre-configured to point to a single customer gateway, which in this case is a Google Cloud HA VPN interface. With this configuration, the minimum number of tunnels to meet the 99.99% service availability SLA is four.

Routing options and combined bandwidth over the VPN tunnels vary based on the Site-to-Site VPN option used on the AWS side:

The following diagram shows the architecture.

Architecture overview.

Objectives

  • Create a VPC network on Google Cloud.
  • Create an HA VPN gateway and Cloud Router on Google Cloud.
  • Create customer gateways on AWS.
  • Create a VPN connection with dynamic routing on AWS.
  • Create an external VPN gateway and VPN tunnels on Google Cloud.
  • Verify and test the VPN connection between VPC networks on Google Cloud and AWS.

Costs

This tutorial uses billable components of Google Cloud, including the following:

For an estimate of the costs for the Google Cloud components, use the Google Cloud pricing calculator.

This tutorial uses billable components of Amazon Web Services, including the following:

  • AWS Transit Gateway
  • AWS Site-to-Site VPN

For an estimate of the costs for the AWS components, use the AWS pricing calculator.

Before you begin

  1. In the Google Cloud Console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  2. Make sure that billing is enabled for your Cloud project. Learn how to confirm that billing is enabled for your project.

  3. Enable the Compute Engine API.

    Enable the API

  4. In the Cloud Console, activate Cloud Shell.

    Activate Cloud Shell

  5. Make sure that you have the required administrative roles to configure networking components:

    • Network Admin: compute.networkAdmin
    • Security Admin: compute.securityAdmin
    • Compute Admin: compute.admin

    For more information about the purposes of these roles, see IAM roles for networking-related job functions.

Create the HA VPN gateway and Cloud Router on Google Cloud

In this section, you create a VPC network, an HA VPN gateway, and a Cloud Router on Google Cloud.

  1. In Cloud Shell, make sure that you're working in the Cloud project that you created or selected:

    gcloud config set project YOUR_PROJECT_ID
    
    export PROJECT_ID=`gcloud config list --format="value(core.project)"`
    

    Replace YOUR_PROJECT_ID with your Cloud project ID.

  2. Create a custom VPC network with a single subnet:

    gcloud compute networks create NETWORK \
        --subnet-mode SUBNET_MODE \
        --bgp-routing-mode BGP_ROUTING_MODE
    

    Replace the following:

    The command should look similar to the following example:

    gcloud compute networks create gc-vpc \
        --subnet-mode custom \
        --bgp-routing-mode global
    
  3. Create one subnet to host the test VMs:

    gcloud compute networks subnets create SUBNET_NAME \
        --network NETWORK \
        --region SUBNET_REGION \
        --range IP_ADDRESS_RANGE
    

    Replace the following:

    The commands should look similar to the following example:

    gcloud compute networks subnets create subnet-east4  \
        --network gc-vpc \
        --region us-east4 \
        --range 10.1.1.0/24
    
  4. Create the HA VPN gateway:

    gcloud compute vpn-gateways create HA_VPN_GATEWAY_NAME \
        --network NETWORK \
        --region REGION
    

    Replace HA_VPN_GATEWAY_NAME with the name for the HA VPN gateway.

  5. Create a Cloud Router:

    gcloud compute routers create ROUTER_NAME \
        --region REGION \
        --network NETWORK \
        --asn GOOGLE_ASN \
        --advertisement-mode custom \
        --set-advertisement-groups all_subnets
    

    Replace the following:

    • ROUTER_NAME: The name for your Cloud Router
    • GOOGLE_ASN: The private ASN (autonomous system number) for the Cloud Router that you're creating. It can be any private ASN in the range 64512-65534 or 4200000000-4294967294 that you aren't already using as a peer ASN in the same region and network.

    The command should look similar to the following example:

    gcloud compute routers create cloud-router \
        --region us-east4 \
        --network gc-vpc \
        --asn 65534 \
        --advertisement-mode custom \
        --set-advertisement-groups all_subnets
    

This procedure creates a VPN gateway with two interfaces. Take note of the external addresses so that you can use them when setting up your environment on the AWS side.

Create gateways and VPN connections on AWS

In this section, you create customer gateways, a target gateway, and VPN connections with dynamic routing.

You run AWS commands using the AWS Command Line Interface.

  1. Create two customer gateways using the following AWS command:

    aws ec2 create-customer-gateway --type ipsec.1 --public-ip INTERFACE_0_IP_ADDRESS --bgp-asn GOOGLE_ASN
    
    aws ec2 create-customer-gateway --type ipsec.1 --public-ip INTERFACE_1_IP_ADDRESS --bgp-asn GOOGLE_ASN
    

    Replace INTERFACE_0_IP_ADDRESS and INTERFACE_1_IP_ADDRESS with the public IP addresses from the final step in the previous section.

  2. Create a target gateway and attach it to your VPC network.

    The target gateway can be a virtual private gateway or a transit gateway. For more information, see Create a target gateway.

    Follow the instructions for either a virtual private gateway or a transit gateway:

    • Virtual private gateway:

      1. Create a virtual private gateway with a specific AWS-side ASN:

        aws ec2 create-vpn-gateway --type ipsec.1 --amazon-side-asn AWS_SIDE_ASN
        

        Replace AWS_SIDE_ASN with the ASN for the AWS side.

        This command should look similar to the following example:

        aws ec2 create-vpn-gateway --type ipsec.1 --amazon-side-asn 65001
        
      2. Attach the virtual private gateway to your VPC network:

        aws ec2 attach-vpn-gateway --vpn-gateway-id VPN_GATEWAY_ID --vpc-id VPC_ID
        
    • Transit gateway:

      1. Create a transit gateway:

        aws ec2 create-transit-gateway --description TRANSIT_GATEWAY_DESCRIPTION \
            --options=AmazonSideAsn=65001,AutoAcceptSharedAttachments=enable,DefaultRouteTableAssociation=enable,DefaultRouteTablePropagation=enable,VpnEcmpSupport=enable,DnsSupport=enable
        

        Replace TRANSIT_GATEWAY_DESCRIPTION with a description for your transit gateway.

      2. Attach your VPC network to the transit gateway:

        aws ec2 create-transit-gateway-vpc-attachment \
            --transit-gateway-id TRANSIT_GATEWAY_ID \
            --vpc-id VPC_ID \
            --subnet-id SUBNET_ID
        
  3. Create a VPN connection with dynamic routing.

    The method for creating a VPN connection with dynamic routing differs depending on whether your target gateway is a virtual private gateway or a transit gateway. For more information, see Create a Site-to-Site VPN connection.

    Follow the instructions for either a virtual private gateway or a transit gateway:

    • Virtual private gateway:

      Create a VPN connection with dynamic routing between the virtual private gateway and the customer gateways, and apply tags to the VPN connection:

          aws ec2 create-vpn-connection \
              --type ipsec.1 \
              --customer-gateway-id CUSTOMER_GATEWAY_1 \
              --vpn-gateway-id VPN_GATEWAY_ID \
              --options TunnelOptions='[{TunnelInsideCidr=AWS_T1_IP,PreSharedKey=SHARED_SECRET_1},{TunnelInsideCidr=AWS_T2_IP,PreSharedKey=SHARED_SECRET_2}]'
      
          aws ec2 create-vpn-connection \
              --type ipsec.1 \
              --customer-gateway-id CUSTOMER_GATEWAY_2 \
              --vpn-gateway-id VPN_GATEWAY_ID \
              --options TunnelOptions='[{TunnelInsideCidr=AWS_T3_IP,PreSharedKey=SHARED_SECRET_3},{TunnelInsideCidr=AWS_T4_IP,PreSharedKey=SHARED_SECRET_4}]'
      
    • Transit gateway:

      Create a VPN connection with dynamic routing between the transit gateway and the customer gateways:

      aws ec2 create-vpn-connection \
          --type ipsec.1 \
          --customer-gateway-id CUSTOMER_GATEWAY_1 \
          --transit-gateway-id TRANSIT_GATEWAY_ID \
          --options TunnelOptions='[{TunnelInsideCidr=AWS_T1_IP,PreSharedKey=SHARED_SECRET_1},{TunnelInsideCidr=AWS_T2_IP,PreSharedKey=SHARED_SECRET_2}]'
      
      aws ec2 create-vpn-connection \
          --type ipsec.1 \
          --customer-gateway-id CUSTOMER_GATEWAY_2 \
          --transit-gateway-id TRANSIT_GATEWAY_ID \
          --options TunnelOptions='[{TunnelInsideCidr=AWS_T3_IP,PreSharedKey=SHARED_SECRET_3},{TunnelInsideCidr=AWS_T4_IP,PreSharedKey=SHARED_SECRET_4}]'
      

      Replace the following:

      • CUSTOMER_GATEWAY_1: Google Cloud VPN gateway, interface 0
      • CUSTOMER_GATEWAY_2: Google Cloud VPN gateway, interface 1
      • AWS_T1_IP: Inside IP address for virtual private gateway for connection 1, tunnel 1
      • AWS_T2_IP: Inside IP address for virtual private gateway for connection 1, tunnel 2
      • AWS_T3_IP: Inside IP address for virtual private gateway for connection 2, tunnel 1
      • AWS_T4_IP: Inside IP address for virtual private gateway for connection 2, tunnel 2
      • SHARED_SECRET_1: Pre-shared key for connection 1, tunnel 1
      • SHARED_SECRET_2: Pre-shared key for connection 1, tunnel 2
      • SHARED_SECRET_3: Pre-shared key for connection 2, tunnel 1
      • SHARED_SECRET_4: Pre-shared key for connection 2, tunnel 2

      These commands create four tunnels to Google Cloud.

  4. Download the configuration files for the two VPN connections.

    In subsequent steps, you use values from the configuration files to create and configure resources on the Google Cloud side.

Create VPN tunnels and Cloud Router interfaces on Google Cloud

In this section, you use the information from the AWS VPN connections that you created in the previous section to create and configure components on Google Cloud.

When configuring VPN tunnels to AWS, use the IKEv2 encryption protocol and select fewer transform sets on the AWS side. For example, select a combination of single Phase 1 and Phase 2 encryption algorithms, integrity algorithms, and Diffie-Hellman (DH) group numbers. Otherwise, the Cloud VPN tunnel can fail to rekey because of the large security association (SA) payload size for the default AWS transform sets. This large payload size can cause IP fragmentation of IKE packets on the AWS side, which Cloud VPN does not support. For more information, see Tunnel options for your Site-to-Site VPN connection.

  1. In Cloud Shell, create an external VPN gateway with four interfaces for the AWS outside IP addresses:

    gcloud compute external-vpn-gateways create PEER_GATEWAY_NAME --interfaces \
      0=AWS_GW_IP_1,1=AWS_GW_IP_2,2=AWS_GW_IP_3,3=AWS_GW_IP_4
    

    Replace the following:

    • AWS_GW_IP_1: Outside IP address for virtual private gateway for connection 1, tunnel 1
    • AWS_GW_IP_2: Outside IP address for virtual private gateway for connection 1, tunnel 2
    • AWS_GW_IP_3: Outside IP address for virtual private gateway for connection 2, tunnel 1
    • AWS_GW_IP_4: Outside IP address for virtual private gateway for connection 2, tunnel 2
  2. Create four VPN tunnels:

    • Tunnel 1:

      gcloud compute vpn-tunnels create tunnel-1 \
          --peer-external-gateway PEER_GATEWAY_NAME \
          --peer-external-gateway-interface 0 \
          --region REGION \
          --ike-version IKE_VERSION \
          --shared-secret SHARED_SECRET_1 \
          --router ROUTER_NAME \
          --vpn-gateway HA_VPN_GATEWAY_NAME \
          --interface 0
      
    • Tunnel 2:

      gcloud compute vpn-tunnels create tunnel-2 \
          --peer-external-gateway PEER_GATEWAY_NAME \
          --peer-external-gateway-interface 1 \
          --region REGION \
          --ike-version IKE_VERSION \
          --shared-secret SHARED_SECRET_2 \
          --router ROUTER_NAME \
          --vpn-gateway HA_VPN_GATEWAY_NAME \
          --interface 0
      
    • Tunnel 3:

      gcloud compute vpn-tunnels create tunnel-3 \
          --peer-external-gateway PEER_GATEWAY_NAME \
          --peer-external-gateway-interface 2 \
          --region REGION \
          --ike-version IKE_VERSION \
          --shared-secret SHARED_SECRET_3 \
          --router ROUTER_NAME \
          --vpn-gateway HA_VPN_GATEWAY_NAME \
          --interface 1
      
    • Tunnel 4:

      gcloud compute vpn-tunnels create tunnel-4 \
          --peer-external-gateway PEER_GATEWAY_NAME \
          --peer-external-gateway-interface 3 \
          --region REGION \
          --ike-version IKE_VERSION \
          --shared-secret SHARED_SECRET_4 \
          --router ROUTER_NAME \
          --vpn-gateway HA_VPN_GATEWAY_NAME \
          --interface 1
      
  3. Create four Cloud Router interfaces.

    In the following commands, replace each GOOGLE_BGP_IP_TUNNEL placeholder with the tunnel's inside IP address on the Google Cloud side. You can find the values in the AWS VPN configuration files as the customer gateway address for each tunnel. Each of these addresses must be in the /30 CIDR range within the 169.254.0.0/16 network range.

    • Cloud Router interface 1:

      gcloud compute routers add-interface ROUTER_NAME \
          --interface-name int-1 \
          --vpn-tunnel tunnel-1 \
          --ip-address GOOGLE_BGP_IP_TUNNEL_1 \
          --mask-length 30 \
          --region REGION
      
    • Cloud Router interface 2:

      gcloud compute routers add-interface ROUTER_NAME \
          --interface-name int-2 \
          --vpn-tunnel tunnel-2 \
          --ip-address GOOGLE_BGP_IP_TUNNEL_2 \
          --mask-length 30 \
          --region REGION
      
    • Cloud Router interface 3:

      gcloud compute routers add-interface ROUTER_NAME \
          --interface-name int-3 \
          --vpn-tunnel tunnel-3 \
          --ip-address GOOGLE_BGP_IP_TUNNEL_3 \
          --mask-length 30 \
          --region REGION
      
    • Cloud Router interface 4:

      gcloud compute routers add-interface ROUTER_NAME \
          --interface-name int-4 \
          --vpn-tunnel tunnel-4 \
          --ip-address GOOGLE_BGP_IP_TUNNEL_4 \
          --mask-length 30 \
          --region REGION
      
  4. Add BGP peers.

    In the following commands, replace PEER_ASN with the ASN for the AWS side of the BGP session.

    • AWS connection 1, tunnel 1

      gcloud compute routers add-bgp-peer ROUTER_NAME \
          --peer-name aws-conn1-tunn1 \
          --peer-asn PEER_ASN \
          --interface int-1 \
          --peer-ip-address AWS_T1_IP \
          --region REGION
      
    • AWS connection 1, tunnel 2

      gcloud compute routers add-bgp-peer ROUTER_NAME \
          --peer-name aws-conn1-tunn2 \
          --peer-asn PEER_ASN \
          --interface int-2 \
          --peer-ip-address AWS_T2_IP \
          --region REGION
      
    • AWS connection 2, tunnel 1

      gcloud compute routers add-bgp-peer ROUTER_NAME \
          --peer-name aws-conn2-tunn1 \
          --peer-asn PEER_ASN \
          --interface int-3 \
          --peer-ip-address AWS_T3_IP \
          --region REGION
      
    • AWS connection 2, tunnel 2

      gcloud compute routers add-bgp-peer ROUTER_NAME \
          --peer-name aws-conn2-tunn2 \
          --peer-asn PEER_ASN \
          --interface int-4 \
          --peer-ip-address AWS_T4_IP \
          --region REGION
      

Verify the configuration

  1. In Cloud Shell, verify the Cloud Router status:

    gcloud compute routers get-status ROUTER_NAME \
        --region REGION \
        --format='flattened(result.bgpPeerStatus[].name, result.bgpPeerStatus[].ipAddress, result.bgpPeerStatus[].peerIpAddress)'
    

    The output is similar to the following:

    result.bgpPeerStatus[].peerIpAddress)'
    result.bgpPeerStatus[0].ipAddress:     169.254.171.18
    result.bgpPeerStatus[0].name:          aws-conn1-tunn1
    result.bgpPeerStatus[0].peerIpAddress: 169.254.171.17
    result.bgpPeerStatus[1].ipAddress:     169.254.156.154
    result.bgpPeerStatus[1].name:          aws-conn1-tunn2
    result.bgpPeerStatus[1].peerIpAddress: 169.254.156.153
    result.bgpPeerStatus[2].ipAddress:     169.254.123.38
    result.bgpPeerStatus[2].name:          aws-conn2-tunn1
    result.bgpPeerStatus[2].peerIpAddress: 169.254.123.37
    result.bgpPeerStatus[3].ipAddress:     169.254.48.186
    result.bgpPeerStatus[3].name:          aws-conn2-tunn2
    result.bgpPeerStatus[3].peerIpAddress: 169.254.48.185
    
  2. List all tunnels:

    gcloud compute vpn-tunnels list
    

    The output is similar to the following:

    NAME      REGION    GATEWAY    PEER_ADDRESS
    tunnel-1  us-east4  ha-vpn-gw  34.205.x.x
    tunnel-2  us-east4  ha-vpn-gw  52.203.x.x
    tunnel-3  us-east4  ha-vpn-gw  3.208.x.x
    tunnel-4  us-east4  ha-vpn-gw  52.204.x.x
    
  3. Check the tunnel status:

    gcloud compute vpn-tunnels describe tunnel-1 \
         --region REGION \
         --format='flattened(status,detailedStatus)'
    

    The output is similar to the following:

    detailed_status: Tunnel is up and running.
    status:          ESTABLISHED
    
  4. List dynamic routes learned by Cloud Router:

    gcloud compute routers get-status ROUTER_NAME \
        --region REGION \
        --format="flattened(result.bestRoutes)"
    

    The output is similar to the following:

    result.bestRoutes[0].creationTimestamp: 2021-01-19T20:42:07.366-08:00
    result.bestRoutes[0].destRange:         10.2.2.0/24
    result.bestRoutes[0].kind:              compute#route
    result.bestRoutes[0].nextHopIp:         169.254.171.17
    result.bestRoutes[0].priority:          100
    result.bestRoutes[1].creationTimestamp: 2021-01-19T20:42:07.366-08:00
    result.bestRoutes[1].destRange:         10.2.2.0/24
    result.bestRoutes[1].kind:              compute#route
    result.bestRoutes[1].nextHopIp:         169.254.156.153
    result.bestRoutes[1].priority:          100
    result.bestRoutes[2].creationTimestamp: 2021-01-19T20:56:26.588-08:00
    result.bestRoutes[2].destRange:         10.2.2.0/24
    result.bestRoutes[2].kind:              compute#route
    result.bestRoutes[2].nextHopIp:         169.254.123.37
    result.bestRoutes[2].priority:          100
    result.bestRoutes[3].creationTimestamp: 2021-01-19T20:56:26.588-08:00
    result.bestRoutes[3].destRange:         10.2.2.0/24
    result.bestRoutes[3].kind:              compute#route
    result.bestRoutes[3].nextHopIp:         169.254.48.185
    result.bestRoutes[3].priority:          100
    

Testing connectivity

  1. Create test VMs on each side of the tunnels to test ping requests.

    Make sure that you have firewall rules in place to allow ICMP traffic.

  2. Test the connection using the ping command.

  3. Measure bandwidth between your test machines using iperf.

    • Server side:

      iperf3 -s
      
    • Client side:

      iperf3 -c SERVER_IP_ADDRESS -P NUMBER_OF_PARALLEL_SESSIONS
      

Cleaning up

Delete the Google Cloud project

To avoid incurring charges to your Google Cloud account for the resources used in this tutorial, you can delete your project:

  1. In the Cloud Console, go to the Manage resources page.

    Go to Manage resources

  2. In the project list, select the project that you want to delete, and then click Delete.
  3. In the dialog, type the project ID, and then click Shut down to delete the project.

Delete AWS resources

  1. Delete VPN connections.
  2. Delete the transit gateway.
  3. Delete the test VMs.

What's next