Create HA VPN connections between Google Cloud and AWS

This tutorial demonstrates how to create highly available VPN connections between Google Cloud and Amazon Web Services (AWS) for direct communication between VPC networks across the two cloud platforms.

This tutorial assumes that you're familiar with basic concepts of virtual private cloud (VPC) networks, Border Gateway Protocol (BGP), virtual private networks (VPNs), and IPsec tunnels.

Google Cloud provides a highly available (HA) VPN service to connect your VPC network to environments running outside of Google Cloud, such as on-premises or on AWS through an IPsec VPN connection. HA VPN provides an SLA of 99.99% service availability when configured based on Google best practices.

Architecture overview

The architecture described in this document includes the following components:

Cloud Router: A fully distributed and managed Google Cloud service to provide dynamic routing using BGP for your VPC networks.
HA VPN gateway: A Google-managed VPN gateway running on Google Cloud. Each HA VPN gateway is a regional resource that has two interfaces, each with its own external IP addresses: interface 0 and 1.
VPN tunnels: Connections from the HA VPN gateway to the peer VPN gateway on AWS through which encrypted traffic passes.
Peer VPN gateway: Two AWS Site-to-Site VPN endpoints, which can be from an AWS virtual private gateway or AWS transit gateway.

Each of the peer VPN gateway connections comes with two tunnels that are pre-configured to point to a single customer gateway, which in this case is a Google Cloud HA VPN interface. With this configuration, the minimum number of tunnels to meet the 99.99% service availability SLA is four.

Routing options and combined bandwidth over the VPN tunnels vary based on the Site-to-Site VPN option used on the AWS side:

Transit gateway: If you create the AWS transit gateway with no BGP preference, ECMP equally distributes traffic across active tunnels.

To use the VPN connectivity across multiple Google Virtual Private Cloud networks, see the best practices for building a hub-and-spoke system in Google Cloud.

For more information about AWS transit gateways, see the Amazon Virtual Private Cloud documentation.
Virtual private gateway: If you use an AWS virtual private gateway, only one tunnel across all connections on the gateway is selected. To use more than one tunnel, instead use an AWS transit gateway so that ECMP is available.

For details about VPN route priority with AWS, see the AWS Site-to-Site VPN routing options documentation.

For more information about AWS virtual private gateways, see AWS Site-to-Site VPN tunnel documentation.

The following diagram shows the architecture.

Architecture overview.

Objectives

Create a VPC network on Google Cloud.
Create an HA VPN gateway and Cloud Router on Google Cloud.
Create customer gateways on AWS.
Create a VPN connection with dynamic routing on AWS.
Create an external VPN gateway and VPN tunnels on Google Cloud.
Verify and test the VPN connection between VPC networks on Google Cloud and AWS.

Example Terraform module

You can use the gcp-to-aws-ha-vpn-terraform-module module to provision HA VPN between Google Cloud and AWS.

Costs

This tutorial uses billable components of Google Cloud, including the following:

For an estimate of the costs for the Google Cloud components, use the Google Cloud pricing calculator.

This tutorial uses billable components of Amazon Web Services, including the following:

AWS Transit Gateway
AWS Site-to-Site VPN

For an estimate of the costs for the AWS components, use the AWS pricing calculator.

Before you begin

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

Go to project selector
Make sure that billing is enabled for your Google Cloud project.
Enable the Compute Engine API.
Enable the API

In the Google Cloud console, activate Cloud Shell.

Activate Cloud Shell
Make sure that you have the required administrative roles to configure networking components:
- Network Admin: compute.networkAdmin
- Security Admin: compute.securityAdmin
- Compute Admin: compute.admin
For more information about the purposes of these roles, see IAM roles for networking-related job functions.

Create the HA VPN gateway and Cloud Router on Google Cloud

In this section, you create a VPC network, an HA VPN gateway, and a Cloud Router on Google Cloud.

In Cloud Shell, make sure that you're working in the Google Cloud project that you created or selected:
```
gcloud config set project YOUR_PROJECT_ID

export PROJECT_ID=`gcloud config list --format="value(core.project)"`
```
Replace YOUR_PROJECT_ID with your Google Cloud project ID.
Create a custom VPC network with a single subnet:
```
gcloud compute networks create NETWORK \
    --subnet-mode SUBNET_MODE \
    --bgp-routing-mode BGP_ROUTING_MODE
```
Replace the following:
- NETWORK: The name of the network
- SUBNET_MODE: The subnet mode
- BGP_ROUTING_MODE: The BGP routing mode
The command should look similar to the following example:
```
gcloud compute networks create gc-vpc \
    --subnet-mode custom \
    --bgp-routing-mode global
```

Create one subnet to host the test VMs:

gcloud compute networks subnets create SUBNET_NAME \
    --network NETWORK \
    --region SUBNET_REGION \
    --range IP_ADDRESS_RANGE

Replace the following:

SUBNET_NAME: The subnet name
SUBNET_REGION: The region in which to create the subnet
IP_ADDRESS_RANGE: The IP address range for the subnet

The commands should look similar to the following example:

gcloud compute networks subnets create subnet-east4  \
    --network gc-vpc \
    --region us-east4 \
    --range 10.1.1.0/24

Create the HA VPN gateway:

gcloud compute vpn-gateways create HA_VPN_GATEWAY_NAME \
    --network NETWORK \
    --region REGION

Replace HA_VPN_GATEWAY_NAME with the name for the HA VPN gateway.

Create a Cloud Router:
```
gcloud compute routers create ROUTER_NAME \
    --region REGION \
    --network NETWORK \
    --asn GOOGLE_ASN \
    --advertisement-mode custom \
    --set-advertisement-groups all_subnets
```
Replace the following:
- ROUTER_NAME: The name for your Cloud Router
- GOOGLE_ASN: The private ASN (autonomous system number) for the Cloud Router that you're creating. It can be any private ASN in the range 64512-65534 or 4200000000-4294967294 that you aren't already using as a peer ASN in the same region and network.
The command should look similar to the following example:
```
gcloud compute routers create cloud-router \
    --region us-east4 \
    --network gc-vpc \
    --asn 65534 \
    --advertisement-mode custom \
    --set-advertisement-groups all_subnets
```

This procedure creates a VPN gateway with two interfaces. Take note of the external addresses so that you can use them when setting up your environment on the AWS side.

Create gateways and VPN connections on AWS

In this section, you create customer gateways, a target gateway, and VPN connections with dynamic routing.

You run AWS commands using the AWS Command Line Interface.

Create two customer gateways using the following AWS command:

aws ec2 create-customer-gateway --type ipsec.1 --public-ip INTERFACE_0_IP_ADDRESS --bgp-asn GOOGLE_ASN

aws ec2 create-customer-gateway --type ipsec.1 --public-ip INTERFACE_1_IP_ADDRESS --bgp-asn GOOGLE_ASN

Replace INTERFACE_0_IP_ADDRESS and INTERFACE_1_IP_ADDRESS with the public IP addresses from the final step in the previous section.

Create a target gateway and attach it to your VPC network.

The target gateway can be a virtual private gateway or a transit gateway. For more information, see Create a target gateway.

Follow the instructions for either a virtual private gateway or a transit gateway:
- Virtual private gateway:
  1. Create a virtual private gateway with a specific AWS-side ASN:
```
aws ec2 create-vpn-gateway --type ipsec.1 --amazon-side-asn AWS_SIDE_ASN
```
    Replace AWS_SIDE_ASN with the ASN for the AWS side.
    
    This command should look similar to the following example:
```
aws ec2 create-vpn-gateway --type ipsec.1 --amazon-side-asn 65001
```
  2. Attach the virtual private gateway to your VPC network:
```
aws ec2 attach-vpn-gateway --vpn-gateway-id VPN_GATEWAY_ID --vpc-id VPC_ID
```
- Transit gateway:
  1. Create a transit gateway:
```
aws ec2 create-transit-gateway --description TRANSIT_GATEWAY_DESCRIPTION \
    --options=AmazonSideAsn=65001,AutoAcceptSharedAttachments=enable,DefaultRouteTableAssociation=enable,DefaultRouteTablePropagation=enable,VpnEcmpSupport=enable,DnsSupport=enable
```
    Replace TRANSIT_GATEWAY_DESCRIPTION with a description for your transit gateway.
  2. Attach your VPC network to the transit gateway:
```
aws ec2 create-transit-gateway-vpc-attachment \
    --transit-gateway-id TRANSIT_GATEWAY_ID \
    --vpc-id VPC_ID \
    --subnet-id SUBNET_ID
```
Create a VPN connection with dynamic routing.

The method for creating a VPN connection with dynamic routing differs depending on whether your target gateway is a virtual private gateway or a transit gateway. For more information, see Create a Site-to-Site VPN connection.

Follow the instructions for either a virtual private gateway or a transit gateway:
- Virtual private gateway:
  
  Create a VPN connection with dynamic routing between the virtual private gateway and the customer gateways, and apply tags to the VPN connection:
```
    aws ec2 create-vpn-connection \
        --type ipsec.1 \
        --customer-gateway-id CUSTOMER_GATEWAY_1 \
        --vpn-gateway-id VPN_GATEWAY_ID \
        --options TunnelOptions='[{TunnelInsideCidr=AWS_T1_IP,PreSharedKey=SHARED_SECRET_1},{TunnelInsideCidr=AWS_T2_IP,PreSharedKey=SHARED_SECRET_2}]'

    aws ec2 create-vpn-connection \
        --type ipsec.1 \
        --customer-gateway-id CUSTOMER_GATEWAY_2 \
        --vpn-gateway-id VPN_GATEWAY_ID \
        --options TunnelOptions='[{TunnelInsideCidr=AWS_T3_IP,PreSharedKey=SHARED_SECRET_3},{TunnelInsideCidr=AWS_T4_IP,PreSharedKey=SHARED_SECRET_4}]'
```
- Transit gateway:
  
  Create a VPN connection with dynamic routing between the transit gateway and the customer gateways:
```
aws ec2 create-vpn-connection \
    --type ipsec.1 \
    --customer-gateway-id CUSTOMER_GATEWAY_1 \
    --transit-gateway-id TRANSIT_GATEWAY_ID \
    --options TunnelOptions='[{TunnelInsideCidr=AWS_T1_IP,PreSharedKey=SHARED_SECRET_1},{TunnelInsideCidr=AWS_T2_IP,PreSharedKey=SHARED_SECRET_2}]'

aws ec2 create-vpn-connection \
    --type ipsec.1 \
    --customer-gateway-id CUSTOMER_GATEWAY_2 \
    --transit-gateway-id TRANSIT_GATEWAY_ID \
    --options TunnelOptions='[{TunnelInsideCidr=AWS_T3_IP,PreSharedKey=SHARED_SECRET_3},{TunnelInsideCidr=AWS_T4_IP,PreSharedKey=SHARED_SECRET_4}]'
```
  Replace the following:
  - CUSTOMER_GATEWAY_1: Google Cloud VPN gateway, interface 0
  - CUSTOMER_GATEWAY_2: Google Cloud VPN gateway, interface 1
  - AWS_T1_IP: Inside IP address for virtual private gateway for connection 1, tunnel 1
  - AWS_T2_IP: Inside IP address for virtual private gateway for connection 1, tunnel 2
  - AWS_T3_IP: Inside IP address for virtual private gateway for connection 2, tunnel 1
  - AWS_T4_IP: Inside IP address for virtual private gateway for connection 2, tunnel 2
  - SHARED_SECRET_1: Pre-shared key for connection 1, tunnel 1
  - SHARED_SECRET_2: Pre-shared key for connection 1, tunnel 2
  - SHARED_SECRET_3: Pre-shared key for connection 2, tunnel 1
  - SHARED_SECRET_4: Pre-shared key for connection 2, tunnel 2
  AWS reserves some CIDR ranges, so you can't use values in these ranges as inside IP addresses (AWS_T1_IP, AWS_T2_IP, AWS_T3_IP, AWS_T4_IP). For information about what CIDR blocks AWS reserves, see Inside tunnel IPv4 CIDR.
  
  These commands create four tunnels to Google Cloud.
Download the configuration files for the two VPN connections.

In subsequent steps, you use values from the configuration files to create and configure resources on the Google Cloud side.

Create VPN tunnels and Cloud Router interfaces on Google Cloud

In this section, you use the information from the AWS VPN connections that you created in the previous section to create and configure components on Google Cloud.

When configuring VPN tunnels to AWS, use the IKEv2 encryption protocol and select fewer transform sets on the AWS side. For example, select a combination of single Phase 1 and Phase 2 encryption algorithms, integrity algorithms, and Diffie-Hellman (DH) group numbers. Otherwise, the Cloud VPN tunnel can fail to rekey because of the large security association (SA) payload size for the default AWS transform sets. This large payload size can cause IP fragmentation of IKE packets on the AWS side, which Cloud VPN does not support. For more information, see Tunnel options for your Site-to-Site VPN connection.

In Cloud Shell, create an external VPN gateway with four interfaces for the AWS outside IP addresses:
```
gcloud compute external-vpn-gateways create PEER_GATEWAY_NAME --interfaces \
  0=AWS_GW_IP_1,1=AWS_GW_IP_2,2=AWS_GW_IP_3,3=AWS_GW_IP_4
```
Replace the following:
- AWS_GW_IP_1: Outside IP address for virtual private gateway for connection 1, tunnel 1
- AWS_GW_IP_2: Outside IP address for virtual private gateway for connection 1, tunnel 2
- AWS_GW_IP_3: Outside IP address for virtual private gateway for connection 2, tunnel 1
- AWS_GW_IP_4: Outside IP address for virtual private gateway for connection 2, tunnel 2

Create four VPN tunnels:

Tunnel 1:

gcloud compute vpn-tunnels create tunnel-1 \
    --peer-external-gateway PEER_GATEWAY_NAME \
    --peer-external-gateway-interface 0 \
    --region REGION \
    --ike-version IKE_VERSION \
    --shared-secret SHARED_SECRET_1 \
    --router ROUTER_NAME \
    --vpn-gateway HA_VPN_GATEWAY_NAME \
    --interface 0

Tunnel 2:

gcloud compute vpn-tunnels create tunnel-2 \
    --peer-external-gateway PEER_GATEWAY_NAME \
    --peer-external-gateway-interface 1 \
    --region REGION \
    --ike-version IKE_VERSION \
    --shared-secret SHARED_SECRET_2 \
    --router ROUTER_NAME \
    --vpn-gateway HA_VPN_GATEWAY_NAME \
    --interface 0

Tunnel 3:

gcloud compute vpn-tunnels create tunnel-3 \
    --peer-external-gateway PEER_GATEWAY_NAME \
    --peer-external-gateway-interface 2 \
    --region REGION \
    --ike-version IKE_VERSION \
    --shared-secret SHARED_SECRET_3 \
    --router ROUTER_NAME \
    --vpn-gateway HA_VPN_GATEWAY_NAME \
    --interface 1

Tunnel 4:

gcloud compute vpn-tunnels create tunnel-4 \
    --peer-external-gateway PEER_GATEWAY_NAME \
    --peer-external-gateway-interface 3 \
    --region REGION \
    --ike-version IKE_VERSION \
    --shared-secret SHARED_SECRET_4 \
    --router ROUTER_NAME \
    --vpn-gateway HA_VPN_GATEWAY_NAME \
    --interface 1

Create four Cloud Router interfaces.

In the following commands, replace each GOOGLE_BGP_IP_TUNNEL placeholder with the tunnel's inside IP address on the Google Cloud side. You can find the values in the AWS VPN configuration files as the customer gateway address for each tunnel. Each of these addresses must be in the /30 CIDR range within the 169.254.0.0/16 network range.

Cloud Router interface 1:

gcloud compute routers add-interface ROUTER_NAME \
    --interface-name int-1 \
    --vpn-tunnel tunnel-1 \
    --ip-address GOOGLE_BGP_IP_TUNNEL_1 \
    --mask-length 30 \
    --region REGION

Cloud Router interface 2:

gcloud compute routers add-interface ROUTER_NAME \
    --interface-name int-2 \
    --vpn-tunnel tunnel-2 \
    --ip-address GOOGLE_BGP_IP_TUNNEL_2 \
    --mask-length 30 \
    --region REGION

Cloud Router interface 3:

gcloud compute routers add-interface ROUTER_NAME \
    --interface-name int-3 \
    --vpn-tunnel tunnel-3 \
    --ip-address GOOGLE_BGP_IP_TUNNEL_3 \
    --mask-length 30 \
    --region REGION

Cloud Router interface 4:

gcloud compute routers add-interface ROUTER_NAME \
    --interface-name int-4 \
    --vpn-tunnel tunnel-4 \
    --ip-address GOOGLE_BGP_IP_TUNNEL_4 \
    --mask-length 30 \
    --region REGION

Add BGP peers.

In the following commands, replace PEER_ASN with the ASN for the AWS side of the BGP session.

AWS connection 1, tunnel 1

gcloud compute routers add-bgp-peer ROUTER_NAME \
    --peer-name aws-conn1-tunn1 \
    --peer-asn PEER_ASN \
    --interface int-1 \
    --peer-ip-address AWS_T1_IP \
    --region REGION

AWS connection 1, tunnel 2

gcloud compute routers add-bgp-peer ROUTER_NAME \
    --peer-name aws-conn1-tunn2 \
    --peer-asn PEER_ASN \
    --interface int-2 \
    --peer-ip-address AWS_T2_IP \
    --region REGION

AWS connection 2, tunnel 1

gcloud compute routers add-bgp-peer ROUTER_NAME \
    --peer-name aws-conn2-tunn1 \
    --peer-asn PEER_ASN \
    --interface int-3 \
    --peer-ip-address AWS_T3_IP \
    --region REGION

AWS connection 2, tunnel 2

gcloud compute routers add-bgp-peer ROUTER_NAME \
    --peer-name aws-conn2-tunn2 \
    --peer-asn PEER_ASN \
    --interface int-4 \
    --peer-ip-address AWS_T4_IP \
    --region REGION

Verify the configuration

In Cloud Shell, verify the Cloud Router status:

gcloud compute routers get-status ROUTER_NAME \
    --region REGION \
    --format='flattened(result.bgpPeerStatus[].name, result.bgpPeerStatus[].ipAddress, result.bgpPeerStatus[].peerIpAddress)'

The output is similar to the following:

result.bgpPeerStatus[].peerIpAddress)'
result.bgpPeerStatus[0].ipAddress:     169.254.171.18
result.bgpPeerStatus[0].name:          aws-conn1-tunn1
result.bgpPeerStatus[0].peerIpAddress: 169.254.171.17
result.bgpPeerStatus[1].ipAddress:     169.254.156.154
result.bgpPeerStatus[1].name:          aws-conn1-tunn2
result.bgpPeerStatus[1].peerIpAddress: 169.254.156.153
result.bgpPeerStatus[2].ipAddress:     169.254.123.38
result.bgpPeerStatus[2].name:          aws-conn2-tunn1
result.bgpPeerStatus[2].peerIpAddress: 169.254.123.37
result.bgpPeerStatus[3].ipAddress:     169.254.48.186
result.bgpPeerStatus[3].name:          aws-conn2-tunn2
result.bgpPeerStatus[3].peerIpAddress: 169.254.48.185

List all tunnels:

gcloud compute vpn-tunnels list

The output is similar to the following:

NAME      REGION    GATEWAY    PEER_ADDRESS
tunnel-1  us-east4  ha-vpn-gw  34.205.x.x
tunnel-2  us-east4  ha-vpn-gw  52.203.x.x
tunnel-3  us-east4  ha-vpn-gw  3.208.x.x
tunnel-4  us-east4  ha-vpn-gw  52.204.x.x

Check the tunnel status:

gcloud compute vpn-tunnels describe tunnel-1 \
     --region REGION \
     --format='flattened(status,detailedStatus)'

The output is similar to the following:

detailed_status: Tunnel is up and running.
status:          ESTABLISHED

List dynamic routes learned by Cloud Router:

gcloud compute routers get-status ROUTER_NAME \
    --region REGION \
    --format="flattened(result.bestRoutes)"

The output is similar to the following:

result.bestRoutes[0].creationTimestamp: 2021-01-19T20:42:07.366-08:00
result.bestRoutes[0].destRange:         10.2.2.0/24
result.bestRoutes[0].kind:              compute#route
result.bestRoutes[0].nextHopIp:         169.254.171.17
result.bestRoutes[0].priority:          100
result.bestRoutes[1].creationTimestamp: 2021-01-19T20:42:07.366-08:00
result.bestRoutes[1].destRange:         10.2.2.0/24
result.bestRoutes[1].kind:              compute#route
result.bestRoutes[1].nextHopIp:         169.254.156.153
result.bestRoutes[1].priority:          100
result.bestRoutes[2].creationTimestamp: 2021-01-19T20:56:26.588-08:00
result.bestRoutes[2].destRange:         10.2.2.0/24
result.bestRoutes[2].kind:              compute#route
result.bestRoutes[2].nextHopIp:         169.254.123.37
result.bestRoutes[2].priority:          100
result.bestRoutes[3].creationTimestamp: 2021-01-19T20:56:26.588-08:00
result.bestRoutes[3].destRange:         10.2.2.0/24
result.bestRoutes[3].kind:              compute#route
result.bestRoutes[3].nextHopIp:         169.254.48.185
result.bestRoutes[3].priority:          100

Test connectivity

Create test VMs on each side of the tunnels to test ping requests.

Make sure that you have firewall rules in place to allow ICMP traffic.
- For instructions to create VMs in Compute Engine, see the getting started guide.
- For instructions to create VMs on AWS, see Launch a virtual machine.
Test the connection using the ping command.
Measure bandwidth between your test machines using iperf.
- Server side:
```
iperf3 -s
```
- Client side:
```
iperf3 -c SERVER_IP_ADDRESS -P NUMBER_OF_PARALLEL_SESSIONS
```

Clean up

Delete the Google Cloud and AWS resources that you have created during this tutorial.

Delete the Google Cloud project

To avoid incurring charges to your Google Cloud account for the resources used in this tutorial, you can delete your project:

In the Google Cloud console, go to the Manage resources page.
Go to Manage resources
In the project list, select the project that you want to delete, and then click Delete.
In the dialog, type the project ID, and then click Shut down to delete the project.

Delete AWS resources

What's next

Learn more about Google Cloud VPN.
Learn more about best practices and reference architectures for VPC design.