Patterns for connecting other cloud service providers with GCP

This document helps cloud architects and operations professionals decide how to connect Google Cloud Platform (GCP) with other cloud service providers (CSP) such as Amazon Web Services (AWS) and Microsoft Azure. In a multi-cloud design, these connections allow data transfers between Virtual Private Cloud (VPC) networks. This document also provides information on how to implement the option that you choose.

Many organizations operate on multiple cloud platforms, either as a temporary measure during a migration or because the organization has a long-term multi-cloud strategy.

For data exchange between GCP and other CSPs, there are multiple options discussed in this document:

These options differ in terms of transfer speed, latency, reliability, service level agreements (SLAs), complexity, and costs. This document describes each option and its advantages and disadvantages in detail and ends with a comparison of all of the options.

This document covers data transfers between virtual machines residing in GCP and other CSP's VPCs. For data stored in other GCP products such as Cloud Storage and BigQuery, see the section covering these products.

This document can act as a guide to evaluate the options to transfer data between GCP and one or more other CSPs, based on your requirements and capabilities.

The concepts in this document apply in multiple cases:

  • When you are planning to transfer large amounts of data for a short period of time, for example, for a data migration project.
  • If you run continuous data transfers between multiple cloud providers, for example, because your compute workloads run on another CSP while your big data workloads use GCP.

Initial considerations

Before you choose how to connect your cloud environments, it's important that you look at which regions you choose in each environment and that you have a strategy for transferring data that doesn't reside in VPC environments.

Choose cloud regions

If both GCP and other CSP's resources are in regions that are geographically close to each other, there is a cost and latency advantage for data transfers between cloud providers.

The following diagram illustrates the flow of data between GCP and other CSPs. Architecture of data flowing between GCP and other CSPs.

Regardless of the method of transfer, data flows in the following directions:

  • From the GCP region where resources are hosted to Google's edge POP.
  • Through a third-party facility between GCP and the other CSP.
  • From the other CSP's edge POP to the region where resources are located inside the other CSP's network.
  • Along the same paths mentioned previously in the list, but in the opposite direction. For example, from GCP to the other CSP as well as from the other CSP to GCP.

The end-to-end path determines the latency of the data transfer. For some solutions, the network costs also increase when the edge POPs of both providers aren't in one metropolitan area. Details are listed in the following pricing section of each solution.

Make sure you choose suitable regions in each cloud that can host your intended workloads. Visit the Locations page for GCP and a similar pages for other CSPs, such as the AWS region table or Azure products by region. For general help in selecting one or multiple locations in GCP, review Best practices for Compute Engine region selection.

Transfer of data in Cloud Storage and BigQuery

Only data residing inside GCP's VPC environment passes a Cloud VPN tunnel or a Cloud Interconnect connection by default.

If you want to transfer data to and from other Google services, you can use Private Google Access for on-premises hosts from the other CSP's environment.

If you want to transfer another CSP's object storage, database, data warehouse, or other products, check their documentation to see if data can pass through their interconnect or managed VPN product. Otherwise, you might be able to pass this data through proxy virtual machines that you set up in the respective CSP's environment to make it pass through the connection you want.

For transferring data to Cloud Storage or BigQuery, you can also use Storage transfer service or BigQuery transfer service.

Transfer through external IP addresses over the internet

The easiest way to transfer data between GCP and another CSP is to use the internet and transfer the data by using external IP addresses.

The following diagram illustrates the elements for this solution.

Architecture of data transferring between GCP and another CSP through external IP address over the internet.

Between Google's network edge and the other CSP's network edge, data passes through the internet or uses a direct peering between GCP and the other CSP. Data can only pass between resources that have public IP addresses assigned.

Implementation

In this setup, all virtual machines that transfer data between GCP and other cloud service providers must have a public IP address. On one end, the firewall must be opened to allow a connection from the public IP address of the other cloud provider. No extra steps are necessary because the data exchange happens over the existing internet connectivity.

Transfer speed and latency

While there is no guaranteed speed and latency on the path through the internet, typically, major CSPs and Google exchange data directly in multiple locations around the world. Capacity is shared with other customers and services, but, often due to the direct connection between both providers, latency is similar or lower than other options.

We recommend that you test the latency and bandwidth between GCP and the other CSPs in your chosen regions. You can do a quick benchmark by using tools such as iperf or netperf, or run a more complete custom benchmark based on your app. While conditions might change over time, the benchmark can provide an indication of the performance you can expect and if this solution fulfills your needs. If you require a dedicated bandwidth between both environments, you should choose another solution.

Security

Traffic over the internet isn't encrypted and might pass through third-party internet service providers (ISPs), autonomous systems, and facilities. Therefore, you should encrypt sensitive traffic at the application layer or choose another solution.

Reliability and SLA

GCP generally has multiple diverse paths for internet connectivity from regions, and there are direct peering connections with other major CSPs at multiple locations around the globe.

However, GCP doesn't provide any SLAs for connectivity to other CSPs over the internet. While you should check SLAs for your other CSPs, they typically only refer to internet connectivity as a whole and not specific providers.

Pricing

For data transfers over the internet, normal internet egress rates apply for traffic leaving GCP. In cases where latency isn't critical, using the Standard Network Service Tier provides a lower pricing tier.

The other CSPs have their own charges for data transfers. In many cases, they only charge for traffic egressing their network. Consult your CSP's price list for example for AWS, see EC2 data transfer charges and for Azure, see Bandwidth Pricing Details.

Managed VPN between cloud providers

You can use managed VPN services from both cloud providers to have an encrypted channel between VPCs in both cloud environments and to transfer data by using private IP addresses. This is an extension to the previous solution of transferring over the internet without requiring any hardware or partners.

The following diagram illustrates the elements for this solution.

Architecture of data transferring between GCP and another CSP by using a managed VPN,

Using this solution, data is encrypted on GCP using Cloud VPN and a VPN solution for the other CSP. The data transfer between GCP and the other CSP uses the internet like in the previous solution.

Implementation

Google offers Cloud VPN as a managed VPN service for encrypted IPsec tunnels, which can be used on the Google end. Other CSPs offer their own managed VPN products, which you can use to build IPsec VPN tunnels between both environments. For example, AWS offers AWS Site-to-Site VPN and Azure offers Azure VPN gateway. You can connect your VPCs between the environments by using one or multiple VPN tunnels.

IP addresses between both environments shouldn't overlap because there is no Network address translation (NAT) functionality available. On Cloud Router, overlapping routes could be removed from being exchanged between environments, but then no communication between the services using these IP addresses is possible.

With Cloud Router in global dynamic routing mode, you can reach all regions in a global VPC network with a single VPN tunnel, but for other CSPs you might need one VPN tunnel per region. If you have multiple VPCs in a cloud environment that aren't peered, you have to connect all VPCs that need to communicate with each other independently by using a VPN.

GCP offers interoperability guides, which have step-by-step instructions for setting up VPN tunnels to other major cloud providers:

Transfer speed and latency

When using managed VPN tunnels, data still follows the same internet path as if data was directly exchanged by using public IP addresses. The latency observed should be similar to the previous option with only a small latency overhead for the VPN tunnel. The available bandwidth is limited by the maximum bandwidth per VPN tunnel on GCP (1.5 Gbps), the other CSP, and the available bandwidth over the internet path, whichever is lowest.

For higher bandwidth, you can deploy multiple tunnels in parallel. For more information on how to deploy such a solution, see Building high-throughput VPNs.

You can test latency and bandwidth as described in the last section, but conditions might vary over time, and there are no guarantees on latency or bandwidth.

Security

Traffic over IPsec VPN tunnels is encrypted by using ciphers accepted by both CSPs. For more information, see Cloud VPN supported IKE ciphers, AWS VPN FAQ, and Azure VPN IPsec/IKE parameters.

Reliability and SLA

Cloud VPN offers an SLA of 99.9%-99.99% depending on if Classic VPN or HA VPN is selected. Other CSPs sometimes offer SLAs on their managed VPN product, for example, AWS Site-to-Site VPN SLA and Azure SLA for VPN Gateway. However, these SLAs only cover the VPN gateways availability and don't include connectivity to other CSPs over the internet, so there is no SLA on the end-to-end solution.

To increase reliability, consider using multiple VPN gateways and tunnels on both GCP and the other CSPs.

Pricing

For the managed VPN service, charges apply. For GCP, an hourly charge applies see Cloud VPN pricing. For other CSPs, consult their price lists, for example, see AWS Site-to-Site VPN connection pricing or Azure VPN Gateway pricing.

In addition to hourly pricing for the VPN service, you have to pay for data transferred through the VPN gateways. For GCP and many CSPs, standard internet data transfer charges apply as detailed in the previous section. In many cases, data transfer charges exceed the fixed cost for this solution.

Partner Interconnect with multi-cloud enabled partners

Partner Interconnect lets you connect a Virtual Private Cloud to other CSP's VPCs through the network of select partners that offer direct multi-cloud solutions. You connect by deploying one or more virtual routing instances that take care of the necessary Border Gateway Protocol (BGP) setup.

The following diagram shows a redundant setup using two Partner Interconnect connections.

Architecture of data transferring between GCP and another CSP by using two Partner Interconnect.

Routes are exchanged between Cloud Router and the gateway on the other CSP's side through a virtual routing instance that is managed by the partner providing the interconnect. Traffic flows through the partner network between GCP and the other CSP.

Implementation

This solution requires you to set up multiple components:

  • On the GCP side, you set up Partner Interconnect with an interconnect service provider that's serving your GCP regions and offering multi-cloud connectivity to the other CSP.
  • On the other CSP, you have to use their interconnect product to connect to the same partner. For example, on AWS you can use Direct Connect and on Azure you can use ExpressRoute.
  • On the service provider partner side, you have to configure the virtual routing equipment providing the BGP sessions to GCP and to the other CSP.

IP address space between both CSP environments cannot overlap because there is no NAT functionality available.

Google offers implementation guides with step-by-step instructions for some partners, for example, Connecting multi-cloud VPCs with Megaport Cloud Router.

Transfer speed and latency

This solution offers dedicated capacity between GCP and other CSPs. Depending on the partner and the other CSP, the available attachment capacity might vary. On the GCP side, Partner Interconnect is available with an attachment capacity between 50 Mbps and 10 Gbps.

Latency for this solution is the sum of the following:

  • Latency between the region in which your resources are hosted on GCP and the interconnect location where the partner connects to GCP.
  • Latency in the partners network to, from, and through the virtual routing instance towards the other CSP.
  • Latency from the other CSP's edge location where the interconnect with the partner takes place to the region where the resources are hosted in the CSP.

For the lowest possible latency, the edge locations of GCP and the other CSP should be in the same metropolitan area, along with the virtual routing instance. For example, you might have a low latency connection if both CSP's cloud regions as well as the edge POP and the virtual routing instance are located in the Ashburn, Virginia area.

While GCP and many other CSPs offer no latency guarantees for traffic towards their network edge, because there is a dedicated path and capacity through the partner, typically the latency in this solution is less variable than the previous options.

Security

Because traffic over Partner Interconnect isn't encrypted, the use of application layer encryption for sensitive traffic is recommended.

If all traffic over Partner Interconnect needs to be encrypted, virtual appliances available from GCP partners in the GCP Marketplace can provide solutions for encryption on the GCP side. Cloud VPN cannot be used over Cloud Interconnect – Dedicated or Partner Interconnect. Some other CSPs allow you to use their managed VPN service over an interconnect, for example, AWS Site-to-Site VPN can be used over AWS Direct Interconnect. Alternatively, you can also use a virtual appliance encrypting the traffic on the other CSPs end.

Reliability and SLA

Depending on the other CSP and the Partner Interconnect SLAs, this solution is the only option that lets you have an SLA for each part of the connectivity solution.

When using Partner Interconnect redundantly, Google offers 99.9% - 99.99% monthly SLAs depending on the topology chosen. There is no SLA on a single Partner Interconnect connection.

See the other CSP's documentation for the SLA on their interconnect product, for example, AWS Direct Connect SLA or on Azure SLA for ExpressRoute.

Consult the documentation or terms of the service provider for the Partner Interconnect for their SLA on the availability of the connectivity and virtual routing instance. For example, see the Megaport Global Services Agreement.

Pricing

On the GCP side, there is a monthly fee for each Partner Interconnect attachment, depending on the bandwidth. Traffic egressing through the Partner Interconnect gets charged at a lower rate than internet traffic. For more information, see the Partner Interconnect pricing page.

Consult the other CSP's pricing page for their interconnect product, for example AWS Direct Connect pricing or Azure ExpressRoute pricing. Typically, pricing also has a monthly charge for the interconnect and data transfer charges through the interconnect at a lower rate than over the internet.

The partner provides interconnect services charges according to their own pricing, which can be found on their website or by consulting their sales team for a quote. Typically, if all data transfers happen in the same metropolitan area, charges are much lower than if data has to travel a longer distance on a partner network.

When data is transferred regularly at sufficiently high volumes, depending on the other CSP's prices, this solution can sometimes offer the lowest total cost due to the discounted egress rates. Even when adding in monthly costs for the interconnect for Partner Interconnect, the other CSP, and the service provider partner, using this solution can provide significant savings. As partners and other CSP's prices can change without notice, make your own comparison using up-to-date quotes from all involved parties.

Dedicated Interconnect through a common POP

By using one or more physical routing devices at a common interconnect facility between GCP and the other CSP, you can connect both cloud providers by using Dedicated Interconnect on the GCP side and an equivalent product at the other CSP. The interconnect location isn't necessarily at the same location as the region in which your resources are located.

The following diagram shows a redundant setup using two Dedicated Interconnect connections:

Architecture of data transferring between GCP and another CSP by using two Dedicated Interconnect connections.

Routes are exchanged between Cloud Router and the gateway on the other CSP's side through a physical router that you place in a common interconnect facility. Traffic flows through this router between GCP and the other CSP.

Implementation

This solution requires that you host and maintain physical routing devices at a colocation facility where Google and the CSP you want to connect to are present. From this routing device, you order two physical connections with the facility: one toward GCP using Dedicated Interconnect, and one towards the other service provider using an equivalent product, for example, AWS Direct Connect or Azure ExpressRoute. On your routing device, you need to configure BGP to allow route exchanges between the two cloud environments.

Due to various factors, including incompatibilities during link setup as well as in BGP parameters, it isn't possible to directly connect the cloud environments in a physical facility without active routing equipment in-between.

Check the Colocation facility locations list from GCP and your other CSP, for example AWS Direct Connect Locations or Azure ExpressRoute peering locations, to identify suitable locations for this setup.

Typically IP address space allocation between both providers' VPCs shouldn't overlap, unless your physical routing device is able to perform NAT between the environments or you restrict some route exchanges between the environments.

This solution is effective if you use the dedicated connectivity to also connect back to your on-premises environment, in addition to the connection to another CSP.

In other cases, this solution is complex because it requires you to own and maintain physical equipment and have a contract with a colocation facility. We only recommend this solution if at least one of the following is true:

  • You already have equipment in a suitable facility for another purpose and an existing contract with the facility.
  • You transfer large amounts of data regularly so this is a cost effective option or you have bandwidth requirements that partners cannot fulfill.

Transfer speed and latency

This solution offers dedicated capacity between GCP and another CSP. On the GCP side, Dedicated Interconnect is available by using one or multiple 10 or 100 Gbps physical connections.

The latency for this solution is a sum of the following:

  • Latency between the region in which your resources are hosted on GCP and the interconnect location where you interconnect with GCP.
  • Latency through the facility and your physical equipment, which is usually negligible when compared with any latency due to fiber length.
  • Latency from the interconnect location through the other CSP's network to the region where the resources are hosted in the CSP.

While no latency guarantees are offered, this solution typically allows for the lowest latency and highest transfer speeds between GCP and other cloud environments over private IP addresses.

Security

Because traffic over Dedicated Interconnect isn't encrypted, we recommend using application layer encryption for sensitive traffic.

If all traffic needs to be encrypted, like with the previous solution, virtual appliances available from GCP partners in the GCP Marketplace can provide solutions for encrypting the traffic to the other CSP's environment.

Reliability and SLA

When using Dedicated Interconnect redundantly, Google offers 99.9%-99.99% monthly SLAs depending on the chosen topology. There is no SLA on a single Dedicated Interconnect connection.

For more information, see the other CSP's documentation for the SLA on their interconnect product, for example, AWS Direct Connect SLA or Azure SLA for ExpressRoute.

The colocation facility or hardware vendor for the physical routing equipment might also offer SLAs on their services.

Pricing

On the GCP side, there is a monthly fee for each Dedicated Interconnect port as well as for each VLAN attachment connecting to a VPC environment. Traffic egressing through Dedicated Interconnect is charged at a lower rate than internet traffic. For more information, see the Dedicated Interconnect pricing page.

Consult the other CSP's pricing page for their interconnect product, for example, AWS Direct Connect pricing or Azure ExpressRoute pricing. Typically, pricing also has a monthly charge for the interconnect and data transfer charges through the interconnect at a lower rate than over the internet.

In addition, you need to factor in charges for the colocation facility services providing space, electrical power, and physical connections towards both cloud environments as well as any cost and ongoing service contracts with the vendor for the physical routing equipment. If the connection between both CSPs cannot happen in the same facility and you need to procure connectivity between facilities, pricing might be much higher for these services.

Comparison of options

The presented options vary in speed, availability, complexity, security, and pricing. You should evaluate all options thoroughly according to your needs.

The following diagram guides you through the process of choosing one of the solutions mentioned in this document through a simple flow chart.

Decision flow chart to help you determing which interconnect solution to choose.

Typically, we can recommend the following choices:

  • For many scenarios where data is exchanged occasionally or at low volume and transfers aren't critical, transferring data over the internet is the simplest option and still provides relatively low latency and high bandwidth.
  • If encryption or transfer of data using private IP addresses is required, consider using Cloud VPN and a managed VPN service on the other CSP's side.
  • If you transfer larger amounts of data, using Partner Interconnect with a multi-cloud enabled partner provides multiple benefits: Dedicated capacity, lower cost for data transfers, and depending on the topology, an SLA for each part of the solution.
  • If you connect your on-premises equipment to multiple clouds, using Dedicated Interconnect through a common POP is a common option. It comes with additional complexity of managing your own hardware and relationships with a colocation facility. Unless you already have the infrastructure in place, this solution is reserved to cases where typical data transfer rates are 10 Gbps or more.

What's next

Was this page helpful? Let us know how we did:

Send feedback about...