Building a cross-cloud architecture for Spanner

Spanner is a fully managed database service for both relational and non-relational workloads that offers strong consistency at global scale, high performance at virtually unlimited scale, and high availability with up to five 9s SLA. And as cross-cloud and hybrid cloud strategies gain traction, customers are increasingly seeking the flexibility to utilize Spanner's exceptional database capabilities while running their compute stack on a third-party cloud provider.

Key considerations for such cross-cloud setups include:

• Latency: Customers want their application response times to be low for OLTP workloads.
• Security: With data traversing cloud boundaries and the sensitivity of information stored in Spanner, secure data transfer is paramount.
• Availability: Spanner boasts a high SLA, which our customers appreciate. The cross-cloud setup must similarly maintain a high SLA.
• Ease of setup: Establishing cross-cloud access that addresses the first three areas can be complex.
• Cost effectiveness: With data traffic flowing between clouds, the cross-cloud solution must be cost-effective.

This blog explores how to use Google Cloud’s Cross-Cloud Interconnect to achieve the aforementioned goals.

Cross-Cloud Interconnect

In the simplest model, applications can connect to Spanner at spanner.googleapis.com over the public internet. However, customers with increased reliability, latency, security and privacy requirements prefer to connect their workloads to their database environments privately.

For application owners who leverage best-of-breed services from different public cloud providers, they need seamless, robust network connectivity across multiple environments. Google Cloud recently introduced Cross-Cloud Interconnect to its network connectivity portfolio to simplify the consumption of differentiated Google Cloud Services like Spanner, from different public cloud environments.

Cross-Cloud Interconnect provides:

• Simplicity, as it does not require the customer to procure, set up or manage any physical infrastructure in colocation facilities.
• High availability, with up to 99.99% connectivity SLA to the third-party cloud’s doorstep.
• Enterprise grade performance, with:
  • Lower inter-cloud latency, providing comparable performance to single provider, cross-zone setups.
  • Dedicated bandwidth to the third-party cloud environment with 10Gbps and 100Gbps dedicated options
  • Fast provisioning times enabling setting up end-to-end, cross-cloud connectivity in a few days
• Lower cost of ownership, as it allows customers to benefit from lower data transfer costs over Interconnect, rather than internet data transfer fees.

In this blog, we cover different private connectivity options, and walk you through how to set up cross-cloud connectivity to Spanner from a third-party public cloud environment, as shown in the diagram below:

Connectivity options

You have a variety of options to connect your on-premises or public cloud environments to your Google Cloud VPCs:

• HA VPN, offering the ability to build secure overlay tunnels over the public internet from on-prem and public cloud environments. This is the recommended solution for customers with low data transfer volumes, and no strict requirement for private underlays.
• Cloud Interconnect, offering the ability to use private circuits to connect to on-prem and public cloud environments. You can choose from different Cloud Interconnect flavors:
  • Dedicated Interconnect: providing dedicated, direct physical connections between a customer’s on-prem network and their Google VPC network. It enables customers to transfer large amounts of data between networks, which can be more cost-effective than purchasing additional bandwidth over the public internet. In order to use this option, a customer’s network should physically meet Google's network in a colocation facility. This is the recommended solution for customers that want to connect their on-premises environment to Google Cloud with strict requirements for private underlays, high bandwidth connectivity, and the ability to extend their network to meet Google at a Cloud Interconnect Point of Presence.
  • Partner Interconnect: providing connectivity between a customer’s on-premises network and their Google VPC network through a supported service provider. A Partner Interconnect can be useful in many networking situations, and is most commonly used when the customer’s bandwidth requirements don't warrant an entire 10 Gbps connection. This is the recommended solution for customers that want to connect their on-prem environment to Google Cloud with strict requirements for private underlays and sub-10Gbps connection speeds.
  • Cross-Cloud Interconnect: providing dedicated, direct physical connections between a customer’s network in a third-party public cloud and their Google Cloud VPC network. It enables customers to transfer large amounts of data between cloud environments, at lower latency, which is more cost-effective than transiting over the public internet. This is the recommended solution for customers that want to connect their third-party cloud environment to Google Cloud with strict requirements for private underlays, low latency, high bandwidth connectivity, and the ability to extend their network to meet Google at a Cloud Interconnect Point of Presence.

You can then use private endpoints to connect on-prem or public clouds to Google APIs and services. For more details and options, please refer to Google Cloud’s connectivity documentation. We go into how to do this for Spanner in the next section.

Setting up cross-cloud connectivity to Spanner

Establishing cross-cloud connectivity from a third-party cloud provider to Spanner is a straightforward process. The following section provides a step-by-step guide on setting up this connection.

1. Physical connectivity

The first thing you will need to set up is a Virtual Private Cloud (VPC) network that you can connect to from other clouds. You can think of a VPC network as similar to a physical network, except that it is virtualized within Google Cloud. It consists of a list of regional virtual subnetworks (subnets) in data centers, all connected by a global wide area network. A VPC also enables you to privately connect networks outside of Google to Google Cloud products.

Similarly, you will need to have a VPC network on the other Cloud Provider into which to terminate connectivity.

Next, you can connect your Google Cloud VPC network to your third-party public cloud environment through Cross-Cloud Interconnect. Go to the Interconnect console and choose Cross-Cloud Interconnect to order a new Cross-Cloud connection:

Next, you can choose the public cloud provider you’d like to connect to, along with the location and capacity.

If you are using regional Spanner, then you should build the connection in the same location as the Spanner region to get the best latency. For multi-regional Spanner, we recommend setting up Interconnects in all the read-write and read-only regions. For example, if you use nam3 configuration, you can set up interconnects in us-east4 and us-east1 regions.

Similarly, you will need to order private circuits from the third-party public cloud to which you’d like to connect. Detailed instructions can be found here.

VLAN attachment and Cloud Routers

Once you’ve set up the physical connections, you can create VLAN attachments, which create the logical connection you need to reach your gateways in the third-party public cloud environment. They also determine which subnetworks can reach your peer networks through the Cross-Cloud Interconnect connection using Cloud Routers. A Cloud Router enables you to dynamically exchange routes between your VPC and peer network by using Border Gateway Protocol (BGP). It automatically learns new subnet IP address ranges in your VPC network and announces them to your peer network.

When you create a VLAN attachment, you should specify a Cloud Router that's in the same region that you are interconnecting with Google. The VLAN attachment automatically allocates a VLAN ID and BGP peering IP addresses. You can use that information to configure your third-party public cloud peer router and establish a BGP session with your Cloud Router.

To achieve 99.99% availability, we recommend setting up at least four Cross-Cloud Interconnects — two connections in one region and two connections in another region. Interconnect connections that are in the same region must be placed in different availability zones. Similarly, we recommend setting up at least two Cloud Routers in at least two distinct Google Cloud regions. This ensures that, in the case of failures, Google Cloud can reroute traffic through the other region to Spanner. Each Cloud Router must be attached to a pair of Interconnect connections in a region. The dynamic routing mode for the VPC network must be global so that Cloud Router can advertise all subnets in the VPC and propagate learned routes to all subnets regardless of the subnet's region.

Next, you will need to create four VLAN attachments — one for each interconnect to add redundancy. The attachments in a region should use the Cloud Router configured in that region.

For high availability, we recommend that you have two routers in your peer network, each router connecting to one of the Google Cloud regions. With this, there are multiple paths for traffic to traverse from the VPC network to your peer network.

Detailed instructions can be found here.

Private endpoints

You can use private endpoints to connect third-party public cloud environments to Google APIs and services. By default, applications will connect to spanner.googleapis.com. For better reliability, performance, lower latency and total cost of ownership, we recommend leveraging private endpoints with Private Service Connect.

With Private Service Connect, you can create private endpoints using global internal IP addresses within your VPC network. You can assign DNS names to these internal IP addresses with meaningful names, for example, spanner.p.googleapis.com. These names and IP addresses are internal to your VPC network and any on-premises networks that are connected to it, and you can control which traffic goes to which endpoint.

Recap

To recap, these are the steps needed to set up a highly-available, cross-cloud architecture across Spanner and applications that are on other public cloud providers.

1. Set up a VPC network that you can use to connect from on-premises or other clouds.
2. Use Cross-Cloud Interconnect to get high-speed, highly available physical connectivity between your VPC and third party public cloud networks.
3. Create VLAN attachments and Cloud routers to enable your VPC network to reach your peer network.
4. Use private endpoints to enable peer hosts to connect to Google APIs and services.

Learn more

• If you are new to Spanner, try Spanner at no charge with a 90-day free trial instance.
• Learn more about multi-regional Spanner configurations by reading Demystifying Cloud Spanner multi-region configurations.
• Learn more about why customers are using Cross-Cloud Interconnect for their connectivity needs.