Hybrid load balancing overview

Cloud Load Balancing supports load-balancing traffic to endpoints that extend beyond Google Cloud, such as on-premises data centers and other public clouds that you can use hybrid connectivity to reach.

A hybrid strategy is a pragmatic solution for you to adapt to changing market demands and incrementally modernize your applications. This could be a temporary hybrid deployment to enable migration to a modern cloud-based solution or a permanent fixture of your organization's IT infrastructure.

Setting up hybrid load balancing also enables you to bring the benefits of Cloud Load Balancing's networking capabilities to services running on your existing infrastructure outside of Google Cloud.

Hybrid load balancing is supported on the following Google Cloud load balancers:

Use-case: Routing traffic to an on-premises location or another cloud

The simplest use case for this feature is routing traffic from a Google Cloud load balancer to Google Cloud, an on-premises location, or another cloud. Clients can originate traffic either from the public internet, from within Google Cloud, or from an on-premises client.

Public clients

You can use HTTP(S) Load Balancing to route traffic from external clients to a backend in Google Cloud, on-premises, or in another cloud. With HTTP(S) Load Balancing, you can also enable value-added networking capabilities for your services on-prem. You can:

  • Use Google's global edge infrastructure to terminate user connections closer to the user, thus decreasing latency.
  • Protect your service with Google Cloud Armor, an edge DDoS defense/WAF security product available to all services accessed through an external HTTP(S) load balancer.
  • Enable your service to optimize delivery using Cloud CDN. With Cloud CDN, you can cache content close to your users. Cloud CDN provides capabilities like cache invalidation and Cloud CDN signed URLs.
  • Use Google-managed SSL certificates. You can reuse certificates and private keys that you already use for other Google Cloud products. This eliminates the need to manage separate certificates.

The following diagram demonstrates a hybrid deployment with an external HTTP(S) load balancer.

Hybrid connectivity with HTTP(S) Load Balancing (click to enlarge)
Hybrid connectivity with External HTTP(S) Load Balancing (click to enlarge)

In this diagram, traffic from clients on the public internet enters your private on-premise or cloud network through a Google Cloud load balancer, such as our global external HTTP(S) load balancer. When traffic reaches the load balancer, you can apply network edge services such as Google Cloud Armor DDoS protection or Identity-Aware Proxy (IAP) user authentication.

How the request is routed (whether to a Google Cloud backend or to an on-premises/cloud endpoint) depends on how your URL map is configured. Depending on your URL map, the load balancer selects a backend service for the request. If the selected backend service has been configured with a hybrid connectivity NEG (used for non-Google Cloud endpoints only), the load balancer forwards the traffic across Cloud VPN or Cloud Interconnect to its intended destination.

Internal clients (within Google Cloud or on-premises)

You can also set up a hybrid deployment for clients internal to Google Cloud. In this case, client traffic originates from the Google Cloud VPC network, your on-premise network, or from another cloud, and is routed to endpoints that can be in Google Cloud, on-premises, or in another cloud. Note that the Internal HTTP(S) Load Balancing is a regional load balancer, which means that it can only route traffic to endpoints within the same GCP zone or region as the load balancer's resources.

The following diagram demonstrates a hybrid deployment with an internal HTTP(S) load balancer.

Hybrid connectivity with Internal HTTP(S) Load Balancing (click to enlarge)
Hybrid connectivity with Internal HTTP(S) Load Balancing (click to enlarge)

Use-case: Migrate to Cloud

Migrating an existing service to cloud enables you to free up on-prem capacity and reduce the cost and burden of maintaining on-prem infrastructure. You can temporarily set up a hybrid deployment that allows you to route traffic to both your current on-premises service and a corresponding Google Cloud service endpoint.

The following diagram demonstrates this setup with Internal HTTP(S) Load Balancing.

Migrate to Google Cloud (click to enlarge)
Migrate to Google Cloud (click to enlarge)

If you are using Internal HTTP(S) Load Balancing to handle internal clients, you can configure the Google Cloud load balancer to use weight-based traffic splitting to split traffic across the two services. Traffic splitting lets you start by sending 0% of traffic to the Google Cloud service and 100% to the on-premises service. You can then gradually increase the proportion of traffic sent to the Google Cloud service. Eventually, you send 100% of traffic to the Google Cloud service, and you can retire the on-premises service.

Hybrid architecture

This section describes the load balancing architecture and resources required to configure a hybrid load balancing deployment.

On-premises and other cloud services are like any other Cloud Load Balancing backend. The key difference is that you use a hybrid connectivity NEG to configure the endpoints of these backends. The endpoints must be valid IP:port combinations that your clients can reach through hybrid connectivity, such as Cloud VPN or Cloud Interconnect.

The following diagram depicts the Google Cloud resources required to enable hybrid load balancing for External HTTP(S) Load Balancing.

External HTTP(S) load balancer resources for hybrid connectivity (click to enlarge)
External HTTP(S) load balancer resources for hybrid connectivity (click to enlarge)

The following diagram depicts the Google Cloud resources required to enable hybrid load balancing for Internal HTTP(S) Load Balancing.

Internal HTTP(S) load balancer resources for hybrid connectivity (click to enlarge)
Internal HTTP(S) load balancer resources for hybrid connectivity (click to enlarge)

Regional vs Global

Cloud Load Balancing routing depends on the scope of the configured load balancer:

External HTTP(S) Load Balancing. For traffic from the internet, the external HTTP(S) load balancer can be configured for either global or regional routing depending on the network tier that is used. You should create the load balancer's hybrid NEG backend in the same region where hybrid connectivity has been configured. Non-Google Cloud endpoints must also be configured accordingly to take advantage of proximity-based load balancing.

Internal HTTP(S) Load Balancing. For traffic from internal clients, the internal HTTP(S) load balancer is a regional load balancer. That is, it can only route traffic to endpoints within the same region as the load balancer. The internal HTTP(S) load balancer's components must be configured in the same region where hybrid connectivity has been configured. Because Internal HTTP(S) Load Balancing does not support global access, clients accessing the load balancer must also be in the same region.

For example, if the Cloud VPN Gateway or Cloud Interconnect VLAN Attachment have been configured in us-central1, the resources required by the internal HTTP(S) load balancer (backend service, hybrid NEG, forwarding rule, etc.) must be created in the us-central1 region. Clients accessing the load balancer must also be in the us-central1 region.

Network connectivity requirements

Before you configure a hybrid load balancing deployment, you must have the following resources set up:

  • Google Cloud VPC network. A VPC network configured inside Google Cloud. This is the network where the hybrid load balancing resources (forwarding rule, target proxy, backend service, etc.) will be created. On-premises, other cloud, and Google Cloud subnet IP addresses and IP address ranges must not overlap. When IP addresses overlap, subnet routes are prioritized over remote connectivity.
  • Hybrid connectivity. Your Google Cloud and on-premises or other cloud environments must be connected through hybrid connectivity, using either Cloud Interconnect VLAN attachment or Cloud VPN tunnels with Cloud Router. We recommend you use a high availability connection. A Cloud Router enabled with Global dynamic routing learns about the specific endpoint via BGP and programs it into your Google Cloud VPC network. Regional dynamic routing is not supported. Static routes are also not supported.

    Cloud Interconnect/Cloud VPN and Cloud Router must be configured in the same VPC network that you intend to use for the hybrid load balancing deployment. The Cloud Router must also advertise the following routes to your on-premises environment:
    • Ranges used by Google's health check probes: 35.191.0.0/16 and 130.211.0.0/22.
    • For Internal HTTP(S) Load Balancing only, the range of the region's proxy-only subnet.
  • Network endpoints (IP:Port) on your on-prem or other cloud. One or more IP:Port network endpoints configured within your on-premises or other cloud environments, routable via Cloud Interconnect or Cloud VPN. If there are multiple paths to the IP endpoint, routing will follow the behavior described in the VPC routes overview and the Cloud Router overview.
  • Firewall rules on your on-prem or other cloud. The following firewall rules must be created on your on-prem or other cloud environment:
    • Ingress allow firewall rules to allow traffic from Google's health-checking probes to your endpoints. For external HTTP(S) load balancer, internal HTTP(S) load balancer, TCP proxy load balancer, and SSL proxy load balancer, the ranges to be allowed are: 35.191.0.0/16 and 130.211.0.0/22. Note that these ranges must also be advertised by Cloud Router to your on-premises network. For more details, see Probe IP ranges and firewall rules.
    • Ingress allow firewall rules to allow traffic that is being load-balanced to reach the endpoints.
    • For Internal HTTP(S) Load Balancing, you also need to create a firewall rule to allow traffic from the region's proxy-only subnet to reach the endpoints.

Load balancer components

Depending on the type of load balancer, you can set up a hybrid load balancing deployment using either the Standard or the Premium Network Service Tiers.

A hybrid load balancer requires special configuration only for the backend service. The frontend configuration is the same as any other load balancer. Additionally, internal HTTP(S) load balancers require a proxy-only subnet to run Envoy proxies on your behalf.

Frontend configuration

No special frontend configuration is required for hybrid load balancing. Forwarding rules are used to route traffic by IP address, port, and protocol to a target proxy. The target proxy then terminates connections from clients.

URL maps are used by HTTP(S) load balancers to set up URL-based routing of requests to the appropriate backend services.

For more details on each of these components, refer the architecture sections of the specific load balancer overviews:

Backend service

Backend services provide configuration information to the load balancer. Load balancers use the information in a backend service to direct incoming traffic to one or more attached backends.

To set up a hybrid load balancing deployment, you configure the load balancer with backends that are both within Google Cloud, as well as outside of Google Cloud.

  • Non-Google Cloud backends (on-prem or other cloud)

    Any destination that you can reach using Google's hybrid connectivity products (either Cloud VPN or Cloud Interconnect), and that can be reached with a valid IP:Port combination, can be configured as an endpoint for the load balancer.

    Configure your non-Google Cloud backends as follows:

    1. Add each non-Google Cloud network endpoint's IP:Port combination to a hybrid connectivity network endpoint group (NEG). Make sure this IP address and port is reachable from Google Cloud by using hybrid connectivity (either by Cloud VPN or Cloud Interconnect). For hybrid connectivity NEGs, you set the network endpoint type to NON_GCP_PRIVATE_IP_PORT.
    2. While creating the NEG, specify a Google Cloud zone that minimizes the geographic distance between Google Cloud and your on-premises or other cloud environment. For example, if you are hosting a service in an on-premises environment in Frankfurt, Germany, you can specify the europe-west3-a Google Cloud zone when you create the NEG.
    3. Add this hybrid connectivity NEG as a backend for the backend service.
  • Google Cloud backends

    Configure your Google Cloud endpoints as follows:

    1. Create a separate backend service for the Google Cloud backends.
    2. Configure multiple backends (zonal NEGs or instance groups) within the same region in which you have set up hybrid connectivity.

Additional points for consideration:

  • Each hybrid connectivity NEG can only contain network endpoints of the same type (NON_GCP_PRIVATE_IP_PORT).

  • The backend service cannot also use other NEG types or instance groups as backends. All backends on a backend service must be of the same type. If you have backends on Google Cloud, you must create a separate backend service for them. This is because Google Cloud-based backends will use a different type of backend (instance group or zonal NEG) and backend services with mixed backend types are not supported.

  • The backend service's load balancing scheme must be either EXTERNAL for external HTTP(S) load balancers, TCP proxy load balancers and SSL proxy load balancers, or INTERNAL_MANAGED for internal HTTP(S) load balancers. INTERNAL_SELF_MANAGED is supported for Traffic Director multi-environment deployments with hybrid connectivity NEGs.

  • The backend service protocol must be one of HTTP, HTTPS, or HTTP2. For the list of backend service protocols supported by each load balancer, see Protocols from the load balancer to the backend.

  • The balancing mode for the backend must be RATE for external and internal HTTP(S) Load Balancing, and CONNECTION for TCP/SSL Proxy Load Balancing. For details on balancing modes, see Backend services overview.

  • To add more network endpoints, update the backends attached to your backend service.

Health checks

Each backend service must be associated with a health check that checks the health of the backends. For the health check probes to function correctly, you must create firewall rules that allow traffic from Google's probe IP ranges (130.211.0.0/22 and 35.191.0.0/16) to reach your endpoints.

For backends outside Google Cloud, create firewall rules on your on-premises and other cloud networks. Contact your network administrator for this. The Cloud Router used for hybrid connectivity must also advertise the ranges used by Google's health check probes. The ranges to be advertised are: 35.191.0.0/16 and 130.211.0.0/22.

For backends within Google Cloud, create firewall rules on Google Cloud as demonstrated in this example.

Limitations

  • Hybrid connectivity NEGs are not supported in the Google Cloud Console. To create, delete, or manage hybrid connectivity NEGs, you must use the gcloud command-line tool or the REST API.
  • The Cloud Router used for hybrid connectivity must be enabled with Global dynamic routing. Regional dynamic routing and static routes are not supported.
  • Internal HTTP(S) Load Balancing and hybrid connectivity must be configured in the same region. If they are configured in different regions, you might see backends as healthy but clients' requests will not be forwarded to the backend.
  • The considerations for encrypted connections from the load balancer to the backends documented here. also apply to non-Google Cloud backend endpoints configured in the hybrid connectivity NEG.

    Make sure to also review the security settings on your hybrid connectivity configuration. Currently, HA Cloud VPN connections are encrypted by default (IPSec). Cloud Interconnect connections are not encrypted by default. For more details, see the Encryption in Transit in Google Cloud whitepaper.

Logging

Requests proxied to an endpoint in a hybrid NEG are logged to Cloud Logging in the same way that requests for other HTTP(S) Load Balancing backends are logged. For more information, see HTTP(S) Load Balancing logging and monitoring.

If you enable Cloud CDN for your load balancer, cache hits are also logged.

Quota

You can configure as many hybrid NEGs with network endpoints as permitted by your existing network endpoint group quota. For more information, see NEG backends and Endpoints per NEG.

What's next