External HTTP(S) Load Balancing overview

This document introduces the concepts that you need to understand to configure Google Cloud external HTTP(S) Load Balancing.

HTTP(S) Load Balancing is a global, proxy-based Layer 7 load balancer that enables you to run and scale your services worldwide behind a single external IP address. External HTTP(S) Load Balancing distributes HTTP and HTTPS traffic to backends hosted on Compute Engine and Google Kubernetes Engine (GKE).

External HTTP(S) Load Balancing is implemented on Google Front Ends (GFEs). GFEs are distributed globally and operate together using Google's global network and control plane. In the Premium Tier, GFEs offer multi-region load balancing, directing traffic to the closest healthy backend that has capacity and terminating HTTP(S) traffic as close as possible to your users. With Standard Tier, the load balancing is handled regionally.

This page describes the architecture of a standalone external HTTP(S) load balancer. If you want to expose your applications in GKE to the internet, we recommend that you use the built-in GKE Ingress for External HTTP(S) Load Balancing.


The following resources define an external HTTP(S) load balancer:

  • An external forwarding rule specifies an external IP address, port, and global target HTTP(S) proxy. Clients use the IP address and port to connect to the load balancer.

  • A target HTTP(S) proxy receives a request from the client. The HTTP(S) proxy evaluates the request by using the URL map to make traffic routing decisions. The proxy can also authenticate communications by using SSL certificates.

  • For HTTPS load balancing, the target HTTPS proxy uses SSL certificates to prove its identity to clients. A target HTTPS proxy supports up to the documented number of SSL certificates.

  • The HTTP(S) proxy uses a URL map to make a routing determination based on HTTP attributes (such as the request path, cookies, or headers). Based on the routing decision, the proxy forwards client requests to specific backend services or backend buckets. The URL map can specify additional actions, such as sending redirects to clients.

  • A backend service or backend bucket distributes requests to healthy backends.

  • One or more backends must be connected to the backend service or backend bucket.

  • A health check periodically monitors the readiness of your backends. This reduces the risk that requests might be sent to backends that can't service the request.

  • A firewall for your backends to accept health check probes.

Forwarding rules and addresses

Forwarding rules route traffic by IP address, port, and protocol to a load balancing configuration consisting of a target proxy, URL map, and one or more backend services.

Each forwarding rule provides a single IP address that can be used in DNS records for your application. No DNS-based load balancing is required. You can either specify the IP address to be used or let Cloud Load Balancing assign one for you.

  • The forwarding rule for an HTTP load balancer can only reference TCP ports 80 and 8080.
  • The forwarding rule for an HTTPS load balancer can only reference TCP port 443.

The type of forwarding rule and IP address required by external HTTP(S) load balancers depends on which Network Service Tier the load balancer is in.

For the complete list of protocols supported by HTTP(S) Load Balancing forwarding rules, see Load balancer features.

Target proxies

Target proxies terminate HTTP(S) connections from clients. One or more forwarding rules direct traffic to the target proxy, and the target proxy consults the URL map to determine how to route traffic to backends.

Do not rely on the proxy to preserve the case of request or response header names. For example, a Server: Apache/1.0 response header might appear at the client as server: Apache/1.0.

Target proxy types Proxy-added headers Custom headers supported Cloud Trace supported
Global HTTP,
Global HTTPS
The proxies set HTTP request/response headers as follows:
  • Via: 1.1 google (requests and responses)
  • X-Forwarded-Proto: [http | https] (requests only)
  • X-Cloud-Trace-Context: <trace-id>/<span-id>;<trace-options> (requests only)
    Contains parameters for Cloud Trace.
  • X-Forwarded-For: [<supplied-value>,]<client-ip>,<load-balancer-ip> (see X-Forwarded-For header) (requests only)

Host header

When the load balancer makes the HTTP request, the load balancer preserves the Host header of the original request.

X-Forwarded-For header

The load balancer appends two IP addresses separated by a single comma to the X-Forwarded-For header in the following order:

  • The IP address of the client that connects to the load balancer
  • The IP address of the load balancer's forwarding rule

If there is no X-Forwarded-For header on the incoming request, these two IP addresses are the entire header value:

X-Forwarded-For: <client-ip>,<load-balancer-ip>

If the request includes an X-Forwarded-For header, the load balancer preserves the supplied value before the <client-ip>,<load-balancer-ip>:

X-Forwarded-For: <supplied-value>,<client-ip>,<load-balancer-ip>

When running HTTP reverse proxy software on the load balancer's backends, the software might append one or both of the following IP addresses to the end of the X-Forwarded-For header:

  • The IP address of the Google Front End (GFE) that connected to the backend. These IP addresses are in the and ranges.

  • The IP address of the backend system itself

Thus, an upstream process after your load balancer's backend might receive an X-Forwarded-For header of the form:


HTTP/3 and QUIC protocol support

HTTP/3 is a next-generation internet protocol. It is built on top of QUIC, a protocol developed from the original Google QUIC) (gQUIC) protocol. HTTP/3 is supported between the external HTTP(S) load balancer, Cloud CDN, and clients.


  • IETF QUIC is a transport layer protocol that provides congestion control similar to TCP and is the security equivalent to SSL/TLS for HTTP/2, with improved performance.
  • HTTP/3 is an application layer built on top of IETF QUIC, and it relies on QUIC to handle multiplexing, congestion control, and retries.
  • HTTP/3 allows faster client connection initiation, eliminates head-of-line blocking in multiplexed streams, and supports connection migration when a client's IP address changes.
  • HTTP/3 affects connections between clients and the load balancer, not connections between the load balancer and its backends.
  • HTTP/3 connections use the BBR congestion control protocol.

Enabling HTTP/3 on your external HTTP(S) load balancer can improve web page load times, reduce video rebuffering, and improve throughput on higher latency connections.

Configuring HTTP/3

You can explicitly enable HTTP/3 support for your external HTTP(S) load balancer by setting quicOverride to ENABLE. In the future, HTTP/3 will be enabled by default for all external HTTP(S) load balancer clients.

Clients that do not support HTTP/3 or gQUIC do not negotiate an HTTP/3 connection. You do not need to explicitly disable HTTP/3 unless you have identified broken or older client implementations.

The external HTTP(S) load balancer provides three modes for configuring HTTP/3 as shown in the following table.

quicOverride value Behavior

HTTP/3 and Google QUIC are not advertised to clients.

Note: This might change in the future, and HTTP/3 will be advertised to clients by default.


Support for HTTP/3 and Google QUIC are advertised to clients. HTTP/3 is advertised at a higher priority. Clients that support both protocols should prefer HTTP/3 over Google QUIC.

Note: TLS 0-RTT (also known as TLS Early Data) is implicitly supported when Google QUIC is negotiated by the client, but it is not currently supported when HTTP/3 is used.

DISABLE Explicitly disables advertising HTTP/3 and Google QUIC to clients.

To explicitly enable (or disable) HTTP/3, follow these steps.

Console: HTTPS

  1. In the Google Cloud Console, go to the Load balancing page.

    Go to Load balancing

  2. Select the external HTTP(S) load balancer that you want to edit.

  3. Click Frontend configuration.

  4. Select the frontend IP address and port that you want to edit. To edit HTTP/3 configurations, the IP address and port must be HTTPS (port 443).

Enable HTTP/3

  1. Select the QUIC negotiation drop-down.
  2. To explicitly enable HTTP/3 for this frontend, select Enabled.
  3. If you have multiple frontend rules representing IPv4 and IPv6, make sure to enable HTTP/3 on each rule.

Disable HTTP/3

  1. Select the QUIC negotiation drop-down.
  2. To explicitly disable HTTP/3 for this frontend, select Disabled.
  3. If you have multiple frontend rules representing IPv4 and IPv6, make sure to disable HTTP/3 for each rule.

gcloud: HTTPS

Before you run this command, you must create an SSL certificate resource for each certificate.

 gcloud compute target-https-proxies create HTTPS_PROXY_NAME \
     --global \

Replace QUIC_SETTING with one of the following:

  • NONE (default): allows Google to control when QUIC is negotiated

    Currently, when you select NONE, QUIC is disabled. By selecting this option, you are allowing Google to automatically enable QUIC negotiations and HTTP/3 in the future for this load balancer. In the Cloud Console, this option is called Automatic (Default).

  • ENABLE: advertises HTTP/3 and Google QUIC to clients

  • DISABLE: does not advertise HTTP/3 or Google QUIC to clients


POST https://www.googleapis.com/v1/compute/projects/PROJECT_ID/global/targetHttpsProxies/TARGET_PROXY_NAME/setQuicOverride

  "quicOverride": QUIC_SETTING

Replace QUIC_SETTING with one of the following:

  • NONE (default): allows Google to control when QUIC is negotiated

    Currently, when you select NONE, QUIC is disabled. By selecting this option, you are allowing Google to automatically enable QUIC negotiations and HTTP/3 in the future for this load balancer. In the Cloud Console, this option is called Automatic (Default).

  • ENABLE: advertises HTTP/3 and Google QUIC to clients

  • DISABLE: does not advertise HTTP/3 or Google QUIC to clients

How HTTP/3 is negotiated

When HTTP/3 is enabled, the load balancer advertises this support to clients, allowing clients that support HTTP/3 to attempt to establish HTTP/3 connections with the HTTPS load balancer.

  • Properly implemented clients always fall back to HTTPS or HTTP/2 when they cannot establish a QUIC connection.
  • Clients that support HTTP/3 use their cached prior knowledge of HTTP/3 support to save unnecessary round-trips in the future.
  • Because of this fallback, enabling or disabling QUIC in the load balancer does not disrupt the load balancer's ability to connect to clients.

Support is advertised in the Alt-Svc HTTP response header. When HTTP/3 is configured as ENABLE on an external HTTP(S) load balancer's targetHttpsProxy resource, responses from the external HTTP(S) load balancer include the following alt-svc header value:

alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,
h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,
quic=":443"; ma=2592000; v="46,43"

If HTTP/3 has been explicitly set to DISABLE, responses do not include an alt-svc response header.

When you have QUIC enabled on your HTTPS load balancer, some circumstances can cause your client to fall back to HTTPS or HTTP/2 instead of negotiating QUIC. These include the following:

  • When a client supports versions of HTTP/3 that are not compatible with the HTTP/3 versions supported by the HTTPS load balancer.
  • When the load balancer detects that UDP traffic is blocked or rate-limited in a way that would prevent HTTP/3 (QUIC) from working.
  • The client does not support HTTP/3 at all, and thus does not attempt to negotiate an HTTP/3 connection.

When a connection falls back to HTTPS or HTTP/2 because of these circumstances, we do not count this as a failure of the load balancer.

Before you enable HTTP/3, ensure that the previously described behaviors are acceptable for your workloads.

URL maps

URL maps define matching patterns for URL-based routing of requests to the appropriate backend services. A default service is defined to handle any requests that do not match a specified host rule or path matching rule. In some situations, such as the multi-region load balancing example, you might not define any URL rules and rely only on the default service. For request routing, the URL map allows you to divide your traffic by examining the URL components to send requests to different sets of backends.

SSL certificates

Transport Layer Security (TLS) is an encryption protocol used in SSL certificates to protect network communications.

Google Cloud uses SSL certificates to provide privacy and security from a client to a load balancer. If you are using HTTPS-based load balancing, you must install one or more SSL certificates on the target HTTPS proxy.

For more information about SSL certificates, see the following:

SSL policies

SSL policies give you the ability to control the features of SSL that your HTTPS load balancer negotiates with HTTPS clients.

By default, HTTPS Load Balancing uses a set of SSL features that provides good security and wide compatibility. Some applications require more control over which SSL versions and ciphers are used for their HTTPS or SSL connections. You can define SSL policies that control the features of SSL that your load balancer negotiates and associate an SSL policy with your target HTTPS proxy.

Backend services and buckets

An external HTTP(S) load balancer can have backend services and backend buckets.

Backend services provide configuration information to the load balancer. Load balancers use the information in a backend service to direct incoming traffic to one or more attached backends. For an example showing how to set up a load balancer with a backend service and a Compute Engine backend, see Setting up an external HTTP(S) load balancer with a Compute Engine backend.

Backend buckets direct incoming traffic to Cloud Storage buckets. For an example showing how to add a bucket to a load balancer, see Setting up a load balancer with backend buckets.

If you want to configure an external HTTP(S) load balancer by using HTTP/2 with Google Kubernetes Engine Ingress or by using gRPC and HTTP/2 with Ingress, see HTTP/2 for load balancing with Ingress.

The following table specifies the backend features supported by HTTP(S) Load Balancing.

Supported backends on a backend service Supports backend buckets Supports Google Cloud Armor Supports Cloud CDN Supports IAP
Instance groups Zonal NEGs Internet NEGs Serverless NEGs

when using Premium Tier

For more information, see the following:

Health checks

Each backend service specifies a health check for backend instances.

For the health check probes, you must create a firewall rule that allows traffic to reach your instances. The firewall rule must allow the following source ranges:


External HTTP(S) load balancers use global health checks.

For more information about health checks, see the following:

Protocol to the backends

When you configure a backend service for the external HTTP(S) load balancer, you set the protocol that the backend service uses to communicate with the backends. You can choose HTTP, HTTPS, or HTTP/2. The load balancer uses only the protocol that you specify. The load balancer does not fall back to one of the other protocols if it is unable to negotiate a connection to the backend with the specified protocol.

If you use HTTP/2, you must use TLS. HTTP/2 without encryption is not supported.

Although it is not required, it is a best practice to use a health check whose protocol matches the protocol of the backend service. For example, an HTTP/2 health check most accurately tests HTTP/2 connectivity to backends.

Using gRPC with your Google Cloud applications

gRPC is an open-source framework for remote procedure calls. It is based on the HTTP/2 standard. Use cases for gRPC include the following:

  • Low latency, highly scalable, distributed systems
  • Developing mobile clients that communicate with a cloud server
  • Designing new protocols that must be accurate, efficient, and language independent
  • Layered design to enable extension, authentication, and logging

To use gRPC with your Google Cloud applications, you must proxy requests end-to-end over HTTP/2. To do this with an external HTTP(S) load balancer:

  1. Configure an HTTPS load balancer.
  2. Enable HTTP/2 as the protocol from the load balancer to the backends.

The load balancer negotiates HTTP/2 with clients as part of the SSL handshake by using the ALPN TLS extension.

The load balancer may still negotiate HTTPS with some clients or accept insecure HTTP requests on an external HTTP(S) load balancer that is configured to use HTTP/2 between the load balancer and the backend instances. Those HTTP or HTTPS requests are transformed by the load balancer to proxy the requests over HTTP/2 to the backend instances.

You must enable TLS on your backends. For more information, see Encryption from the load balancer to the backends.

If you want to configure an external HTTP(S) load balancer by using HTTP/2 with Google Kubernetes Engine Ingress or by using gRPC and HTTP/2 with Ingress, see HTTP/2 for load balancing with Ingress.

For information about troubleshooting problems with HTTP/2, see Troubleshooting issues with HTTP/2 to the backends.

For information about HTTP/2 limitations, see HTTP/2 limitations.

Firewall rules

The backend instances must allow connections from the following sources:

  • The load balancer Google Front End (GFE) for all requests sent to your backends
  • Health check probes

To allow this traffic, you must create ingress firewall rules. The ports for these firewall rules must allow traffic as follows:

  • To the destination port for each backend service's health check.

  • For instance group backends: Determined by the mapping between the backend service's named port and the port numbers associated with that named port on each instance group. The numbers can vary among instance groups assigned to the same backend service.

  • For GCE_VM_IP_PORT NEG backends: To the port numbers of the endpoints.

Firewall rules are implemented at the VM instance level, not on GFE proxies. You cannot use Google Cloud firewall rules to prevent traffic from reaching the load balancer. You can use Google Cloud Armor to achieve this.

For more information about health check probes and why it's necessary to allow traffic from them, see Probe IP ranges and firewall rules.

For external HTTP(S) load balancers, the following table specifies the health check and request source ranges:

Health check source ranges GFE request source ranges
The source of GFE traffic depends on the backend type:
  • Instance groups, zonal NEGs (GCE_VM_IP_PORT), and hybrid connectivity NEGs (NON_GCP_PRIVATE_IP_PORT):
  • SERVERLESS NEGs and backend buckets: Google's production network handles packet routing.

Source IP addresses for client packets

The source IP address for packets, as seen by the backends, is not the Google Cloud external IP address of the load balancer. In other words, there are two TCP connections:

  • Connection 1, from original client to the load balancer (GFE):

    • Source IP address: the original client (or external IP address if the client is behind NAT or a forward proxy).
    • Destination IP address: your load balancer's IP address.
  • Connection 2, from the load balancer (GFE) to the backend VM or endpoint:

    • Source IP address: an IP address in one of the ranges specified in Firewall rules.

    • Destination IP address: the internal IP address of the backend VM or container in the Virtual Private Cloud (VPC) network.

Return path

Google Cloud uses special routes not defined in your VPC network for health checks. For more information, see Load balancer return paths.

How connections work in HTTP(S) Load Balancing

External HTTP(S) Load Balancing is a service, implemented by many proxies called Google Front Ends (GFEs). There isn't just a single proxy. In Premium Tier, the same global external IP address is advertised from various points of presence, and traffic is directed to the client's nearest GFE.

Depending on where your clients are, multiple GFEs can initiate HTTP(S) connections to your backends. Packets sent from GFEs have source IP addresses from the same range used by health check probers: and

Depending on the backend service configuration, the protocol used by each GFE to connect to your backends can be HTTP, HTTPS, or HTTP/2. If HTTP or HTTPS, the HTTP version is HTTP 1.1. HTTP keepalive is enabled by default, as specified in the HTTP 1.1 specification. The GFE uses a keepalive timeout of 600 seconds, and you cannot configure this. You can, however, configure the request/response timeout by setting the backend service timeout. For more information, see timeouts and retries.

HTTP keepalives attempt to efficiently use the same TCP session; however, there's no guarantee. Though closely related, an HTTP keepalive and a TCP idle timeout are not the same thing.

The numbers of HTTP connections and TCP sessions vary depending on the number of GFEs connecting, the number of clients connecting to the GFEs, the protocol to the backends, and where backends are deployed.

For more information, see How HTTP(S) Load Balancing works in the solutions guide: Application Capacity Optimizations with Global Load Balancing.

Client communications with the load balancer

  • Clients can communicate with the load balancer by using the HTTP 1.1 or HTTP/2 protocol.
  • When HTTPS is used, modern clients default to HTTP/2. This is controlled on the client, not on the HTTPS load balancer.
  • You cannot disable HTTP/2 by making a configuration change on the load balancer. However, you can configure some clients to use HTTP 1.1 instead of HTTP/2. For example, with curl, use the --http1.1 parameter.
  • HTTP(S) Load Balancing supports the HTTP/1.1 100 Continue response.

For the complete list of protocols supported by HTTP(S) Load Balancing forwarding rules, see Load balancer features.

Open ports

The external HTTP(S) load balancers are reverse proxy load balancers. The load balancer terminates incoming connections, and then opens new connections from the load balancer to the backends. These load balancers are implemented using Google Front End (GFE) proxies worldwide.

GFEs have several open ports to support other Google services that run on the same architecture. To see a list of some of the ports likely to be open on GFEs, see Forwarding rule: Port specifications. There might be other open ports for other Google services running on GFEs.

Running a port scan on the IP address of a GFE-based load balancer is not useful from an auditing perspective for the following reasons:

  • A port scan (for example, with nmap) generally expects no response packet or a TCP RST packet when performing TCP SYN probing. GFEs will send SYN-ACK packets in response to SYN probes for a variety of ports if your load balancer uses a Premium Tier IP address. However, GFEs only send packets to your backends in response to packets sent to your load balancer's IP address and the destination port configured on its forwarding rule. Packets sent to different load balancer IP addresses or your load balancer's IP address on a port not configured in your forwarding rule do not result in packets being sent to your load balancer's backends. GFEs implement security features such as Google Cloud Armor. Even without a Google Cloud Armor configuration, Google infrastructure and GFEs provide defense-in-depth for DDoS attacks and SYN floods.

  • Packets sent to the IP address of your load balancer could be answered by any GFE in Google's fleet; however, scanning a load balancer IP address and destination port combination only interrogates a single GFE per TCP connection. The IP address of your load balancer is not assigned to a single device or system. Thus, scanning the IP address of a GFE-based load balancer does not scan all the GFEs in Google's fleet.

With that in mind, the following are some more effective ways to audit the security of your backend instances:

  • A security auditor should inspect the forwarding rules configuration for the load balancer's configuration. The forwarding rules define the destination port for which your load balancer accepts packets and forwards them to the backends. For GFE-based load balancers, each external forwarding rule can only reference a single destination TCP port. For external HTTP(S) load balancers using TCP port 443, UDP port 443 is used when the connection is upgraded to QUIC (HTTP/3).

  • A security auditor should inspect the firewall rule configuration applicable to backend VMs. The firewall rules that you set block traffic from the GFEs to the backend VMs, but do not block incoming traffic to the GFEs. For best practices, see the firewall rules section.

TLS termination

The HTTPS load balancer terminates TLS in locations that are distributed globally, so as to minimize latency between clients and the load balancer. If you require geographic control over where TLS is terminated, you should use Google Cloud Network Load Balancing instead, and terminate TLS on backends that are located in regions appropriate to your needs.

Timeouts and retries

HTTP(S) Load Balancing has two distinct types of timeouts:

  • A configurable HTTP backend service timeout, which represents the amount of time the load balancer waits for your backend to return a complete HTTP response. The default value for the backend service timeout is 30 seconds. The full range of timeout values allowed is 1-2,147,483,647 seconds.

    Consider increasing this timeout under any of these circumstances:

    • You expect a backend to take longer to return HTTP responses.
    • You see an HTTP 408 response with jsonPayload.statusDetail client_timed_out.
    • The connection is upgraded to a WebSocket.

For example, a practical timeout value for external HTTP(S) load balancers is 1 day (86400 seconds). Note that events such as GFE restarts might cause sessions to terminate sooner than this timeout.

For WebSocket traffic sent through the load balancer, the backend service timeout is interpreted as the maximum amount of time that a WebSocket connection can remain open, whether idle or not. For more information, see Backend service settings.

  • A TCP session timeout, whose value is fixed at 10 minutes (600 seconds). This session timeout is sometimes called a keepalive or idle timeout, and its value is not configurable by modifying your backend service. You must configure the web server software used by your backends so that its keepalive timeout is longer than 600 seconds to prevent connections from being closed prematurely by the backend. This timeout does not apply to WebSockets.

This table illustrates changes necessary to modify keepalive timeouts for common web server software:

Web server software Parameter Default setting Recommended setting
Apache KeepAliveTimeout KeepAliveTimeout 5 KeepAliveTimeout 620
nginx keepalive_timeout keepalive_timeout 75s; keepalive_timeout 620s;

The load balancer retries failed GET requests in certain circumstances, such as when the backend service timeout is exhausted. It does not retry failed POST requests. Retries are limited to two attempts. Retried requests only generate one log entry for the final response.

A GFE doesn't attempt a retry if more than 80% of the backend instances are unhealthy. If there is a single backend instance in a group and the connection to that backend instance fails, the percentage of unhealthy backend instances is 100%, so the GFE doesn't retry the request. For more information, see HTTP(S) Load Balancing logging and monitoring.

The WebSocket protocol is supported with Ingress.

Illegal request and response handling

The external HTTP(S) load balancer blocks both client requests and backend responses from reaching the backend or the client, respectively, for a number of reasons. Some reasons are strictly for HTTP/1.1 compliance and others are to avoid unexpected data being passed to or from the backends. None of the checks can be disabled.

The load balancer blocks the following for HTTP/1.1 compliance:

  • It cannot parse the first line of the request.
  • A header is missing the : delimiter.
  • Headers or the first line contain invalid characters.
  • The content length is not a valid number, or there are multiple content length headers.
  • There are multiple transfer encoding keys, or there are unrecognized transfer encoding values.
  • There's a non-chunked body and no content length specified.
  • Body chunks are unparseable. This is the only case where some data reaches the backend. The load balancer closes the connections to the client and backend when it receives an unparseable chunk.

The load balancer blocks the request if any of the following are true:

  • The total size of request headers and the request URL exceeds the limit for the maximum request header size for external HTTP(S) Load Balancing.
  • The request method does not allow a body, but the request has one.
  • The request contains an Upgrade header, and the Upgrade header is not used to enable WebSocket connections.
  • The HTTP version is unknown.

The load balancer blocks the backend's response if any of the following are true:

  • The total size of response headers exceeds the limit for maximum response header size for external HTTP(S) Load Balancing.
  • The HTTP version is unknown.

Traffic distribution

When you add a backend instance group or NEG to a backend service, you specify a balancing mode, which defines a method measuring backend load and a target capacity. External HTTP(S) Load Balancing supports two balancing modes:

  • RATE, for instance groups or NEGs, is the target maximum number of requests (queries) per second (RPS, QPS). The target maximum RPS/QPS can be exceeded if all backends are at or above capacity.

  • UTILIZATION is the backend utilization of VMs in an instance group.

Before a Google Front End (GFE) sends requests to backend instances, the GFE estimates which backend instances have capacity to receive requests. This capacity estimation is made proactively, not at the same time as requests are arriving. The GFEs receive periodic information about the available capacity and distribute incoming requests accordingly.

What capacity means depends in part on the balancing mode. For the RATE mode, it is relatively simple: a GFE determines exactly how many requests it can assign per second. UTILIZATION-based load balancing is more complex: the load balancer checks the instances' current utilization and then estimates a query load that each instance can handle. This estimate changes over time as instance utilization and traffic patterns change.

Both factors—the capacity estimation and the proactive assignment—influence the distribution among instances. Thus, Cloud Load Balancing behaves differently from a simple round-robin load balancer that spreads requests exactly 50:50 between two instances. Instead, Google Cloud load balancing attempts to optimize the backend instance selection for each request.

For more information, see Traffic distribution.

For more information about the balancing modes, see Balancing mode.

How requests are distributed

Whether traffic is distributed regionally or globally depends on which network service tier is in use.

For Premium Tier:

  • Google advertises your load balancer's IP address from all points of presence, worldwide. Each load balancer IP address is global anycast.
  • If you configure a backend service with backends in multiple regions, Google Front Ends (GFEs) attempt to direct requests to healthy backend instance groups or NEGs in the region closest to the user. Details for the process are documented on this page.

For Standard Tier:

  • Google advertises your load balancer's IP address from points of presence associated with the forwarding rule's region. The load balancer uses a regional external IP address.

  • You can configure backends in the same region as the forwarding rule. The process documented here still applies, but GFEs only direct requests to healthy backends in that one region.

Request distribution process:

The balancing mode and choice of target define backend fullness from the perspective of each zonal GCE_VM_IP_PORT NEG, zonal instance group, or zone of a regional instance group. Distribution within a zone is then done with consistent hashing.

The load balancer uses the following process:

  1. The forwarding rule's external IP address is advertised by edge routers at the borders of Google's network. Each advertisement lists a next hop to a Layer 3/4 load balancing system (Maglev) as close to the user as possible.
  2. Maglev systems inspect the source IP address of the incoming packet. They direct the incoming request to the Maglev systems that Google's geo-IP systems determine are as close to the user as possible.
  3. The Maglev systems route traffic to a first-layer Google Front End (GFE). The first-layer GFE terminates TLS if required and then routes traffic to second-layer GFEs according to this process:
    1. The URL map selects a backend service.
    2. If a backend service uses instance group or GCE_VM_IP_PORT NEG backends, the first layer-GFEs prefer second-layer GFEs that are located in or near the region that contains the instance group or NEG.
    3. For backend buckets and backend services with hybrid NEGs, serverless NEGs, and internet NEGs, the first-layer GFEs choose second-layer GFEs in a subset of regions such that the round trip time between the two GFEs is minimized.

      Second-layer GFE preference is not a guarantee, and it can dynamically change based on Google's network conditions and maintenance.

      Second-layer GFEs are aware of health check status and actual backend capacity usage.

  4. The second-layer GFE directs requests to backends in zones within its region.
  5. For Premium Tier, sometimes second-layer GFEs send requests to backends in zones of different regions. This behavior is called spillover.
  6. Spillover is governed by two principles:

    • Spillover is possible when all backends known to a second-layer GFE are at capacity or are unhealthy.
    • The second-layer GFE has information for healthy, available backends in zones of a different region.

    The second-layer GFEs are typically configured to serve a subset of backend locations.

    Spillover behavior does not exhaust all possible Google Cloud zones. If you need to direct traffic away from backends in a particular zone or in an entire region, you must set the capacity scaler to zero. Configuring backends to fail health checks does not guarantee that the second-layer GFE spills over to backends in zones of a different region.

  7. When distributing requests to backends, GFEs operate at a zonal level.

    With a low number of requests per second, second-layer GFEs sometimes prefer one zone in a region over the other zones. This preference is normal and expected. The distribution among zones in the region doesn't become even until the load balancer receives more requests per second.

Session affinity

Session affinity provides a best-effort attempt to send requests from a particular client to the same backend for as long as the backend is healthy and has the capacity, according to the configured balancing mode.

Google Cloud HTTP(S) Load Balancing offers three types of session affinity:

  • NONE. Session affinity is not set for the load balancer.
  • Client IP affinity sends requests from the same client IP address to the same backend.
  • Generated cookie affinity sets a client cookie when the first request is made, and then sends requests with that cookie to the same backend.

When you use session affinity, we recommend the RATE balancing mode rather than UTILIZATION. Session affinity works best if you set the balancing mode to requests per second (RPS).

WebSocket support

Google Cloud HTTP(S)-based load balancers have native support for the WebSocket protocol when you use HTTP or HTTPS as the protocol to the backend. The load balancer does not need any configuration to proxy WebSocket connections.

The WebSocket protocol provides a full-duplex communication channel between clients and servers. An HTTP(S) request initiates the channel. For detailed information about the protocol, see RFC 6455.

When the load balancer recognizes a WebSocket Upgrade request from an HTTP(S) client followed by a successful Upgrade response from the backend instance, the load balancer proxies bidirectional traffic for the duration of the current connection. If the backend instance does not return a successful Upgrade response, the load balancer closes the connection.

The timeout for a WebSocket connection depends on the configurable backend service timeout of the load balancer, which is 30 seconds by default. This timeout applies to WebSocket connections regardless of whether they are in use.

Session affinity for WebSockets works the same as for any other request. For information, see Session affinity.

HTTP/2 max concurrent streams

The HTTP/2 SETTINGS_MAX_CONCURRENT_STREAMS setting describes the maximum number of streams that an endpoint accepts, initiated by the peer. The value advertised by an HTTP/2 client to a Google Cloud load balancer is effectively meaningless, because the load balancer doesn't initiate streams to the client.

In cases where the load balancer uses HTTP/2 to communicate with a server that is running on a VM, the load balancer respects the SETTINGS_MAX_CONCURRENT_STREAMS value advertised by the server. If a value of zero is advertised, the load balancer can't forward requests to the server, and this might result in errors.

HTTP/2 limitations

  • HTTP/2 between the load balancer and the instance can require significantly more TCP connections to the instance than HTTP(S). Connection pooling, an optimization that reduces the number of these connections with HTTP(S), is not currently available with HTTP/2.
  • HTTP/2 between the load balancer and the backend does not support running the WebSocket Protocol over a single stream of an HTTP/2 connection (RFC 8441).
  • HTTP/2 between the load balancer and the backend does not support server push.
  • gRPC error rate and request volume aren't visible in the Google Cloud API or the Cloud Console. If the gRPC endpoint returns an error, the load balancer logs and monitoring data report the 'OK 200' HTTP response code.

What's next