Internet network endpoint groups overview

Cloud Load Balancing supports proxying traffic to external backends outside Google Cloud. To define an external backend for a load balancer, you use a global resource called an internet network endpoint group (NEG).

You can use this type of deployment when you want to serve content from an external backend, but you want your gcloud CLI load balancer to be the frontend. This allows you to do the following:

• Use Google Edge infrastructure for terminating your user connections.
• Direct the connections to your external backend.
• Use Cloud CDN to cache content for your external backend.
• Deliver traffic to your public endpoint across Google's private backbone, which improves reliability and can decrease latency between client and server.

Figure 1 shows an external HTTP(S) load balancer with multiple backend types, one of which is an external backend configured with an internet NEG.

Figure 1. Internet network endpoint groups in load balancing (click to enlarge).

Internet NEG backends are supported with the global external HTTP(S) load balancer, the global external HTTP(S) load balancer (classic), and Traffic Director. The sections that follow explain how external backends are used with external HTTP(S) load balancers. When you use an external backend with Traffic Director, DNS, health checking, and traffic routing behave differently. For more information, see Traffic Director with internet network endpoint groups.

Support for internet NEG backends with global external HTTP(S) load balancers is in Preview.

Note: Cloud Load Balancing also supports connecting to backends on-premises or in other cloud environments outside Google Cloud by using hybrid NEGs. However, connections to hybrid NEG endpoints make use of hybrid connectivity. In contrast, connections to internet NEG endpoints use the internet while keeping your traffic on Google's high performance backbone for the longest possible distance, for reduced latency and improved throughput.

Terminology

The following terms are sometimes used interchangeably because they have the same or similar meanings:

• external backend: A backend that resides outside of Google Cloud and is reachable across the internet. The endpoint in an internet NEG.
• custom origin: Same as an external backend. In CDN, origin is the industry-standard term for a backend instance that serves web content.
• internet network endpoint group (NEG): The Google Cloud API resource that you use to specify an external backend.
• external endpoint: Same as an external backend.

This document uses the term external backend except when referring to the internet NEG API resource.

Load balancer components

This section describes the load balancing architecture and resources required to configure a load balancer with an external backend. The load balancer requires special configuration only for the backend service. The frontend configuration is the same as any other global external HTTP(S) load balancer.

Figure 2 shows the Google Cloud resources required to set up a global external HTTP(S) load balancer with an external backend.

Figure 2. Global external HTTP(S) load balancer with external backend (click to enlarge).

You can only use internet NEGs on the Premium network service tier.

Frontend configuration

No special frontend configuration is required for creating a load balancer with an internet NEG backend. Forwarding rules are used to route traffic by IP address, port, and protocol to a target proxy. The target proxy then terminates connections from clients.

URL maps are used by HTTP(S) load balancers to set up URL-based routing of requests to the appropriate backend services.

For more details about each of these components, refer to the architecture sections of the External HTTP(S) load balancer overview.

Internet NEG

An internet NEG is a global resource used to define an external backend for the load balancer. The external backend referenced by the internet NEG must be reachable over the internet. The endpoint cannot be reachable only over Cloud VPN or Cloud Interconnect. If the external backend references a Google API or service, the service must be reachable through TCP port 80 or 443 by using the HTTP, HTTPS, or HTTP/2 protocol.

There are two ways to configure the external endpoint referenced by the NEG: INTERNET_FQDN_PORT or INTERNET_IP_PORT.

Endpoint address	Type	Definition	When to use
Hostname and an optional port	INTERNET_FQDN_PORT	For external HTTP(S) load balancers, a publicly resolvable fully qualified domain name and an optional port. For example backend.example.com:443 (default ports: 80 for HTTP and 443 for HTTPS and HTTP/2). The domain name defined must be resolvable by Google's public DNS infrastructure. For Traffic Director, the VPC network's name-resolution order is used to resolve the FQDN.	Use this endpoint with external HTTP(S) load balancers when your external backend can be resolved through a fully-qualified domain name with public DNS. Use this endpoint with Traffic Director to send traffic to external services.
IP address and an optional port	INTERNET_IP_PORT	A publicly routable IP address and an optional port. For example 8.8.8.8 or 8.8.8.8:443 (default ports: 80 for HTTP and 443 for HTTPS and HTTP/2). The IP address cannot be an RFC 1918 address.	Use this endpoint only with external HTTP(S) load balancers to specify a publicly accessible IP address and a port to connect to.

IP address resolution for INTERNET_FQDN_PORT endpoints

When an INTERNET_FQDN_PORT endpoint points to a DNS record that returns multiple IP addresses, the IP address is resolved in the following way:

• The load balancer uses a DNS resolver in a Google Cloud region that is closest to its client on the internet. If the DNS record for your INTERNET_FQDN_PORT endpoint returns different IP addresses based on the location of the client, make sure that each of those IP addresses can be reached by the load balancer.

• The load balancer attempts to connect to the first IP address in the DNS response. If that IP address isn't reachable, the load balancer returns an HTTP 502 (Bad Gateway) response. This is true even if other IP addresses from the DNS response are available.

For more information about the IP ranges and locations used by Google's DNS resolver infrastructure, see the Google public DNS documentation. Names that cannot be resolved by the public DNS system are not usable as external backends.

Backend service

Backend services provide configuration information to the load balancer. Load balancers use the information in a backend service to direct incoming traffic to one or more attached backends.

To set up a load balancer with an external backend, you configure the backend service with an internet NEG backend. When you add an internet NEG to a backend service, the following limitations apply:

• The backend service cannot also use other backend types (such as zonal NEGs or instance groups) as backends.

• You can add only one internet NEG backend to the same backend service.

• You can add only one endpoint to an internet NEG.

  Because only one endpoint is allowed in each internet NEG, load balancing isn't actually performed. The load balancer serves as the frontend only, and it proxies traffic to the specified external backend. This means that you can't use any of the load balancing modes, such as rate or utilization.

• The backend service cannot reference a health check.

• The backend service's load balancing scheme must be either EXTERNAL_MANAGED for the global external HTTP(S) load balancer or EXTERNAL for the global external HTTP(S) load balancer (classic).

• The backend service protocol must be one of HTTP, HTTPS, or HTTP2.

  We strongly recommend you use either HTTPS or HTTP/2 as the protocol when configuring a backend service with an internet NEG so that communication between the load balancer and the backend is encrypted and authenticated when transiting the public internet.

  Additionally, when using HTTPS or HTTP/2 as the backend protocol, make sure that you use an INTERNET_FQDN_PORT endpoint to create your external backend. This has two advantages:

  • It ensures that the load balancer validates the SSL server certificate presented by the external backend and verifies that the following information is true:

    • The certificate is signed by well-known certificate authorities (CAs).
    • The certificate is not expired.
    • The certificate signature is valid.
    • The configured FQDN matches one of the Subject Alternative Names (SANs) in the certificate.

    If you create the external backend by using an INTERNET_IP_PORT endpoint, SSL server certificate validation is not performed.

  • The SSL Server Name Indication (SNI) extension is only supported with INTERNET_FQDN_PORT endpoints. The configured FQDN is sent an SNI in the client hello during the SSL handshake between the load balancer and the external endpoint. The SNI is not sent when you use an INTERNET_IP_PORT endpoint because IP address literals aren't allowed in the HostName field of an SNI payload.

The following backend service features are supported when using an internet NEG:

• Cloud CDN. Use Cloud CDN to cache content for your external backend.
• Google Cloud Armor. Use Google Cloud Armor to protect your deployments against denial of service attacks.
• Identity-Aware Proxy (IAP). Use IAP to authenticate requests sent from the load balancer to the backend. For details, see Authenticate requests to the external backend section on this page. Note that IAP isn't compatible with Cloud CDN.
• Custom request headers

Health checks

A backend service with an internet NEG doesn't support health checks. Google Cloud doesn't provide health checking for any external backend.

If your external backend becomes unreachable or if the configured hostname (FQDN) cannot be resolved, the load balancer returns an HTTP 502 (Bad Gateway) response to its clients.

Enable the external backend to receive requests

To allow requests from the load balancer to reach your external backend, you must allowlist the IP address ranges that Google uses to send requests to external backends. To look up the IP addresses to be allowlisted, query the _cloud-eoips.googleusercontent.com DNS TXT record by using a tool like dig or nslookup.

For details, see Allow the external backend to receive traffic from Google Cloud.

Authenticate requests to the external backend

To authenticate requests sent to your external backend, you can do the following:

• Set a custom header to indicate that the request came from a Google Cloud external HTTP(S) load balancer by using a custom request header. For example, you can use 16 or more cryptographically random bytes as a shared key.

  For example, you can configure the external backend to expect a particular value for the HTTP request's Host header, and you can configure the backend service to set the Host header to that expected value. If you don't configure a custom request header, the backend service preserves the headers that the client used to connect to the load balancer and includes the same header in its response.

  When you're using a global external HTTP(S) load balancer, there are certain limitations associated with configuring the Host header. For details, see Create custom headers in backend services. For a specific example, see Set up a global external HTTP(S) load balancer (classic) with an external backend.

• Enable IAP and verify that the signed JWT in the request header is signed by Google, and that the aud (audience) claim contains the project number where your external HTTP(S) load balancer is defined. IAP isn't compatible with Cloud CDN.

• Enable private origin authentication, which gives an external HTTP(S) load balancer long-term access to a private Amazon Simple Storage Service (Amazon S3) bucket or other compatible object stores. Private origin authentication is compatible with Cloud CDN.

Logs

Requests proxied to an external backend are logged to Cloud Logging in the same way that requests for other backends are logged. For more information, see External HTTP(S) load balancer logging and monitoring.

If you enable Cloud CDN for an external backend, cache hits are also logged.

Header processing with external backends

When an external HTTP(S) load balancer proxies requests to an external backend, it adjusts HTTP headers in the following ways:

• Some headers are coalesced. When there are multiple instances of the same header key (for example, Via), the load balancer combines their values into a single comma-separated list for a single header key. Only the headers whose values can be represented as a comma-separated list are coalesced. Other headers, such as Set-Cookie, are never coalesced.

• Headers are proper-cased when the backend service's protocol is HTTP or HTTPS:

  • The first letter of the header's key and every letter following a hyphen (-) is capitalized to preserve compatibility with HTTP/1.1 clients. For example, user-agent is changed to User-Agent, and content-encoding is changed to Content-Encoding.

  • Certain headers, such as Accept-CH (client hints), are converted to match standard mixed letter representation.

• Some headers are added, or values are appended to them. The external HTTP(S) load balancers always add or modify certain headers such as Via and X-Forwarded-For.

Limitations

• You can't use internet NEGs as backends for regional external HTTP(S) load balancers.
• Review the backend service section for limitations associated with configuring internet NEGs as backends.
• No health checking is performed for external backends. If your external backend becomes unreachable or if it is specified as an FQDN but cannot be resolved, the load balancer sends a 502 (Bad Gateway) message in response to user requests.
• When you modify a load balancer to change the backend from an internet NEG to any other backend type, or change the backend from any other backend type to an internet NEG, your application experiences a temporary downtime of about 30-90 seconds. During this time, clients see 502 errors with the error code failed_to_connect_to_backend. This is expected behavior.
• The following advanced traffic management features aren't supported with internet NEG backends:
  • Request mirroring
  • Retry policies
  • Traffic policies (including load balancing locality policy, session affinity, and outlier detection)

Quota

You can configure as many NEGs with external network endpoints as permitted by your existing network endpoint group quota. For more information, see the NEG backends quota and the Endpoints per NEG quota.

Pricing

Egress traffic to an internet NEG's endpoint is charged at internet egress rates for Premium Tier networking. The source is based on the client location, and the destination is based on the location of your public endpoint.

For more information, see Cloud Load Balancing pricing.

What's next

• Set up a global external HTTP(S) load balancer with an internet NEG.
• Set up a global external HTTP(S) load balancer (classic) with an internet NEG.
• Set up Cloud CDN with an external backend.
• Read the Traffic Director with internet network endpoint groups overview.