Internet network endpoint groups overview

An internet network endpoint group (NEG) defines an external backend for a load balancer. An external backend is a backend that resides outside of Google Cloud. You can use this type of backend with an external HTTP(S) load balancer. You should do this when you want to serve content from an external backend, and you want your external HTTP(S) load balancer to be the frontend.

This allows you to:

Use Google Edge infrastructure for terminating your user connections.
Direct the connections to your external backend.
Use Cloud CDN to cache content for your external backend.
Deliver traffic to your public endpoint across Google's private backbone, which improves reliability and can decrease latency between client and server.

This document discusses using external backends with the global external HTTP(S) load balancer (classic). You cannot use external backends with the global external HTTP(S) load balancer with advanced traffic management capability, the regional external HTTP(S) load balancer, or with any other load balancer types. However, you can use external backends with Traffic Director. To learn how, see Traffic Director with internet network endpoint groups.

For information about zonal NEGs, see Zonal network endpoint group overview.

For information about serverless NEGs, see Serverless network endpoint group overview.

Terminology

The following terms are sometimes used interchangeably because they have the same or similar meanings:

external backend: A backend that resides outside of Google Cloud and is reachable across the internet. The endpoint in an internet NEG.
custom origin: Same as an external backend. In CDN, origin is the industry-standard term for a backend instance that serves web content.
internet network endpoint group (NEG): The Google Cloud API resource that you use to specify an external backend.
external endpoint: Same as an external backend.

This document uses the term external backend except when referring to the internet NEG API resource.

Overview

Internet NEGs are global resources that are hosted within on-premises infrastructure or on infrastructure provided by third-party providers.

Endpoint types

When you create an internet NEG, you specify the network endpoint type of INTERNET_FQDN_PORT or INTERNET_IP_PORT:

Endpoint address	Type	Definition	When to use
Hostname and an optional port	`INTERNET_FQDN_PORT`	For external HTTP(S) Load Balancing, a publicly resolvable fully qualified domain name and an optional port, for example `backend.example.com:443` (default ports: `80` for HTTP and `443` for HTTPS). For Traffic Director, the VPC network's name-resolution order is used to resolve the FQDN.	Use this endpoint with external HTTP(S) Load Balancing when your external backend can be resolved through a fully-qualified domain name with public DNS. Use this endpoint with Traffic Director to send traffic to external services.
IP address and an optional port	`INTERNET_IP_PORT`	A publicly accessible IP address and an optional port, for example `8.8.8.8` or `8.8.8.8:443` (default ports: `80` for HTTP and `443` for HTTPS)	Use this endpoint only with external HTTP(S) Load Balancing to specify a publicly accessible IP address and a port to connect to.

In an internet NEG, you can't have more than one endpoint. Internet network endpoint groups don't support the GCE_VM_IP_PORT endpoint type.

The sections that follow explain how external backends are used with external HTTP(S) load balancers. When you use an external backends with Traffic Director, DNS, health checking, and traffic routing behave differently. For more information, see Traffic Director with internet network endpoint groups.

Load balancing components and specifications

You can use an external backend in a load balancer using the Premium network service tier.

However, because only one endpoint is allowed in each internet NEG, load balancing isn't actually performed. This means that you can't use any of the load balancing modes, such as rate or utilization. The load balancer serves as the frontend only, and it proxies traffic to the specified external backend.

The forwarding selection is based on a URL map. For target HTTP(S) proxies, the backend service used is determined by checking the request host name and path in the URL map. External HTTP(S) load balancers can have multiple backend services referenced from the URL map.

Each external HTTP(S) load balancer has its own global external forwarding rule to direct traffic to the appropriate target proxy object.
If the URL map sends the request to a backend service that contains an internet NEG, the backend service directs traffic to that external backend.

The following illustration shows an external HTTP(S) load balancer with multiple backend types, one of which is an external backend.

Internet network endpoint groups in load balancing (click to enlarge)

Backend service

As shown in the previous section, an internet NEG is one type of backend supported by a backend service of an external HTTP(S) load balancer. It allows you to use Google's global edge infrastructure to terminate user requests in front of an external backend.

When you add an internet NEG to a backend service:

The backend service cannot also use zonal NEGs or instance groups as backends. All backends on a backend service must be of the same type.
You can add only one internet NEG backend to the same backend service.
You can add only one endpoint to an internet NEG.
The backend service cannot reference a health check.
The backend service's load balancing scheme must be EXTERNAL and its protocol must be one of HTTP, HTTPS, or HTTP2.
Backend service features are supported when using an internet NEG. Such features include:

Health checks

A backend service with an internet NEG doesn't support a health check. Google Cloud doesn't provide health checking for any external backend.

If your external backend becomes unreachable or if the configured hostname (FQDN) cannot be resolved, the external HTTP(S) load balancer returns an HTTP 502 (Bad Gateway) response to its clients.

Authenticating requests

To allow an external HTTP(S) load balancer to send requests to your external backend:

Query the _cloud-eoips.googleusercontent.com DNS TXT record using a tool like dig or nslookup.

Examples:
```
dig _cloud-eoips.googleusercontent.com txt @8.8.8.8
```
```
nslookup -q=TXT _cloud-eoips.googleusercontent.com 8.8.8.8
```
Note the CIDRs (following ip4:) and ensure these ranges are allowed by your firewall or cloud access control list (ACL). For an example, see Allowlisting the necessary IP ranges.
You can set a custom header to indicate that the request came from a Google Cloud external HTTP(S) load balancer using a custom request header
- For example, you can use 16 or more cryptographically random bytes as a shared key.
You can also enable Identity-Aware Proxy (IAP), and verify that the signed JWT in the request header is signed by Google and that the aud (audience) claim contains the project number where your external HTTP(S) load balancer is defined. Note that IAP isn't compatible with Cloud CDN.

SSL Server certification validation and SAN validation

When using HTTPS or HTTP/2 as the backend protocol, we strongly recommend that you use INTERNET_FQDN_PORT to create your external backend.

When you create an external backend using INTERNET_FQDN_PORT, the load balancer validates the SSL server certificate presented by the external backend and verifies that:

The certificate is signed by well-known CAs.
The certificate is not expired.
The certificate signature is valid.
The configured FQDN matches one of the Subject Alternative Names (SANs) in the certificate.

Support for the SSL Server Name Indication (SNI) extension

SNI is supported only when you use INTERNET_FQDN_PORT with HTTPS or HTTP/2 as the backend protocol. In this case, the configured FQDN is sent an SNI in the client hello during the SSL handshake between load balancer and the external endpoint. The SNI is not sent when you configure the endpoint as an INTERNET_IP_PORT, because IP address literals aren't allowed in the HostName field of an SNI payload.

IP address resolution for FQDN endpoints

When an INTERNET_FQDN_PORT endpoint points to a DNS record that returns multiple IP addresses, the IP address is resolved in the following way:

The external HTTP(S) load balancer attempts to connect to the first IP address in the DNS response. If that IP address isn't reachable, the load balancer returns an HTTP 502 (Bad Gateway) response. This is true even if other IP addresses from the DNS response are available.
The external HTTP(S) load balancer uses a DNS resolver in a Google Cloud region that is closest to its client on the internet. If the DNS record for your INTERNET_FQDN_PORT endpoint returns different IP addresses based on the location of the client, make sure that each of those IP addresses can be reached by the load balancer.

For more information on the IP ranges and locations used by Google's DNS resolver infrastructure, see the Google public DNS documentation.

Logs

Requests proxied to an external backend are logged to Cloud Logging in the same way that requests for other HTTP(S) Load Balancing backends are logged. For more information, see HTTP(S) Load Balancing logging and monitoring.

If you enable Cloud CDN for an external backend, cache hits are also logged.

Header processing

When an external HTTP(S) load balancer proxies requests to an external backend, it adjusts HTTP headers in the following ways:

Some headers are coalesced. When there are multiple instances of the same header key (for example, Via), the load balancer combines their values into a single comma-separated list for a single header key. Only the headers whose values can be represented as a comma-separated list are coalesced. Other headers such as Set-Cookie, are never coalesced.
Header are proper-cased when the backend service's protocol is HTTP or HTTPS:
- The first letter of the header's key, and every letter following a hyphen (-), is capitalized to preserve compatibility with HTTP/1.1 clients. For example, user-agent is changed to User-Agent, and content-encoding is changed to Content-Encoding.
- Certain headers (such TE (transfer encoding), Accept-CH (client hints) are converted to match standard mixed letter representation.
Some headers are added, or values are appended to them. The external HTTP(S) load balancers always add or modify certain headers, such as Via and X-Forwarded-For.

Limitations

You cannot use internet NEGs as backends for Regional external HTTP(S) load balancers.
An external backend with an FQDN defined must be resolvable by Google Public DNS. Names that cannot be resolved by the public DNS system will not be usable as an external backend.
An external backend must be a publicly-routable IPv4 address or must resolve to one:
- An external backend cannot be an RFC 1918 address.
- It must be reachable over the internet. The endpoint cannot be only reachable over Cloud VPN or Cloud Interconnect.
- If the external backend references a Google API or service, the service must be reachable via TCP port 80 or 443 using the HTTP, HTTPS or HTTP/2 protocol.
You can only use internet NEGs on the Premium network service tier, which is the default tier. For more information, see the Network Service Tiers documentation.
You must use IP:{optional port} for the INTERNET_IP_PORT endpoint or FQDN:{optional port} for the INTERNET_FQDN_PORT.
When using a NEG with endpoint type INTERNET_FQDN_PORT or INTERNET_IP_PORT, you can add only one endpoint to the NEG. This means that when using a NEG with a INTERNET_FQDN_PORT or INTERNET_IP_PORT network endpoint type in a backendService, you can't attach multiple NEGs to the backendService.
Load balancing is currently not supported on external backends – the requests are only proxied to the endpoint. Google Edge infrastructure terminates your user connections and then directs the connections to your external backend.
You can use internet NEGs without Cloud CDN enabled. With this configuration, you still can't attach multiple internet NEGs to the backend service. That is, the restriction of a single endpoint remains.
No health checking is performed for the external backend. If your external backend becomes unreachable or if it is specified as an FQDN but cannot be resolved, Cloud CDN sends a 502 (Bad Gateway) message in response to user requests.
When using an external backend that expects a particular value for the HTTP request's Host header, you must configure the backend service to set the Host header to that expected value. If you don't configure a custom request header, a backend service preserves the Host header that the client used to connect to the Google Cloud external HTTP(S) load balancer. For general information about custom request headers, see Creating custom request headers. For a specific example, see Configuring a load balancer with an external backend.

Quota

You can configure as many NEGs with external network endpoints as permitted by your existing network endpoint group quota. For more information, see NEG backends and Endpoints per NEG.

Pricing

Egress traffic to an internet NEG's endpoint (type INTERNET_FQDN_PORT or INTERNET_IP_PORT) is charged at internet egress rates for Premium Tier networking.

The source is based on the client location, and the destination is based on the location of your public endpoint.

For more information, see Cloud CDN pricing for external backends.