Internet network endpoint groups overview

A network endpoint group (NEG) defines a set of backend endpoints for a load balancer. An internet NEG is a backend that resides outside of Google Cloud. You can use an internet NEG as the backend for a backend service for a Google Cloud external HTTP(S) load balancer. You should do this when you want to serve content from an origin that is hosted outside of Google Cloud, and you want your external HTTP(S) load balancer to be the frontend.

This allows you to:

• Use Google Edge infrastructure for terminating your user connections.
• Direct the connections to your custom origin.
• Use Cloud CDN for your custom origin.
• Deliver traffic to your public endpoint across Google's private backbone, which improves reliability and can decrease latency between client and server.

This document discusses using internet network endpoint groups with external HTTP(S) Load Balancing. You cannot use internet NEGs with other load balancer types.

The terms "custom origin," "external endpoint," and "internet endpoint" are often used interchangeably because they have the same meaning. This document uses the terms internet NEG as the resource that contains an internet endpoint.

For information about zonal NEGs, see Zonal network endpoint group overview.

For information about serverless NEGs, see Serverless network endpoint group overview.

Overview

Internet network endpoint groups (NEGs) are global resources that are hosted within on-premises infrastructure or on infrastructure provided by third-party providers.

You can use an internet NEG as a backend in a backend service for an external HTTP(S) load balancer, and assign the internet NEG as the origin for Cloud CDN.

Endpoint types

When you create an internet network endpoint group, you specify the network endpoint type of INTERNET_FQDN_PORT or INTERNET_IP_PORT:

Endpoint address	Type	Definition	When to use
Hostname and an optional port	INTERNET_FQDN_PORT	A publicly resolvable fully qualified domain name and an optional port, for example backend.example.com:443 (default ports: 80 for HTTP and 443 for HTTPS)	Use this endpoint when your external origin can be resolved through a fully-qualified domain name with public DNS.
IP address and an optional port	INTERNET_IP_PORT	A publicly accessible IP address and an optional port, for example 8.8.8.8 or 8.8.8.8:443 (default ports: 80 for HTTP and 443 for HTTPS)	Use this endpoint to specify a publicly accessible IP address and a port to connect to.

In an internet NEG, you can't have more than one endpoint. Internet network endpoint groups don't support the GCE_VM_IP_PORT endpoint type.

Load balancing components and specifications

You can use an internet network endpoint group in a load balancer using the Premium network service tier.

However, because only one internet endpoint is allowed in each internet NEG, load balancing isn't actually performed. This means that you can't use any of the load balancing modes, such as rate or utilization. The load balancer serves as the frontend only, and it proxies traffic to the specified internet endpoint.

The forwarding selection is based on a URL map. For target HTTP(S) proxies, the backend service used is determined by checking the request host name and path in the URL map. HTTP(S) load balancers can have multiple backend services referenced from the URL map.

• Each external HTTP(S) load balancer has its own global external forwarding rule to direct traffic to the appropriate target proxy object.

• If the URL map sends the request to a backend service that contains an internet NEG, the backend service directs traffic to that internet NEG.

The following illustration shows an external HTTP(S) load balancer with multiple backend types, one of which is an internet NEG.

Internet network endpoint groups in load balancing (click to enlarge)

Backend service

As shown in the previous section, an internet NEG is one type of backend supported by a backend service of an external HTTP(S) load balancer. It allows you to use Google's global edge infrastructure to terminate user requests in front of a custom origin.

Important: We strongly recommend you use HTTPS or HTTP/2 as the protocol when configuring a backend service with an internet NEG, so that communication between the external HTTP(S) load balancer and your backend is encrypted and authenticated when transiting the public internet.

When you add an internet NEG as a backend on a backend service:

• The backend service cannot also use zonal NEGs or instance groups as backends. All backends on a backend service must be of the same type.

• You can add only one internet NEG backend to the same backend service.

• You can add only one endpoint to an internet NEG.

• The backend service cannot reference a health check.

• The backend service's load balancing scheme must be EXTERNAL and its protocol must be one of HTTP, HTTPS, or HTTP2.

• Backend service features are supported when using an internet NEG as a backend. Such features include:

  • Cloud CDN
  • Google Cloud Armor
  • Identity-Aware Proxy
  • User-defined request headers

Health checks

A backend service that uses an internet NEG as a backend doesn't support a health check. Google Cloud doesn't provide health checking for any internet endpoint or custom origin.

If your internet endpoint becomes unreachable or if the configured hostname (FQDN) cannot be resolved, the external HTTP(S) load balancer returns an HTTP 502 (Bad Gateway) response to its clients.

Authenticating requests

To allow an external HTTP(S) load balancer to send requests to your internet NEG:

• Query the _cloud-eoips.googleusercontent.com DNS TXT record using a tool like dig or nslookup. Note the CIDRs (following ip4:) and ensure these ranges are allowed by your firewall or cloud access control list (ACL). For an example, see Allowlisting the necessary IP ranges.
• You can set a custom header to indicate that the request came from a Google Cloud external HTTP(S) load balancer using a custom request header
  • For example, you can use 16 or more cryptographically random bytes as a shared key.
• You can also enable Identity-Aware Proxy (IAP), and verify that the signed JWT in the request header is signed by Google and that the aud (audience) claim contains the project number where your external HTTP(S) load balancer is defined. Note that IAP isn't compatible with Cloud CDN.

SSL Server certification validation and SAN validation

When using HTTPS or HTTP/2 as the backend protocol, we strongly recommend that you use INTERNET_FQDN_PORT to create your internet endpoint.

When you create an internet endpoint using INTERNET_FQDN_PORT, the load balancer validates the SSL server certificate presented by the internet endpoint and verifies that:

• The certificate is signed by well-known CAs.
• The certificate is not expired.
• The certificate signature is valid.
• The configured FQDN matches one of the Subject Alternative Names (SANs) in the certificate.

Important: If you create the internet endpoint using INTERNET_IP_PORT, SSL server certificate validation, including SAN validation, is not performed. Therefore, when using the HTTPS or HTTP/2 backend protocol with the custom origin, we strongly recommended that you use the INTERNET_FQDN_PORT endpoint type to define the custom origin.

Support for the SSL Server Name Indication (SNI) extension

SNI is supported only when you use INTERNET_FQDN_PORT with HTTPS or HTTP/2 as the backend protocol. In this case, the configured FQDN is sent an SNI in the client hello during the SSL handshake between load balancer and the external endpoint. The SNI is not sent when you configure the endpoint as an INTERNET_IP_PORT, because IP address literals aren't allowed in the HostName field of an SNI payload.

IP address resolution for FQDN endpoints

When an INTERNET_FQDN_PORT endpoint points to a DNS record that returns multiple IP addresses, the IP address is resolved in the following way:

• The external HTTP(S) load balancer attempts to connect to the first IP address in the DNS response. If that IP address isn't reachable, the load balancer returns an HTTP 502 (Bad Gateway) response. This is true even if other IP addresses from the DNS response are available.

• The external HTTP(S) load balancer uses a DNS resolver in a Google Cloud region that is closest to its client on the internet. If the DNS record for your INTERNET_FQDN_PORT endpoint returns different IP addresses based on the location of the client, make sure that each of those IP addresses can be reached by the load balancer.

For more information on the IP ranges and locations used by Google's DNS resolver infrastructure, see the Google public DNS documentation.

Logs

Requests proxied to an internet endpoint are logged to Cloud Logging in the same way that requests for other HTTP(S) Load Balancing backends are logged. For more information, see HTTP(S) Load Balancing logging and monitoring.

If you enable Cloud CDN for an internet endpoint, cache hits are also logged.

Header processing

When an external HTTP(S) load balancer proxies requests to an internet endpoint, it adjusts HTTP headers in the following ways:

• Some headers are coalesced. When there are multiple instances of the same header key (for example, Via), the load balancer combines their values into a single comma-separated list for a single header key. Only the headers whose values can be represented as a comma-separated list are coalesced. Other headers such as Set-Cookie, are never coalesced.

• Header are proper-cased when the backend service's protocol is HTTP or HTTPS:

  • The first letter of the header's key, and every letter following a hyphen (-), is capitalized to preserve compatibility with HTTP/1.1 clients. For example, user-agent is changed to User-Agent, and content-encoding is changed to Content-Encoding.

  • Certain headers (such TE (transfer encoding), Accept-CH (client hints) are converted to match standard mixed letter representation.

• Some headers are added, or values are appended to them. The external HTTP(S) load balancers always add or modify certain headers, such as Via and X-Forwarded-For.

Limitations

• An internet endpoint with an FQDN defined must be resolvable by Google Public DNS. Names that cannot be resolved by the public DNS system will not be usable as an internet endpoint.
• An internet endpoint must be a publicly-routable IPv4 address or must resolve to one:
  • An internet endpoint cannot be an RFC 1918 address.
  • It must be reachable over the internet. The endpoint cannot be only reachable over Cloud VPN or Cloud Interconnect.
  • If the internet endpoint references a Google API or service, the service must be reachable via TCP port 80 or 443 using the HTTP, HTTPS or HTTP/2 protocol.
• You can only use internet NEGs on the Premium network service tier, which is the default tier. For more information, see the Network Service Tiers documentation.
• You must use IP:{optional port} for the INTERNET_IP_PORT endpoint or FQDN:{optional port} for the INTERNET_FQDN_PORT.
• When using a NEG with endpoint type INTERNET_FQDN_PORT or INTERNET_IP_PORT, you can add only one endpoint to the NEG. This means that when using a NEG with a INTERNET_FQDN_PORT or INTERNET_IP_PORT network endpoint type in a backendService, you can't attach multiple NEGs to the backendService.
• Load balancing is currently not supported on internet endpoints – the requests are only proxied to the endpoint. Google Edge infrastructure terminates your user connections and then directs the connections to your internet endpoint.
• You can use internet NEGs without Cloud CDN enabled. With this configuration, you still can't attach multiple NEGs to the backend service. That is, the restriction of a single endpoint remains.
• No health checking is performed for the internet NEG. If your internet endpoint becomes unreachable or if it is specified as an FQDN but cannot be resolved, Cloud CDN sends a 502 (Bad Gateway) message in response to user requests.
• When using a custom origin that expects a particular value for the HTTP request's Host header, you must configure the backend service to set the Host header to that expected value. If you don't configure a custom request header, a backend service preserves the Host header that the client used to connect to the Google Cloud external HTTP(S) load balancer. For general information about custom request headers, see Creating custom request headers. For a specific example, see Configuring a load balancer with a custom origin.

Quota

You can configure as many NEGs with external network endpoints as permitted by your existing network endpoint group quota. For more information, see NEG backends and Endpoints per NEG.

Pricing

Egress traffic to an internet NEG's endpoint (type INTERNET_FQDN_PORT or INTERNET_IP_PORT) is charged at internet egress rates for Premium Tier networking.

The source is based on the client location, and the destination is based on the location of your public endpoint.

For more information, see Cloud CDN pricing for custom origins.

What's next

• Setting up an external HTTP(S) load balancer with an internet NEG