Network Endpoint Groups in Load Balancing Concepts

In addition to instance groups, you can use network endpoint groups (NEGs) as the backend for a backend service. This document discusses using network endpoint groups in HTTP(S), TCP proxy, and SSL proxy load balancers. You cannot use NEGs as a backend with internal load balancers or network load balancers.

Overview

Network endpoint groups (NEGs) are zonal resources that represent collections of IP address and port combinations for GCP resources within a single subnet. Each IP address and port combination is called a network endpoint.

Network endpoint groups can be used as backends in backend services for HTTP(S), TCP proxy, and SSL proxy load balancers. You cannot use NEGs as a backend with internal load balancers. Because NEG backends allow you to specify IP addresses and ports, you can distribute traffic in a granular fashion among applications or containers running within VM instances.

Endpoint relationships

When you create a network endpoint group, you select a zone, a network, and a subnet. Every endpoint IP address must be in the same subnet as the NEG.

If the network you select is an auto mode network, you can omit specifying the subnet. However, a subnet is still associated with the NEG. If you specify an auto mode network but do not specify a subnet when creating a NEG, the subnet it uses is the automatically-created subnet in the region that contains the zone that you selected for the NEG.

Network endpoint groups currently only support the GCE_VM_IP_PORT type. This means that individual endpoints must have IP addresses associated with GCP VMs. When adding endpoints to a NEG:

  • You must specify the name of a VM instance. The VM instance must be in the same zone as the NEG, and its network interface in the VPC network must be in a subnet in the same region that contains that zone.

  • You can have up to 10,000 endpoints in a NEG. All 10,000 endpoints can be on the same VM. Each endpoint is created by referencing the VM when adding an endpoint to an existing NEG. Endpoints created in different NEGs can point to the same VM.

    • You can specify an IP address or an IP address and port combination in addition to specifying the VM instance. Any IP address you specify must either be the primary internal IP address of the VM or an alias IP for the VM in the same subnet as the NEG.

    • If you do not specify an IP address when you add an endpoint, the VM's primary internal IP address in the VPC network is used.

    • If you do not specify a port when adding an endpoint, you must have defined a default port for the NEG. Endpoints use the default port unless they specify a port of their own. This means that you must either specify a default port when you create a NEG, or you must specify a port for each and every endpoint you add to it.

    • Every endpoint in a NEG must be a unique IP address and port combination. If a port is not specified for an IP address, the combination is created by using the IP and the default port.

Endpoints, containers, and services

To create a unique network endpoint for a container or application running in a VM, you must either use the primary IP address of the VM or a secondary IP assigned to the VM using the alias IP address feature. The software running in the container or the application running in the VM should be configured to bind to the IP address used by the network endpoint.

NEGs are especially useful for GKE. For information on using NEGs with GKE, refer to Using Container-native Load Balancing.

NEGs are also useful because they allow you to create logical groupings of IP addresses and ports representing software services instead of entire VMs. IP addresses for microservices (running in GCP VMs) managed by other orchestrators like Apache Mesos or Cloud Foundry can be endpoints.

Load balancing

Backend services

NEGs can be used as backends for backend services in a load balancer. When you use a NEG as a backend for a backend service, all other backends in that backend service must also be NEGs. You cannot use instance groups and NEGs as backends in the same backend service.

You can add the same network endpoint (IP address and port combination) to more than one NEG. You can use the same NEG as a backend for more than one backend service.

Backend services using NEGs for backends can only use balancing modes of RATE or CONNECTION. You cannot use a balancing mode of UTILIZATION for backend services that use NEGs as backends.

HTTP(S), TCP Proxy, and SSL Proxy load balancing

You can use network endpoint groups in load balancers using either Standard or Premium network service tiers.

The following illustrations show configuration components for HTTP(S) load balancing and TCP/SSL Proxy load balancing where NEGs are the backends:

  • A global forwarding rule directs traffic from a global external IP address to the appropriate target proxy object.

  • For target HTTP(S) proxies, the backend service used is determined by checking the request host name and path in the URL map. HTTP(S) load balancers can have multiple backend services referenced from the URL map.

  • For target TCP or target SSL proxies, only one backend service can be defined.

  • The backend service directs traffic to its backend NEGs. For each request, the load balancer picks a network endpoint from one of the NEGs and sends the traffic there.

Network endpoint groups in load balancing (click to enlarge)
Network endpoint groups in load balancing (click to enlarge)

Load balancing with containers

The following example demonstrates how to distribute traffic among microservices running in containers on your VMs. The VMs are configured to use alias IP ranges from their subnets, and those ranges are the addresses used by the containers. Either HTTP(S), TCP Proxy, or SSL Proxy load balancing is used.

Load balancing network endpoint groups with containers (click to enlarge)
Load balancing network endpoint groups with containers (click to enlarge)

This example can be set up as follows:

  1. Configure containers or services on the VMs. If multiple containers should run on each VM or if you need IP addresses for the containers, configure alias IP addresses for the VMs. If you are configuring services, you need two or more services running on the same VM such that at least the port numbers are different.
  2. Create network endpoint groups. If you are using Kubernetes or Google Kubernetes Engine, this step is not needed because the Ingress controller creates the NEGs.
  3. Add network endpoints to the network endpoint groups.
  4. Create a health check.
  5. Create a backend service.
  6. For HTTP(S) Load Balancing, create a URL map and connect the backend service to it.
  7. Create the appropriate type of target proxy, such as a target HTTP proxy, target HTTPS proxy, target SSL proxy, or target TCP proxy. Link the target proxy to the URL map (for HTTP(S) load balancing) or the backend service (for TCP Proxy and SSL Proxy load balancing).
  8. Create a global forwarding rule and link it to the target proxy.

See Load balancing network endpoint group example for an example configured using the gcloud command-line tool.

Restrictions

  • You cannot use NEGs with legacy networks.
  • The IP address for a network endpoint must be a primary or alias IP that belongs to the specified VM instance.

Limits

  • Only HTTP(S), TCP Proxy, and SSL Proxy Load Balancing are supported by NEGs.
  • Only RATE balancing mode is supported by NEGs for HTTP(s) load balancing, while CONNECTION is supported for TCP/SSL load balancing. Utilization-based load balancing is not supported.
  • Each NEG can contain up to 10,000 endpoints.
  • The maximum number of NEGs you can have per project is at least 100. Contact your GCP sales team if you need to increase this limit.
  • A backend service that uses NEGs as backends cannot also use instance groups as backends.
  • Each backend service can have up to 50 NEGs. The NEGs can be in the same zone or different zones.
  • Only internal (RFC 1918) IP addresses can currently be added to a NEG.

Troubleshooting

Traffic does not reach the endpoints

After the service is configured, new endpoints generally become reachable after attaching them to the NEG, provided they respond to health checks.

If traffic cannot reach the endpoints, which results in a 502 error code for HTTP(s) or rejected connections for TCP/SSL load balancers, check the following:

  • Verify that firewall rules allow incoming TCP traffic to your endpoints from following ranges: 130.211.0.0/22 and 35.191.0.0/16.
  • Verify that your endpoints are healthy by using gcloud, as shown below, or by calling the getHealth API on the backend service resource or the listEndpoints API on the NEG with the showHealth parameter set to SHOW. The following gcloud command shows health information by network endpoint:
    gcloud beta compute network-endpoint-groups list-network-endpoints --zone=ZONE

What's next

Was this page helpful? Let us know how we did:

Send feedback about...

Load Balancing