You can use network endpoint groups (NEGs) as the backend for a backend service. The primary use case for this configuration is deploying containers on your VMs so that you can run services in the containers. You can also distribute traffic in a granular fashion to applications running on the VMs.
This document discusses using network endpoint groups with the following load balancer types:
You cannot use NEGs as a backend with the following load balancer types:
Network endpoint groups (NEGs) are zonal resources that represent collections of IP address and port combinations for GCP resources within a single subnet. Each IP address and port combination is called a network endpoint.
Network endpoint groups can be used as backends in backend services for HTTP(S), Internal HTTP(S) Load Balancing, TCP proxy, and SSL proxy load balancers. You cannot use NEGs as a backend with internal TCP/UDP load balancers. Because NEG backends allow you to specify IP addresses and ports, you can distribute traffic in a granular fashion among applications or containers running within VM instances.
When you create a network endpoint group, you select a zone, a network, and a subnet. Every endpoint IP address must be in the same subnet as the NEG.
If the network you select is an auto mode network, you can omit specifying the subnet. However, a subnet is still associated with the NEG. If you specify an auto mode network but do not specify a subnet when creating a NEG, the subnet it uses is the automatically-created subnet in the region that contains the zone that you selected for the NEG.
Network endpoint groups currently only support the
This means that individual endpoints must have IP addresses associated with
GCP VMs. When adding endpoints to a NEG:
You must specify the name of a VM instance. The VM instance must be in the same zone as the NEG, and its network interface in the VPC network must be in a subnet in the same region that contains that zone.
You can have up to 10,000 endpoints in a NEG. All 10,000 endpoints can be on the same VM. Each endpoint is created by referencing the VM when adding an endpoint to an existing NEG. Endpoints created in different NEGs can point to the same VM.
You can specify an IP address or an IP address and port combination in addition to specifying the VM instance. Any IP address you specify must either be the primary internal IP address of the VM or an alias IP for the VM in the same subnet as the NEG.
If you do not specify an IP address when you add an endpoint, the VM's primary internal IP address in the VPC network is used.
If you do not specify a port when adding an endpoint, you must have defined a default port for the NEG. Endpoints use the default port unless they specify a port of their own. This means that you must either specify a default port when you create a NEG, or you must specify a port for each and every endpoint you add to it.
Every endpoint in a NEG must be a unique IP address and port combination. If a port is not specified for an IP address, the combination is created by using the IP and the default port.
Endpoints, containers, and services
To create a unique network endpoint for a container or application running in a VM, you must either use the primary IP address of the VM or a secondary IP assigned to the VM using the alias IP address feature. The software running in the container or the application running in the VM should be configured to bind to the IP address used by the network endpoint.
NEGs are especially useful for GKE. For information on using NEGs with GKE, refer to Using Container-native Load Balancing.
NEGs are also useful because they allow you to create logical groupings of IP addresses and ports representing software services instead of entire VMs. IP addresses for microservices (running in GCP VMs) managed by other orchestrators like Apache Mesos or Cloud Foundry can be endpoints.
NEGs can be used as backends for backend services in a load balancer. When you use a NEG as a backend for a backend service, all other backends in that backend service must also be NEGs. You cannot use instance groups and NEGs as backends in the same backend service.
You can add the same network endpoint (IP address and port combination) to more than one NEG. You can use the same NEG as a backend for more than one backend service.
Backend services using NEGs for backends can only use balancing modes of
CONNECTION. You cannot use a balancing mode of
for backend services that use NEGs as backends.
HTTP(S), Internal HTTP(S), TCP Proxy, and SSL Proxy load balancing
You can use network endpoint groups in load balancers using either Standard or Premium network service tiers.
The following illustrations show configuration components for HTTP(S) load balancers, TCP/SSL Proxy load balancers and Internal HTTP(S) load balancers where NEGs are the backends:
Each Premium Tier – HTTP(S), SSL Proxy, and TCP Proxy load balancer has its own global external forwarding rule to direct traffic to the appropriate target proxy object.
Each standard Tier – HTTP(S), SSL Proxy, and TCP Proxy load balancer has its own regional external forwarding rule to direct traffic to the appropriate target proxy object.
Each internal HTTP(S) (Beta) load balancer has its own regional internal managed forwarding rule to direct traffic to the appropriate target proxy object.
For target HTTP(S) proxies, the backend service used is determined by checking the request host name and path in the URL map. External HTTP(S) and internal HTTP(S)load balancers can have multiple backend services referenced from the URL map.
For target TCP or target SSL proxies, only one backend service can be defined.
The backend service directs traffic to its backend NEGs. For each request, the load balancer picks a network endpoint from one of the NEGs and sends the traffic there.
Load balancing with containers
The following example demonstrates how to distribute traffic among microservices running in containers on your VMs. The VMs are configured to use alias IP ranges from their subnets, and those ranges are the addresses used by the containers. Either HTTP(S), Internal HTTP(S), TCP Proxy, or SSL Proxy load balancing is used.
This example can be set up as follows:
- Configure containers or services on the VMs. If multiple containers should run on each VM or if you need IP addresses for the containers, configure alias IP addresses for the VMs. If you are configuring services, you need two or more services running on the same VM such that at least the port numbers are different.
- Create network endpoint groups. If you are using Kubernetes or Google Kubernetes Engine, this step is not needed because the Ingress controller creates the NEGs.
- Add network endpoints to the network endpoint groups.
- Create a health check.
- Create a backend service.
- For HTTP(S) Load Balancing, create a URL map and connect the backend service to it.
- Create the appropriate type of target proxy, such as a target HTTP proxy, target HTTPS proxy, target SSL proxy, or target TCP proxy. Link the target proxy to the URL map (for HTTP(S) load balancing) or the backend service (for TCP Proxy and SSL Proxy load balancing).
- Create a forwarding rule and link it to the target proxy.
See Load balancing network endpoint group example
for an example configured using the
gcloud command-line tool.
- You cannot use NEGs with legacy networks.
- The IP address for a network endpoint must be a primary or alias IP that belongs to the specified VM instance.
- NEGs are only usable as backends for load balancers. The following load balancer types support NEGs: HTTP(S), Internal HTTP(S), TCP Proxy, and SSL Proxy.
- Only RATE balancing mode is supported by NEGs for HTTP(s) load balancing, while CONNECTION is supported for TCP/SSL load balancing. Utilization-based load balancing is not supported.
- Each NEG can contain up to 10,000 endpoints.
- The maximum number of NEGs you can have per project is at least 100. Contact your GCP sales team if you need to increase this limit.
- A backend service that uses NEGs as backends cannot also use instance groups as backends.
- Each backend service can have up to 50 NEGs. The NEGs can be in the same zone or different zones.
- Only internal (RFC 1918) IP addresses can currently be added to a NEG.
Traffic does not reach the endpoints
After the service is configured, new endpoints generally become reachable after attaching them to the NEG, provided they respond to health checks.
If traffic cannot reach the endpoints, which results in a 502 error code for HTTP(s) or rejected connections for TCP/SSL load balancers, check the following:
- Verify that firewall rules allow incoming TCP traffic
to your endpoints from following ranges:
- Verify that your endpoints are healthy by using
gcloud, as shown below, or by calling the
getHealthAPI on the backend service resource or the
listEndpointsAPI on the NEG with the
showHealthparameter set to
SHOW. The following
gcloudcommand shows health information by network endpoint:
gcloud compute network-endpoint-groups list-network-endpoints --zone=ZONE
- For information on configuring NEGs, see Setting Up Network Endpoint Groups in Load Balancing.
- For information on using network endpoint groups in Google Kubernetes Engine, see Using Container-native Load Balancing.