A network endpoint group (NEG) is a configuration object that specifies a group of backend endpoints or services. Zonal NEGs are zonal resources that represent collections of either IP addresses or IP address/port combinations for Google Cloud resources within a single subnet.
NEGs are useful because they allow you to create logical groupings of IP addresses and ports representing software services instead of entire VMs. IP addresses for microservices (running in Google Cloud VMs) managed by other orchestrators like Apache Mesos or Cloud Foundry can be endpoints.
For information about other NEG types see:
- Network endpoint groups overview
- Internet network endpoint groups overview
- Serverless network endpoint groups overview
There are two types of zonal NEGs, depending on the type of network endpoints that make up the NEG. Each of the two types of zonal NEG supports different use cases and load balancer types.
- GCE_VM_IP_PORT NEGs. Supported as backends for External HTTP(S), Internal HTTP(S), TCP proxy, and SSL proxy load balancers.
- GCE_VM_IP NEGs. Supported as backends for internal TCP/UDP load balancers.
You cannot use zonal NEGs as a backend with Network Load Balancing.
GCE_VM_IP NEGs
These zonal NEGs contain one or more internal network endpoints that resolve to a Compute Engine VM's primary IP address. You cannot specify a port with this type of zonal NEG.
These endpoints can only be used as backends in backend services for internal TCP/UDP load balancers.
With GCE_VM_IP
NEGs, you can only attach endpoints that belong to a VM's
primary internal IP address in NEG's VPC network. Primary
internal IP addresses from any NIC of a multi-NIC VM can be added to a NEG as
long as the NEG uses the same VPC network as the NIC.
GCE_VM_IP_PORT NEGs
These zonal NEGs contain one or more internal network endpoints that resolve to a VM's primary internal IP address or an IP address in one of its alias IP ranges. For example, GKE uses NEGs of this type, whose endpoints are an IP address from the node's alias IP range plus a port – a Pod IP address and container port. Each network endpoint is specified using an IP address and port combination.
These NEGs can be used as backends in backend
services for External HTTP(S), Internal HTTP(S)
Load Balancing, TCP proxy, and SSL proxy load balancers. You cannot use
GCE_VM_IP_PORT
endpoints with internal TCP/UDP load balancers or
network load balancers. Because these zonal NEG
backends allow you to specify IP addresses and ports, you can distribute
traffic in a granular fashion among applications or containers running
within VM instances.
To create a unique GCE_VM_IP_PORT
network endpoint for a container or
application running in a
VM, you must either use the primary IP address of the VM or a secondary IP
assigned to the VM using the alias IP address feature. The
software running in the container or the application running in the VM should
be configured to bind to the IP address used by the network endpoint.
GCE_VM_IP_PORT
NEGs are especially useful for GKE.
For information on using zonal NEGs with GKE, refer to
Using Container-native Load
Balancing.
Endpoint relationships
When you create a NEG, you select a zone, a network, and a subnet. Every endpoint IP address must be in the same subnet as the zonal NEG.
If the network you select is an auto mode network, you can omit specifying the subnet. However, a subnet is still associated with the zonal NEG. If you specify an auto mode network but do not specify a subnet when creating a zonal NEG, the subnet it uses is the automatically-created subnet in the region that contains the zone that you selected for the zonal NEG.
The type of zonal NEG you create is specified when you create the NEG
(either GCE_VM_IP
or GCE_VM_IP_PORT
). This determines what types of
endpoints the NEG supports.
For both GCE_VM_IP
and GCE_VM_IP_PORT
zonal NEGs:
You must specify the name for each VM endpoint.
Each endpoint VM must be located in the same zone as the NEG.
Every endpoint in the NEG must be a unique IP address and port combination. A unique endpoint IP address and port combination can be referenced by more than one NEG.
Each endpoint VM must have a network interface (NIC) in the same VPC network as the NEG. Endpoint IP addresses must be associated with the same subnet specified in the NEG.
Each NEG supports up to the maximum number of endpoints per NEG. The endpoints can be distributed among that many unique VMs or all located on one VM.
For GCE_VM_IP_PORT
NEGs, when adding an endpoint you can choose to specify
an IP address and port, just an IP address, or neither:
If you specify an IP address and port, the IP address can be either the VM NIC's primary internal IP address or an alias IP on the NIC. The port is your choice.
If you specify just an IP address, the IP address can be either the VM NIC's primary internal IP address or an alias IP on the NIC. The port used is the NEG's default port.
If you omit both, Google Cloud selects the NIC's primary internal IP address and uses the NEG's default port.
Load balancing with zonal NEGs
Zonal NEGs can be used as backends for backend
services in a load balancer.
When you use a zonal NEG as
a backend for a backend service, all other backends in that backend service must
also be zonal NEGs of the same type (either all GCE_VM_IP
or GCE_VM_IP_PORT
).
You cannot use instance groups and zonal NEGs as backends in
the same backend service.
You can add the same network endpoint to more than one zonal NEG. You can use the same zonal NEG as a backend for more than one backend service.
GCE_VM_IP_PORT
zonal NEGs can use either the RATE
balancing
mode or the
CONNECTION
balancing
mode,
depending on the backend service's protocol. Supported load balancers require
defining a target capacity.
GCE_VM_IP
zonal NEGs must use the CONNECTION
balancing mode and you cannot
define a target capacity because internal TCP/UDP load balancers do not support
the target capacity setting.
You cannot use a balancing mode of UTILIZATION
for zonal NEG backends.
Internal TCP/UDP Load Balancing
Zonal NEGs with GCE_VM_IP
endpoints can be used as backends for backend
services only for internal TCP/UDP load balancers.
The primary use-cases for NEGs with GCE_VM_IP
endpoints are:
Simplified VM management for internal TCP/UDP load balancers
Like instance groups, you can use the same NEG as a backend for multiple internal TCP/UDP load balancers. Unlike instance groups, a VM endpoint can be a member of multiple NEGs, and each of those NEGs can be used as a backend for one or more internal TCP/UDP load balancers.
GKE subsetting
GKE uses GCP_VM_IP
zonal NEGs and subsetting to improve
the scalability of internal TCP/UDP load balancers in the following way:
Without subsetting, GKE creates one unmanaged instance group per zone, consisting of the cluster's nodes from all node pools in that zone. These zonal instance groups are used as backends for one or more internal LoadBalancer Services (and for external Ingresses that don't use NEGs themselves).
With subsetting, GKE creates GCE_VM_IP
zonal NEGs for
each internal LoadBalancer Service. The same endpoint can be a member of more
than one zonal NEG. Unlike instance groups, Google Cloud can load balance
to more multiple zonal NEGs that contain the same endpoint.
Subsetting more efficiently distributes traffic to internal LoadBalancer Services in clusters with more than 250 nodes. For example, a 300-node GKE cluster might have one internal LoadBalancer Service with 25 nodes in a NEG because there are 25 serving Pods for that Service. Not all 300 nodes need to be added to an instance group backend for this Service.
Note that quotas for NEGs, forwarding rules, backend services, and other Google Cloud networking resources still apply.
External HTTP(S), Internal HTTP(S), TCP Proxy, and SSL Proxy load balancing
Zonal NEGs with GCE_VM_IP_PORT
endpoints can be used as backends for backend
services in the following types of load balancers: External HTTP(S),
Internal HTTP(S), TCP Proxy, and SSL Proxy load balancing.
The primary use case for GCE_VM_IP_PORT
NEGs zonal NEGs is container-native load
balancing so that you can distribute traffic directly to
containers running on VMs – for example to Pod IP addresses in
GKE clusters.
The following example demonstrates how load balancers distribute traffic among microservices running in containers on your VMs. The VMs are configured to use alias IP ranges from their subnets, and those ranges are the addresses used by the containers.
The following illustrations show configuration components for HTTP(S) load balancers, TCP/SSL Proxy load balancers and Internal HTTP(S) load balancers where zonal NEGs are the backends:
Each Premium Tier HTTP(S), SSL Proxy, and TCP Proxy load balancer has its own global external forwarding rule to direct traffic to the appropriate target proxy object.
Each Standard Tier HTTP(S), SSL Proxy, and TCP Proxy load balancer has its own regional external forwarding rule to direct traffic to the appropriate target proxy object.
Each internal HTTP(S) load balancer has its own regional internal managed forwarding rule to direct traffic to the appropriate target proxy object.
For target HTTP(S) proxies, the backend service used is determined by checking the request host name and path in the URL map. External HTTP(S) and internal HTTP(S)load balancers can have multiple backend services referenced from the URL map.
For target TCP or target SSL proxies, only one backend service can be defined.
The backend service directs traffic to its backend zonal NEGs. For each request, the load balancer picks a network endpoint from one of the zonal NEGs and sends the traffic there.
Use-case: Container-native load balancing
Container-native load balancing enables load balancers to target Pods directly and to make load distribution decisions at the Pod-level instead of at the VM-level. There are two ways to configure container-native load balancing: either use NEGs managed by GKE Ingress, or use standalone NEGs.
Kubernetes Ingress with NEGs (Recommended)
When NEGs are used with Ingress, the Ingress controller facilitates the creation of all aspects of the L7 load balancer. This includes creating the virtual IP address, forwarding rules, health checks, firewall rules, and more. To learn how to configure this, see Container-native load balancing through Ingress.
Ingress is the recommended way to use NEGs for container-native load balancing as it has many features that simplify the management of NEGs. Standalone NEGs can be used if NEGs managed by Ingress do not serve your use case.
For instructions about how to set up a load balancer through Ingress, see Container-native load balancing through Ingress.
Standalone NEGs
When zonal NEGs are deployed with load balancers provisioned by anything other than GKE Ingress, they are considered standalone NEGs. Standalone zonal NEGs are deployed and managed through the NEG controller, while the forwarding rules, health checks, and other load balancing components are deployed manually.
For examples on using standalone zonal NEGs with GKE, see:
- Attach an external HTTP(S) load balancer to standalone zonal NEGs
- Attach an internal HTTP(S) load balancer to standalone zonal NEGs
Limitations
- You cannot use zonal NEGs with legacy networks.
- A backend service that uses NEGs as backends cannot also use instance groups as backends.
Limitations for GCE_VM_IP
zonal NEGs:
- Health checks are not supported for backend services with
GCE_VM_IP
zonal NEGs. - The
default-port
property is not supported forGCE_VM_IP
zonal NEGs. - You cannot use the Cloud Console to create or manage
GCE_VM_IP
NEGs. Use eithergcloud
or the REST API.
Quotas
- For information about NEG quotas—such as NEGs per project, NEGs per backend service, and endpoints per NEG—see the load balancing quotas page.
Troubleshooting
Traffic does not reach the endpoints
After the service is configured, new endpoints generally become reachable after attaching them to the zonal NEG, provided they respond to health checks.
If traffic cannot reach the endpoints, which results in a 502 error code for HTTP(s) or rejected connections for TCP/SSL load balancers, check the following:
- Verify that firewall rules allow incoming TCP traffic
to your endpoints from following ranges:
130.211.0.0/22
and35.191.0.0/16
. - Verify that your endpoints are healthy by using
gcloud
, as shown below, or by calling thegetHealth
API on the backend service resource or thelistEndpoints
API on the zonal NEG with theshowHealth
parameter set toSHOW
. The followinggcloud
command shows health information by network endpoint:
gcloud compute network-endpoint-groups list-network-endpoints NAME \ --zone=ZONE
What's next
- For information about configuring zonal NEGs, see Setting Up zonal network endpoint groups in load balancing.
- For information about using zonal network endpoint groups in Google Kubernetes Engine, see Using container-native load balancing.