Create a backend service-based external load balancer

Autopilot Standard

This page shows you how to deploy an external LoadBalancer Service that builds a backend service-based external passthrough Network Load Balancer. Before reading this page, you should be familiar with the following concepts:

Backend service-based external passthrough Network Load Balancer

As a cluster administrator, you can create an external LoadBalancer Service so that clients outside the cluster can send packets to the Service's Pods. The following diagram illustrates two backend service-based external passthrough Network Load Balancers created for two external LoadBalancer Services (store-v1-lb-svc and store-v2-lb-svc). Both load balancers distribute packets to the nodes in the cluster, and the nodes route packets to serving Pods.

External LoadBalancer Services powered by regional backend service-based external passthrough Network Load Balancers

This guide shows you how to set up an external LoadBalancer Service named store-v1-lb-svc with the following steps:

Create a cluster with the HttpLoadBalancing add-on enabled.
Create a Service that includes the cloud.google.com/l4-rbs annotation. This annotation instructs GKE to create a backend service-based external passthrough Network Load Balancer that uses a regional backend service.
Verify that the load balancer successfully delivers packets to the store-v1-lb-svc Service's Pods. Also verify that GKE created the components of the backend service-based external passthrough Network Load Balancer:
- Forwarding rule
- Regional backend service
- Instance group
- Health check
- VPC firewall rules
Delete the store-v1-lb-svc external LoadBalancer Service.

Before you begin

Before you start, make sure you have performed the following tasks:

Enable the Google Kubernetes Engine API.

Enable Google Kubernetes Engine API

If you want to use the Google Cloud CLI for this task, install and then initialize the gcloud CLI. If you previously installed the gcloud CLI, get the latest version by running gcloud components update.
Note: For existing gcloud CLI installations, make sure to set the compute/region and compute/zone properties. By setting default locations, you can avoid errors in gcloud CLI like the following: One of [--zone, --region] must be supplied: Please specify location.

Set up your cluster

Create a cluster

Use the gcloud CLI to create a new cluster that supports the creation of backend service-based external passthrough Network Load Balancers:

gcloud container clusters create-auto CLUSTER_NAME \
    --release-channel=RELEASE_CHANNEL \
    --cluster-version=VERSION \
    --location=COMPUTE_LOCATION

Replace the following:

CLUSTER_NAME: the name of the new cluster.
RELEASE_CHANNEL: the name of the GKE release channel for the cluster.
VERSION: the GKE version for the cluster, which must be 1.24.9 or later.
COMPUTE_LOCATION: the Compute Engine region of the cluster.

Your new cluster has the HttpLoadBalancing add-on enabled by default. This add-on is required so that the control plane can create and manage backend service-based external passthrough Network Load Balancers.

Upgrade an existing cluster

Use the gcloud CLI to update an existing cluster so that it can support the creation of backend service-based external passthrough Network Load Balancers.

Upgrade your control plane to GKE version 1.24.9 or later:
```
gcloud container clusters upgrade CLUSTER_NAME \
    --cluster-version=VERSION \
    --master \
    --location=COMPUTE_LOCATION
```
Replace the following:
- CLUSTER_NAME: the name of your cluster.
- VERSION: the GKE version, which must be 1.24.9 or later. The version must be a valid minor version in your cluster's release channel. For more information, refer to Manually upgrading the control plane.
- COMPUTE_LOCATION: the Compute Engine location for the new cluster.

Create the external LoadBalancer Service

Save the following sample Deployment as store-deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: store
spec:
  replicas: 2
  selector:
    matchLabels:
      app: store
  template:
    metadata:
      labels:
        app: store
    spec:
      containers:
      - image: gcr.io/google_containers/echoserver:1.10
        imagePullPolicy: Always
        name: echoserver
        ports:
          - name: http
            containerPort: 8080
        readinessProbe:
          httpGet:
            path: /healthz
            port: 8080
            scheme: HTTP

Apply the manifest to the cluster:
```
kubectl apply -f store-deployment.yaml
```

Verify that there are two serving Pods for the Deployment:

kubectl get pods

The output is similar to the following:

NAME                     READY   STATUS    RESTARTS   AGE
store-cdb9bb4d6-s25vw      1/1     Running   0          10s
store-cdb9bb4d6-vck6s      1/1     Running   0          10s

Save the following Service manifest as store-v1-lb-svc.yaml:
```
apiVersion: v1
kind: Service
metadata:
  name: store-v1-lb-svc
  annotations:
    cloud.google.com/l4-rbs: "enabled"
spec:
  type: LoadBalancer
  externalTrafficPolicy: Cluster
  selector:
    app: store
  ports:
  - name: tcp-port
    protocol: TCP
    port: 8080
    targetPort: 8080
```
This external LoadBalancer Service uses the default externalTrafficPolicy of Cluster. For details about how the externalTrafficPolicy defines node grouping, which nodes pass their load balancer health checks, and packet processing, see LoadBalancer Service concepts.

If you are using a IPv4/IPv6 dual-stack cluster, add the spec.ipFamilyPolicy and ipFamilies to define how GKE allocates IP addresses to the Service. Consider the following conditions when using the ipFamilyPolicy and ipFamilies specs:
- When you create a backend service-based external passthrough Network Load Balancer, GKE automatically adds the cloud.google.com/l4-rbs annotation to new Services created on IPv4/IPv6 dual-stack clusters. However, if you add the cloud.google.com/l4-rbs: "enabled" annotation to existing Service manifests, LoadBalancer Services that already exist in the cluster keep using target-pool based external passthrough Network Load Balancers, which are IPv4 only. For additional information, see Node grouping.
- GKE can allocate either single-stack (IPv4 only or IPv6 only), or dual-stack LoadBalancer Services. A dual-stack LoadBalancer Service is implemented with two separate external passthrough Network Load Balancer forwarding rules: one to handle TCP traffic over IPv4 and another to handle TCP traffic over IPv6. For more information, see Services.
Apply the manifest to the cluster:
```
kubectl apply -f store-v1-lb-svc.yaml
```
Verify that your Service is running:
```
kubectl get svc store-v1-lb-svc
```
The output is similar to the following:
```
NAME               TYPE           CLUSTER-IP        EXTERNAL-IP     PORT(S)          AGE
store-v1-lb-svc   LoadBalancer   10.44.196.160     35.193.28.231   8080:32466/TCP   11m
```
GKE assigned an EXTERNAL_IP for the external passthrough Network Load Balancer.

Note: It might take a few minutes for GKE to allocate an external IP address before the load balancer is ready to serve your application.

Test connecting to the load balancer:

curl EXTERNAL_IP:PORT

Replace the following:

EXTERNAL_IP: the allocated IP address for the external passthrough Network Load Balancer.
PORT: the allocated port number for the external passthrough Network Load Balancer.

The output is similar to the following:

Hostname: store-v1-lb-svc-cdb9bb4d6-hflxd

Pod Information:
  -no pod information available-

Server values:
  server_version=nginx: 1.13.3 - lua: 10008

Request Information:
  client_address=10.128.0.50
  method=GET
  real path=/
  query=
  request_version=1.1
  request_scheme=http
  request_uri=EXTERNAL_IP

Request Headers:
  accept=*/*
  host=EXTERNAL_IP
  user-agent=curl/7.81.0

Request Body:
  -no body in request-

Verify the external LoadBalancer Service and its components

Check your LoadBalancer Service and its set of annotations describing its Google Cloud resources:

kubectl describe svc store-v1-lb-svc

The output is similar to the following:

Name:                     store-v1-lb-svc
Namespace:                default
Labels:                   <none>
Annotations:              cloud.google.com/l4-rbs: enabled
                          service.kubernetes.io/backend-service: k8s2-c086604n-default-store-v1-lb-svc-aip4ty1x
                          service.kubernetes.io/firewall-rule: k8s2-c086604n-default-store-v1-lb-svc-aip4ty1x
                          service.kubernetes.io/firewall-rule-for-hc: k8s2-c086604n-l4-shared-hc-fw
                          service.kubernetes.io/healthcheck: k8s2-c086604n-l4-shared-hc
                          service.kubernetes.io/tcp-forwarding-rule: a683373f85bfe433ba929a50ca8d72e2
Selector:                 app=store
Type:                     LoadBalancer
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.44.196.160
IPs:                      10.44.196.160
LoadBalancer Ingress:     35.193.28.231
Port:                     tcp-port  8080/TCP
TargetPort:               8080/TCP
NodePort:                 tcp-port  32466/TCP
Endpoints:                10.48.0.5:8080,10.48.2.8:8080
Session Affinity:         None
External Traffic Policy:  Cluster
Events:
  Type    Reason                Age                   From                     Message
  ----    ------                ----                  ----                     -------
  Normal  ADD                   2m42s                 loadbalancer-controller  default/store-v1-lb-svc
  Normal  EnsuringLoadBalancer  102s (x2 over 2m42s)  service-controller       Ensuring load balancer
  Normal  Annotations           102s                  loadbalancer-controller  map[cloud.google.com/l4-rbs:enabled kubectl.kubernetes.io/last-applied-configuration:{"apiVersion":"v1","kind":"Service","metadata":{"annotations":
{"cloud.google.com/l4-rbs":"enabled"},"name":"store-v1-lb-svc","namespace":"default"}
,"spec":{"externalTrafficPolicy":"Cluster","ports":
[{"name":"tcp-port","port":8080,"protocol":"TCP","targetPort":8080}],
"selector":{"app":"store"},"type":"LoadBalancer"}}
] -> map[cloud.google.com/l4-rbs:enabled
kubectl.kubernetes.io/last-applied-configuration:{"apiVersion":"v1","kind":
"Service","metadata":{"annotations":{"cloud.google.com/l4-rbs":"enabled"},
"name":"store-v1-lb-svc","namespace":"default"},"spec":{"externalTrafficPolicy"
:"Cluster","ports":[{"name":"tcp-port","port":8080,"protocol":"TCP","targetPort"
:8080}],"selector":{"app":"store"},"type":"LoadBalancer"}}
service.kubernetes.io/backend-service:k8s2-c086604n-default-store-v1-lb-svc-aip4ty1x
service.kubernetes.io/firewall-rule:k8s2-c086604n-default-store-v1-lb-svc-aip4ty1x
service.kubernetes.io/firewall-rule-for-hc:k8s2-c086604n-l4-shared-hc-fw
service.kubernetes.io/healthcheck:k8s2-c086604n-l4-shared-hc
service.kubernetes.io/tcp-forwarding-rule:a683373f85bfe433ba929a50ca8d72e2]
Normal  SyncLoadBalancerSuccessful  16s (x3 over 102s)  loadbalancer-controller  Successfully ensured L4 External LoadBalancer resources

There are several fields that indicate that a backend service-based external passthrough Network Load Balancer and its Google Cloud resources were successfully created:

Events field. This field is empty when the LoadBalancer Service and its resources were created successfully. If an error has occurred, it is listed here.
List of Annotations enabled: GKE adds the following list of read-only annotations to the Service manifest. Each annotation whose name begins with service.kubernetes.io/ is used to indicate the name of a Google Cloud resource created as part of or to support the load balancer.
The service.kubernetes.io/backend-service annotation indicates the name of the load balancer's backend service.
The service.kubernetes.io/healthcheck annotation indicates the name of the load balancer health check used by the backend service.
The service.kubernetes.io/tcp-forwarding-rule or service.kubernetes.io/udp-forwarding-rule annotation indicates the name of the load balancer's forwarding rule.
The service.kubernetes.io/firewall-rule annotation indicates the name of the firewall rule created to permit traffic to the cluster nodes. Source ranges for this firewall rule are customizable using spec.loadBalancerSourceRanges[]. For additional detail about firewall rules for LoadBalancer Services, see Firewall rules and source IP address allowlist.
The service.kubernetes.io/firewall-rule-for-hc annotation indicates the name of the firewall rule required for load balancer health checks.

Verify that load balancer resources and firewall rules have been created for the external LoadBalancer Service:

To see the forwarding rule, run the following command:
```
  gcloud compute forwarding-rules describe FWD_RULE_NAME \
    --region=REGION_NAME
```
Replace the following:
- FWD_RULE_NAME: the forwarding rule name provided by either the service.kubernetes.io/tcp-forwarding-rule or service.kubernetes.io/udp-forwarding-rule read-only annotations. To check these annotations, run kubectl describe svc SERVICE_NAME.
- REGION_NAME: the Google Cloud region containing the cluster. For zonal clusters, the region contains the zone used by the cluster.
To see the backend service, run the following command:
```
gcloud compute backend-services describe BACKEND_SERVICE_NAME \
  --region=REGION_NAME
```
Replace the following:
- BACKEND_SERVICE_NAME: the name of the backend service provided by the service.kubernetes.io/backend-service read-only annotation. To check this read-only annotation, run kubectl describe svc SERVICE_NAME.
- REGION_NAME: he Google Cloud region containing the cluster. For zonal clusters, the region contains the zone used by the cluster.
To see the load balancer health check, run the following command:
```
gcloud compute health-checks describe HEALTH_CHECK_NAME \
  --region=REGION_NAME
```
Replace the following:
- HEALTH_CHECK_NAME: the load balancer's health check name. The name of the health check is provided by the service.kubernetes.io/healthcheck read-only annotation. To check this read-only annotation, run kubectl describe svc SERVICE_NAME.
- REGION_NAME: the Google Cloud region containing the cluster. For zonal clusters, the region contains the zone used by the cluster.
To see the firewall rules, run the following commands:
```
gcloud compute firewall-rules describe FIREWALL_RULE_NAME \
gcloud compute firewall-rules describe HEALTH_CHECK_FIREWALL_RULE_NAME
```
Replace the following:
- FIREWALL_RULE_NAME: the name of the firewall rule that permits traffic to the load balancer. The name of this firewall rule is provided by the service.kubernetes.io/firewall-rule read-only annotation. To check this read-only annotation, run kubectl describe svc SERVICE_NAME.
- HEALTH_CHECK_FIREWALL_RULE_NAME: the name of the firewall rule that permits health checks of the load balancer's backends (the cluster's nodes). The name of this firewall rule is provided by the service.kubernetes.io/firewall-rule-for-hc read-only annotation. To check this read-only annotation, run kubectl describe svc SERVICE_NAME.

Delete the external LoadBalancer Service and its components

Delete the store-v1-lb-svc external LoadBalancer Service.

kubectl delete service store-v1-lb-svc

GKE deletes the following resources:

The load balancer's forwarding rule.
The load balancer's backend service.
The load balancer's health check.
The VPC firewall rules necessary for the load balancer and its health check traffic.
The zonal unmanaged instance group backends, only if GKE does not need to use them as backends for other load balancers created by the cluster.

What's next

For a general overview of Load Balancer Services, see LoadBalancer Services.
For a description of Load Balancer Services parameters, see LoadBalancer Service parameters.