This page shows you how to deploy an external LoadBalancer Service that builds a backend service-based external passthrough Network Load Balancer. Before reading this page, ensure that you're familiar with the following concepts:
Backend service-based external passthrough Network Load Balancer
As a cluster administrator, you can create an external LoadBalancer Service so
that clients outside the cluster can send packets to the Service's Pods. The
following diagram illustrates two backend service-based external passthrough Network Load Balancers
created for two external LoadBalancer Services (store-v1-lb-svc
and
store-v2-lb-svc
). Both load balancers distribute packets to the nodes in the
cluster, and the nodes route packets to serving Pods.
This guide shows you how to set up an external LoadBalancer Service named
store-v1-lb-svc
with the following steps:
- Create a cluster with the
HttpLoadBalancing
add-on enabled. - Create a Service that includes the
cloud.google.com/l4-rbs
annotation. This annotation instructs GKE to create a backend service-based external passthrough Network Load Balancer that uses a regional backend service. Verify that the load balancer successfully delivers packets to the
store-v1-lb-svc
Service's Pods. Also verify that GKE created the components of the backend service-based external passthrough Network Load Balancer:- Forwarding rule
- Regional backend service
- Instance group
- Health check
- VPC firewall rules
Delete the
store-v1-lb-svc
external LoadBalancer Service.
Before you begin
Before you start, make sure you have performed the following tasks:
- Enable the Google Kubernetes Engine API. Enable Google Kubernetes Engine API
- If you want to use the Google Cloud CLI for this task,
install and then
initialize the
gcloud CLI. If you previously installed the gcloud CLI, get the latest
version by running
gcloud components update
.
Set up your cluster
Create a cluster
Use the gcloud CLI to create a new cluster that supports the creation of backend service-based external passthrough Network Load Balancers:
gcloud container clusters create-auto CLUSTER_NAME \
--release-channel=RELEASE_CHANNEL \
--cluster-version=VERSION \
--location=COMPUTE_LOCATION
Replace the following:
CLUSTER_NAME
: the name of the new cluster.RELEASE_CHANNEL
: the name of the GKE release channel for the cluster.VERSION
: the GKE version for the cluster, which must be 1.24.9 or later.COMPUTE_LOCATION
: the Compute Engine region of the cluster.
Your new cluster has the HttpLoadBalancing
add-on enabled by default. This
add-on is required so that the control plane can create and manage backend service-based external passthrough Network Load Balancers.
Upgrade an existing cluster
Use the gcloud CLI to update an existing cluster so that it can support the creation of backend service-based external passthrough Network Load Balancers.
Upgrade your control plane to GKE version 1.24.9 or later:
gcloud container clusters upgrade CLUSTER_NAME \ --cluster-version=VERSION \ --master \ --location=COMPUTE_LOCATION
Replace the following:
CLUSTER_NAME
: the name of your cluster.VERSION
: the GKE version, which must be 1.24.9 or later. The version must be a valid minor version in your cluster's release channel. For more information, refer to Manually upgrading the control plane.COMPUTE_LOCATION
: the Compute Engine location for the new cluster.
Create the external LoadBalancer Service
Save the following sample Deployment as
store-deployment.yaml
:apiVersion: apps/v1 kind: Deployment metadata: name: store spec: replicas: 2 selector: matchLabels: app: store template: metadata: labels: app: store spec: containers: - image: gcr.io/google_containers/echoserver:1.10 imagePullPolicy: Always name: echoserver ports: - name: http containerPort: 8080 readinessProbe: httpGet: path: /healthz port: 8080 scheme: HTTP
Apply the manifest to the cluster:
kubectl apply -f store-deployment.yaml
Verify that there are two serving Pods for the Deployment:
kubectl get pods
The output is similar to the following:
NAME READY STATUS RESTARTS AGE store-cdb9bb4d6-s25vw 1/1 Running 0 10s store-cdb9bb4d6-vck6s 1/1 Running 0 10s
Save the following Service manifest as
store-v1-lb-svc.yaml
:apiVersion: v1 kind: Service metadata: name: store-v1-lb-svc annotations: cloud.google.com/l4-rbs: "enabled" spec: type: LoadBalancer externalTrafficPolicy: Cluster selector: app: store ports: - name: tcp-port protocol: TCP port: 8080 targetPort: 8080
This external LoadBalancer Service uses the default
externalTrafficPolicy
ofCluster
. For details about how theexternalTrafficPolicy
defines node grouping, which nodes pass their load balancer health checks, and packet processing, see LoadBalancer Service concepts.If you are using a IPv4/IPv6 dual-stack cluster, add the
spec.ipFamilyPolicy
andipFamilies
to define how GKE allocates IP addresses to the Service. Consider the following conditions when using theipFamilyPolicy
andipFamilies
specs:- When you create a backend service-based external passthrough Network Load Balancer,
GKE automatically adds the
cloud.google.com/l4-rbs
annotation to new Services created on IPv4/IPv6 dual-stack clusters. However, if you add thecloud.google.com/l4-rbs: "enabled"
annotation to existing Service manifests, LoadBalancer Services that already exist in the cluster keep using target-pool based external passthrough Network Load Balancers, which are IPv4 only. For additional information, see Node grouping. - GKE can allocate either single-stack (IPv4 only or IPv6 only), or dual-stack LoadBalancer Services. A dual-stack LoadBalancer Service is implemented with two separate external passthrough Network Load Balancer forwarding rules: one to handle TCP traffic over IPv4 and another to handle TCP traffic over IPv6. For more information, see Services.
- When you create a backend service-based external passthrough Network Load Balancer,
GKE automatically adds the
Apply the manifest to the cluster:
kubectl apply -f store-v1-lb-svc.yaml
Verify that your Service is running:
kubectl get svc store-v1-lb-svc
The output is similar to the following:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE store-v1-lb-svc LoadBalancer 10.44.196.160 35.193.28.231 8080:32466/TCP 11m
GKE assigned an
EXTERNAL_IP
for the external passthrough Network Load Balancer.Test connecting to the load balancer:
curl EXTERNAL_IP:PORT
Replace the following:
EXTERNAL_IP
: the allocated IP address for the external passthrough Network Load Balancer.PORT
: the allocated port number for the external passthrough Network Load Balancer.
The output is similar to the following:
Hostname: store-v1-lb-svc-cdb9bb4d6-hflxd Pod Information: -no pod information available- Server values: server_version=nginx: 1.13.3 - lua: 10008 Request Information: client_address=10.128.0.50 method=GET real path=/ query= request_version=1.1 request_scheme=http request_uri=EXTERNAL_IP Request Headers: accept=*/* host=EXTERNAL_IP user-agent=curl/7.81.0 Request Body: -no body in request-
Verify the external LoadBalancer Service and its components
Check your LoadBalancer Service and its set of annotations describing its Google Cloud resources:
kubectl describe svc store-v1-lb-svc
The output is similar to the following:
Name: store-v1-lb-svc Namespace: default Labels: <none> Annotations: cloud.google.com/l4-rbs: enabled service.kubernetes.io/backend-service: k8s2-c086604n-default-store-v1-lb-svc-aip4ty1x service.kubernetes.io/firewall-rule: k8s2-c086604n-default-store-v1-lb-svc-aip4ty1x service.kubernetes.io/firewall-rule-for-hc: k8s2-c086604n-l4-shared-hc-fw service.kubernetes.io/healthcheck: k8s2-c086604n-l4-shared-hc service.kubernetes.io/tcp-forwarding-rule: a683373f85bfe433ba929a50ca8d72e2 Selector: app=store Type: LoadBalancer IP Family Policy: SingleStack IP Families: IPv4 IP: 10.44.196.160 IPs: 10.44.196.160 LoadBalancer Ingress: 35.193.28.231 Port: tcp-port 8080/TCP TargetPort: 8080/TCP NodePort: tcp-port 32466/TCP Endpoints: 10.48.0.5:8080,10.48.2.8:8080 Session Affinity: None External Traffic Policy: Cluster Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal ADD 2m42s loadbalancer-controller default/store-v1-lb-svc Normal EnsuringLoadBalancer 102s (x2 over 2m42s) service-controller Ensuring load balancer Normal Annotations 102s loadbalancer-controller map[cloud.google.com/l4-rbs:enabled kubectl.kubernetes.io/last-applied-configuration:{"apiVersion":"v1","kind":"Service","metadata":{"annotations": {"cloud.google.com/l4-rbs":"enabled"},"name":"store-v1-lb-svc","namespace":"default"} ,"spec":{"externalTrafficPolicy":"Cluster","ports": [{"name":"tcp-port","port":8080,"protocol":"TCP","targetPort":8080}], "selector":{"app":"store"},"type":"LoadBalancer"}} ] -> map[cloud.google.com/l4-rbs:enabled kubectl.kubernetes.io/last-applied-configuration:{"apiVersion":"v1","kind": "Service","metadata":{"annotations":{"cloud.google.com/l4-rbs":"enabled"}, "name":"store-v1-lb-svc","namespace":"default"},"spec":{"externalTrafficPolicy" :"Cluster","ports":[{"name":"tcp-port","port":8080,"protocol":"TCP","targetPort" :8080}],"selector":{"app":"store"},"type":"LoadBalancer"}} service.kubernetes.io/backend-service:k8s2-c086604n-default-store-v1-lb-svc-aip4ty1x service.kubernetes.io/firewall-rule:k8s2-c086604n-default-store-v1-lb-svc-aip4ty1x service.kubernetes.io/firewall-rule-for-hc:k8s2-c086604n-l4-shared-hc-fw service.kubernetes.io/healthcheck:k8s2-c086604n-l4-shared-hc service.kubernetes.io/tcp-forwarding-rule:a683373f85bfe433ba929a50ca8d72e2] Normal SyncLoadBalancerSuccessful 16s (x3 over 102s) loadbalancer-controller Successfully ensured L4 External LoadBalancer resources
There are several fields that indicate that a backend service-based external passthrough Network Load Balancer and its Google Cloud resources were successfully created:
Events
field. This field is empty when the LoadBalancer Service and its resources were created successfully. If an error has occurred, it is listed here.List of
Annotations
enabled: GKE adds the following list of read-only annotations to the Service manifest. Each annotation whose name begins withservice.kubernetes.io/
is used to indicate the name of a Google Cloud resource created as part of or to support the load balancer.The
service.kubernetes.io/backend-service
annotation indicates the name of the load balancer's backend service.The
service.kubernetes.io/healthcheck
annotation indicates the name of the load balancer health check used by the backend service.The
service.kubernetes.io/tcp-forwarding-rule
orservice.kubernetes.io/udp-forwarding-rule
annotation indicates the name of the load balancer's forwarding rule.The
service.kubernetes.io/firewall-rule
annotation indicates the name of the firewall rule created to permit traffic to the cluster nodes. Source ranges for this firewall rule are customizable usingspec.loadBalancerSourceRanges[]
. For additional detail about firewall rules for LoadBalancer Services, see Firewall rules and source IP address allowlist.The
service.kubernetes.io/firewall-rule-for-hc
annotation indicates the name of the firewall rule required for load balancer health checks.
Verify that load balancer resources and firewall rules have been created for the external LoadBalancer Service:
To see the forwarding rule, run the following command:
gcloud compute forwarding-rules describe FWD_RULE_NAME \ --region=REGION_NAME
Replace the following:
FWD_RULE_NAME
: the forwarding rule name provided by either theservice.kubernetes.io/tcp-forwarding-rule
orservice.kubernetes.io/udp-forwarding-rule
read-only annotations. To check these annotations, runkubectl describe svc SERVICE_NAME
.REGION_NAME
: the Google Cloud region containing the cluster. For zonal clusters, the region contains the zone used by the cluster.
To see the backend service, run the following command:
gcloud compute backend-services describe BACKEND_SERVICE_NAME \ --region=REGION_NAME
Replace the following:
BACKEND_SERVICE_NAME
: the name of the backend service provided by theservice.kubernetes.io/backend-service
read-only annotation. To check this read-only annotation, runkubectl describe svc SERVICE_NAME
.REGION_NAME
: he Google Cloud region containing the cluster. For zonal clusters, the region contains the zone used by the cluster.
To see the load balancer health check, run the following command:
gcloud compute health-checks describe HEALTH_CHECK_NAME \ --region=REGION_NAME
Replace the following:
HEALTH_CHECK_NAME
: the load balancer's health check name. The name of the health check is provided by theservice.kubernetes.io/healthcheck
read-only annotation. To check this read-only annotation, runkubectl describe svc SERVICE_NAME
.REGION_NAME
: the Google Cloud region containing the cluster. For zonal clusters, the region contains the zone used by the cluster.
To see the firewall rules, run the following commands:
gcloud compute firewall-rules describe FIREWALL_RULE_NAME \ gcloud compute firewall-rules describe HEALTH_CHECK_FIREWALL_RULE_NAME
Replace the following:
FIREWALL_RULE_NAME
: the name of the firewall rule that permits traffic to the load balancer. The name of this firewall rule is provided by theservice.kubernetes.io/firewall-rule
read-only annotation. To check this read-only annotation, runkubectl describe svc SERVICE_NAME
.HEALTH_CHECK_FIREWALL_RULE_NAME
: the name of the firewall rule that permits health checks of the load balancer's backends (the cluster's nodes). The name of this firewall rule is provided by theservice.kubernetes.io/firewall-rule-for-hc
read-only annotation. To check this read-only annotation, runkubectl describe svc SERVICE_NAME
.
Delete the external LoadBalancer Service and its components
Delete the store-v1-lb-svc
external LoadBalancer Service.
kubectl delete service store-v1-lb-svc
GKE deletes the following resources:
- The load balancer's forwarding rule.
- The load balancer's backend service.
- The load balancer's health check.
- The VPC firewall rules necessary for the load balancer and its health check traffic.
- The zonal unmanaged instance group backends, only if GKE does not need to use them as backends for other load balancers created by the cluster.
What's next
- For a general overview of Load Balancer Services, see LoadBalancer Services.
- For a description of Load Balancer Services parameters, see LoadBalancer Service parameters.
- Troubleshoot load balancing in GKE.