This document describes parameters for Service manifests which control LoadBalancer Service behavior and configuration. Before reading this document, ensure that you're familiar with Google Kubernetes Engine (GKE) LoadBalancer Service Concepts.
Service parameters
GKE supports the following parameters for LoadBalancer Services.
Parameter | Service field and description | Internal | External | Version Support |
---|---|---|---|---|
Internal passthrough Network Load Balancer | networking.gke.io/load-balancer-type: "Internal"
Instructs GKE to create an internal passthrough Network Load Balancer. For details, see LoadBalancer Service Concepts. |
All supported versions. | ||
Backend service-based external passthrough Network Load Balancer | cloud.google.com/l4-rbs: "enabled"
Instructs GKE to create a backend service-based external passthrough Network Load Balancer. For details, see LoadBalancer Service Concepts. |
GKE 1.25.5+ | ||
Internal Traffic Policy | spec.internalTrafficPolicy
When set to This parameter is not supported in clusters running GKE Dataplane V2. |
GKE 1.22+ | ||
External Traffic Policy | spec.externalTrafficPolicy
Controls which node VMs pass load balancer health checks and how
packets are routed to ready and serving Pods in the cluster. Also
controls how nodes are grouped into For details, see LoadBalancer Service Concepts. |
GKE 1.14+ (1.23.4-gke.400+ for Windows node pool). | ||
Health check port | spec.healthCheckNodePort
Deploys a load balancer health check for LoadBalancer Services. This
parameter is only valid if |
All supported versions. | ||
Firewall rules and source IP address allowlist | spec.loadBalancerSourceRanges
Configures optional firewall rules in GKE and in the VPC network to only allow certain source ranges. |
All supported versions. | ||
Static IP addresses |
Specifies a static IPv4 address, static IPv6 address range, or both, that are assigned to the forwarding rules of the load balancer. See Considerations for sharing a common IP address for important configuration requirements and implementation details. |
|
||
Network Service Tiers | cloud.google.com/network-tier
Specifies which
Network Service Tiers
GKE uses for the external forwarding rule and IP
address. The annotation valid values are |
GKE 1.19 and later. | ||
Custom subnet |
|
(only applies to IPv6) |
|
|
Global access | networking.gke.io/internal-load-balancer-allow-global-access: "true"
Enables the IP address of the forwarding rule to be accessible by clients in any region of the VPC network or a connected network. |
Preview in GKE 1.16+. Generally available in GKE 1.17.9-gke.600+. | ||
All ports | No annotation required, but GKE subsetting must be enabled. GKE automatically configures the forwarding rule to
use all ports if at least six (up to 100) unique ports are specified
in |
GKE version 1.18.19-gke.1400 or later | ||
ipFamilyPolicy |
Defines how GKE allocates IP addresses to a Service.
You can define the To learn more, see IPv4/IPv6 dual-stack Services. |
GKE clusters in version 1.29 or later support dual-stack network for LoadBalancer Services. | ||
ipFamilies (Optional) |
Defines the IP address family to allocate either single-stack or dual-stack Services. Use any of the following values:
|
GKE clusters in version 1.29 or later support dual-stack network for LoadBalancer Services. |
Health check port
As described in Load balancer health checks and externalTrafficPolicy
,
GKE always deploys a load balancer health check when it creates
an external passthrough Network Load Balancer or internal passthrough Network Load Balancer.
Whether you can set up healthCheckNodePort
parameter or not depends on the
following externalTrafficPolicy
configuration:
externalTrafficPolicy |
Health check port |
---|---|
Cluster |
You cannot use |
Local |
You can select a custom port by using the
|
Firewall rules and source IP address allowlist
When you create a LoadBalancer Service, GKE creates a VPC firewall rule corresponding to the Service. Each firewall rule has the following characteristics:
- The firewall rule's direction is ingress, and its action is allow. The implied deny ingress firewall rules in Google Cloud mean that GKE uses an allowlist model when creating ingress firewall rules.
- GKE sets the firewall rule's protocol and destination port to
those specified in the Service's
spec.ports[]
list. - GKE sets the destination for the firewall rule by setting its target parameter to the virtual IP address of the LoadBalancer.
- If the Service includes
spec.loadBalancerSourceRanges[]
, GKE sets the firewall rule's source parameter to the IP addresses in that list. If the Service does not includeloadBalancerSourceRanges[]
, GKE sets the firewall rule's source parameter to all IP addresses(0.0.0.0/0)
.
The firewall rule created for a LoadBalancer Service allows packets, matching that Service's protocol and destination ports, to the Service's virtual IP address.
Services using common ports
When a cluster contains two or more Services that share at least one common
protocol and destination port, the effective set of source ranges is the union
of loadBalancerSourceRanges
for all Services that specify that protocol and
destination port combination. This is because the target parameter of an ingress
firewall rule defines the destination IP addresses as
all IP addresses associated with a VM.
Consider a cluster that has two LoadBalancer Services:
- The first Service's
spec.ports[0].port
is TCP port80
, and itsspec.loadBalancerSourceRanges=[100.10.0.0/16]
. The resulting load balancer corresponding to this Service has IP address192.0.2.2
. - The second Service's
spec.ports[0].port
is TCP port80
,spec.ports[1].port
is TCP port90
, and itsspec.loadBalancerSourceRanges=[172.16.0.0/24]
. The resulting load balancer corresponding to this Service has IP address198.51.100.3
.
GKE creates two ingress-allow firewall rules in the cluster's Virtual Private Cloud network. Both firewall rules specify all nodes of the cluster as their targets:
- The first firewall rule allows packets to TCP destination port
80
from source100.10.0.0/16
. - The second firewall rule allows packets to TCP destination ports
80
and90
from source172.16.0.0/24
.
The first forwarding rule routes traffic matching the 192.0.2.2:80
destination. The second forwarding rule routes traffic matching both the
198.51.100.3:80
and 198.51.100.3:90
destinations. All three of the following
are valid destinations on each node: 192.0.2.2:80
, 198.51.100.3:80
, and
198.51.100.3:90
. This means that:
- Both Services accept packets to TCP port
80
from the union of the IP address source ranges, from either100.10.0.0/16
or172.16.0.0/24
. - The second Service accepts packets to TCP port
90
from172.16.0.0/24
.
The effective set of source ranges for all Services using the same protocol and
destination port combination becomes all IP addresses if
spec.loadBalancerSourceRanges
is omitted on at least one Service using that
protocol and destination port combination. For example, If the second Service
had omitted spec.loadBalancerSourceRanges
, the second firewall's source would
be 0.0.0.0/0
, and:
- Both Services would accept packets to TCP port 80 from the union of the IP
address source ranges, from either
100.10.0.0/16
or0.0.0.0/0
. Because the0.0.0.0/0
range includes the100.10.0.0/16
range, the effective source for packets to TCP port 80 is any IP address. - The second Service would accept packets to TCP port
90
from0.0.0.0/0
(any IP address).
Static IP addresses
You can create a static IP
address and configure
GKE to assign that static address to the load balancer's
forwarding rule. Using a static IP address ensures that the load balancer's IP
address remains the same even if you make changes to the LoadBalancer Service.
Without a static IP address, GKE might assign a different IP
address to the load balancer forwarding rule when you update a LoadBalancer
Service. The forwarding rule IP address is not the same as the Service's
spec.clusterIP
address. The ClusterIP address for a Service never changes when
you update a LoadBalancer Service.
Static IP address parameters
To instruct a LoadBalancer Service to use a static IP address, use either the
spec.loadBalancerIP
parameter or the
networking.gke.io/load-balancer-ip-addresses
annotation. In GKE
version 1.29 and later, the annotation takes precedence over
spec.loadBalancerIP
if the Service manifest contains both the parameter and
the annotation.
Parameter or annotation and value | Requirements and capabilities |
---|---|
spec.loadBalancerIP: IPv4_ADDRESS
|
You can specify a static internal IPv4 address for an IPv4-only Internal LoadBalancer Service. You can specify a static external IPv4 address for an IPv4-only External LoadBalancer Service. The parameter works with all supported GKE versions. |
networking.gke.io/load-balancer-ip-addresses: IP_ADDRESS_RESOURCE_NAME
|
You can specify static IPv4 address, static IPv6 address range, or both for IPv4-only, IPv6-only, and dual-stack Internal and External LoadBalancer Services. The annotation requires GKE 1.29 or later and the following additional requirements:
|
Considerations for sharing a common IP address
Two or more LoadBalancer Services can reference the same static IP address if the forwarding rule for each load balancer uses a unique combination of IP address, protocol, port specification, and Network Service Tiers specification, as indicated in the table in this section. Additionally:
Static IPv6 addresses are actually
/96
IPv6 address ranges, but GKE only configures nodes to accept packets on the first IPv6 address (/128
) in the/96
range.For two or more Internal LoadBalancer Services to use the same internal IPv4 address or internal IPv6 address range, the static IP address must be created with the
SHARED_LOADBALANCER_VIP
purpose.
Internal LoadBalancer Service | External LoadBalancer Service | |
---|---|---|
Port specification | Forwarding rules for internal passthrough Network Load Balancers support up to five discrete port numbers, or they can be configured to use all ports. When an internal LoadBalancer Service has six or more
When a forwarding rule uses all ports, no other forwarding rule (therefore, no other internal LoadBalancer Service) can use the same IP address. |
GKE creates a target pool based external passthrough Network Load Balancer if
the LoadBalancer Service manifest lacks the
Forwarding rules for target pool based external passthrough Network Load Balancers must use contiguous port ranges. The contiguous port range includes all ports the Service needs, but the range might also include additional ports not used by the Service. For example, an external LoadBalancer Service powered by a target pool based external passthrough Network Load Balancer which specifies ports 80 and 443 in its Service manifest uses a load balancer forwarding rule with a port range of 80-443. This port range prevents other external LoadBalancer Services from using any of the ports 80, 443, and any of the numbers between 80 and 443. GKE creates a backend service based external passthrough Network Load Balancer if
the LoadBalancer Service manifest includes the
|
Network Service Tiers | Not configurable—internal addresses are always Premium tier. | Configurable for static regional external IPv4 addresses. Static regional external IPv6 address ranges can only be created in the Premium tier. The Network Service Tiers specification of the static external IP address must match either:
The project's default tier is Premium, unless you have configured it differently. |
LoadBalancer subnet
You can configure an Internal LoadBalancer Service to use an ephemeral or static IPv4 address, IPv6 address range, or both, in a custom subnet located in the same region and VPC network as the cluster. Use a custom subnet for an Internal LoadBalancer Service to:
- Group internal passthrough Network Load Balancers created by Internal LoadBalancer Services from two or more GKE clusters in the same VPC network and region.
- Create Internal LoadBalancer Services whose internal passthrough Network Load Balancers have IPv4 addresses separate from the cluster's node IPv4 addresses.
- In a dual-stack cluster, create Internal LoadBalancer Services whose internal passthrough Network Load Balancers have IPv6 address ranges separate from the cluster's node and Pod IPv6 addresses. A custom LoadBalancer subnet is mandatory to support Internal LoadBalancer Services if the cluster's subnet has an external IPv6 address range.
You can configure an External LoadBalancer Service to use an ephemeral or static IPv6 address range in a custom subnet located in the same region and VPC network as the cluster. Use a custom subnet to create External LoadBalancer Services whose external passthrough Network Load Balancers have IPv6 address ranges separate from the cluster's node and Pod IPv6 addresses. A custom LoadBalancer subnet is mandatory to support External LoadBalancer Services in a dual-stack private cluster because the cluster's subnet has an internal IPv6 address range.
Custom subnet annotations
Use one of the following annotations to instruct a LoadBalancer Service to use
an ephemeral or static IP address in a custom subnet. If a LoadBalancer Service
manifest includes both annotations, the networking.gke.io/load-balancer-subnet
annotation takes precedence provided that its annotation requirements are met.
Annotation and value | Requirements and capabilities |
---|---|
networking.gke.io/internal-load-balancer-subnet: SUBNET_RESOURCE_NAME
|
You can only use the annotation to specify a custom subnet for an IPv4-only Internal LoadBalancer Service. The annotation works with all supported GKE versions. |
networking.gke.io/load-balancer-subnet: SUBNET_RESOURCE_NAME
|
You can specify a custom subnet for an IPv4-only, IPv6-only, or dual-stack Internal LoadBalancer Service. You can specify a custom subnet for an IPv6-only or dual-stack External LoadBalancer Service. The annotation requires GKE 1.29 or later and the following additional requirements:
|
Subnet and IPv4 address for an internal LoadBalancer Service
The table below describes valid subnet specification and IPv4 address combinations for an IPv4-only or dual-stack internal LoadBalancer Service.
Static IPv4 address
|
Ephemeral IPv4 address | |
---|---|---|
Custom subnet
|
Custom subnet and static IPv4 address: The static internal IPv4 address must have been created in the custom subnet's primary IPv4 address range. | Custom subnet and ephemeral IPv4 address: GKE uses an unallocated internal IPv4 address in the custom subnet primary IPv4 address range. |
Cluster subnet | Cluster subnet and static IPv4 address: The static internal IPv4 address must have been created in the cluster subnet's primary IPv4 address range. | Cluster subnet and ephemeral IPv4 address: GKE uses an unallocated internal IPv4 address in the cluster subnet primary IPv4 address range. |
Subnet and IPv6 address range for an internal LoadBalancer Service
The table below describes valid subnet specification and IPv6 address range
combinations for an IPv6-only or dual-stack internal LoadBalancer Service. Even
though the internal passthrough Network Load Balancer IPv6 forwarding rule uses an internal /96
IPv6
address range, GKE only configures nodes to accept packets whose
destinations match the first IPv6 address (/128
) of the forwarding rule's
/96
range.
Static IPv6 address range
|
Ephemeral IPv6 address range | |
---|---|---|
Custom dual-stack subnet
|
Custom subnet and static IPv6 address range: The static
internal /96 IPv6 address range must have been created in
the custom subnet's internal /64 IPv6 address range.
|
Custom subnet and ephemeral IPv6 address range:
GKE uses an unallocated internal /96 IPv6
address range from the custom subnet's internal /64 IPv6
address range.
|
Cluster dual-stack subnet
|
Cluster subnet and static IPv6 address range: The static
internal /96 IPv6 address range must have been created in
the cluster subnet's internal /64 IPv6 address range.
|
Cluster subnet and ephemeral IPv6 address range:
GKE uses an unallocated internal /96 IPv6
address range from the cluster subnet's internal /64 IPv6
address range.
|
Subnet and IPv4 address for an external LoadBalancer Service
For IPv4-only and dual-stack external LoadBalancer Services, the external IPv4 address—whether a static external IPv4 address or an ephemeral external IPv4 address—doesn't come from a subnet.
Subnet and IPv6 address range for an external LoadBalancer Service
The table below describes valid subnet specification and IPv6 address range
combinations for an IPv6-only or dual-stack external LoadBalancer Service. Even
though the external passthrough Network Load Balancer IPv6 forwarding rule uses an external /96
IPv6
address range, GKE only configures nodes to accept packets whose
destinations match the first IPv6 address (/128
) of the forwarding rule's
/96
range.
Static IPv6 address range
|
Ephemeral IPv6 address range | |
---|---|---|
Custom dual-stack subnet
|
Custom subnet and static IPv6 address range: The static
external /96 IPv6 address range must have been created in
the custom subnet's external /64 IPv6 address range. Static
external IPv6 address ranges can only be created in Premium tier.
|
Custom subnet and ephemeral IPv6 address range:
GKE uses an unallocated external /96 IPv6
address range from the custom subnet's external /64 IPv6
address range.
|
Cluster dual-stack subnet
|
Cluster subnet and static IPv6 address range: The static
external /96 IPv6 address range must have been created in
the cluster subnet's external /64 IPv6 address range. Static
external IPv6 address ranges can only be created in Premium tier.
|
Cluster subnet and ephemeral IPv6 address range:
GKE uses an unallocated external /96 IPv6
address range from the cluster subnet's external /64 IPv6
address range.
|
Global access
When the networking.gke.io/internal-load-balancer-allow-global-access
annotation is false
or unspecified for an internal LoadBalancer Service,
GKE creates an internal passthrough Network Load Balancer whose forwarding rule has
global access disabled. When global access is disabled, clients that need to
access the load balancer must be located in the same region and
VPC network, or a network connected to the cluster's
VPC network.
When the networking.gke.io/internal-load-balancer-allow-global-access
annotation is true
for an internal LoadBalancer Service, GKE
enables the global access option on the internal passthrough Network Load Balancer's forwarding rule.
Clients located in any region of the VPC network or a network
connected to the cluster's VPC network can access the load
balancer.
For more information about how global access applies to clients in a connected network, see:
- Client access in the internal passthrough Network Load Balancer documentation
- Internal passthrough Network Load Balancers and connected networks
All ports forwarding rules
Forwarding rules for internal passthrough Network Load Balancers support five unique port numbers or all ports.
In GKE clusters with GKE subsetting disabled, an
internal LoadBalancer Service can only support five unique ports in the
Service's spec.ports[].port
.
In GKE clusters with GKE subsetting enabled, an
internal LoadBalancer Service can only support up to 100 ports in the Service's
spec.ports[].port
. For internal LoadBalancer Services with between six and 100
unique spec.ports[].port
entries, GKE configures the
internal passthrough Network Load Balancer's forwarding rule to use all ports since its creation. The
GKE controller enables all ports on the forwarding rule because
the Service has more than five ports. When configuring a forwarding rule to use
all ports, GKE only creates ingress allow firewall rules for the
specific ports configured in spec.ports[].port
on the Service.
For more information about internal passthrough Network Load Balancer forwarding rules and valid port specifications, see Forwarding rules and port specifications.
IPv4/IPv6 dual-stack LoadBalancer Service
You can create an internal or external LoadBalancer Service that can be single-stack (IPv4 only or IPv6 only) or dual-stack. Single-stack LoadBalancer Services create a single forwarding rule with either an IPv4 address or an IPv6 address. Dual-stack LoadBalancer Services create two forwarding rules: one with an IPv4 address, and another with an IPv6 address. To create a IPv4/IPv6 dual-stack LoadBalancer Service, deploy it on a IPv4/IPv6 dual-stack cluster and complete any of the following depending on the type of load balancer you use:
- For internal LoadBalancer Services, you must have GKE subsetting enabled. For more information see, enabling internal load balancer subsetting.
- For external LoadBalancer Services, you must use a backend service-based external passthrough Network Load Balancer which uses regional backend services. For more information, see backend service-based external network load balancer.
For each
Service, you can define ipFamilyPolicy
and ipFamilies
specs. To learn more,
see IPv4/IPv6 dual-stack.
Restrictions of dual-stack LoadBalancer Services
- LoadBalancer Services with IPv6 addresses are only supported on clusters with
stack type
ipv4-ipv6
. For more information, learn how to Use a dual-stack IP address for a VPC-native cluster. LoadBalancer Services created with a single-stack address cannot be upgraded to dual-stack Services.
LoadBalancer Services created with a dual-stack addresses can be changed to single stack according to the following conditions:
- A Service with ipFamilies
["IPv4","IPv6"]
can be changed to a Service with ipFamiliesIPv4
but notIPv6
. - A Service with ipFamilies
["IPv6","IPv4"]
can be changed to a Service with ipFamiliesIPv6
but notIPv4
.
- A Service with ipFamilies