This page shows you how to configure the load balancer that GKE creates when you deploy a Gateway in a Google Kubernetes Engine (GKE) cluster.
When you deploy a Gateway for your GKE cluster or for a fleet of GKE clusters, the GatewayClass configuration determines which load balancer GKE creates. This managed load balancer is pre-configured with default settings that you can modify using a Policy.
You can customize Gateway resources to fit with your infrastructure or application(s) requirements by attaching Policies to Gateways, Services or ServiceImports (in the context of multi-cluster Gateways). After you apply or modify a Policy, you don't need to delete or recreate your Gateway, Route, or Service resources, the Policy is processed by the Gateway controller and the underlying load balancer resource is (re)configured according to the (new) Policy.
Before you begin
Before you start, make sure you have performed the following tasks:
- Enable the Google Kubernetes Engine API. Enable Google Kubernetes Engine API
- If you want to use the Google Cloud CLI for this task,
install and then
initialize the
gcloud CLI. If you previously installed the gcloud CLI, get the latest
version by running
gcloud components update
.
GKE Gateway controller requirements
- For Standard, GKE version 1.24 or later.
- For Autopilot, GKE version 1.26 or later.
- Google Cloud CLI version 407.0.0 or later.
- The Gateway API is supported on VPC-native clusters only.
- If you are using the internal GatewayClasses, you must enable a proxy-only subnet.
- Your cluster must have the
HttpLoadBalancing
add-on enabled. - If you are using Istio, you must upgrade Istio to one of the following
versions:
- 1.15.2 or later
- 1.14.5 or later
- 1.13.9 or later.
Restrictions and Limitations
In addition to the GKE Gateway controller restrictions and limitations, the following limitations apply specifically to Policies applied on the Gateway resources:
GCPGatewayPolicy
resources can only be attached to agateway.networking.k8s.io
Gateway
.GCPGatewayPolicy
resources must exist in the same namespace as the targetGateway
.When using a single cluster Gateway,
GCPBackendPolicy
, andHealthCheckPolicy
resources must refer to aService
resource. When using a multi-cluster Gateway,GCPBackendPolicy
, andHealthCheckPolicy
, resources must refer to aServiceImport
resource.HealthCheckPolicy
, andGCPBackendPolicy
resources must exist in the same namespace as the targetService
orServiceImport
resource.GCPBackendPolicy
andHealthCheckPolicy
resources are structured in a way that they can reference only one backend service.
Configure global access for your regional internal Gateway
This section describes a functionality that is available on GKE clusters running version 1.24 or later.
To enable global access with your internal Gateway, attach a policy to the Gateway resource.
The following GCPGatewayPolicy
manifest enables regional internal Gateway for
global access:
apiVersion: networking.gke.io/v1
kind: GCPGatewayPolicy
metadata:
name: my-gateway-policy
namespace: default
spec:
default:
allowGlobalAccess: true
targetRef:
group: gateway.networking.k8s.io
kind: Gateway
name: my-gateway
Configure SSL Policies to secure client-to-load-balancer traffic
This section describes a functionality that is available on GKE clusters running version 1.24 or later.
To secure your client-to-load-balancer traffic, configure the SSL policy by adding
the name of your policy to the GCPGatewayPolicy
. By default, the Gateway does not
have any SSL Policy defined and attached.
Make sure that you create an SSL policy
prior to referencing the policy in your GCPGatewayPolicy
.
The following GCPGatewayPolicy
manifest specifies a security policy named
gke-gateway-ssl-policy
:
apiVersion: networking.gke.io/v1
kind: GCPGatewayPolicy
metadata:
name: my-gateway-policy
namespace: team1
spec:
default:
sslPolicy: gke-gateway-ssl-policy
targetRef:
group: gateway.networking.k8s.io
kind: Gateway
name: my-gateway
Configure health checks
This section describes a functionality that is available on GKE clusters running version 1.24 or later.
You can use a HealthCheckPolicy
to control the load balancer health check
settings. Each type of health check (http
, https
, grpc
, and http2
) has a collection
of parameters that can be defined and Google Cloud creates a unique health
check for each backend service for each GKE Service.
The following HealthCheckPolicy
manifest shows all the fields available when
configuring a health check policy:
Service
apiVersion: networking.gke.io/v1
kind: HealthCheckPolicy
metadata:
name: lb-healthcheck
namespace: lb-service-namespace
spec:
default:
checkIntervalSec: INTERVAL
timeoutSec: TIMEOUT
healthyThreshold: HEALTHY_THRESHOLD
unhealthyThreshold: UNHEALTHY_THRESHOLD
logConfig:
enabled: ENABLED
config:
type: PROTOCOL
httpHealthCheck:
portSpecification: PORT_SPECIFICATION
port: PORT
portName: PORT_NAME
host: HOST
requestPath: REQUEST_PATH
response: RESPONSE
proxyHeader: PROXY_HEADER
httpsHealthCheck:
portSpecification: PORT_SPECIFICATION
port: PORT
portName: PORT_NAME
host: HOST
requestPath: REQUEST_PATH
response: RESPONSE
proxyHeader: PROXY_HEADER
grpcHealthCheck:
grpcServiceName: GRPC_SERVICE_NAME
portSpecification: PORT_SPECIFICATION
port: PORT
portName: PORT_NAME
http2HealthCheck:
portSpecification: PORT_SPECIFICATION
port: PORT
portName: PORT_NAME
host: HOST
requestPath: REQUEST_PATH
response: RESPONSE
proxyHeader: PROXY_HEADER
targetRef:
group: ""
kind: Service
name: lb-service
Multi-cluster Service
apiVersion: networking.gke.io/v1
kind: HealthCheckPolicy
metadata:
name: lb-healthcheck
namespace: lb-service-namespace
spec:
default:
checkIntervalSec: INTERVAL
timeoutSec: TIMEOUT
healthyThreshold: HEALTHY_THRESHOLD
unhealthyThreshold: UNHEALTHY_THRESHOLD
logConfig:
enabled: ENABLED
config:
type: PROTOCOL
httpHealthCheck:
portSpecification: PORT_SPECIFICATION
port: PORT
portName: PORT_NAME
host: HOST
requestPath: REQUEST_PATH
response: RESPONSE
proxyHeader: PROXY_HEADER
httpsHealthCheck:
portSpecification: PORT_SPECIFICATION
port: PORT
portName: PORT_NAME
host: HOST
requestPath: REQUEST_PATH
response: RESPONSE
proxyHeader: PROXY_HEADER
grpcHealthCheck:
grpcServiceName: GRPC_SERVICE_NAME
portSpecification: PORT_SPECIFICATION
port: PORT
portName: PORT_NAME
http2HealthCheck:
portSpecification: PORT_SPECIFICATION
port: PORT
portName: PORT_NAME
host: HOST
requestPath: REQUEST_PATH
response: RESPONSE
proxyHeader: PROXY_HEADER
targetRef:
group: net.gke.io
kind: ServiceImport
name: lb-service
Replace the following:
INTERVAL
: specifies the check-interval, in seconds, for each health check prober. This is the time from the start of one prober's check to the start of its next check. If you omit this parameter, the Google Cloud default is 5 seconds. For more information, see Multiple probes and frequency.TIMEOUT
: specifies the amount of time that Google Cloud waits for a response to a probe. The value ofTIMEOUT
must be less than or equal to theINTERVAL
. Units are seconds. Each probe requires an HTTP 200 (OK) response code to be delivered before the probe timeout.HEALTHY_THRESHOLD
andUNHEALTHY_THRESHOLD
: specifies the number of sequential connection attempts that must succeed or fail, for at least one prober, to change the health state from healthy to unhealthy or unhealthy to healthy. If you omit one of these parameters, the Google Cloud default is 2.PROTOCOL
: specifies a protocol used by probe systems for health checking. For more information, see Success criteria for HTTP, HTTPS, and HTTP/2 and Success criteria for gRPC. This parameter is required.ENABLED
: specifies if logging is enabled or disabled.PORT_SPECIFICATION
: specifies if the health check uses a fixed port (USE_FIXED_PORT
), named port (USE_NAMED_PORT
) or serving port (USE_SERVING_PORT
). If not specified, the health check follows the behavior specified in theport
andportName
fields. If neitherport
orportName
are specified, this field defaults toUSE_SERVING_PORT
.PATH
: specifies the request-path to which the probe system should connect for HTTP, HTTPS, or HTTP2 health checks. If you omit this parameter, the Google Cloud default is/
.PORT
: A HealthCheckPolicy only supports specifying the load balancer health check port by using a port number. If you omit this parameter, the Google Cloud default is 80. Because the load balancer sends probes to the Pod's IP address directly, you should select a port matching acontainerPort
of a serving Pods, even if thecontainerPort
is referenced by atargetPort
of the Service. You are not limited tocontainerPorts
referenced by a Service'stargetPort
.PORT_NAME
: specifies the port name as defined in InstanceGroup.NamedPort.name. If bothport
andportName
are defined, Google Cloud considers theport
value first.HOST
: the value of the host header in the health check request. This value uses the RFC 1123 definition of a hostname except numeric IP addresses are not allowed. If not specified or left empty, this value defaults to the IP address of the health check.REQUEST_PATH
: specifies the request path of the health check request. If not specified or left empty, defaults to/
.RESPONSE
: specifies the bytes to match against the beginning of the response data. If not specified or left empty, GKE interprets any response as healthy. The response data can only be ASCII.PROXY_HEADER
: specifies the proxy header type. You can useNONE
orPROXY_V1
. Defaults toNONE
.GRPC_SERVICE_NAME
: an optional name of the gRPC Service. Omit this field to specify all Services.
For more information about HealthCheckPolicy fields, see the healthChecks
reference.
Configure Google Cloud Armor security policy to secure your backend services
This section describes a functionality that is available on GKE clusters running version 1.24 or later.
Configure the Google Cloud Armor security policy by adding the name of your security
policy to the GCPBackendPolicy
to secure your backend services. By default,
the Gateway does not have any Google Cloud Armor security policy defined and attached.
Make sure that you create a Google Cloud Armor security policy
prior to referencing the policy in your GCPBackendPolicy
.
The following GCPBackendPolicy
manifest specifies a security policy named
example-security-policy
:
Service
apiVersion: networking.gke.io/v1
kind: GCPBackendPolicy
metadata:
name: my-backend-policy
namespace: lb-service-namespace
spec:
default:
securityPolicy: example-security-policy
targetRef:
group: ""
kind: Service
name: lb-service
Multi-cluster Service
apiVersion: networking.gke.io/v1
kind: GCPBackendPolicy
metadata:
name: my-backend-policy
namespace: lb-service-namespace
spec:
default:
securityPolicy: example-security-policy
targetRef:
group: net.gke.io
kind: ServiceImport
name: lb-service
Configure IAP
This section describes a functionality that is available on GKE clusters running version 1.24 or later.
Identity-Aware Proxy (IAP) enforces access control policies on backend services associated with an HTTPRoute. With this enforcement, only authenticated users or applications with the correct Identity and Access Management (IAM) role assigned can access these backend services.
By default, there is no IAP applied to your backend services, you
need to explicitly configure IAP in a GCPBackendPolicy
.
To configure IAP with Gateway, do the following:
Enable IAP for GKE Do not configure the backend (Configuring BackendConfig) because
BackendConfig
is only valid in the case of an Ingress deployment.Create a secret for your IAP:
Go to the Credentials page. Button: Go to the Credentials page.
Click the name of the client and download the OAuth client file.
From the OAuth client file, copy the OAuth secret on the clipboard.
Create a file called
iap-secret.txt
.Paste the OAuth secret in the
iap-secret.txt
file using the following command:echo -n CLIENT_SECRET > iap-secret.txt kubectl create secret generic SECRET_NAME --from-file=key=iap-secret.txt
To specify IAP policy referencing a secret:
Create the following
GCPBackendPolicy
manifest, replace theSECRET_NAME
andCLIENT_ID
respectively. Save the manifest asbackend-policy.yaml
:Service
apiVersion: networking.gke.io/v1 kind: GCPBackendPolicy metadata: name: backend-policy spec: default: iap: enabled: true oauth2ClientSecret: name: SECRET_NAME clientID: CLIENT_ID targetRef: group: "" kind: Service name: lb-service
Multi-cluster Service
apiVersion: networking.gke.io/v1 kind: GCPBackendPolicy metadata: name: backend-policy spec: default: iap: enabled: true oauth2ClientSecret: name: SECRET_NAME clientID: CLIENT_ID targetRef: group: net.gke.io kind: ServiceImport name: lb-service
Apply the
backend-policy.yaml
manifest:kubectl apply -f backend-policy.yaml
Verify your configuration:
Confirm that the policy was applied after creating your
GCPBackendPolicy
with IAP:kubectl get gcpbackendpolicy
The output is similar to the following:
NAME AGE backend-policy 45m
To get more details, use the describe command:
kubectl describe gcpbackendpolicy
The output is similar to the following:
Name: backend-policy Namespace: default Labels: <none> Annotations: <none> API Version: networking.gke.io/v1 Kind: GCPBackendPolicy Metadata: Creation Timestamp: 2023-05-27T06:45:32Z Generation: 2 Resource Version: 19780077 UID: f4f60a3b-4bb2-4e12-8748-d3b310d9c8e5 Spec: Default: Iap: Client ID: 441323991697-luotsrnpboij65ebfr13hlcpm5a4heke.apps.googleusercontent.com Enabled: true oauth2ClientSecret: Name: my-iap-secret Target Ref: Group: Kind: Service Name: lb-service Status: Conditions: Last Transition Time: 2023-05-27T06:48:25Z Message: Reason: Attached Status: True Type: Attached Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal ADD 46m sc-gateway-controller default/backend-policy Normal SYNC 44s (x15 over 43m) sc-gateway-controller Application of GCPGatewayPolicy "default/backend-policy" was a success
Configure backend service timeout
This section describes a functionality that is available on GKE clusters running version 1.24 or later.
The following GCPBackendPolicy
manifest specifies a
backend service timeout
period of 40 seconds. The timeoutSec
field defaults to 30 seconds.
Service
apiVersion: networking.gke.io/v1
kind: GCPBackendPolicy
metadata:
name: my-backend-policy
namespace: lb-service-namespace
spec:
default:
timeoutSec: 40
targetRef:
group: ""
kind: Service
name: lb-service
Multi-cluster Service
apiVersion: networking.gke.io/v1
kind: GCPBackendPolicy
metadata:
name: my-backend-policy
namespace: lb-service-namespace
spec:
default:
timeoutSec: 40
targetRef:
group: net.gke.io
kind: ServiceImport
name: lb-service
Configure session affinity
This section describes a functionality that is available on GKE clusters running version 1.24 or later.
You can configure session affinity based on the following criteria:
- Client IP address
- Generated cookie
The following GCPBackendPolicy
manifest specifies a session affinity based on the
client IP address:
Service
apiVersion: networking.gke.io/v1
kind: GCPBackendPolicy
metadata:
name: my-backend-policy
namespace: lb-service-namespace
spec:
default:
sessionAffinity:
type: CLIENT_IP
targetRef:
group: ""
kind: Service
name: lb-service
Multi-cluster Service
apiVersion: networking.gke.io/v1
kind: GCPBackendPolicy
metadata:
name: my-backend-policy
namespace: lb-service-namespace
spec:
default:
sessionAffinity:
type: CLIENT_IP
targetRef:
group: net.gke.io
kind: ServiceImport
name: lb-service
The following GCPBackendPolicy
manifest specifies a session affinity based on a
generated cookie
and configures the cookies TTL to 50 seconds:
Service
apiVersion: networking.gke.io/v1
kind: GCPBackendPolicy
metadata:
name: my-backend-policy
namespace: lb-service-namespace
spec:
default:
sessionAffinity:
type: GENERATED_COOKIE
cookieTtlSec: 50
targetRef:
group: ""
kind: Service
name: lb-service
Multi-cluster Service
apiVersion: networking.gke.io/v1
kind: GCPBackendPolicy
metadata:
name: my-backend-policy
namespace: lb-service-namespace
spec:
default:
sessionAffinity:
type: GENERATED_COOKIE
cookieTtlSec: 50
targetRef:
group: net.gke.io
kind: ServiceImport
name: lb-service
You can use the following options for sessionAffinity.type
: CLIENT_IP
,
CLIENT_IP_PROTO
, CLIENT_IP_PORT_PROTO
, GENERATED_COOKIE
, HEADER_FIELD
,
HTTP_COOKIE
, NONE
.
Configure connection draining timeout
This section describes a functionality that is available on GKE clusters running version 1.24 or later.
You can configure
connection draining timeout
using GCPBackendPolicy
. Connection draining timeout is the time, in seconds, to
wait for connections to drain. The timeout duration can be from 0 to 3600 seconds.
The default value is 0, which also disables connection draining.
The following GCPBackendPolicy
manifest specifies a connection draining timeout
of 60 seconds:
Service
apiVersion: networking.gke.io/v1
kind: GCPBackendPolicy
metadata:
name: my-backend-policy
namespace: lb-service-namespace
spec:
default:
connectionDraining:
drainingTimeoutSec: 60
targetRef:
group: ""
kind: Service
name: lb-service
Multi-cluster Service
apiVersion: networking.gke.io/v1
kind: GCPBackendPolicy
metadata:
name: my-backend-policy
namespace: lb-service-namespace
spec:
default:
connectionDraining:
drainingTimeoutSec: 60
targetRef:
group: net.gke.io
kind: ServiceImport
name: lb-service
For the specified duration of the timeout, GKE waits for existing requests to the removed backend to complete. The load balancer does not send new requests to the removed backend. After the timeout duration is reached, GKE closes all remaining connections to the backend.
Configure HTTP access logging
This section describes a functionality that is available on GKE clusters running version 1.24 or later.
Gateway can log all HTTP requests from clients to Cloud Logging.
You can enable and disable
access logging
using GCPBackendPolicy
. You can also configure the access logging sampling rate.
To configure access logging, use the logging
field in your GCPBackendPolicy
. If
logging
is omitted, access logging is disabled by default.
The following GCPBackendPolicy
manifest enables access logging and sets the
sample rate to 50% of the HTTP requests for a given Service resource:
Service
apiVersion: networking.gke.io/v1
kind: GCPBackendPolicy
metadata:
name: my-backend-policy
namespace: lb-service-namespace
spec:
default:
logging:
enabled: true
sampleRate: 500000
targetRef:
group: ""
kind: Service
name: lb-service
Multi-cluster Service
apiVersion: networking.gke.io/v1
kind: GCPBackendPolicy
metadata:
name: my-backend-policy
namespace: lb-service-namespace
spec:
default:
logging:
enabled: true
sampleRate: 500000
targetRef:
group: net.gke.io
kind: ServiceImport
name: lb-service
This manifest has the following fields:
enable: true
: enables access logging. Logs are available in Logging.sampleRate: 500000
: specifies that 50% of packets are logged. You can use a value between 0 and 1,000,000. GKE converts this value to a floating point value in the range [0, 1] by dividing by 1,000,000. This field is only relevant ifenable
is set totrue
.sampleRate
is an optional field, but if it's configured thenenable: true
must also be set. Ifenable
is set totrue
andsampleRate
is not provided then GKE setsenable
tofalse
.
What's next
- Learn more about the Gateway controller.
- Learn how to reference a Gateway from a resource.