Gateway


This page describes the Google Kubernetes Engine (GKE) implementation of the Kubernetes Gateway API using the GKE Gateway controller. To deploy Gateways in GKE, see Deploying Gateways or Deploying multi-cluster Gateways.

The Gateway API is an open source standard for service networking. The Gateway API evolves the Ingress resource and improves upon it in the following ways:

  • Role-oriented: Gateway is composed of API resources that correspond to the organizational roles of cluster operator, developer, and infrastructure provider. This allows cluster operators to define how shared infrastructure can be used by many different and non-coordinating developer teams.

  • Portable: The Gateway API is an open source standard with many implementations. It's designed by using the concept of flexible conformance, which promotes a highly portable core API (like Ingress) that still has the flexibility and extensibility to support native capabilities of the environment and implementation. This enables the concepts and core resources to be consistent across implementations and environments, reducing complexity and increasing user familiarity.

  • Expressive: The Gateway API resources provide built-in functionality for header-based matching, traffic weighting, and other capabilities that are only possible in Ingress through custom annotations.

Gateway API resources

The Gateway API is a role-oriented resource model, designed for the personas who interact with Kubernetes networking. As shown by the following diagram, this model enables different non-coordinating service owners to share the same underlying network infrastructure safely in a way that centralizes policy and control for the platform administrator.

GKE provides Gateway classes. Cluster operators create
Gateway resources based on these classes. Application developers create
HTTPRoute resources that bind to Gateway resources.

The Gateway API contains the following resource types:

  • GatewayClass: Defines a cluster-scoped resource that's a template for creating load balancers in a cluster. GKE provides GatewayClasses that can be used in GKE clusters.
  • Gateway: Defines where and how the load balancers listen for traffic. Cluster operators create Gateways in their clusters based on a GatewayClass. GKE creates load balancers that implement the configuration defined in the Gateway resource.
  • HTTPRoute: Defines protocol-specific rules for routing requests from a Gateway to Kubernetes services. GKE supports HTTPRoutes for HTTP(S)-based traffic routing. Application developers create HTTPRoutes to expose their HTTP applications using Gateways.

GatewayClass

A GatewayClass is a resource that defines a template for TCP/UDP (level 4) load balancers and HTTP(S) (level 7) load balancers in a Kubernetes cluster. GKE provides GatewayClasses as cluster-scoped resources. Cluster operators specify a GatewayClass when creating Gateways in their clusters.

The different GatewayClasses correspond to different Google Cloud load balancers. When you create a Gateway based on a GatewayClass, a corresponding load balancer is created to implement the specified configuration. Some GatewayClasses support multi-cluster load balancing.

The following table lists the GatewayClasses available in GKE clusters and their underlying load balancer type. For complete details on the GatewayClasses, see the GatewayClass capabilities and specifications.

Class name Description
gke-l7-rilb Regional internal HTTP(S) load balancers built on Internal HTTP(S) Load Balancing
gke-l7-gxlb Global external HTTP(S) load balancers built on External HTTP(S) Load Balancing
gke-l7-rilb-mc Multi-cluster regional load balancers built on Internal HTTP(S) Load Balancing
gke-l7-gxlb-mc Multi-cluster global load balancers built on External HTTP(S) Load Balancing

Each GatewayClass is subject to the limitations of the underlying load balancer.

Gateway

Cluster operators create Gateways to define where and how the load balancers listen for traffic. Gateways take their behavior (that is, how they are implemented) from their associated GatewayClass.

The Gateway specification includes the GatewayClass for the Gateway, which ports and protocols to listen on, and which Routes can bind to the Gateway. A Gateway selects routes based on the Route metadata; specifically the kind, namespace, and labels of Route resources.

For an example of deploying a Gateway, see Deploying Gateways. For an example of deploying a multi-cluster Gateway, see Deploying multi-cluster Gateways.

HTTPRoute

An HTTPRoute defines how HTTP and HTTPS requests received by a Gateway are directed to Services. Application developers create HTTPRoutes to expose their applications through Gateways.

An HTTPRoute defines which Gateways it can route traffic from, which Services to route to, and rules that define what traffic the HTTPRoute matches. Gateway and Route binding is bidirectional, which means that both resources must select each other for them to bind. HTTPRoutes can match requests based on details in the request header.

Comparison of Ingress and Gateway

Gateway and Ingress are both open source standards for routing traffic. Gateway was designed by the Kubernetes community, drawing on lessons learned from the Ingress and the service mesh ecosystems. Gateway is an evolution of Ingress that provides the same function, delivered as a superset of the Ingress capabilities. Both can be used simultaneously without conflict, though over time Gateway and Route resources will deliver more functionality not available in Ingress, compelling users to start using Gateway where they might have previously used Ingress.

In GKE all Ingress resources are directly convertible to Gateway and HTTPRoute resources. A single Ingress corresponds to both a Gateway (for frontend configuration) and an HTTPRoute (for routing configuration). The following example shows what the corresponding Gateway and HTTPRoute configuration looks like. Note that the Gateway and HTTPRoute resources can be created separately, also by different users. Gateways can have many Routes and a Route can also attach to more than one Gateway. The relationships between Gateways and Routes is discussed in Gateway usage patterns.

Ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-ingress
  annotations:
    kubernetes.io/ingress.class: "gce-internal"
spec:
  rules:
  - host: "example.com"
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: site
            port:
              number: 80
      - pathType: Prefix
        path: /shop
        backend:
          service:
            name: store
            port:
              number: 80

Route

apiVersion: networking.x-k8s.io/v1alpha1
kind: HTTPRoute
metadata:
  name: my-route
  labels:
    gateway: my-gateway
spec:
  hostnames:
  - "example.com"
  rules:
  - forwardTo:
    - serviceName: site
      port: 80
  - matches:
    - path:
        value: /shop
    forwardTo:
    - serviceName: store
      port: 80

Gateway

apiVersion: networking.x-k8s.io/v1alpha1
kind: Gateway
metadata:
  name: my-gateway
spec:
  gatewayClassName: gke-l7-rilb
  listeners:
  - protocol: HTTP
    port: 80
    routes:
      kind: HTTPRoute
      selector:
        matchLabels:
          gateway: my-gateway

Gateway ownership and usage patterns

Gateway and Route resources provide flexibility in how they are owned and deployed within an organization. This means that a single load balancer can be deployed and managed by an infrastructure team, but routing under a particular domain or path can be delegated to another team in another Kubernetes Namespace. The GKE Gateway controller supports multi-tenant usage of a load balancer, shared across Namespaces, clusters, and regions. Gateways also don't have to be shared if more distributed ownership is required. The following are some of the most common usage patterns for Gateways in GKE.

Self-managed Gateway

A single owner can deploy a Gateway and Route just for their applications and use them exclusively. Gateways and Routes deployed in this manner are similar to Ingress. The following diagram shows two different service owners who deploy and manage their own Gateways. Similar to Ingress, each Gateway corresponds to its own unique IP address and load balancer. TLS, routing, and other policies are fully controlled by the service owner.

A single owner can have full control of both the Gateway and the Route

This usage pattern is common for Ingress but is challenging to scale across many teams because of the lack of shared resources. The Gateway API resource model enables following usage patterns which provide a spectrum of options for distributed control and ownership.

Platform-managed Gateway per Namespace

Separation between Gateway and Route resources lets platform administrators control Gateways on behalf of service owners. Platform admins can deploy a Gateway per Namespace or team, giving that Namespace exclusive access to use the Gateway. This gives the service owner full control over the routing rules without any risk of conflict from other teams. This lets the platform administrator control aspects such as IP allocation, port exposure, protocols, domains, and TLS. Platform admins can also decide which kinds of Gateways are available to teams, such as internal or external/public Gateways. This usage pattern creates a clean separation of responsibilities between different roles.

Cross-Namespace routing is what lets Routes to attach to Gateways across Namespace boundaries. Gateways can restrict from which Namespaces Routes can attach. Similarly, Routes specify the Gateways that they attach to, but they can only attach to a Gateway that has permitted the Route's Namespace. This bi-directional attachment provides each side with flexible controls that enable this diversity of usage patterns.

In the following diagram, the platform administrator has deployed a Gateway for exclusive access for each Namespace. For example, the store Gateway is configured so that only Routes from the store Namespace can attach to it. Each Gateway represents a unique, load balanced IP address and so it lets each team deploy any number of Routes against the Gateway for any domains or routes it chooses.

A Gateway per Namespace provides exclusive access of a Gateway to that Namespace.

Shared Gateway per cluster

Sharing Gateways across Namespaces offers an even more centralized form of ownership to platform administrators. This lets different service owners, running in different Namespaces, share the same IP address, DNS domain, certificates, or paths for fine grained routing between services. Gateways provide control to platform administrators over which Namespaces can route for a specific domain. This is similar to the previous example except Gateways permit Route attachment from more than one Namespace.

In the following diagram, the platform administrator has deployed two Gateways into the infra Namespace. The external Gateway permits Routes from the web and mobile Namespaces to attach to the Gateway. Routes from the accounts Namespace cannot use the external Gateway because the accounts Namespace is only for internal services. The internal Gateway lets internal clients communicate privately within the VPC using private IP addresses.

A Gateway per cluster allows different Namespaces inside a cluster to share a single Gateway

The m.example.com domain is delegated to the mobile Namespace which lets mobile service owners configure any routing rules they need under the m.example.com domain. This gives the service owners a greater degree of control for introducing new API endpoints and controlling traffic without requesting a change from administrators.

Shared Gateway per Fleet

Using multi-cluster Gateways, a Gateway can be shared across both Namespaces, clusters, and regions. Large organizations with geographically distributed apps might benefit from multi-cluster Gateways because they can granularly control global traffic while also delegating routing ownership. Similar to the previous examples, a platform administrator manages the Gateway and delegates routing. The major addition to this use case is that Routes reference multi-cluster Services, which are deployed across clusters. Traffic can be routed in an explicit manner, traffic to store.example.com/us goes to gke-us Pods, or in an implicit manner, traffic to example.com/* gets routed to the closest cluster to the client. This flexibility lets service owners define the optimal routing strategy for their application.

A Gateway per fleet provides multi-cluster, multi-regional load balancing

GKE Gateway controller

The GKE Gateway controller is Google's implementation of the Gateway API for Cloud Load Balancing. Similar to the GKE Ingress controller, the Gateway controller watches a Kubernetes API for Gateway API resources and reconciles Cloud Load Balancing resources to implement the networking behavior specified by the Gateway resources.

There are two versions of the GKE Gateway controller:

  • Single-cluster: manages single-cluster Gateways for a single GKE cluster.
  • Multi-cluster: manages multi-cluster Gateways for one or more GKE clusters.

Both Gateway controllers are Google-hosted controllers that watch the Kubernetes API for GKE clusters. Unlike the GKE Ingress controller, the Gateway controllers are not hosted on GKE control planes or in the user project, enabling them to be more scalable and robust.

The Gateway controllers themselves are not a networking data plane and they do not process any traffic. They sit out of band from traffic and manage various data planes that process traffic. The following diagram shows the architecture of the single-cluster and multi-cluster GKE Gateway controllers. The underlying controller that is used depends on the GatewayClass of the deployed Gateway.

The multi-cluster and single-cluster Gateway controllers deploy and manage load balancers for GKE, but do not process network traffic themselves.

Controller Single-cluster Gateway controller Multi-cluster Gateway controller
Managed by Google Google
Cluster scope Single cluster Gateways Multi-cluster Gateways
Deployment location Deployed regionally in the same region as its GKE cluster. Deployed globally across multiple Google Cloud regions.
How to enable Enabled by default in GKE. Enabled through the Multi Cluster Ingress API and registration into a fleet. See Enabling multi-cluster Gateways.
Supported GatewayClasses
  • gke-l7-rilb
  • gke-l7-gxlb
  • gke-l7-rilb-mc
  • gke-l7-gxlb-mc

You can use multiple Gateway controllers, including controllers not provided by Google, in a GKE cluster simultaneously. Every GatewayClass is supported by one and only one Gateway controller, which enables single and multi-cluster load balancing to be used simultaneously.

Migrating from Ingress to Gateway

Migration between Ingress and Gateway is possible by deploying both Ingress and Gateway resources simultaneously, pointed at the same set of Services. This creates two entrypoints for the same backend applications. Each Ingress and Gateway corresponds to and individual load balancer IP address. Test the Gateway path and once it is validated use DNS to switch traffic from the Ingress to the Gateway.

Direct conversion of an Ingress to a Gateway is not supported. However, if you are currently using Ingress, it's recommended to run both simultaneously so that you can safely ensure that the transition to Gateway is done without any traffic interruption.

Pricing

All Compute Engine resources deployed through the Gateway controller are charged against the project in which your GKE clusters reside. The single-cluster Gateway controller is offered at no additional charge as a part of GKE Standard and Autopilot pricing. You can use the multi-cluster Gateway controller without additional charge during Preview. Upon GA, multi-cluster Gateways will be charged according to the multi-cluster Ingress and Gateway pricing.

What's next