Cloud Service Mesh GKE service mesh overview

Preview

This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA products and features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

This configuration is supported for Preview customers but we do not recommended it for new Cloud Service Mesh users. For more information, see the Cloud Service Mesh overview.

This document is for Google Kubernetes Engine users who want to deploy a Cloud Service Mesh service mesh using the Kubernetes Gateway API.

You can configure Cloud Service Mesh for GKE using the Kubernetes Gateway APIs, enabling service-to-service communications, traffic management, global load balancing, and security policy enforcement for service mesh use cases.

Kubernetes APIs and Google Cloud APIs

You can configure a Cloud Service Mesh service mesh using two different APIs:

• Google Cloud APIs, which can be the service routing APIs or the older APIs
• Kubernetes Gateway APIs

This document and the associated setup guides provide instructions for using the Kubernetes Gateway API to configure a Cloud Service Mesh service mesh.

We recommend that you use the Kubernetes Gateway APIs on Google Kubernetes Engine and we recommend against using both APIs to configure routing in the same service mesh on GKE.

The service routing API uses the same resource names as the resources in the Kubernetes Gateway APIs, which makes it easier for you when you are using the two APIs. The Kubernetes resources that you configure are functionally equivalent to the Google Cloud resources represented by the service routing API for Cloud Service Mesh.

The following sections describe the resources and architecture used by the Cloud Service Mesh integration with the Kubernetes Gateway APIs.

Gateway API

The Gateway API is a collection of resources that model service networking in Kubernetes. The Kubernetes Gateway API is an open source project that focuses on supporting ingress and load balancer use cases by providing a generic routing API. The generic routing API has many implementations. Cloud Service Mesh custom resource definitions (CRDs) are added as an extension to the open source Gateway API. The CRDs support service mesh use cases and use the same generic routing API that is introduced by the Gateway API.

The Gateway API is hierarchically organized, with a Gateway parent resource and associated GatewayClass, to which you attach routes. GKE includes a TDMesh resource that is a peer of the Gateway resource. You can attach the same Route types to the TDMesh resource. The TDMesh resource is where you attach routes and policies for service meshes.

Gateway API, Gateway resource, Mesh resource, and Routes (click to enlarge)

Fleet

A fleet consists of one or more GKE clusters that are grouped logically. A fleet lets you manage capabilities and applies policies consistently across multiple clusters. When you use a fleet, you can manage a Cloud Service Mesh service mesh spanning multiple clusters.

Architecture

Cloud Service Mesh supports the Gateway API on GKE by programming your clusters' data planes to implement the networking behaviors specified in Gateway API resources. Cloud Service Mesh itself is a Google-managed control plane that does not process any data plane traffic. Envoy proxies running as sidecars to your workloads or proxyless gRPC clients process traffic in the data plane. Cloud Service Mesh configures both Envoy proxies and proxyless gRPC clients through the xDSv3 API.

Cloud Service Mesh offers a managed, globally available control plane solution that is more robust and scalable than running in-cluster controllers. Because it's a global solution, Cloud Service Mesh can load balance traffic across workloads distributed across multiple GKE clusters. In the following illustration, Cloud Service Mesh manages traffic to services in three clusters that are in a single fleet, using Gateway API resources.

A Cloud Service Mesh multi-cluster service mesh configured using the Gateway API (click to enlarge)

You designate one cluster in your fleet as the config cluster. The config cluster is where Gateway API resources are stored. Cloud Service Mesh only watches resources that are in the config cluster, and ignores resources that are in other clusters in the fleet. See Config cluster design in the GKE documentation for more detailed information about the config cluster.

With GKE multi-cluster Services, Gateway API resources in the config cluster can reference Kubernetes services in any cluster within a fleet. See Multi-cluster Services for more information about Multi-Cluster Service Discovery.

Resources

Cloud Service Mesh supports both Envoy proxies and proxyless gRPC in the data plane of a service mesh. Both clients receive configuration from Cloud Service Mesh for a particular service mesh, by specifying the name and corresponding project number of the TDMesh resource in their respective bootstrap configuration. The setup guides for Cloud Service Mesh with the Kubernetes Gateway APIs provide demonstration data plane configurations with Envoy and proxyless gRPC.

TDMesh resource

The TDMesh resource is a Cloud Service Mesh custom resource. It's an extension to the open source Gateway APIs to support Cloud Service Mesh's service mesh use cases. Using the TDMesh resource, you create a service mesh instance in your fleet. Routes attached to the TDMesh resource specify service-to-service routing behaviors in the service mesh.

Route resources

A subset of the Gateway API Route resources can be attached to a TDMesh resource to specify service-level routing within the service mesh. Cloud Service Mesh supports the following Route resources:

• HTTPRoute
• TCPRoute
• TDGRPCRoute (Cloud Service Mesh custom resource)

For example, you can create an HTTPRoute to specify that HTTP requests destined to host payments.svc.internal are routed to the Kubernetes service service-payments. When you attach the HTTPRoute resource to a TDMesh resource that data plane instances are subscribed to, HTTP requests sent by workloads within the mesh are routed accordingly.

This release augments the generic Route resources in the Gateway API with a new route type, TDGRPCRoute. The new route type provides a first-class experience for routing gRPC requests, by matching on native gRPC primitives, such as method and service definitions.

Using the Gateway API resources in GKE to configure Cloud Service Mesh (click to enlarge)

Limitations

• Cloud Service Mesh configures the following default behaviors for all Kubernetes services in the service mesh. You cannot change these behaviors.
  • TCP health checks are configured on service ports referenced by any Gateway API Route resources.
  • A default 30-second timeout is configured for all incoming requests to services.
  • Session affinity is disabled.
• The Envoy auto injector only supports one mesh per fleet.
• Cloud Service Mesh's security features cannot be enabled using the Gateway API.
• You must configure the TDMesh and Route resources on GKE using only the Gateway API. You cannot use the Google Cloud console, the gcloud CLI, or the REST APIs.
• All clusters must be in one project. A service mesh spanning across clusters in multiple projects is not supported.
• You cannot configure or view a GKE service mesh using the Google Cloud console.
• Control plane observability with Cloud Logging and Cloud Monitoring is not supported.

What's next

• Learn about preparing to set up a service mesh with the Gateway API.