Ingress traffic for your mesh

A service mesh facilitates communications among the services running in the mesh. But how do you get traffic into your mesh? This step—directing traffic from outside of your mesh into your mesh through an entry point—can be solved using a gateway.

This document describes how to use Cloud Load Balancing as a gateway to get traffic into your mesh.

Overview

When you design your service mesh, you need to consider traffic coming from these sources:

  • Traffic that originates inside of your mesh
  • Traffic that originates outside of your mesh

Traffic that originates inside of your mesh travels on the service mesh data plane to reach a backend or endpoint associated with the destination service. But traffic that originates outside of your mesh needs to first reach the service mesh data plane.

In the following example of traffic that originates inside of your mesh, Traffic Director configures your sidecar proxies. These sidecar proxies form the data plane of your service mesh. If Service A wants to communicate with Service B:

  1. Service A makes a request to Service B by name.
  2. This request is intercepted and redirected to Service A's sidecar proxy.
  3. The sidecar proxy then sends the request to an endpoint associated with Service B.
Service mesh-internal traffic handled by the service mesh data plane (click to enlarge)
Traffic internal to the service mesh is handled by the mesh's data plane (click to enlarge)

In the following example, traffic originates outside of your service mesh and doesn't travel along the service mesh data plane.

Traffic external to the service mesh isn't handled by the service mesh data plane (click to enlarge)
Traffic external to the service mesh isn't handled by the service mesh data plane (click to enlarge)

In this example, the client is outside of your service mesh. Because it doesn't directly participate in the mesh, the client doesn't know which endpoints belong to services inside the mesh. In other words, because the client doesn't send outbound requests using a Traffic Director-configured proxy, it doesn't know which IP address-port pairs to use when sending traffic to Service A or Service B. Without that information, the client can't reach services inside your mesh.

The gateway patterns described in this doc help you solve this problem: how do you get traffic into your mesh?

This document offers:

  • High-level considerations for your gateway.
  • An overview of options when you select a gateway for your mesh.
  • Architectural recommendations that you can apply to your gateway topology.

Considerations for your gateway

This section provides an overview of issues to consider when you select a gateway:

  • How can clients reach my gateway?
  • What policies do I want to apply to traffic that reaches my gateway?
  • How does my gateway distribute traffic to services in my mesh?

Enabling clients to reach the gateway to your mesh

Clients, whether on the public internet, in your on-premises environment, or within Google Cloud, need a way to reach a service within your mesh. This is typically achieved by using a publicly or privately routable IP address and port that resolve to a gateway. Clients outside of your mesh use this IP address and port to send requests to services in your mesh, through your gateway.

Cloud Load Balancing provides various load balancing options that you can use as the gateway to your mesh. The main questions to ask when you choose a Google Cloud load balancer to act as your gateway are these:

  • Are my clients on the public internet, in an on-premises environment, or part of my Virtual Private Cloud (VPC) network?
  • Which communication protocol(s) do my clients use?

Choosing a gateway for your mesh provides an overview of Cloud Load Balancing options, depending on client location and communication protocol.

Handling traffic at the gateway

Because your gateway sits at the edge of your mesh—between clients that are outside of your mesh and services that are inside of your mesh—the gateway is a natural place to apply policies as traffic is entering your mesh. These policies include:

  • Traffic management (for example, routing, redirects and request transformation)
  • Security (for example, TLS termination and Google Cloud Armor DDoS protection)
  • Cloud CDN caching

Choosing a gateway for your mesh highlights policies that are relevant at the edge of your mesh.

Sending traffic from the gateway to a service in your mesh

After your gateway applies policies to incoming traffic, the gateway decides where to send the traffic. You configure this using traffic management and load balancing policies. The gateway might, for example, inspect the request header to identify the mesh service that should receive the traffic. After the gateway identifies the service, it distributes traffic to a specific backend according to a load balancing policy.

Choosing a gateway for your mesh outlines the backends to which a gateway can send traffic.

Choosing a gateway for your mesh

Google Cloud offers a wide range of load balancers that can act as the gateway to your mesh. This section discusses selecting a gateway, comparing the following options along dimensions relevant to the gateway pattern:

In this section, we refer to first-level and second-level gateways. These terms are used to describe the use of one gateway or two gateways to handle ingress traffic to your mesh.

You might need just one level, a single load balancer that acts as a gateway to the mesh. Sometimes, though, it makes sense to have multiple gateways. In these configurations, one gateway handles traffic coming into Google Cloud and a separate, second-level gateway handles traffic as it enters the service mesh. For example, you might want to apply Google Cloud Armor security policies to traffic entering Google Cloud and advanced traffic management policies to traffic that is entering the mesh. The pattern of using a second Traffic Director-configured gateway is discussed in Deploying a second-level gateway at the edge of your mesh.

The following table compares the capabilities available, depending on the gateway option you select:

Gateway Client location Protocols Policies Backends / endpoints
Internal HTTP(S) Load Balancing Google Cloud-based clients in the same region as the load balancer

On-premises clients whose requests arrive in the same Google Cloud region as the load balancer (for example, using Cloud VPN or Cloud Interconnect)
  • HTTP/1.1
  • HTTP/2
  • HTTPS
Advanced traffic management
TLS termination using self-managed certificates
Backends in the same Google Cloud region as the load balancer, running on:

Virtual machine backends on Compute Engine

Container instances on:
  • Google Kubernetes Engine (GKE)
  • Kubernetes
HTTP(S) Load Balancing Clients on the public internet
  • HTTP/1.1
  • HTTP/2
  • HTTPS
  • Traffic management
  • Cloud CDN (including Cloud Storage bucket backends)
  • TLS termination using Google- or self-managed certificates
  • SSL policies
  • Google Cloud Armor for DDoS and web attack prevention
  • IAP support for user authentication
Backends in any Google Cloud region, running on:

Virtual machines on Compute Engine

Container instances on:
  • Google Kubernetes Engine (GKE)
  • Kubernetes
Internal TCP/UDP Load Balancing Google Cloud-based clients in any region; this requires Global Access if clients are in a different region from the load balancer.

On-premises clients whose requests arrive in any Google Cloud region (for example, using Cloud VPN or Cloud Interconnect)
TCP Backends in the same Google Cloud region as the load balancer, running on:

Virtual machines on Compute Engine
Network Load Balancing Clients on the public internet One of TCP or UDP Backends in the same Google Cloud region as the load balancer, running on:

Virtual machines on Compute Engine
SSL Proxy Load Balancing and TCP Proxy Load Balancing Clients on the public internet One of SSL or TCP TLS termination using Google- or self-managed certificates (SSL proxy only)

SSL policies (SSL proxy only)
Backends in any Google Cloud region, running on:

Virtual machines on Compute Engine

Container instances on:
  • Google Kubernetes Engine (GKE)
  • Kubernetes
Edge proxy (on VM or container instances) configured by Traffic Director Clients must be in a location where one of the following applies:
  • They can send a request to a Google Cloud-managed load balancer, which then sends the request to the edge proxy. For more information, see Deploying a second-level gateway at the edge of your mesh in this document.
  • They can send a request through a proxy – for example, a sidecar proxy – that is configured by Traffic Director.
  • They can send a request directly to the IP address and port of a VM or container instance that is running the edge proxy.
  • HTTP/1.1
  • HTTP/2
Advanced traffic management (including regex support) Backends in any Google Cloud region, running on:

Virtual machines on Compute Engine

Container instances on:
  • Google Kubernetes Engine (GKE)
  • Kubernetes

The Load balancer features page provides a detailed feature-by-feature comparison. The Traffic Director features pages provides a detailed overview of Traffic Director features.

Deploying and configuring gateways

A final consideration in selecting your gateway is the developer experience and tooling that you want to use. Google Cloud offers multiple approaches for creating and managing your gateway.

gcloud command-line tool and Compute Engine APIs

You can use the gcloud command-line tool and Compute Engine APIs to configure Google Cloud's managed load balancing products and Traffic Director. The CLI and APIs provide mechanisms to deploy and configure your Google Cloud resources imperatively. You can create scripts to automate repetitive tasks.

Google Cloud Console

You can use the Google Cloud Cloud Console, a graphical interface to configure Traffic Director and Google Cloud's managed load balancers. To configure your gateway pattern, you are likely to need both the Cloud Load Balancing user interface and Traffic Director user interface.

GKE and Ingress for Anthos

GKE and Anthos network controllers also support the deployment of Cloud Load Balancing for native integration with container networking. They provide a Kubernetes-style declarative interface for deploying and configuring gateways. GKE Ingress and Anthos Ingress controllers manage internal and external load balancers for sending traffic to a single cluster or across multiple GKE clusters. The Ingress resource can also be configured to point to Traffic Director-configured services that are deployed in GKE clusters.

Gateway architecture patterns

This section describes high-level patterns and provides architecture diagrams for your gateway.

Patterns

The most common pattern involves using a Google Cloud-managed load balancer as your gateway:

  1. Clients send traffic to a Google Cloud-managed load balancer that acts as your gateway.
    • The gateway applies policies.
  2. The gateway sends the traffic to a service in your mesh.

A more advanced pattern involves gateways at two levels. The gateways work as follows:

  1. Clients send traffic to a Google Cloud-managed load balancer that acts as your first-level gateway.
    • The gateway applies policies.
  2. The gateway sends the traffic to an edge proxy (or pool of edge proxies) configured by Traffic Director.
    • This edge proxy acts as a second-level gateway. This level:
      1. Provides a clear separation of concerns in which, for example, one team may be responsible for ingress traffic entering Google Cloud while another team is responsible for ingress traffic entering that team's mesh.
      2. Enables you to apply policies that may not be supported in the Google Cloud-managed load balancer.
  3. The second-level gateway sends the traffic to a service in your mesh.

The ingress pattern ends once traffic reaches an in-mesh service. Both the common case and advanced patterns are described below.

Enabling ingress traffic from the internet

If your clients are outside of Google Cloud and need to reach Google Cloud through the public internet, you can use one of the following load balancers as your gateway.

For more information to help you decide on an appropriate gateway, see Choosing a gateway for your mesh.

Ingress traffic from clients on the public internet to in-mesh services using a load balancer (click to enlarge)
Ingress traffic from clients on the public internet to in-mesh services using a load balancer (click to enlarge)

In this pattern, the Google Cloud-managed load balancer serves as your gateway. The gateway handles ingress traffic before forwarding it to a service in your mesh.

For example, you might choose HTTP(S) Load Balancing as your gateway to use the following:

  • A publicly-routable global Anycast IP address, which minimizes latency and network traversal costs
  • Google Cloud Armor and TLS termination to secure traffic to your mesh
  • Cloud CDN to serve web and video content
  • Traffic management capabilities such as host- and path-based routing

Enabling ingress traffic from clients in Virtual Private Cloud and connected on-premises networks

If your clients are inside of your VPC network, or if they are on-premises and can reach Google Cloud services using a private connectivity method (such as Cloud VPN or Cloud Interconnect), you can use one of the following load balancers as your gateway.

For more information to help you decide on an appropriate gateway, see Choosing a gateway for your mesh.

Ingress traffic from clients on a VPC network to in-mesh services using a load balancer (click to enlarge)
Ingress traffic from clients on a VPC network to in-mesh services using a load balancer (click to enlarge)

In this pattern, the Google Cloud-managed load balancer serves as your gateway. The gateway handles ingress traffic before forwarding it to a service in your mesh.

For example, you might choose Internal HTTP(S) Load Balancing as your gateway so that you can use these features:

  • A privately-addressable IP address
  • TLS termination to secure your mesh
  • Advanced traffic management capabilities such as weight-based traffic splitting
  • NEGs as backends

Handling ingress traffic using a second-level gateway at the edge of your mesh

Depending on your needs, you may consider a more advanced pattern which adds an additional gateway.

Ingress traffic from external clients to in-mesh services using a load balancer and an edge proxy (click to enlarge)
Ingress traffic from external clients to in-mesh services using a load balancer and an edge proxy (click to enlarge)

This gateway is a Traffic Director-configured edge proxy (or pool of proxies) that sits behind the Google Cloud-managed load balancer. You can host this second-level gateway in your project using a pool of Compute Engine VMs (a MIG) or GKE services.

While this pattern is more advanced, it provides additional benefits:

  • The Google Cloud-managed load balancer applies an initial set of policies (for example, Google Cloud Armor protection, if you are using HTTP(S) Load Balancing).

  • The Traffic Director-configured edge proxy applies a second set of policies that might not be available in the Google Cloud-managed load balancer. These policies include advanced traffic management using regular expressions applied to HTTP headers and weight-based traffic splitting.

This pattern can be set up to reflect your organizational structure. For example:

  1. One team might be responsible for handling ingress traffic to Google Cloud while another team is responsible for handling ingress traffic to its mesh.

  2. If multiple teams offer services on one Shared VPC, with each team owning its own service project, teams can use this pattern to manage and apply policies in their own meshes. Each team can expose a Traffic Director-configured gateway that is reachable on a single IP address and port pair. A team can then independently define and manage the policies that are applied on ingress traffic to the team's mesh.

Note that this pattern can be implemented using any Google Cloud-managed load balancer, as long as the load balancer can send traffic to the backends hosting the Traffic Director-configured gateway(s). See Choosing a gateway for your mesh for information on the backends supported by each load balancer. For information about using Traffic Director with Shared VPC, see Configuring Traffic Director with Shared VPC on multiple GKE clusters.