Traffic Director concepts

Traffic Director for open service mesh

Service meshes have become increasingly popular for deploying microservices and other modern applications. In a service mesh, the data plane uses service proxies, such as Envoy, to control traffic flow, while the service mesh control plane provides policy, configuration, and intelligence to the service proxies.

Service mesh as the control plane (click to enlarge)
Service mesh as the control plane (click to enlarge)

The service mesh reduces the amount of networking code that you must write and maintain to run your services. Instead, the service proxy performs networking functions alongside your business logic. A services control plane is required to manage the service proxies.

Traffic Director is Google Cloud Platform's fully-managed traffic control plane for service meshes. Using Traffic Director, you can easily deploy global load balancing across clusters and VM instances in multiple regions, offload health checking from the service proxies, and configure sophisticated traffic control policies. Traffic Director uses open standard APIs (xDS v2) to communicate with the service proxies in the data plane, ensuring that you are not locked in to a proprietary solution and can use the service mesh control plane of your choice.

Traffic Director as the control plane in a microservices environment (click to enlarge)
Traffic Director as the control plane in a microservices environment (click to enlarge)

Traffic Director features

Traffic management for open service proxies

Traffic Director provides a GCP-managed traffic management control plane for open standard (xDS v2) service proxies such as Envoy.

Works with VMs and containers

Deploy your Traffic Director-managed VM service instances using managed instance groups and your container instances using network endpoint groups.

Global load balancing

With Traffic Director, deploy your service instances in multiple regions for resiliency and reach while requiring only a single service IP address. This means, for instance, that your GKE service can be in multiple clusters, with each cluster in a different region. If the instances closest to the user go down or are overloaded, traffic is seamlessly directed to another available instance. Global load balancing is performed with an algorithm using RPS (Requests Per Second) for VMs and containers. A utilization-based algorithm is supported only with instance groups (VMs).

Health checking at scale

Traffic Director provides GCP-delivered health checking at scale. This offloads health checking from Envoy or other open standard (xDS v2) service proxy to Google's resilient systems, allowing you to scale health checks for deployments of all sizes. In addition, your instances are not overwhelmed by mesh-sized health checks.

Request routing and rich traffic policies (Alpha)

Traffic Director supports advanced request routing features such as traffic splitting, enabling use cases such as canarying, URL rewrites and redirects, fault injection, traffic mirroring, and advanced routing capabilities based on various header values, including cookies. Traffic Director also supports many advanced traffic policies, with the inclusion of many load-balancing schemes, circuit breaking, and backend outlier detections. For access to the Alpha release, please sign up here.

Intelligent and rapid auto-scaling for your services

Traffic Director gives you demand-driven autoscaling, allowing you to pay only for what you use, while scaling up quickly and intelligently without having to contact your cloud provider and without any pre-warming requirements.

Traffic Director benefits

This section describes the benefits of using Traffic Director.

Fully-managed service with an SLA at general availability

As a Google-managed service, the general availability release of Traffic Director will have a production-grade 99.99% SLA: if there is a problem, our operators get paged, not yours. You don't have to deploy and manage the control plane, which means your staff members can focus on your business needs.

Sophisticated traffic management made simple

Use Traffic Director to easily deploy everything from simple load balancing, to advanced features such request routing and percentage-based traffic splitting.

Build resilient services

Keep your service up and running by deploying it across multiple regions as VMs or containers, and use Traffic Director to deliver global load balancing with automatic cross-region overflow and failover.

Scale your deployment seamlessly

Traffic Director is built to seamlessly handle the growth of your deployment. As the number of services grows, Traffic Director seamlessly scales to manage your entire deployment, even for large installations.

Modernize at your pace

Traffic Director works for both VM-based (Compute Engine) and containerized applications (Google Kubernetes Engine or self-managed Kubernetes) and can be incrementally introduced for your services.

Global load balancing with Traffic Director

Traffic Director delivers global load balancing for your internal microservices with service proxies. You can deploy internal microservices (service proxy-based) with instances in multiple regions. Traffic Director provides health, routing, and backend information to the service proxies, enabling them to perform optimal traffic routing to application instances in multiple cloud regions for a service.

In the following illustration, a global load balancing deployment features three global microservices: Front End, Shopping Cart, and Payments. Each service runs on managed instance groups in two regions, us-central1 and asia-southeast1. Traffic Director uses a global load balancing algorithm that directs traffic from the user in California to the microservices deployed in us-central1, while requests from the user in Singapore are directed to the microservices in asia-southeast1.

An incoming user request is routed to the Front End microservice. The service proxy installed on the host with the Front End then directs traffic to the Shopping Cart. The service proxy installed on the host with the Shopping Cart directs traffic to the Payments microservice.

Traffic Director in a global load balancing deployment (click to enlarge)
Traffic Director in a global load balancing deployment (click to enlarge)

In the following example, if Traffic Director receives health check results indicating that the VMs running the Shopping Cart microservice in us-central1 are unhealthy, Traffic Director instructs the service proxy for the Frontend microservices to fail over traffic to the Shopping Cart microservice running in asia-southeast1. Because autoscaling is integrated with traffic management in Google Cloud Platform, Traffic Director notifies the managed instance group in asia-southeast1 of the additional traffic, and the managed instance group increases in size.

Traffic Director detects that all backends of the Payments microservice are healthy, so Traffic Director instructs Envoy's proxy for the Shopping Cart to send a portion of the traffic, up to the customer's configured capacity, to asia-southeast1 and overflow the rest to us-central1.

Failover with Traffic Director in a global load balancing deployment (click to enlarge)
Failover with Traffic Director in a global load balancing deployment (click to enlarge)

Traffic Director and Istio

Istio provides a control plane to secure, connect, and monitor microservices. It has three components: Pilot for traffic management, Mixer for observability and Istio Security for service-to-service security.

Traffic Director delivers a GCP-managed Pilot along with additional capabilities such as global load balancing and centralized health checking. However, note that Traffic Director cannot be configured using Istio APIs in this Beta release; you can use GCP APIs for configuration. Both Traffic Director and Pilot use open standard APIs (xDS v2) to communicate with service proxies.

Pricing

Traffic Director is offered without charge for the Beta release.

Limits

All existing forward rule, backend service, and other load balancing limits and quotas per project apply to Traffic Director deployments.

Limitations

  • Traffic Director only supports GCP APIs for the Beta release. Traffic Director does not support Istio APIs for the Beta release.
  • Traffic Director is supported only with HTTP traffic.
  • Traffic Director traffic control capabilities are in Alpha. You can sign up for the Alpha release here.
  • Traffic Director supports Shared VPC either with all load balancing resources in the host project or all load balancing resources in the service project. However, only the service accounts of projects that have at least one forwarding rule configuration defined with the shared VPC network name can be used to access Traffic Director.
  • Traffic Director does not support VPC peering.
  • Traffic Director Beta supports load balancing for clients only within the GCP network, the name of which is specified in the forwarding rule.
  • You can only configure GCP endpoints with Traffic Director in the Beta release. We do not support endpoints on-prem or in another cloud.
  • This document discusses Envoy proxies, but you can use any open standard API (xDS v2) proxy with Traffic Director. However, note that Google has tested Traffic Director only with the Envoy proxy. During this Beta, Traffic Director only supports Envoy versions 1.9.1 or later.

What's next

Was this page helpful? Let us know how we did:

Send feedback about...

Traffic Director Documentation