Traffic management with route rules and traffic policies

Traffic Director includes route rules and traffic policies that enable you to control traffic within your Traffic Director deployment. These capabilities are based on xDS API-compatible sidecar proxies, such as Envoy and enabled using the load balancing components URL maps, forwarding rules, and backend services.

A route rule matches information you specify to an incoming request and makes a routing decision based on the match. Route rules are executed in order.

After traffic is routed, traffic policies take further action to manage your traffic. Traffic policies generally belong to these categories:

  • Load balancer settings, which control the load balancing algorithm
  • Connection pool settings, which control the volume of connections to an upstream service
  • Outlier detection, which controls the eviction of unhealthy hosts from the load balancing pool

Use case examples

Route rules and traffic policies let you implement traffic splitting, traffic steering, fault injection, circuit breaking, and mirroring.

Traffic splitting splits traffic based on weight. For example, you can send 99% of traffic to one service instance and 1% to another service instance. Traffic splitting is typically used for deploying new versions, A/B testing, service migration, and similar processes.

Traffic Director traffic splitting (click to enlarge)
Traffic Director traffic splitting (click to enlarge)

Traffic steering directs traffic to service instances based on content in HTTP request headers. For example, if a user's device is an Android device, with user-agent:Android in the request header, that traffic is sent to service instances designated to receive Android traffic, and traffic that does not have user-agent:Android is sent to instances that handle other devices.

Traffic Director traffic steering (click to enlarge)
Traffic Director traffic steering (click to enlarge)

Fault injection enables you to test the resiliency of services by simulating service failure scenarios, such as delays and aborted requests. Delay injection configures the client side proxy to introduce delays for a fraction of requests before sending the request to the selected backend service. Abort injection configures the sidecar proxy to directly respond to a fraction of requests with failure response codes, rather than forwarding those requests to the backend service.

Circuit breaking enforces limits on requests to a particular service, such as the maximum number of connections or maximum number requests after which requests are prevented from reaching the service to protect the service from degrading further.

Mirroring is implemented by deploying a debug application that receives a copy of your real traffic. The production application processes the original traffic, while the debug application discards responses. You can use mirroring to test binaries with production traffic and to debug productions errors on the debug application rather than on your production application.

About route rules

Route rules provide several ways to match HTTP requests and perform various actions such as traffic splitting across several backend services, responding with redirects and altering requests or responses. Route rules are configured using the URL map.

Route rules are executed in order, providing flexibility in tying actions to match rules. Note that when the first match is made, Traffic Director stops evaluating the rules and any remaining rules are ignored.

  • Match rules enable Traffic Director to match one or more attributes of a request and take actions specified in the route rule. The following attributes of a request can be used to specify the matching criteria:
    • Host: A host name is the domain name portion of a URL; for example, the host name portion of the URL http://example.net/video/hd is example.net.
    • Paths are the part of the URL that follows the hostname; for example /images in http://service-host-name/images. The rule can specify whether the entire path or only the leading portion of the path must match. The rule can alternatively supply a regular expression that must match the path.
    • Other HTTP request parameters such as HTTP headers, which allows cookie matching, as well as matching based on query parameters (GET variables).
    • Metadata filters: These are opaque match criteria for proxies conforming to Envoy's xDS protocol. For example, sidecar proxies making xDS requests may supply node metadata such as application version, staged deployment version, and other criteria. Traffic Director supplies route configuration only to those sidecar proxies when settings in metadata filters match corresponding settings in the node metadata supplied by the proxy.
  • Route actions configure Traffic Director with specific actions to take when a route rule matches the attributes of a request. Use the following advanced route actions:
    • Redirects: Traffic Director returns a configurable 3xx response code. It also sets the Location response header with the appropriate URI, replacing the host and path as specified in the redirect action.
    • URL rewrites: Traffic Director can re-write the host name portion of the URL, the path portion of the URL, or both, before sending a request to the selected backend service.
    • Header transformations: Traffic Director can add or remove request headers before sending a request to the backend service. It can also add or remove response headers after receiving a response from the backend service. You configure header transformations at various levels of the URL map, including its top level, at a path matcher, in a route rule, and in a weighted backend service. This allows you to specify hierarchical control of headers.
    • Traffic mirroring: In addition to forwarding the request to the selected backend service, Traffic Director sends an identical request to the configured mirror backend service on a "fire and forget" basis. This capability is useful for logging requests.
    • Weighted traffic splitting is a configuration that allows traffic for a matched rule to be distributed to multiple backend services, proportional to a user-defined weight assigned to the individual backend service. This capability is useful for configuring staged deployments or A/B testing. For example, the route action could be configured such that 99% of the traffic is sent to a service that's running a stable version of an application, while 1% of traffic is sent to a separate service running a newer version of that application.
    • Retries configure the conditions under which Traffic Director retries failed requests, how long it waits before retrying and the maximum number of retries permitted.
  • Fault injection: Traffic Director can intentionally introduce errors when servicing requests to simulate failures, including high latency, service overload, service failures, and network partitioning. This feature is useful for testing the resiliency of a service to simulated faults.
    • Delay injection configures the proxy to introduce delays for a user-defined portion of requests before sending the request to the selected backend service.
    • Abort injection configures the proxy to respond directly to a fraction of requests with user-defined HTTP status codes instead of forwarding those requests to the backend service.
  • Security policies: Cross-origin resource sharing (CORS) policies handle Traffic Director settings for enforcing CORS requests.

About traffic policies

Traffic policies are groups of settings that define how load balancing behaves, including the response to failing backend services and how to prevent localized failures from affecting the rest of the service mesh. The following traffic policy features are configured in the backend service.

  • Load balancing policy: Traffic Director performs global load balancing based on available capacity, and, as stated in the Envoy documentation, sets the locality level (per Google Cloud Platform zone) load balancing weights, for the proxy to choose a particular GCP zone for the backend VMs or endpoints in NEGs. The load balancing policies specified in the backend service further determine the algorithm used for backend VMs or endpoints in NEGs within the locality to be chosen after a weighted locality choice. Load balancing algorithms include round robin, ring hash, and backends VMs or endpoints in NEGs with the least request. Note that in Traffic Director, global load balancing is performed using URL maps, forwarding rules, and backend services.
  • Session affinity: Traffic Director offers several session affinity options: HTTP cookie-based affinity, HTTP header-based affinity, client IP address affinity, and generated cookie affinity. Note that affinity settings are not honored if more than one weightedBackendServices is included in routeAction. For more information on session affinity, read Session affinity.
  • Outlier detection is a set of policies that specify the criteria for eviction of unhealthy backend VMs or endpoints in NEGs, along with criteria defining when a backend or endpoint is considered healthy enough to receive traffic again.
  • Circuit breaking sets upper limits on the volume of connections and requests per connection to a backend service.

Traffic control data model

Three existing GCP resources are used to implement capabilities such as advanced route and traffic policies. The following diagram shows which resources are used to implement each feature:

Traffic Director traffic steering (click to enlarge)
Traffic Director data model (click to enlarge)

Forwarding rule resource

Traffic Director uses a forwarding rule whose load balancing scheme is INTERNAL_SELF_MANAGED. The forwarding rule intercepts traffic on a configurable RFC 1918 IP address, then sends it to a target HTTP proxy and URL map. The forwarding rule resource supports traffic control with selective configuration of proxies. Traffic Director supplies route configuration to only those proxies that satisfy criteria specified in the forwarding rule's Metadata filters.

  • MetadataFilters specify the criteria for which an xDS compliant proxy gets the configuration attached to this forwarding rule.

URL map resource

The URL map component is extended to support advanced routing features in the path matcher. You can match requests based on path prefix, full path, path regex, HTTP header, and query parameter. The prefix, full path, and regex all apply to the path portion of the match. For more information, read the Envoy proxy RouteMatch documentation.

  • Path matcher: Traffic Director extends the URL map's path matcher concept. You can continue to use simple path rules as you would for a GCP HTTP(S) load balancer, or you can use route rules instead. Use the two types of rules exclusively. Path rules are evaluated on the longest-path-matches-first basis and these rules can be specified in any order. Route rules are interpreted in order. This allows greater flexibility in defining complex route criteria.
    • Route rules (HttpRouteRule): Traffic Director evaluates route rules in the order in which they are specified. This allows you to define complex routing criteria. The following are the components of a route rule:
      • Route rule match (HttpRouteRuleMatch): Allows you to determine if the route rule applies to a request by matching all or a subset of the request's attributes such as the path, HTTP headers, and query (GET) parameters. Within an HttpRouteRuleMatch, all matching criteria must be met for the rule's actions to take effect. If a rule has multiple HttpRouteRuleMatches, the actions of the rule take effect when a request matches any of the rule's HttpRouteRuleMatches.
      • Route action (HttpRouteAction): Allows you to specify what actions Traffic Director should take when the criteria within HttpRouteRuleMatch are met. These actions include traffic splitting, URL rewrites, retry and mirroring, fault injection, and CORS policies.
      • Redirect action (HttpRedirectAction): You can configure Traffic Director to respond with an HTTP redirect when the criteria within HttpRouteRuleMatch are met.
      • Header action (HttpHeaderAction): You can configure request and response header transformation rules when the criteria within HttpRouteRuleMatch are met.
      • Metadata filters (MetadataFilters): You can specify criteria for which xDS compliant proxies get routing configuration tied to a route rule.
    • Path rules (PathRule): Traffic Director supports path rule objects in path matchers to maintain compatibility with existing URL maps. You can enter path rules in any order. Traffic Director tries to match the request's path to each of the paths from all path rules within that path matcher, on the longest-path-matches-first basis. If a path match occurs, traffic is routed to the backend service of the corresponding path rule.

The URL map hierarchy for advanced route features is as follows:

  • Default backend service or default backend bucket
  • Traffic Director only: default route action
  • Traffic Director only: default redirect
  • List of host rules
  • List of path matchers, each path matcher containing
    • Traffic Director only: default route action
    • Traffic Director only: default redirect
    • Default backend service or default backend bucket for path matcher
    • List of path rules
    • Traffic Director only: list of route rules
      • Traffic Director only: list of match rules
      • Traffic Director only: route action
      • Traffic Director only: redirect
      • Traffic Director only: header action

Backend service resource

Traffic Director uses a backend service whose load balancing scheme is INTERNAL_SELF_MANAGED. A backend service with this scheme supports the settings that implement the bulk of the traffic policies. The following attributes can be specified for an internal self-managed backend service:

  • Load balancing policy (LocalityLoadBalancingPolicy): For an internal self-managed backend service, traffic distribution is accomplished by using a combination of a load balancing mode and a load balancing policy. The backend service first directs traffic to a backend (instance group or NEG) according to the backend's balancing mode, then, once a backend has been selected, Traffic Director distributes traffic according to the load balancing policy. The balancing mode allows Traffic Director to first select a locality, such as a GCP zone; then, the load balancing policy is used to select a specific backend VM or endpoint in a NEG.
  • Session affinity (SessionAffinity): Internal self-managed backend services support four session affinities: client IP address, HTTP cookie-based, HTTP header-based, and generated cookie (generated by Traffic Director itself) affinity.
  • Consistent hash (ConsistentHashLoadBalancerSettings) define criteria for building consistent hashes from cookies and headers for Traffic Director to consistently route new requests to the same backend VMs or endpoints in NEGs.
  • Circuit breakers (CircuitBreakers) define parameters for limiting the volume of traffic to any particular backend VM or endpoint of the backend service. This prevents overloading services with requests that they cannot handle in a meaningful way.
  • Outlier detection (OutlierDetection) defines criteria for determining when a backend VM or endpoint in a NEG is deemed unhealthy and excluded from load balancing considerations, as well as the conditions that must be satisfied to reconsider the backend VM or endpoints for load balancing. A backend VM or endpoint is deemed unhealthy when the health check linked to the backend service marks the backend VM or endpoint in a NEG as unhealthy.

Configuring traffic control

The following sections provide information on setting up traffic control.

Before you configure traffic control

Follow the instructions in Setting Up Traffic Director, including configuring Traffic Director and any VM hosts or GKE clusters you need. Create the required GCP resources.

Setting up traffic splitting

These instructions assume the following:

  • Your Traffic Director deployment has a URL map called review-url-map.
  • The URL map sends all traffic to one backend service, called review1, which serves as the default backend service as well.
  • You plan to route 5% of traffic to a new version of a service. That service is running on a backend VM or endpoint in a NEG associated with the backend service review2.
  • No host rules or path matchers are used.

To set up traffic splitting, follow these steps:

  1. Use the gcloud export command to get the URL map configuration:

     gcloud beta compute url-maps export review-url-map \
         --destination=review-url-map-config.yaml --global
    
  2. Add the following section to the review-url-map-config.yaml file:

    
         hostRules:
         - description: ''
           hosts:
           - '*'
          pathMatcher: matcher1
         pathMatchers:
         - defaultService: $[SERVICE_URL1]
           name: matcher1
           routeRules:
           - matchRules:
             - prefixMatch: ''
             routeAction:
              weightedBackendServices:
              - backendService: $[SERVICE_URL1]
                weight: 95
              - backendService: $[SERVICE_URL2]
                weight: 5
    
    
  3. Update the URL map:

     gcloud beta compute url-maps import review-url-map
         --source=review-url-map-config.yaml --global
    

After you are satisfied with the new version, you can gradually adjust the weights of the two services and eventually send all traffic to review2.

Traffic control enables you to configure session affinity based on a provided cookie. To configure HTTP_COOKIE based session affinity for a backend service named service1, follow these directions.

To set up session affinity using HTTP_COOKIE:

  1. Use the gcloud export command to get the backend service configuration.

     gcloud beta compute backend-services export service1 --destination=service1-config.yaml --global
    
  2. Update the service1-config.yaml file as follows:

     sessionAffinity: 'HTTP_COOKIE'
     localityLbPolicy: 'RING_HASH'
     consistentHash:
      httpCookie:
       name: 'http_cookie'
       path: '/cookie_path'
       ttl:
         seconds: 100
         nanos: 30
      minimumRingSize: 10000
    
  3. Update the backend service config file:

     gcloud beta compute backend-services import service1 --source=service1-config.yaml --global
    

Setting up config filtering based on MetadataFilter match

MetadataFilters are enabled with forwarding rules and HttpRouteRuleMatch. Use this feature to control a particular forwarding rule or route rule so that the control plane sends the forwarding rule or route rule only to proxies whose node metadata matches the metadatafilter setting. If you do not specify any MetadataFilters, the rule is sent to all Envoy proxies.

This feature makes it easy to operate a staged deployment of a configuration. For example, create a forwarding rule named forwarding-rule1, which you want to be pushed only to Envoys whose node metadata contains 'app: review' and 'version: canary'. Follow the steps below to add MetadataFilters to the forwarding rule.

To add MetadataFilter to a forwarding rule, follow these steps:

  1. Use the gcloud export command to get the forwarding rule config.

     gcloud beta compute forwarding-rules export forwarding-rule1 --destination=forwarding-rule1-config.yaml --global
    
  2. Update the forwarding-rule1-config.yaml file.

    
         metadataFilters:
         - filterMatchCriteria: 'MATCH_ALL'
           filterLabels:
           - name: 'app'
             value: 'review'
           - name: 'version'
             value: 'canary'
    
    
  3. Use the gcloud import command to update the forwarding-rule1-config.yaml file.

    gcloud beta compute forwarding-rules import forwarding-rule1 --source=forwarding-rule1-config.yaml --global
    
  4. Use these instructions to add node metadata to Envoy before starting Envoy. Note that only string value is supported.

    a. For a VM-based deployment, in bootstrap_template.yaml, add the following under metadata section:

        app: "review"
        version: "canary"
    

    b. For Google Kubernetes Engine-based or Kubernetes-based deployment, in trafficdirector_istio_sidecar.yaml, add the following under the env section:

        - name: ISTIO_META_app
          value: "review"
        - name: ISTIO_META_version
          value: "canary"
     

Troubleshooting

Use this information for troubleshooting when traffic is not being routed according to the route rules and traffic policies that you configured.

Symptoms:

  • Increased traffic to services in rules above the rule in question.
  • An unexpected increase in 4xx and 5xx HTTP responses for a given route rule.

Solution: Check the order of your route rules. Route rules are interpreted in the order in which they are specified.

Route rules within a URL map are interpreted in the order in which they are specified. This is different from the way that path rules are interpreted by longest prefix match. For a path rule, Traffic Director will only select a single path rule; however, when you use route rules, more than one might apply.

When you define route rules, check to be sure that rules at the top of the list do not inadvertently route traffic that would otherwise have been routed by a subsequent route rule. Consider the following example:

First route rule: route rule match pathPrefix = '/shopping/'; redirect action: send traffic to backend service service-1 Second route rule: route rule match regexMatch = '/shopping/cart/ordering/.*'; redirect action: send traffic to backend service service-2

In this case, a request with the path '/shopping/cart/ordering/cart.html' is routed to service-1. Even though the second rule would have matched, it is ignored because the first rule takes precedence.

Limitations

  • UrlMap.defaultRouteAction and UrlMap.defaultUrlRedirect do not currently work as intended. You must specify UrlMap.defaultService for handling traffic that does not match any of the hosts in UrlMap.hostRules[] in that UrlMap. Similarly, UrlMap.pathMatchers[].defaultRouteAction and UrlMap.pathMatchers[].defaultUrlRedirect do not currently work. You must specify UrlMap.pathMatchers[].defaultService for handling traffic that does not match any of the routeRules for that pathMatcher.
  • RouteRule.service does not currently work as intended. The workaround is to use RouteRule.weightedBackendServices with a single WeightedBackendService.
  • The gcloud import command doesn't delete top-level fields of the resource, such as the backend service and the URL map. For example, if a backend service is created with settings for circuitBreakers, those settings can be updated via a subsequent gcloud import command. However, those settings cannot be deleted from the backend service. The resource itself can be deleted and recreated without the circuitBreakers settings. Similarly, for forwarding rules, metadataFilters can't be added to an exported yaml file then and re-imported. The workaround is to export the config file, add the metadataFilters, delete the forwarding rule, and import the config file.
  • The gcloud export command exports int64 fields as strings, not as integers, in the exported file. You must manually remove the quotation marks around the field to import the exported file.
Was this page helpful? Let us know how we did:

Send feedback about...

Traffic Director Documentation