Traffic management with route rules and traffic policies

Internal HTTP(S) Load Balancing is powered by the same technology as Traffic Director with fully managed Envoy so that you can use route rules and traffic policies to control traffic within your deployment. Internal HTTP(S) Load Balancing delivers advanced traffic control capabilities of Envoy using familiar GCP load balancing components: URL maps and backend services.

A route rule matches information in an incoming request and makes a routing decision based on the match. GCP looks for the first route rule whose matching rules match the request, after which GCP stops looking at any other rule. Then GCP applies the actions in the route rule. A route rule is an alternative to a path matcher and path rule.

After traffic is routed, traffic policies take further action to manage your traffic:

  • Load balancer settings control the load balancing algorithm.
  • Connection pool settings control the volume of connections to an upstream service.
  • Outlier detection controls the eviction of unhealthy hosts from the load balancing pool.

Use case examples

Route rules and traffic policies let you implement traffic splitting, traffic steering, fault injection, circuit breaking, and mirroring.

Traffic splitting allows a route rule to have multiple backend services, where each backend service receives a specified percentage of the requests matched by the route rule. For example, you can send 95% of traffic to one service instance and 5% to another service instance. Traffic splitting is typically used for deploying new versions, A/B testing, service migration, and similar processes.

Google Cloud Load Balancing traffic splitting (click to enlarge)
Google Cloud Load Balancing traffic splitting (click to enlarge)

Traffic steering directs traffic to service instances based on content in HTTP request headers. For example, if a user's device is an Android device with user-agent:Android in the request header, traffic steering sends that traffic to service instances designated to receive Android traffic, and sends traffic that doesn't have user-agent:Android to instances that handle other devices.

Google Cloud Load Balancing traffic steering (click to enlarge)
Google Cloud Load Balancing traffic steering (click to enlarge)

Fault injection enables you to test the resiliency of services by simulating service failure scenarios, such as delays and aborted requests. Delay injection configures the proxy to introduce delays for a fraction of requests before sending the request to the selected backend service. Abort injection configures the proxy to directly respond to a fraction of requests with failure response codes, rather than forwarding those requests to the backend service.

Circuit breaking enforces a limit on all requests to a particular service if a different service is unable to reach that service for a configurable number of consecutive attempts.

Mirroring allows you to send a copy of the requests matched by a route rule to a second backend service. The proxy doesn't wait for a response from the second backend service, and discards the responses it does receive from the second backend service. Mirroring is useful for testing a new version of a backend service. You can also use it to debug production errors on a debug version of your backend service, rather than on the production version.

About route rules

Route rules provide several ways to match HTTP requests and perform various actions such as traffic splitting across several backend services, responding with redirects and altering requests or responses. Route rules are executed in order, providing flexibility in tying actions to match rules in URL maps. Note that when the first match is made, Internal HTTP(S) Load Balancing stops evaluating the rules and any remaining rules are ignored.

  • Match rules enable Internal HTTP(S) Load Balancing to match one or more attributes of a request and take actions specified in the route rule. The following attributes of a request can be used to specify the matching criteria:

    • Host: A host name is the domain name portion of a URL; for example, the host name portion of the URL http://example.net/video/hd is example.net. In the request, the host name comes from the Host header, as shown in this example curl command:
    curl -v http://10.1.2.9/video/hd --header 'Host: example.com'
    

    where 10.1.2.9 is the load-balanced IP address.

    • Paths follow the hostname; for example /images. The rule can specify whether the entire path or only the leading portion of the path needs to match.

    • Other HTTP request parameters such as HTTP headers, which allows cookie matching, as well as matching based on query parameters (GET variables).

  • Route actions configure Internal HTTP(S) Load Balancing with specific actions to take when a route rule matches the attributes of a request. Use the following advanced route actions:

    • Redirects: Internal HTTP(S) Load Balancing returns a configurable 3xx response code. It also sets the Location response header with the appropriate URI, replacing the host and path as specified in the redirect action.

    • URL rewrites: Internal HTTP(S) Load Balancing can re-write the host name portion of the URL, the path portion of the URL, or both, before sending a request to the selected backend service.

    • Header transformations: Internal HTTP(S) Load Balancing can add or remove request headers before sending a request to the backend service. It can also add or remove response headers after receiving a response from the backend service.

    • Traffic mirroring: In addition to forwarding the request to the selected backend service, Internal HTTP(S) Load Balancing sends an identical request to the configured mirror backend service on a "fire and forget" basis. Mirroring is useful for testing a new version of a backend service. You can also use it to debug production errors on a debug version of your backend service, rather than on the production version.

    • Weighted traffic splitting is a configuration that allows traffic for a matched rule to be distributed to multiple backend services, proportional to a user-defined weight assigned to the individual backend service. This capability is useful for configuring staged deployments or A/B testing. For example, the route action could be configured such that 99% of the traffic is sent to a service that's running a stable version of an application, while 1% of traffic is sent to a separate service running a newer version of that application.

  • Retries configure the conditions under which the load balancer retries failed requests, how long the load balancer waits before retrying, and the maximum number of retries permitted.

  • Fault injection: Internal HTTP(S) Load Balancing can intentionally introduce errors when servicing requests to simulate failures, including high latency, service overload, service failures, and network partitioning. This feature is useful for testing the resiliency of a service to simulated faults.

  • Delay injection configures the proxy to introduce delays for a user-defined portion of requests before sending the request to the selected backend service.

  • Abort injection configures the proxy to respond directly to a fraction of requests with user-defined HTTP status codes instead of forwarding those requests to the backend service.

  • Security policies: Cross-origin resource sharing (CORS) policies handle Internal HTTP(S) Load Balancing settings for enforcing CORS requests.

About traffic policies

Traffic policies are groups of settings that define how load balancing behaves, including the response to failing backend services and how to prevent localized failures from affecting the rest of the service mesh. The following traffic policy features are configured in the backend service.

  • Load balancing policy: Internal HTTP(S) Load Balancing performs load balancing based on available capacity, and, as stated in the Envoy documentation, sets the locality level (per Google Cloud Platform zone) load balancing weights, for the proxy to choose a particular GCP zone for the backend VMs or endpoints in NEGs. The load balancing policies specified in the backend service further determine the algorithm used for backend VMs or endpoints in NEGs within the locality to be chosen after a weighted locality choice. Load balancing algorithms include round robin, ring hash, and backends VMs or endpoints in NEGs with the least request. Note that in Internal HTTP(S) Load Balancing, load balancing is performed using URL maps and backend services.

  • Session affinity: Internal HTTP(S) Load Balancing offers several session affinity options: HTTP cookie-based affinity, HTTP header-based affinity, client IP address affinity, and generated cookie affinity. Note that affinity settings are not honored if more than one weightedBackendServices is included in routeAction. For more information on session affinity, read Session affinity.

  • Outlier detection is a set of policies that specify the criteria for eviction of unhealthy backend VMs or endpoints in NEGs, along with criteria defining when a backend or endpoint is considered healthy enough to receive traffic again.

  • Circuit breaking sets upper limits on the volume of connections and requests per connection to a backend service.

Traffic control data model

Existing GCP resources are used to implement capabilities such as advanced route and traffic policies. The following diagram shows which resources are used to implement each feature:

Google Cloud Load Balancing traffic steering (click to enlarge)
Google Cloud Load Balancing data model (click to enlarge)

URL map resource

The URL map component is extended to support advanced routing features in the path matcher. You can match requests based on path prefix, full path, HTTP header, and query parameter. The prefix, and full path all apply to the path portion of the match. For more information, read the Envoy proxy RouteMatch documentation.

  • Path matcher: Internal HTTP(S) Load Balancing extends the URL map's path matcher concept. You can continue to use simple path rules as you would for a GCP HTTP(S) load balancer, or you can use route rules instead. Use the two types of rules exclusively. A single URL Map may contain only one or the other. Path rules are evaluated on the longest-path-matches-first basis and these rules can be specified in any order. Route rules are interpreted in order. This allows greater flexibility in defining complex route criteria.

    • Route rules (HttpRouteRule): Internal HTTP(S) Load Balancing evaluates route rules in the order in which they are specified. This allows you to define complex routing criteria. The following are the components of a route rule:

      • Route rule match (HttpRouteRuleMatch): Allows you to determine if the route rule applies to a request by matching all or a subset of the request's attributes, such as the path, HTTP headers, and query (GET) parameters.

        Within an HttpRouteRuleMatch, all matching criteria must be met for the rule's actions to take effect. If a rule has multiple HttpRouteRuleMatches, the actions of the rule take effect when a request matches any of the rule's HttpRouteRuleMatches.

      • Route action (HttpRouteAction): Allows you to specify what actions Internal HTTP(S) Load Balancing should take when the criteria within HttpRouteRuleMatch are met. These actions include traffic splitting, URL rewrites, retry and mirroring, fault injection, and CORS policies.

      • Redirect action (HttpRedirectAction): You can configure Internal HTTP(S) Load Balancing to respond with an HTTP redirect when the criteria within HttpRouteRuleMatch are met.

      • Header action (HttpHeaderAction): You can configure request and response header transformation rules when the criteria within HttpRouteRuleMatch are met.

    • Path rules (PathRule): Internal HTTP(S) Load Balancing supports path rule objects in path matchers to maintain compatibility with existing URL maps. You can enter path rules in any order. Internal HTTP(S) Load Balancing tries to match the request's path to each of the paths from all path rules within that path matcher, on the longest-path-matches-first basis. If a path match occurs, traffic is routed to the backend service of the corresponding path rule.

The URL map hierarchy for advanced route features is as follows:

  • Default backend service
  • Internal HTTP(S) Load Balancing only: default route action
  • Internal HTTP(S) Load Balancing only: default redirect
  • Internal HTTP(S) Load Balancing only: header action
  • List of host rules
  • List of path matchers, each path matcher containing
    • Internal HTTP(S) Load Balancing only: default route action
    • Internal HTTP(S) Load Balancing only: default redirect
    • Internal HTTP(S) Load Balancing only: header action
    • Default backend service for path matcher
    • List of path rules
    • Internal HTTP(S) Load Balancing only: list of route rules
      • Internal HTTP(S) Load Balancing only: list of match rules
      • Internal HTTP(S) Load Balancing only: backend service
      • Internal HTTP(S) Load Balancing only: route action
      • Internal HTTP(S) Load Balancing only: redirect
      • Internal HTTP(S) Load Balancing only: header action

Backend service resource

Internal HTTP(S) Load Balancing uses a backend service with a load balancing scheme of INTERNAL_MANAGED. A backend service with this scheme supports the settings that implement the bulk of the traffic policies. The following attributes can be specified for an internal managed backend service:

  • Load balancing policy (LocalityLoadBalancingPolicy): For an internal managed backend service, traffic distribution uses a load balancing mode and a load balancing policy. The backend service first directs traffic to a backend group (instance group, managed instance group, or network endpoint group) according to the balancing mode. Then, after a backend group is selected, the load balancer distributes traffic according to the load balancing policy. The balancing mode allows Internal HTTP(S) Load Balancing to first select a locality, such as a GCP zone; then, the load balancing policy is used to select a specific backend VM or endpoint in a NEG.

  • Session affinity (SessionAffinity): Internal managed backend services support four session affinities: client IP address, HTTP cookie-based, HTTP header-based, and generated cookie affinity.

  • Consistent hash (ConsistentHashLoadBalancerSettings) define criteria for building consistent hashes from cookies and headers for Internal HTTP(S) Load Balancing to consistently route new requests to the same backend VMs or endpoints in NEGs.

  • Circuit breakers (CircuitBreakers) define parameters for limiting the volume of traffic to any particular backend VM or endpoint of the backend service. This prevents overloading services with requests that they cannot handle in a meaningful way.

  • Outlier detection (OutlierDetection) defines criteria for determining when a backend VM or endpoint in a NEG is deemed unhealthy and excluded from load balancing considerations, as well as the conditions that must be satisfied to reconsider the backend VM or endpoints for load balancing. A backend VM or endpoint is deemed unhealthy when the health check linked to the backend service marks the backend VM or endpoint in a NEG as unhealthy.

Configuring traffic control

The following sections provide information on setting up traffic control.

Before you configure traffic control

Follow the instructions in Preparing for Internal HTTP(S) Load Balancing setup, and configure any VM hosts or GKE clusters you need. Create the required GCP resources.

Setting up traffic splitting

This example demonstrates the following steps:

  1. Create distinct templates for different services.

  2. Create instance groups for those templates.

  3. Create routing rules that set up 95% / 5% canarying.

  4. Send curl commands showing that the traffic split percentages roughly match the configuration.

Defining the services

The following bash function creates a backend service, including the instance template and the managed instance group.

These instructions assume that an HTTP health check (l7-ilb-basic-check) has been created. Refer to the instructions in Setting Up Internal HTTP(S) Load Balancing for Compute Engine VMs.

function make_service() {
  local name="$1"
  local region="$2"
  local zone="$3"
  local network="$4"
  local subnet="$5"
  local subdir="$6"

  www_dir="/var/www/html/$subdir"

  (set -x; \
  gcloud compute instance-templates create "${name}-template" \
    --region="$region" \
    --network="$network" \
    --subnet="$subnet" \
    --tags=l7-ilb-tag,http-server \
    --image-family=debian-9 \
    --image-project=debian-cloud \
    --metadata=startup-script="#! /bin/bash
  sudo apt-get update
  sudo apt-get install apache2 -y
  sudo mkdir -p $www_dir
  /bin/hostname | sudo tee $www_dir/index.html
  sudo service apache2 restart
  "; \
  gcloud beta compute instance-groups managed create \
    "${name}-instance-group" \
    --zone="$zone" \
    --size=2 \
    --template="${name}-template"; \
  gcloud beta compute backend-services create "${name}-service" \
    --load-balancing-scheme=INTERNAL_MANAGED \
    --protocol=HTTP \
    --health-checks=l7-ilb-basic-check \
    --health-checks-region="$region" \
    --region="$region"; \
  gcloud beta compute backend-services add-backend "${name}-service" \
    --balancing-mode='UTILIZATION' \
    --instance-group="${name}-instance-group" \
    --instance-group-zone="$zone" \
    --region="$region")
}

Creating the services

Now call the function to make three services, red, green, and blue. The red service acts as the default service for requests to /. The green and blue services are both set up on /prefix to handle 95% and 5% of the traffic, respectively.

make_service red us-west1 us-west1-a lb-network backend-subnet ""
make_service green us-west1 us-west1-a lb-network backend-subnet prefix
make_service blue us-west1 us-west1-a lb-network backend-subnet prefix

Creating the URL map file

Write a URL map file canary.yaml with the desired canary setup.

These instructions assume the following:

  • The region is us-west1.
  • Your Internal HTTP(S) Load Balancing deployment has a URL map called l7-ilb-map.
  • A target proxy and forwarding rule have been created.
  • The forwarding rule has address 10.1.2.9.

    See instructions in Setting Up Internal HTTP(S) Load Balancing for Compute Engine VMs.

  • The URL map sends all traffic to one backend service, called red-service, which is the default backend service.

  • You set up an alternate path that sends 5% of the traffic to blue-service and 95% of traffic to green-service.

  • A path matcher is used.

  • You are using Cloud Shell or another environment with bash installed.

You must replace project-id with your GCP project ID.

defaultService: https://www.googleapis.com/compute/beta/projects/project-id/regions/us-west1/backendServices/red-service
hostRules:
- description: ''
  hosts:
  - '*'
  pathMatcher: matcher1
kind: compute#urlMap
name: l7-ilb-map
pathMatchers:
- defaultService: https://www.googleapis.com/compute/beta/projects/project-id/regions/us-west1/backendServices/red-service
  name: matcher1
  routeRules:
  - matchRules:
    - prefixMatch: /prefix
    routeAction:
      weightedBackendServices:
      - backendService: https://www.googleapis.com/compute/beta/projects/project-id/regions/us-west1/backendServices/green-service
        weight: 95
      - backendService: https://www.googleapis.com/compute/beta/projects/project-id/regions/us-west1/backendServices/blue-service
        weight: 5
region: https://www.googleapis.com/compute/beta/projects/project-id/regions/us-west1
selfLink: https://www.googleapis.com/compute/beta/projects/project-id/regions/us-west1/urlMaps/l7-ilb-map

Importing the URL map

gcloud beta compute url-maps import l7-ilb-map \
    --region=us-west1 \
    --source=canary.yaml

Testing the configuration

To test the canary configuration, first ensure that requests to 10.1.2.9/ (the load balancer's IP address set up earlier) are handled by the default red configuration.

Then check to make sure that requests sent to 10.1.2.9/prefix are split as expected.

Creating a client VM

Refer to Creating a VM instance in the zone to test connectivity.

Sending requests to 10.1.2.9

SSH into the client and run the following command:

for LB_IP in 10.1.2.9; do
    RESULTS=
    for i in {1..1000}; do RESULTS="$RESULTS:`curl ${LB_IP}`"; done >/dev/null 2>&1
    IFS=':'
    echo "***"
    echo "*** Results of load balancing to $LB_IP: "
    echo "***"
    for line in $RESULTS; do echo $line; done | grep -Ev "^$" | sort | uniq -c
    echo
done
Checking the results
***
***Results of load balancing to 10.1.2.9:
***
502 red-instance-group-9jvq
498 red-instance-group-sww8

Sending requests to 10.1.2.9/prefix

Send requests to 10.1.2.9/prefix and note the traffic splitting.

for LB_IP in 10.1.2.9; do
    RESULTS=
    for i in {1..1000}; do RESULTS="$RESULTS:`curl ${LB_IP}/prefix/index.html`"; done >/dev/null 2>&1
    IFS=':'
    echo "***"
    echo "*** Results of load balancing to $LB_IP/prefix: "
    echo "***"
    for line in $RESULTS; do echo $line; done | grep -Ev "^$" | sort | uniq -c
    echo
done
Checking the results
***
***Results of load balancing to 10.1.2.9/prefix:
***
21 blue-instance-group-8n49
27 blue-instance-group-vlqc
476 green-instance-group-c0wv
476 green-instance-group-rmf4

The canary setup successfully sends 95% of /prefix requests to service green and 5% to service blue.

Traffic control enables you to configure session affinity based on a provided cookie. To configure HTTP_COOKIE based session affinity for a backend service named red-service, follow these directions.

To set up session affinity using HTTP_COOKIE:

  1. Use the gcloud export command to get the backend service configuration.

    gcloud beta compute backend-services export red-service \
        --destination=red-service-config.yaml \
        --region=us-west1
    
  2. Update the red-service-config.yaml file as follows:

    sessionAffinity: 'HTTP_COOKIE'
    localityLbPolicy: 'RING_HASH'
    consistentHash:
     httpCookie:
      name: 'http_cookie'
      path: '/cookie_path'
      ttl:
        seconds: 100
        nanos: 30
     minimumRingSize: 10000
    
  3. In the red-service-config.yaml file, delete the line that says:

    sessionAffinity: NONE
    
  4. Update the backend service configuration file:

    gcloud beta compute backend-services import red-service \
        --source=red-service-config.yaml \
        --region=us-west1
    

Troubleshooting

Use this information for troubleshooting when traffic is not being routed according to the route rules and traffic policies that you configured.

For information about logging and monitoring, see Internal HTTP(S) logging and monitoring.

Symptoms:

  • Increased traffic to services in rules above the rule in question.
  • An unexpected increase in 4xx and 5xx HTTP responses for a given route rule.

Solution: Check the order of your route rules. Route rules are interpreted in the order in which they are specified.

Route rules within a URL map are interpreted in the order in which they are specified. This is different from the way that path rules are interpreted by longest prefix match. For a path rule, Internal HTTP(S) Load Balancing will only select a single path rule; however, when you use route rules, more than one might apply.

When you define route rules, check to be sure that rules at the top of the list do not inadvertently route traffic that would otherwise have been routed by a subsequent route rule. The service that receives misdirected traffic would likely reject requests, and the service in your route rules would receive reduced traffic or no traffic at all.

Limitations

  • UrlMap.defaultRouteAction and UrlMap.defaultUrlRedirect do not currently work as intended. You must specify UrlMap.defaultService for handling traffic that does not match any of the hosts in UrlMap.hostRules[] in that UrlMap. Similarly, UrlMap.pathMatchers[].defaultRouteAction and UrlMap.pathMatchers[].defaultUrlRedirect do not currently work. You must specify UrlMap.pathMatchers[].defaultService for handling traffic that does not match any of the routeRules for that pathMatcher.
  • RouteRule.service does not currently work as intended. The workaround is to use RouteRule.weightedBackendServices with a single WeightedBackendService.
  • The gcloud import command doesn't delete top-level fields of the resource, such as the backend service and the URL map. For example, if a backend service is created with settings for circuitBreakers, those settings can be updated via a subsequent gcloud import command. However, those settings cannot be deleted from the backend service. The resource itself can be deleted and recreated without the circuitBreakers settings.
  • The gcloud export command exports int64 fields as strings, not as integers, in the exported file. You must manually remove the quotation marks around the field to import the exported file.
¿Te sirvió esta página? Envíanos tu opinión:

Enviar comentarios sobre…