Setting up traffic management for internal HTTP(S) load balancers

This guide contains examples for setting up traffic management for Internal HTTP(S) Load Balancing. This document shows examples of using traffic management for some specific use cases. Many other use cases are possible.

Before you begin

Configuring traffic management

You can use the Cloud Console, gcloud, or the Cloud Load Balancing API to configure traffic management. Within your chosen configuration environment, you set up traffic management by using YAML configurations. A URL map and a backend service each has its own YAML file. Depending on your desired functionality, you need to write either a URL map YAML, a backend service YAML, or both.

For help writing these YAML files, you can use the following resources:

Accessing the YAML examples in the Cloud Console

To access YAML examples in the Cloud Console:

  1. Go to the Load balancing page in the Google Cloud Console.
    Go to the Load balancing page
  2. Under HTTP(S) Load Balancing, click Start configuration.
  3. Select Only between my VMs. This setting means that the load balancer is internal.
  4. Click Continue.
  5. In the Routing rules configuration, select Advanced host, path and route rule.
  6. Click Add hosts and path matcher.
  7. Click the Code guidance link.

The Path matcher YAML examples page appears.

URL map YAML examples

One allowed in a URL map

Single service

Send all traffic to a single service. Make sure to replace the variables marked by []:

defaultService: regions/[region]/backendServices/[backend-service1]
hostRules:
- hosts:
  - '*'
  pathMatcher: matcher1
name: [url-map-name]
pathMatchers:
- defaultService: regions/[region]/backendServices/[backend-service1]
  name: matcher1
  routeRules:
    - matchRules:
        - prefixMatch: /
      priority: [priority]
      routeAction:
        weightedBackendServices:
          - backendService: regions/[region]/backendServices/[backend-service1]
            weight: 100

Split traffic

Split traffic between multiple services. Make sure to replace the variables marked by []:

defaultService: regions/[region]/backendServices/[backend-service1]
hostRules:
- hosts:
  - '*'
  pathMatcher: matcher1
name: [url-map-name]
pathMatchers:
- defaultService: regions/[region]/backendServices/[backend-service1]
  name: matcher1
  routeRules:
  - matchRules:
    - prefixMatch: /prefix
    priority: 2
    routeAction:
      weightedBackendServices:
      - backendService: regions/[region]/backendServices/[backend-service1]
        weight: 95
      - backendService: regions/[region]/backendServices/[backend-service2]
        weight: 5

URL redirect

Returns a configurable 3xx response code. Also sets the Location response header with the appropriate URI, replacing the host and path as specified in the redirect action. Make sure to replace the variables marked by []:

defaultService: regions/[region]/backendServices/[backend-service1]
kind: compute#urlMap
name: l7-ilb-map
region: region/[region]
hostRules:
- hosts:
  - '*'
  pathMatcher: matcher1
pathMatchers:
- defaultService: regions/[region]/backendServices/[backend-service1]
  name: matcher1
  routeRules:
    - matchRules:
        - prefixMatch: /
      priority: [priority]
      urlRedirect:
        hostRedirect: [REDIRECT_HOST]
        pathRedirect: [REDIRECT_PATH]
        redirectResponseCode: FOUND
        stripQuery: True

Multiple allowed in a URL map

Mirror traffic

In addition to forwarding the request to the selected backend service, send an identical request to the configured mirror backend service on a "fire and forget" basis. The load balancer doesn't wait for a response from the backend to which it sends the mirrored request. Mirroring is useful for testing a new version of a backend service. You can also use it to debug production errors on a debug version of your backend service, rather than on the production version. Make sure to replace the variables marked by []:

defaultService: regions/[region]/backendServices/[backend-service1]
kind: compute#urlMap
name: l7-ilb-map
region: region/[region]
hostRules:
- hosts:
  - '*'
  pathMatcher: matcher1
pathMatchers:
- defaultService: regions/[region]/backendServices/[backend-service1]
  name: matcher1
  routeRules:
    - matchRules:
        - prefixMatch: /
      priority: [priority]
      routeAction:
        weightedBackendServices:
          - backendService: regions/[region]/backendServices/[backend-service1]
            weight: 100
        requestMirrorPolicy:
          backendService: regions/[region]/backendServices/[backend-service2]

Rewrite URL

Rewrite the host name portion of the URL, the path portion of the URL, or both, before sending a request to the selected backend service. Make sure to replace the variables marked by []:

defaultService: regions/[region]/backendServices/[backend-service1]
kind: compute#urlMap
name: l7-ilb-map
region: region/[region]
hostRules:
- hosts:
  - '*'
  pathMatcher: matcher1
pathMatchers:
- defaultService: regions/[region]/backendServices/[backend-service1]
  name: matcher1
  routeRules:
    - matchRules:
        - prefixMatch: / 
      priority: [priority]
      routeAction:
        weightedBackendServices:
          - backendService: regions/[region]/backendServices/[backend-service1]
            weight: 100
        urlRewrite:
          hostRewrite: [REWRITE_HOST]
          pathPrefixRewrite: [REWRITE_PATH]

Retry request

Configure the conditions under which the load balancer retries failed requests, how long the load balancer waits before retrying, and the maximum number of retries permitted. Make sure to replace the variables marked by []:

defaultService: regions/[region]/backendServices/[backend-service1]
kind: compute#urlMap
name: l7-ilb-map
region: region/[region]
hostRules:
- hosts:
  - '*'
  pathMatcher: matcher1
pathMatchers:
- defaultService: regions/[region]/backendServices/[backend-service1]
  name: matcher1
  routeRules:
    - matchRules:
        - prefixMatch: /
      priority: [priority]
      routeAction:
        weightedBackendServices:
          - backendService: regions/[region]/backendServices/[backend-service1]
            weight: 100
        retryPolicy:
          retryConditions: 502, 504
          numRetries: 3
          perTryTimeout:
            seconds: 1
            nanos: 50

Timeout

Specify the timeout for the selected route. Timeout is computed from the time the request is fully processed up until the response fully processed. Timeout includes all retries. Make sure to replace the variables marked by []:

defaultService: regions/[region]/backendServices/[backend-service1]
kind: compute#urlMap
name: l7-ilb-map
region: region/[region]
hostRules:
- hosts:
  - '*'
  pathMatcher: matcher1
pathMatchers:
- defaultService: regions/[region]/backendServices/[backend-service1]
  name: matcher1
  routeRules:
    - matchRules:
        - prefixMatch: /
      priority: [priority]
      routeAction:
        weightedBackendServices:
          - backendService: regions/[region]/backendServices/[backend-service1]
            weight: 100
        timeout:
          seconds: 30
          nanos: 100

Faults

Introduce errors when servicing requests to simulate failures, including high latency, service overload, service failures, and network partitioning. This feature is useful for testing the resiliency of a service to simulated faults. Make sure to replace the variables marked by []`:

defaultService: regions/[region]/backendServices/[backend-service1]
kind: compute#urlMap
name: l7-ilb-map
region: region/[region]
hostRules:
- hosts:
  - '*'
  pathMatcher: matcher1
pathMatchers:
- defaultService: regions/[region]/backendServices/[backend-service1]
  name: matcher1
  routeRules:
    - matchRules:
        - prefixMatch: /
      priority: [priority]
      routeAction:
        weightedBackendServices:
          - backendService: regions/[region]/backendServices/[backend-service1]
            weight: 100
        faultInjectionPolicy:
          delay:
            fixedDelay:
              seconds: 10
              nanos: 20
            percentage: 25
          abort:
            httpStatus: 503
            percentage: 50

CORS

Configure cross-origin resource sharing (CORS) policies to handle Internal HTTP(S) Load Balancing settings for enforcing CORS requests. Make sure to replace the variables marked by []:

defaultService: regions/[region]/backendServices/[backend-service1]
kind: compute#urlMap
name: l7-ilb-map
region: region/[region]
hostRules:
- hosts:
  - '*'
  pathMatcher: matcher1
pathMatchers:
- defaultService: regions/[region]/backendServices/[backend-service1]
  name: matcher1
  routeRules:
    - matchRules:
        - prefixMatch: /
      priority: [priority]
      routeAction:
        weightedBackendServices:
          - backendService: regions/[region]/backendServices/[backend-service1]
            weight: 100
        corsPolicy:
            allowOrigins: my-domain.com
            allowMethods: GET, POST
            allowHeaders: Authorization, Content-Type
            maxAge: 1200
            allowCredentials: True

Headers

Add and remove request headers before sending a request to the backend service. Also add and remove response headers after receiving a response from the backend service. Make sure to replace the variables marked by []:

defaultService: regions/[region]/backendServices/[backend-service1]
kind: compute#urlMap
name: l7-ilb-map
region: region/[region]
hostRules:
- hosts:
  - '*'
  pathMatcher: matcher1
pathMatchers:
- defaultService: regions/[region]/backendServices/[backend-service1]
  name: matcher1
  routeRules:
    - matchRules:
        - prefixMatch: /
      priority: [priority]
      headerAction:
        requestHeadersToAdd:
          - headerName: header-1-name
            headerValue: header-1-value
            replace: True
        requestHeadersToRemove:
          - header-2-name
          - header-3-name
        responseHeadersToAdd:
          - headerName: header-4-name
            headerValue: header-4-value
            replace: True
        responseHeadersToRemove:
          - header-5-name
          - header-6-name

Backend service YAML examples

Outlier detection

Specify the criteria for eviction of unhealthy backend VMs or endpoints in NEGs, along with criteria defining when a backend or endpoint is considered healthy enough to receive traffic again. Make sure to replace the variables marked by []:

kind: compute#backendService
loadBalancingScheme: INTERNAL_MANAGED
localityLbPolicy: RANDOM
name: regions/[region]/backendServices/[backend-service1]
outlierDetection:
  baseEjectionTime:
    nanos: 0
    seconds: '30'
  consecutiveErrors: 5
  consecutiveGatewayFailure: 3
  enforcingConsecutiveErrors: 2
  enforcingConsecutiveGatewayFailure: 100
  enforcingSuccessRate: 100
  interval:
    nanos: 0
    seconds: '1'
  maxEjectionPercent: 50
  successRateMinimumHosts: 5
  successRateRequestVolume: 100
  successRateStdevFactor: 1900
region: region/[region]

Circuit breaking

Circuit breaking lets you set failure thresholds to prevent client requests from overloading your backends. After requests reach a limit that you've set, the load balancer stops allowing new connections or sending additional requests, giving your backends time to recover. Thus, circuit breaking prevents cascading failures by returning an error to the client rather than overloading a backend. This allows some traffic to be served while providing time for managing the overload situation, such as handling a traffic spike by increasing capacity through autoscaling.

Set upper limits on requests per connection as well as the volume of connections to a backend service. Also limit the number of pending requests and retries. Make sure to replace the variables marked by []:

kind: compute#backendService
loadBalancingScheme: INTERNAL_MANAGED
localityLbPolicy: RANDOM
affinityCookieTtlSec: 0
backends:
- balancingMode: UTILIZATION
  capacityScaler: 1.0
  group: region/[region]/instanceGroups/[instance-group]
  maxUtilization: 0.8
circuitBreakers:
  maxConnections: 1000
  maxPendingRequests: 200
  maxRequests: 1000
  maxRequestsPerConnection: 100
  maxRetries: 3
connectionDraining:
  drainingTimeoutSec: 0
healthChecks:
- region/[region]/healthChecks/[health-check]

Setting up traffic splitting

This example demonstrates the following steps:

  1. Create distinct templates for different services.

  2. Create instance groups for those templates.

  3. Create routing rules that set up 95% / 5% traffic splitting.

  4. Send curl commands showing that the traffic split percentages roughly match the configuration.

These instructions assume the following:

  • The region is us-west1.
  • A target proxy and forwarding rule have been created, along with a URL map named l7-ilb-map.

  • The forwarding rule has address 10.1.2.99.

    See instructions in Setting Up Internal HTTP(S) Load Balancing for Compute Engine VMs.

  • The URL map sends all traffic to one backend service, called red-service, which is the default backend service.

  • You set up an alternate path that sends 5% of the traffic to blue-service and 95% of traffic to green-service.

  • A path matcher is used.

  • You are using Cloud Shell or another environment with bash installed.

Defining the services

The following bash function creates a backend service, including the instance template and the managed instance group.

These instructions assume that an HTTP health check (l7-ilb-basic-check) has been created. Refer to the instructions in Setting Up Internal HTTP(S) Load Balancing for Compute Engine VMs.

function make_service() {
  local name="$1"
  local region="$2"
  local zone="$3"
  local network="$4"
  local subnet="$5"
  local subdir="$6"
 
  www_dir="/var/www/html/$subdir"
 
  (set -x; \
  gcloud compute instance-templates create "${name}-template" \
    --region="$region" \
    --network="$network" \
    --subnet="$subnet" \
    --tags=allow-ssh,load-balanced-backend \
    --image-family=debian-9 \
    --image-project=debian-cloud \
    --metadata=startup-script="#! /bin/bash
  apt-get update
  apt-get install apache2 -y
  a2ensite default-ssl
  a2enmod ssl
  sudo mkdir -p $www_dir
  /bin/hostname | sudo tee ${www_dir}index.html
  systemctl restart apache2"; \
  gcloud compute instance-groups managed create \
    "${name}-instance-group" \
    --zone="$zone" \
    --size=2 \
    --template="${name}-template"; \
  gcloud compute backend-services create "${name}-service" \
    --load-balancing-scheme=INTERNAL_MANAGED \
    --protocol=HTTP \
    --health-checks=l7-ilb-basic-check \
    --health-checks-region="$region" \
    --region="$region"; \
  gcloud compute backend-services add-backend "${name}-service" \
    --balancing-mode='UTILIZATION' \
    --instance-group="${name}-instance-group" \
    --instance-group-zone="$zone" \
    --region="$region")
}

Creating the services

Now call the function to make three services, red, green, and blue. The red service acts as the default service for requests to /. The green and blue services are both set up on /prefix to handle 95% and 5% of the traffic, respectively.

make_service red us-west1 us-west1-a lb-network backend-subnet ""
make_service green us-west1 us-west1-a lb-network backend-subnet /prefix
make_service blue us-west1 us-west1-a lb-network backend-subnet /prefix

Creating the URL map

Console

  1. Go to the Load balancing page in the Google Cloud Console.
    Go to the Load balancing page
  2. Click the l7-ilb-map link.
  3. Click Edit .

Configure the new routing rules

  1. Under Routing rules, select Advanced host, path and route rule.
  2. Under New hosts and path matcher, create the default action by setting the Service to red-service.
  3. Click Done.
  4. Click Add hosts and path matcher.
  5. In the Hosts field, enter 10.1.2.99. This is the IP address of your load balancer's forwarding rule.

  6. Paste the following YAML content into the Path matcher box, making sure to replace [PROJECT_ID] with your project ID:

    defaultService: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/us-west1/backendServices/red-service
name: matcher1
routeRules:
- priority: 2
  matchRules:
    - prefixMatch: /prefix
  routeAction:
    weightedBackendServices:
      - backendService: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/us-west1/backendServices/green-service
        weight: 95
      - backendService: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/us-west1/backendServices/blue-service
        weight: 5

  7. Click Done.

  8. Click Update.

gcloud

  1. Export the existing URL map using the gcloud compute url-maps export command:

    gcloud compute url-maps export l7-ilb-map \
  --destination=l7-ilb-map-config.yaml \
  --region=us-west1

  2. Update the URL map file l7-ilb-map-config.yaml by adding this to the end of the file, making sure to replace [PROJECT_ID] with your project ID:

    hostRules:
- hosts:
  - '*'
  pathMatcher: matcher1
pathMatchers:
- defaultService: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/us-west1/backendServices/red-service
  name: matcher1
  routeRules:
  - priority: 2
    matchRules:
      - prefixMatch: /prefix
    routeAction:
      weightedBackendServices:
        - backendService: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/us-west1/backendServices/green-service
          weight: 95
        - backendService: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/us-west1/backendServices/blue-service
          weight: 5

  3. Update the URL map using the gcloud compute url-maps import command:

    gcloud compute url-maps import l7-ilb-map \
   --region=us-west1 \
   --source=l7-ilb-map-config.yaml

Testing the configuration

To test the configuration, first ensure that requests to 10.1.2.99/ (the load balancer's IP address set up earlier) are handled by the default red configuration.

Then check to make sure that requests sent to 10.1.2.99/prefix are split as expected.

Creating a client VM

Refer to Creating a VM instance in the zone to test connectivity.

Sending requests to 10.1.2.99

SSH into the client.

gcloud compute ssh l7-ilb-client-us-west1-a \
    --zone=us-west1-a

Run the following command:

for LB_IP in 10.1.2.99; do
    RESULTS=
    for i in {1..1000}; do RESULTS="$RESULTS:`curl ${LB_IP}`"; done >/dev/null 2>&1
    IFS=':'
    echo "***"
    echo "*** Results of load balancing to $LB_IP: "
    echo "***"
    for line in $RESULTS; do echo $line; done | grep -Ev "^$" | sort | uniq -c
    echo
done
Checking the results
***
***Results of load balancing to 10.1.2.99:
***
502 red-instance-group-9jvq
498 red-instance-group-sww8

Sending requests to 10.1.2.99/prefix

Send requests to 10.1.2.99/prefix and note the traffic splitting.

for LB_IP in 10.1.2.99; do
    RESULTS=
    for i in {1..1000}; do RESULTS="$RESULTS:`curl ${LB_IP}/prefix/index.html`"; done >/dev/null 2>&1
    IFS=':'
    echo "***"
    echo "*** Results of load balancing to $LB_IP/prefix: "
    echo "***"
    for line in $RESULTS; do echo $line; done | grep -Ev "^$" | sort | uniq -c
    echo
done
Checking the results
***
***Results of load balancing to 10.1.2.99/prefix:
***
21 blue-instance-group-8n49
27 blue-instance-group-vlqc
476 green-instance-group-c0wv
476 green-instance-group-rmf4

The canary setup successfully sends 95% of /prefix requests to service green and 5% to service blue.

Traffic control enables you to configure session affinity based on a provided cookie. To configure HTTP_COOKIE based session affinity for a backend service named red-service, follow these directions.

To set up session affinity using HTTP_COOKIE:

  1. Use the gcloud compute backend_services export command to get the backend service configuration.

    gcloud compute backend-services export red-service \
    --destination=red-service-config.yaml \
    --region=us-west1

  2. Update the red-service-config.yaml file as follows:

    sessionAffinity: 'HTTP_COOKIE'
localityLbPolicy: 'RING_HASH'
consistentHash:
 httpCookie:
  name: 'http_cookie'
  path: '/cookie_path'
  ttl:
    seconds: 100
    nanos: 30
 minimumRingSize: 10000

  3. In the red-service-config.yaml file, delete the line that says:

    sessionAffinity: NONE

  4. Update the backend service configuration file:

    gcloud compute backend-services import red-service \
    --source=red-service-config.yaml \
    --region=us-west1

Troubleshooting

Use this information for troubleshooting when traffic is not being routed according to the route rules and traffic policies that you configured.

For information about logging and monitoring, see Internal HTTP(S) logging and monitoring.

Symptoms:

  • Increased traffic to services in rules above the rule in question.
  • An unexpected increase in 4xx and 5xx HTTP responses for a given route rule.

Solution: Check the order of your route rules. Route rules are interpreted in the order in which they are specified.

Route rules within a URL map are interpreted in the order in which they are specified. This is different from the way that path rules are interpreted by longest prefix match. For a path rule, Internal HTTP(S) Load Balancing will only select a single path rule; however, when you use route rules, more than one might apply.

When you define route rules, check to be sure that rules at the top of the list do not inadvertently route traffic that would otherwise have been routed by a subsequent route rule. The service that receives misdirected traffic would likely reject requests, and the service in your route rules would receive reduced traffic or no traffic at all.

What's next