Setting up traffic management for regional external HTTP(S) load balancers

This document shows examples of using traffic management for some specific use cases. Many other use cases are possible.

The document contains examples for the following load balancers:

  • Regional external HTTP(S) load balancer
  • Regional Internal HTTP(S) load balancer

For the regional load balancers' traffic management configuration, the only difference between these two load balancers is the load balancing scheme, as follows:

  • Regional external HTTP(S) load balancers use EXTERNAL_MANAGED.
  • Regional Internal HTTP(S) load balancers use INTERNAL_MANAGED.

Global external HTTP(S) load balancers use the EXTERNAL_MANAGED load balancing scheme and global load balancing components, such as the forwarding rule, URL map, and backend service. For information about traffic management with global external HTTP(S) load balancers, see Traffic management overview for global external HTTP(S) load balancers.

For information about traffic management with global external HTTP(S) load balancer (classic), see Traffic management overview for global external HTTP(S) load balancer (classic).

Before you begin

Make sure that you understand how traffic management works. For more information, read Traffic management overview for regional external HTTP(S) load balancers.

Configure traffic management

Within your chosen configuration environment, you set up traffic management by using YAML configurations. A URL map and a backend service each has its own YAML file. Depending on your desired functionality, you need to write either a URL map YAML file, a backend service YAML file, or both.

For help writing these YAML files, you can use the examples on this page and the Cloud Load Balancing API documentation.

The Cloud Console is not supported.

The regional URL map API and the regional backend service API documentation provides a full list of fields, including semantics regarding relationships, restrictions, and cardinality.

URL map YAML examples

One action in a URL map

Single service

Send all traffic to a single service. Make sure to replace the placeholders :

defaultService: regions/REGION/backendServices/BACKEND_SERVICE_1
hostRules:
- hosts:
  - '*'
  pathMatcher: matcher1
name: URL_MAP_NAME
pathMatchers:
- defaultService: regions/REGION/backendServices/BACKEND_SERVICE_1
  name: matcher1
  routeRules:
    - matchRules:
        - prefixMatch: /PREFIX
      priority: 1
      routeAction:
        weightedBackendServices:
          - backendService: regions/REGION/backendServices/BACKEND_SERVICE_1
            weight: 100

Split traffic

Split traffic between multiple services. Make sure to replace the placeholders:

defaultService: regions/REGION/backendServices/BACKEND_SERVICE_1
hostRules:
- hosts:
  - '*'
  pathMatcher: matcher1
name: URL_MAP_NAME
pathMatchers:
- defaultService: regions/REGION/backendServices/BACKEND_SERVICE_1
  name: matcher1
  routeRules:
  - matchRules:
    - prefixMatch: /PREFIX
    priority: 2
    routeAction:
      weightedBackendServices:
      - backendService: regions/REGION/backendServices/BACKEND_SERVICE_1
        weight: 95
      - backendService: regions/REGION/backendServices/BACKEND_SERVICE_2
        weight: 5

URL redirect

Returns a configurable 3xx response code. Also sets the Location response header with the appropriate URI, replacing the host and path as specified in the redirect action.

defaultService: regions/REGION/backendServices/BACKEND_SERVICE_1
name: regional-lb-map
region: region/REGION
hostRules:
- hosts:
  - '*'
  pathMatcher: matcher1
pathMatchers:
- defaultService: regions/REGION/backendServices/BACKEND_SERVICE_1
  name: matcher1
  routeRules:
    - matchRules:
        - prefixMatch: /PREFIX
      priority: PRIORITY # 1 is highest
      urlRedirect:
        hostRedirect: "new-host-name.com" # Omit to keep the requested host
        pathRedirect: "/new-path" # Omit to keep the requested path; mutually exclusive to prefixRedirect
        prefixRedirect: "/newPrefix" # Omit to keep the requested path; mutually exclusive to pathRedirect
        redirectResponseCode: FOUND
        stripQuery: True

Multiple actions in a URL map

Mirror traffic

In addition to forwarding the request to the selected backend service, send an identical request to the configured mirror backend service on a fire and forget basis. The load balancer doesn't wait for a response from the backend to which it sends the mirrored request. Mirroring is useful for testing a new version of a backend service. You can also use it to debug production errors on a debug version of your backend service, rather than on the production version.

defaultService: regions/REGION/backendServices/BACKEND_SERVICE_1
name: regional-lb-map
region: region/REGION
hostRules:
- hosts:
  - '*'
  pathMatcher: matcher1
pathMatchers:
- defaultService: regions/REGION/backendServices/BACKEND_SERVICE_1
  name: matcher1
  routeRules:
    - matchRules:
        - prefixMatch: /PREFIX
      priority: 1
      routeAction:
        weightedBackendServices:
          - backendService: regions/REGION/backendServices/BACKEND_SERVICE_1
            weight: 100
        requestMirrorPolicy:
          backendService: regions/REGION/backendServices/BACKEND_SERVICE_2

Rewrite URL

Rewrite the host name portion of the URL, the path portion of the URL, or both, before sending a request to the selected backend service. Make sure to replace the placeholders:

defaultService: regions/REGION/backendServices/BACKEND_SERVICE_1
name: regional-lb-map
region: region/REGION
hostRules:
- hosts:
  - '*'
  pathMatcher: matcher1
pathMatchers:
- defaultService: regions/REGION/backendServices/BACKEND_SERVICE_1
  name: matcher1
  routeRules:
    - matchRules:
        - prefixMatch: /PREFIX
      priority: PRIORITY # 1 is highest
      routeAction:
        weightedBackendServices:
          - backendService: regions/REGION/backendServices/BACKEND_SERVICE_1
            weight: 100
        urlRewrite:
          hostRewrite: "new-host-name.com" # Omit to keep the requested host
          pathPrefixRewrite: "/new-path/" # Omit to keep the requested path

Retry request

Configure the conditions under which the load balancer retries failed requests, how long the load balancer waits before retrying, and the maximum number of retries permitted.

defaultService: regions/REGION/backendServices/BACKEND_SERVICE_1
name: regional-lb-map
region: region/REGION
hostRules:
- hosts:
  - '*'
  pathMatcher: matcher1
pathMatchers:
- defaultService: regions/REGION/backendServices/BACKEND_SERVICE_1
  name: matcher1
  routeRules:
    - matchRules:
        - prefixMatch: /PREFIX
      priority: PRIORITY # 1 is highest
      routeAction:
        weightedBackendServices:
          - backendService: regions/REGION/backendServices/BACKEND_SERVICE_1
            weight: 100
        retryPolicy:
          retryConditions: 502, 504
          numRetries: 3
          perTryTimeout:
            seconds: 1
            nanos: 500000000

Timeout

Specify the timeout for the selected route. Timeout is computed from the time the request is fully processed until the response is fully processed. Timeout includes all retries. Make sure to replace the placeholders:

defaultService: regions/REGION/backendServices/BACKEND_SERVICE_1
name: regional-lb-map
region: region/REGION
hostRules:
- hosts:
  - '*'
  pathMatcher: matcher1
pathMatchers:
- defaultService: regions/REGION/backendServices/BACKEND_SERVICE_1
  name: matcher1
  routeRules:
    - matchRules:
        - prefixMatch: /PREFIX
      priority: PRIORITY # 1 is highest
      routeAction:
        weightedBackendServices:
          - backendService: regions/REGION/backendServices/BACKEND_SERVICE_1
            weight: 100
        timeout:
          seconds: 30
          nanos: 500000000

Faults

Introduce errors when servicing requests to simulate failures, including high latency, service overload, service failures, and network partitioning. This feature is useful for testing the resiliency of a service to simulated faults.

defaultService: regions/REGION/backendServices/BACKEND_SERVICE_1
name: regional-lb-map
region: region/REGION
hostRules:
- hosts:
  - '*'
  pathMatcher: matcher1
pathMatchers:
- defaultService: regions/REGION/backendServices/BACKEND_SERVICE_1
  name: matcher1
  routeRules:
    - matchRules:
        - prefixMatch: /PREFIX
      priority: PRIORITY # 1 is highest
      routeAction:
        weightedBackendServices:
          - backendService: regions/REGION/backendServices/BACKEND_SERVICE_1
            weight: 100
        faultInjectionPolicy:
          delay:
            fixedDelay:
              seconds: 10
              nanos: 500000000
            percentage: 25
          abort:
            httpStatus: 503
            percentage: 50

CORS

Configure cross-origin resource sharing (CORS) policies to handle Internal HTTP(S) Load Balancing settings for enforcing CORS requests.

defaultService: regions/REGION/backendServices/BACKEND_SERVICE_1
name: regional-lb-map
region: region/REGION
hostRules:
- hosts:
  - '*'
  pathMatcher: matcher1
pathMatchers:
- defaultService: regions/REGION/backendServices/BACKEND_SERVICE_1
  name: matcher1
  routeRules:
    - matchRules:
        - prefixMatch: /PREFIX
      priority: PRIORITY # 1 is highest
      routeAction:
        weightedBackendServices:
          - backendService: regions/REGION/backendServices/BACKEND_SERVICE_1
            weight: 100
        corsPolicy:
            allowOrigins: my-domain.com
            allowMethods: GET, POST
            allowHeaders: Authorization, Content-Type
            maxAge: 1200
            allowCredentials: True

Headers

Add and remove request headers before sending a request to the backend service. Also add and remove response headers after receiving a response from the backend service.

defaultService: regions/REGION/backendServices/BACKEND_SERVICE_1
name: regional-lb-map
region: region/REGION
hostRules:
- hosts:
  - '*'
  pathMatcher: matcher1
pathMatchers:
- defaultService: regions/REGION/backendServices/BACKEND_SERVICE_1
  name: matcher1
  routeRules:
    - matchRules:
        - prefixMatch: /PREFIX
      priority: PRIORITY # 1 is highest
      routeAction:
        weightedBackendServices:
          - backendService: regions/REGION/backendServices/BACKEND_SERVICE_1
            weight: 100
            headerAction:
              requestHeadersToAdd:
              - headerName: header-1-name
                headerValue: header-1-value
                replace: True
              requestHeadersToRemove:
              - header-2-name
              - header-3-name
              responseHeadersToAdd:
              - headerName: header-4-name
                headerValue: header-4-value
                replace: True
             responseHeadersToRemove:
             - header-5-name
             - header-6-name

Backend service YAML examples

Outlier detection

Specify the criteria for eviction of unhealthy backend VMs or endpoints in NEGs, along with criteria defining when a backend or endpoint is considered healthy enough to receive traffic again. Make sure to replace the placeholders:

loadBalancingScheme: LOAD_BALANCING_SCHEME
localityLbPolicy: RANDOM
name: regions/REGION/backendServices/BACKEND_SERVICE_1
outlierDetection:
  baseEjectionTime:
    nanos: 0
    seconds: '30'
  consecutiveErrors: 5
  consecutiveGatewayFailure: 3
  enforcingConsecutiveErrors: 2
  enforcingConsecutiveGatewayFailure: 100
  enforcingSuccessRate: 100
  interval:
    nanos: 0
    seconds: '1'
  maxEjectionPercent: 50
  successRateMinimumHosts: 5
  successRateRequestVolume: 100
  successRateStdevFactor: 1900
region: region/REGION

Circuit breaking

Circuit breaking lets you set failure thresholds to prevent client requests from overloading your backends. After requests reach a limit that you've set, the load balancer stops allowing new connections or sending additional requests, giving your backends time to recover. Thus, circuit breaking prevents cascading failures by returning an error to the client rather than overloading a backend. This allows some traffic to be served while providing time for managing the overload situation, such as handling a traffic spike by increasing capacity through autoscaling.

Set upper limits on requests per connection as well as the volume of connections to a backend service. Also limit the number of pending requests and retries.

loadBalancingScheme: LOAD_BALANCING_SCHEME # EXTERNAL_MANAGED or INTERNAL_MANAGED
localityLbPolicy: RANDOM
affinityCookieTtlSec: 0
backends:
- balancingMode: UTILIZATION
  capacityScaler: 1.0
  group: region/REGION/instanceGroups/INSTANCE_GROUP
  maxUtilization: 0.8
circuitBreakers:
  maxConnections: 1000
  maxPendingRequests: 200
  maxRequests: 1000
  maxRequestsPerConnection: 100
  maxRetries: 3
connectionDraining:
  drainingTimeoutSec: 0
healthChecks:
- region/REGION/healthChecks/HEALTH_CHECK

Set up traffic splitting

This example demonstrates the following steps:

  1. Create distinct templates for different services.

  2. Create instance groups for those templates.

  3. Create routing rules that set up 95% / 5% traffic splitting.

  4. Send curl commands showing that the traffic split percentages roughly match the configuration.

These instructions assume the following:

  • The region is us-west1.
  • A target proxy and forwarding rule have been created, along with a URL map named regional-lb-map.

  • The URL map sends all traffic to one backend service, called red-service, which is the default backend service.

  • You set up an alternate path that sends 5% of the traffic to blue-service and 95% of traffic to green-service.

  • A path matcher is used.

  • You are using Cloud Shell or another environment with bash installed.

Define the services

The following bash function creates a backend service, including the instance template and the managed instance group.

These instructions assume that an HTTP health check (regional-lb-basic-check) has been created. For instructions, see Setting Up a regional external HTTP(S) load balancer.
function make_service() {
  local name="$1"
  local region="$2"
  local zone="$3"
  local network="$4"
  local subnet="$5"
  local subdir="$6"

  www_dir="/var/www/html/$subdir"

  (set -x; \
  gcloud compute instance-templates create "${name}-template" \
    --region="$region" \
    --network="$network" \
    --subnet="$subnet" \
    --tags=allow-ssh,load-balanced-backend \
    --image-family=debian-9 \
    --image-project=debian-cloud \
    --metadata=startup-script="#! /bin/bash
  apt-get update
  apt-get install apache2 -y
  a2ensite default-ssl
  a2enmod ssl
  sudo mkdir -p $www_dir
  /bin/hostname | sudo tee ${www_dir}index.html
  systemctl restart apache2"; \
  gcloud compute instance-groups managed create \
    "${name}-instance-group" \
    --zone="$zone" \
    --size=2 \
    --template="${name}-template"; \
  gcloud compute backend-services create "${name}-service" \
    --load-balancing-scheme=LOAD_BALANCING_SCHEME\
    --protocol=HTTP \
    --health-checks=regional-lb-basic-check \
    --health-checks-region="$region" \
    --region="$region"; \
  gcloud compute backend-services add-backend "${name}-service" \
    --balancing-mode='UTILIZATION' \
    --instance-group="${name}-instance-group" \
    --instance-group-zone="$zone" \
    --region="$region")
}

Create the services

Call the function to make three services, red, green, and blue. The red service acts as the default service for requests to /. The green and blue services are both set up on /PREFIX to handle 95% and 5% of the traffic, respectively.

make_service red us-west1 us-west1-a lb-network backend-subnet ""
make_service green us-west1 us-west1-a lb-network backend-subnet /PREFIX
make_service blue us-west1 us-west1-a lb-network backend-subnet /PREFIX

Create the URL map

gcloud

  1. Export the existing URL map using the gcloud compute url-maps export command:

    gcloud compute url-maps export regional-lb-map \
      --destination=regional-lb-map-config.yaml \
      --region=us-west1
    
  2. Update the URL map file regional-lb-map-config.yaml by adding this to the end of the file:

    hostRules:
    - hosts:
      - '*'
      pathMatcher: matcher1
    pathMatchers:
    - defaultService: regions/us-west1/backendServices/red-service
      name: matcher1
      routeRules:
      - priority: 2
        matchRules:
          - prefixMatch: /PREFIX
        routeAction:
          weightedBackendServices:
            - backendService: regions/us-west1/backendServices/green-service
              weight: 95
            - backendService: regions/us-west1/backendServices/blue-service
              weight: 5
    
  3. Update the URL map using the gcloud compute url-maps import command:

    gcloud compute url-maps import regional-lb-map \
       --region=us-west1 \
       --source=regional-lb-map-config.yaml
    

Test the configuration

To test the configuration, first ensure that requests to the load balancer's IP address set up earlier are handled by the default red configuration.

Then check to make sure that requests sent to FORWARDING_RULE_IP_ADDRESS/PREFIX are split as expected.

Traffic control enables you to configure session affinity based on a provided cookie. To configure HTTP_COOKIE based session affinity for a backend service named red-service, follow these directions.

  1. Use the gcloud compute backend_services export command to get the backend service configuration.

    gcloud compute backend-services export red-service \
        --destination=red-service-config.yaml \
        --region=us-west1
    
  2. Update the red-service-config.yaml file as follows:

    sessionAffinity: 'HTTP_COOKIE'
    localityLbPolicy: 'RING_HASH'
    consistentHash:
     httpCookie:
      name: 'http_cookie'
      path: '/cookie_path'
      ttl:
        seconds: 100
        nanos: 500000000
     minimumRingSize: 10000
    
  3. In the red-service-config.yaml file, delete the following line:

    sessionAffinity: NONE
    
  4. Update the backend service configuration file:

    gcloud compute backend-services import red-service \
        --source=red-service-config.yaml \
        --region=us-west1
    

Troubleshooting

Use this information for troubleshooting when traffic is not being routed according to the route rules and traffic policies that you configured.

For information about logging and monitoring, see External HTTP(S) logging and monitoring.

Symptoms:

  • Increased traffic to services in rules above the rule in question.
  • An unexpected increase in 4xx and 5xx HTTP responses for a given route rule.

Solution: Check the order of your route rules. Route rules are interpreted in the order in which they are specified.

Route rules within a URL map are interpreted in the order in which they are specified. This is different from the way that path rules are interpreted by longest prefix match. For a path rule, Internal HTTP(S) Load Balancing will only select a single path rule; however, when you use route rules, more than one might apply.

When you define route rules, check to be sure that rules at the top of the list do not inadvertently route traffic that would otherwise have been routed by a subsequent route rule. The service that receives misdirected traffic would likely reject requests, and the service in your route rules would receive reduced traffic or no traffic at all.

What's next