Setting up proxyless gRPC services with advanced traffic management

This document provides end-to-end instructions for configuring Traffic Director with these routing features:

  • Route matching
  • Traffic splitting

For conceptual information about these features, see Advanced traffic management.

To illustrate these features, you deploy a gRPC Wallet example. As shown in the diagram below, you create backend services for wallet-service, stats-service and account-service.

Example gRPC Wallet traffic routing configuration (click to enlarge)
Example gRPC Wallet traffic routing configuration (click to enlarge)

In the example, you deploy multiple versions of each service to illustrate request routing based on rules that you configure. To simulate the building and deploying of different versions of a service, you use server flags to change the behavior of binaries that you build just once.

  • The --port flag specifies the port on which the service on the backend VM listens.
  • The --hostname_suffix flag specifies a value that is appended to the hostname of the VM responding to a request. The resulting value is added as the hostname metadata in the response. This helps you identify which instance of a backend service responded to the client request.
  • The --premium_only flag with the value true specifies that the service is a premium version of the stats service.
  • The --v1_behavior flag with the value true specifies that the wallet binary behaves as a v1 version.
Service Instance group Instance(s) Server flags
account account 2 --port=50053
--hostname_suffix="account"
stats stats 2 --port=50052
--hostname_suffix="stats"
--account_server="xds:///account.grpcwallet.io"
stats-premium stats-premium 2 --port=50052
--hostname_suffix="stats_premium"
--account_server="xds:///account.grpcwallet.io"
--premium_only=true
wallet-v1 wallet-v1 2 --port=50051
--hostname_suffix="wallet_v1"
--v1_behavior=true
--account_server="xds:///account.grpcwallet.io"
--stats_server="xds:///stats.grpcwallet.io"
wallet-v2 wallet-v2 2 --port=50051
--hostname_suffix "wallet_v2"
--account_server="xds:///account.grpcwallet.io"
--stats_server="xds:///stats.grpcwallet.io"

You then configure Traffic Director to route requests to these services from a test client, based on the following routing rules:

Host Match rules Route action
wallet.grpcwallet.io Default wallet-v1
Full path: /grpc.examples.wallet.Wallet/FetchBalance wallet-v1: 40%
wallet-v2: 60%
Path prefix: /grpc.examples.wallet.Wallet/ wallet-v2
stats.grpcwallet.io Default stats
Path prefix: "/"
Header: {"membership": "premium"} 		stats-premium
account.grpcwallet.io Default account

Creating the gRPC health check and firewall rule

In this section you create a gRPC health check and a firewall rule to allow gRPC health check requests to reach your network. Later, the gRPC health check is associated with backend services to check the health of the backends in those backend services.

gcloud

  1. Create the health check.

    gcloud compute health-checks create grpc grpcwallet-health-check \
  --use-serving-port

  2. Create the firewall rule for the health check.

    gcloud compute firewall-rules create grpcwallet-allow-health-checks \
   --network default --action allow --direction INGRESS \
   --source-ranges 35.191.0.0/16,130.211.0.0/22 \
   --target-tags allow-health-checks \
   --rules tcp:50051-50053

Creating the instance template

In this section you create an instance template to deploy the account gRPC service that is exposed on port 50053.

gcloud

  1. Create the instance template.

    gcloud compute instance-templates create grpcwallet-account-template \
  --scopes=https://www.googleapis.com/auth/cloud-platform \
  --tags=allow-health-checks \
  --image-family=debian-10 \
  --image-project=debian-cloud \
  --metadata-from-file=startup-script=<(echo '#! /bin/bash
set -ex
cd /root
export HOME=/root
sudo apt-get update -y
curl -L https://github.com/GoogleCloudPlatform/traffic-director-grpc-examples/archive/master.tar.gz | tar -xz
cd traffic-director-grpc-examples-master/go/account_server/
sudo apt-get install -y golang git
go build .
sudo systemd-run ./account_server --port 50053 --hostname_suffix account')

Creating the managed instance group

Managed instance groups use autoscaling to create new backend VMs as needed. In this section you create a managed instance group using the instance template you created in the previous section.

gcloud

  1. Create the instance group.

    gcloud compute instance-groups managed create grpcwallet-account-mig-us-central1 \
  --zone us-central1-a \
  --size=2 \
  --template=grpcwallet-account-template

Configuring the named port

In this section, you configure the named port for the gRPC service. The named port is the port on which the gRPC service listens for requests. In this example, the named port is port 50053.

gcloud

  1. Create the named port.

    gcloud compute instance-groups set-named-ports grpcwallet-account-mig-us-central1 \
  --named-ports=grpcwallet-account-port:50053 \
  --zone us-central1-a

Creating the backend service

In this section, you create a global backend service with a load balancing scheme of INTERNAL_SELF_MANAGED and protocol GRPC, and then associate the health check and instance group with the backend service. In this example, you use the managed instance group you created in Creating the managed instance group. This managed instance group runs the account gRPC service. The port in the --port-name flag is the named port you created in Configuring the named port.

gcloud

  1. Create the backend service and associate the health check with the new backend service.

    gcloud compute backend-services create grpcwallet-account-service \
    --global \
    --load-balancing-scheme=INTERNAL_SELF_MANAGED \
    --protocol=GRPC \
    --port-name=grpcwallet-account-port \
    --health-checks grpcwallet-health-check

  2. Add the managed instance group as the backend.

    gcloud compute backend-services add-backend grpcwallet-account-service \
  --instance-group grpcwallet-account-mig-us-central1 \
  --instance-group-zone us-central1-a \
  --global

The steps to create the remaining backend services used in the gRPC Wallet example are similar to the above steps. You create the remaining services by running a shell script. The script deploys the following backend services:

  • stats
  • stats-premium
  • wallet-v1
  • wallet-v2

Run the shell script that creates the additional backend services:

curl -O https://raw.githubusercontent.com/GoogleCloudPlatform/traffic-director-grpc-examples/master/scripts/create_service.sh
chmod +x ./create_service.sh

./create_service.sh go stats 50052 stats '--account_server="xds:///account.grpcwallet.io"'

./create_service.sh go stats 50052 stats-premium '--account_server="xds:///account.grpcwallet.io" --premium_only=true'

./create_service.sh java wallet 50051 wallet-v1 '--account_server="xds:///account.grpcwallet.io" --stats_server="xds:///stats.grpcwallet.io" --v1_behavior=true'

./create_service.sh java wallet 50051 wallet-v2 '--account_server="xds:///account.grpcwallet.io" --stats_server="xds:///stats.grpcwallet.io"'

Creating the routing rules

In this section, you create a URL map that specifies the virtual host names of the services in this example and the associated routing rules. For more information, see Routing rule maps.

In the URL map, the hostRules specify the virtual host names of the services in the example. These are the names a client uses in the channel URI to connect to a specific service. For example, to send a request to the account service, a client uses xds:///account.grpcwallet.io in the channel URI. You must configure a hosts entry with value account.grpcwallet.io in the hostRules.

The pathMatcher associated with a hosts entry specifies the name of a pathMatcher that contains all the routing rules for that virtual host. A pathMatcher configuration consists of matching rules and corresponding action rules. The example results in the following configuration:

  • When a request is sent to account.grpcwallet.io, send the request to the grpcwallet-account-service backend service.
  • When a request is sent to stats.grpcwallet.io:
    • If the request contains the header membership with value premium, then send the request to grpcwallet-stats-premium-service backend service.
    • Else, send the request to the default backend service grpcwallet-stats-service.

  • When a request is sent to wallet.grpcwallet.io:

    • If the ServiceName/MethodName matches /grpc.examples.wallet.Wallet/FetchBalance, then send 40% of such requests to grpcwallet-wallet-v2-service backend service and the remaining requests to grpcwallet-wallet-v1-service backend service.
    • Otherwise, if the ServiceName matches /grpc.examples.wallet.Wallet/, regardless of the method called, send the request to the grpcwallet-wallet-v1-service backend service.

gcloud

Run the following to create the URL map:

export PROJECT_ID=$(gcloud config list --format 'value(core.project)')

gcloud compute url-maps import grpcwallet-url-map << EOF
defaultService: projects/$PROJECT_ID/global/backendServices/grpcwallet-account-service
name: grpcwallet-url-map
hostRules:
- hosts:
  - account.grpcwallet.io
  pathMatcher: grpcwallet-account-path-matcher
- hosts:
  - stats.grpcwallet.io
  pathMatcher: grpcwallet-stats-path-matcher
- hosts:
  - wallet.grpcwallet.io
  pathMatcher: grpcwallet-wallet-path-matcher
pathMatchers:
- defaultService: projects/$PROJECT_ID/global/backendServices/grpcwallet-account-service
  name: grpcwallet-account-path-matcher
- defaultService: projects/$PROJECT_ID/global/backendServices/grpcwallet-stats-service
  name: grpcwallet-stats-path-matcher
  routeRules:
  - matchRules:
    - prefixMatch: /
      headerMatches:
      - headerName: membership
        exactMatch: premium
    priority: 0
    service: projects/$PROJECT_ID/global/backendServices/grpcwallet-stats-premium-service
- defaultService: projects/$PROJECT_ID/global/backendServices/grpcwallet-wallet-v1-service
  name: grpcwallet-wallet-path-matcher
  routeRules:
  - matchRules:
    - fullPathMatch: /grpc.examples.wallet.Wallet/FetchBalance
    priority: 0
    routeAction:
      weightedBackendServices:
      - backendService: projects/$PROJECT_ID/global/backendServices/grpcwallet-wallet-v2-service
        weight: 40
      - backendService: projects/$PROJECT_ID/global/backendServices/grpcwallet-wallet-v1-service
        weight: 60
  - matchRules:
    - prefixMatch: /grpc.examples.wallet.Wallet/
    priority: 1
    routeAction:
      weightedBackendServices:
      - backendService: projects/$PROJECT_ID/global/backendServices/grpcwallet-wallet-v2-service
        weight: 100
EOF

Creating the target proxy and forwarding rule

In this section you create the target gRPC proxy and a forwarding rule.

The target gRPC proxy references the URL map you created in the previous step. The flag --validate-for-proxyless enables configuration checks so that you do not accidentally enable a feature that is not compatible with proxyless gRPC deployments.

gcloud

  1. Create the target gRPC proxy.

    gcloud compute target-grpc-proxies create grpcwallet-proxy \
  --url-map grpcwallet-url-map \
  --validate-for-proxyless

The forwarding rule references the target gRPC proxy you created. The load balancing scheme is set to INTERNAL_SELF_MANAGED to indicate that this forwarding rule is used by Traffic Director. It must be a global forwarding rule. The IP address is set to 0.0.0.0 because a proxyless gRPC client resolves the hostname:port in the target URI by sending an LDS request to Traffic Director instead of doing a DNS lookup. For more information, see Name resolution scheme.

When a port is not specified in the target URI, the default value is 80. For example, a target URI xds:///foo.myservice:8080 matches a forwarding rule that is configured with port 8080. In this example the forwarding rule is configured with port 80.

gcloud

  1. Create the forwarding rule.

    gcloud compute forwarding-rules create grpcwallet-forwarding-rule \
   --global \
   --load-balancing-scheme=INTERNAL_SELF_MANAGED \
   --address=0.0.0.0 --address-region=us-central1 \
   --target-grpc-proxy=grpcwallet-proxy \
   --ports 80 \
   --network default

Verifying the configuration

When the configuration process is complete, verify that the backend services you configured are available by checking the Traffic Director page in the Console. Confirm that the backend services and associated backends are reported as healthy.

Verifying the routing configuration

In this section, you verify that the routing configuration is working correctly. You use the grpcurl tool to test the configuration.

gcloud

  1. Create a client VM on which you run the clients to test the service.

    gcloud compute instances create grpc-wallet-client \
  --zone us-central1-a \
  --scopes=https://www.googleapis.com/auth/cloud-platform \
  --image-family=debian-10 \
  --image-project=debian-cloud \
  --metadata-from-file=startup-script=<(echo '#! /bin/bash
set -e
export GRPC_XDS_BOOTSTRAP=/run/td-grpc-bootstrap.json
# Expose bootstrap variable to SSH connections
echo export GRPC_XDS_BOOTSTRAP=$GRPC_XDS_BOOTSTRAP | sudo tee /etc/profile.d/grpc-xds-bootstrap.sh
# Create the bootstrap file
curl -L https://storage.googleapis.com/traffic-director/td-grpc-bootstrap-0.9.0.tar.gz | tar -xz
./td-grpc-bootstrap-0.9.0/td-grpc-bootstrap | tee $GRPC_XDS_BOOTSTRAP')

  2. Access the VM using SSH.

To access the services without a sidecar proxy, the gRPC client needs to use the xds name resolution scheme. This scheme tells the gRPC library used in the client to use an xDS server to resolve the hostname. To do this, a bootstrap configuration is required. The startup script in the previous section sets the GRPC_XDS_BOOTSTRAP environment variable and uses a helper script to generate the bootstrap file. The values for TRAFFICDIRECTOR_GCP_PROJECT_NUMBER, TRAFFICDIRECTOR_NETWORK_NAME, and zone in the generated bootstrap file are obtained from the metadata server that knows these details about your Compute Engine VM instances. You can provide these values to the helper script manually using the -gcp-project-number and -vpc-network-name options.

Verifying the configuration using the grpcurl tool

First, install the grpcurl tool.

curl -L https://github.com/fullstorydev/grpcurl/releases/download/v1.6.1/grpcurl_1.6.1_linux_x86_64.tar.gz | tar -xz
chmod +x grpcurl

Next, run the following commands to verify that the Account, Stats, and Wallet services are running.

$ ./grpcurl -plaintext xds:///account.grpcwallet.io list
grpc.examples.wallet.account.Account
grpc.health.v1.Health
grpc.reflection.v1alpha.ServerReflection

$ ./grpcurl -plaintext -d '{"token": "2bd806c9"}' xds:///account.grpcwallet.io grpc.examples.wallet.account.Account/GetUserInfo
{
  "name": "Alice",
  "membership": "PREMIUM"
}

$ ./grpcurl -plaintext -H 'authorization:2bd806c9' -H 'membership:premium' xds:///stats.grpcwallet.io grpc.examples.wallet.stats.Stats/FetchPrice
{
  "price": "10295"
}

$ ./grpcurl -plaintext -H 'authorization:2bd806c9' -H 'membership:premium' -d '{"include_balance_per_address": true}' xds:///wallet.grpcwallet.io grpc.examples.wallet.Wallet/FetchBalance
{
  "balance": "5089953"
}

Verifying with grpc-wallet clients

Use the following language-specific instructions to verify the configuration. The commands send multiple RPCs, some with extra metadata, to show that requests are routed to backend services based on the matching rules from the URL map. The command also prints the backend hostname for each response to show which backend service the request was routed to.

C++

  1. To verify the service with a gRPC C++ client, run the following:

    sudo apt-get update
sudo apt-get install -y build-essential git

git clone https://github.com/GoogleCloudPlatform/traffic-director-grpc-examples.git
cd traffic-director-grpc-examples/cpp
../tools/bazel build :client

# This command calls FetchBalance from the Wallet service in a loop, to demonstrate that
# FetchBalance gets responses from wallet-v1 (40%) and wallet-v2 (60%).
../bazel-bin/cpp/client balance --wallet_server="xds:///wallet.grpcwallet.io" --unary_watch=true

# This command calls the streaming RPC WatchBalance from the Wallet
# service. The RPC path matches the service prefix, so all requests are sent to wallet-v2.
../bazel-bin/cpp/client balance --wallet_server="xds:///wallet.grpcwallet.io" --watch=true

# This command calls WatchPrice from the Stats service. It sends the
# user's membership (premium or not) in metadata. Premium requests are
# all sent to stats-premium, and get faster responses. Alice's requests
# always go to premium, and Bob goes to regular.
../bazel-bin/cpp/client price --stats_server="xds:///stats.grpcwallet.io" --watch=true --user=Bob
../bazel-bin/cpp/client price --stats_server="xds:///stats.grpcwallet.io" --watch=true --user=Alice

Go

  1. To verify the service with a gRPC Go client, run the following:

    sudo apt-get update
sudo apt install -y golang git

git clone https://github.com/GoogleCloudPlatform/traffic-director-grpc-examples.git
cd traffic-director-grpc-examples/go/wallet_client
go build .

# This command calls FetchBalance from the Wallet service in a loop,
# to demonstrate that FetchBalance gets responses from wallet-v1 (40%)
# and wallet-v2 (60%).
./wallet_client balance --wallet_server="xds:///wallet.grpcwallet.io" --unary_watch

# This command calls the streaming RPC WatchBalance from the Wallet
# service. The RPC path matches the service prefix, so all requests
# are sent to wallet-v2.
./wallet_client balance --wallet_server="xds:///wallet.grpcwallet.io" --watch

# This command calls WatchPrice from the Stats service. It sends the
# user's membership (premium or not) in metadata. Premium requests are
# all sent to stats-premium, and get faster responses. Alice's requests
# always go to premium and Bob goes to regular.
./wallet_client price --stats_server="xds:///stats.grpcwallet.io" --watch --user=Bob
./wallet_client price --stats_server="xds:///stats.grpcwallet.io" --watch --user=Alice

Java

  1. To verify the service with a gRPC Java client, run the following:

    sudo apt-get update
sudo apt-get install -y openjdk-11-jdk-headless git

git clone https://github.com/GoogleCloudPlatform/traffic-director-grpc-examples.git
cd traffic-director-grpc-examples/java
./gradlew installDist

# This command calls FetchBalance from the Wallet service in a loop,
# to demonstrate that FetchBalance gets responses from wallet-v1 (40%)
# and wallet-v2(60%).
./build/install/wallet/bin/client balance --wallet_server="xds:///wallet.grpcwallet.io" --unary_watch=true

# This command calls the streaming RPC WatchBalance from the Wallet
# service. The RPC path matches the service prefix, so all requests are
# sent to wallet-v2.
./build/install/wallet/bin/client balance --wallet_server="xds:///wallet.grpcwallet.io" --watch=true

# This command calls WatchPrice from the Stats service. It sends the
# user's membership (premium or not) in metadata. Premium requests are
# all sent to stats-premium, and get faster responses. Alice's requests
# always go to premium and Bob goes to regular.
./build/install/wallet/bin/client price --stats_server="xds:///stats.grpcwallet.io" --watch=true --user=Bob
./build/install/wallet/bin/client price --stats_server="xds:///stats.grpcwallet.io" --watch=true --user=Alice

Cleaning up the resources

Run this script to clean up the resources.

bash <(curl -s https://raw.githubusercontent.com/GoogleCloudPlatform/traffic-director-grpc-examples/master/scripts/cleanup.sh)
gcloud compute instances delete grpc-wallet-client --zone us-central1-a -q

What's next

If you encountered unexpected behavior during the configuration process, see Troubleshooting proxyless Traffic Director deployments and Traffic Director limitations with proxyless gRPC applications.