Set up an Envoy sidecar service mesh on GKE

This page describes how to set up an Envoy sidecar service mesh on GKE.

Prerequisites

As a starting point, this guide assumes that you have already:

Set up the Service

Create a sample HTTP service:

kubectl apply -f - <<EOF
kind: Namespace
apiVersion: v1
metadata:
  name: sidecar-example
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: whereami
  namespace: sidecar-example
spec:
  replicas: 2
  selector:
    matchLabels:
      app: whereami
  template:
    metadata:
      labels:
        app: whereami
    spec:
      containers:
      - name: whereami
        image: us-docker.pkg.dev/google-samples/containers/gke/whereami:v1
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: whereami
  namespace: sidecar-example
spec:
  selector:
    app: whereami
  ports:
  - port: 8080
    targetPort: 8080
EOF

Create a baseline HTTPRoute for the service:

apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
  name: whereami-route
  namespace: sidecar-example
spec:
  parentRefs:
  - name: whereami
    kind: Service
    group: ""
  rules:
  - backendRefs:
    - name: whereami
      port: 8080
EOF

Alternatively, the following manifest describes a sample gRPC Service:

apiVersion: v1
kind: Service
metadata:
  name: sample-service
  namespace: sample-ns
  annotations:
    networking.gke.io/app-protocols: '{"50051": "HTTP2"}' # 50051 is backendref.port
spec:
  ports:
  - port: 50051
    targetPort: 50051

Set up the Client

Create a client:

kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    run: client
  name: client
  namespace: sidecar-example
spec:
  replicas: 1
  selector:
    matchLabels:
      run: client
  template:
    metadata:
      labels:
        run: client
    spec:
      containers:
      - name: client
        image: curlimages/curl
        command:
        - sh
        - -c
        - while true; do sleep 1; done
EOF

Verify that the client Pod has an Envoy sidecar container automatically injected:
```
kubectl get pods -n sidecar-example -l run=client
```
The output is similar to:
```
NAME                    READY   STATUS    RESTARTS   AGE
client-xxxx             2/2     Running   0          20s
```
Wait for the client to be ready and have the Status Running before continuing.

Verify Envoy-sidecar service mesh setup. The following command sends a request to the whereami service from the client

CLIENT_POD=$(kubectl get pod -n sidecar-example -l run=client -o=jsonpath='{.items[0].metadata.name}')

# The VIP where the following request will be sent. Because all requests
# from the client container are redirected to the Envoy proxy sidecar, you
# can use any IP address, including 10.0.0.2, 192.168.0.1, and others.
VIP='10.0.0.1'

TEST_CMD="curl -v -H 'host: whereami.sidecar-example.svc.cluster.local' $VIP"

kubectl exec -it $CLIENT_POD -n sidecar-example -c client -- /bin/sh -c "$TEST_CMD"

The output is similar to:

< Trying 10.0.0.1:80...
< Connected to 10.0.0.1 (10.0.0.1) port 80 (#0)
< GET / HTTP/1.1
< Host: whereami
< User-Agent: curl/7.82.0-DEV
< Accept: */*
<
< Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< content-type: application/json
< content-length: 318
< access-control-allow-origin: *
< server: envoy
< date: Tue, 12 Apr 2022 22:30:13 GMT
<
{
  "cluster_name": "${CLUSTER_NAME}",
  "location": "${LOCATION}",
  "host_header": "whereami",
  ...
}