Installing Anthos Service Mesh on premises

This guide explains how to do a clean installation of Anthos Service Mesh version 1.7.8-asm.10 on an existing GKE on VMware cluster. If you have a previous version of Anthos Service Mesh installed, refer to Upgrading Anthos Service Mesh on GKE on VMware. The installation enables the supported features on your cluster for the asm-multicloud configuration profile.

About the control plane components

GKE on VMware comes with the following Istio components preinstalled:

  • Citadel is installed in the kube-system namespace.
  • Pilot and the Istio Ingress Gateway are installed in the gke-system namespace.

GKE on VMware uses these components to enable ingress and to secure communication between Google-controlled components. If you only need ingress functionality, you don't need to install OSS Istio or Anthos Service Mesh. For more information on configuring ingress, see Enabling ingress.

When you install Anthos Service Mesh, its components are installed in the istio-system namespace. Because the Anthos Service Mesh components are in a different namespace, they don't conflict with the GKE on VMware preinstalled Istio components.

Before you begin

Review the following requirements before you begin setup.

Requirements

  • You must have an Anthos subscription. Alternatively, a pay-as-you-go billing option is available for GKE Enterprise on Google Cloud only. For more information, see the GKE Enterprise Pricing guide.

  • Verify that your user cluster that you install Anthos Service Mesh on has at least 4 vCPUs, 15 GB memory, and 4 nodes.

  • You must name your service ports using the following syntax: name: protocol[-suffix] where the square brackets indicate an optional suffix that must start with a dash. For more information, see Naming service ports.

  • Verify that your cluster version is listed in Supported environments. To check your cluster version, you can use the gkectl command line tool. If you don't have gkectl installed, see GKE on-prem downloads.

    gkectl version

Setting up your environment

You need the following tools on the machine you want to install Anthos Service Mesh from. Note that you can install Anthos Service Mesh only on a user cluster, not an admin cluster.

After installing the Google Cloud CLI:

  1. Authenticate with the Google Cloud CLI:

    gcloud auth login

  2. Update the components:

    gcloud components update

  3. Install kubectl:

    gcloud components install kubectl

  4. If you want to deploy and test your installation with the Online Boutique sample application, install kpt:

    gcloud components install kpt

  5. Switch context to your user cluster (if necessary):

    kubectl config use-context CLUSTER_NAME

  6. Grant cluster admin permissions to your user account (your Google Cloud login email address). You need these permissions to create the necessary role based access control (RBAC) rules for Anthos Service Mesh:

    kubectl create clusterrolebinding cluster-admin-binding \
  --clusterrole=cluster-admin \
  --user=USER_ACCOUNT

Downloading the installation file

    Linux

  1. Download the Anthos Service Mesh installation file to your current working directory: 
    curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.7.8-asm.10-linux-amd64.tar.gz
  2. Download the signature file and use openssl to verify the signature: 
    curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.7.8-asm.10-linux-amd64.tar.gz.1.sig
openssl dgst -verify /dev/stdin -signature istio-1.7.8-asm.10-linux-amd64.tar.gz.1.sig istio-1.7.8-asm.10-linux-amd64.tar.gz <<'EOF'
-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEWZrGCUaJJr1H8a36sG4UUoXvlXvZ
wQfk16sxprI2gOJ2vFFggdq3ixF2h4qNBt0kI7ciDhgpwS8t+/960IsIgw==
-----END PUBLIC KEY-----
EOF

    The expected output is: Verified OK

  3. Extract the contents of the file to any location on your file system. For example, to extract the contents to the current working directory: 
    tar xzf istio-1.7.8-asm.10-linux-amd64.tar.gz

    The command creates an installation directory in your current working directory named istio-1.7.8-asm.10 that contains:

    • Sample applications in the samples directory.
    • The istioctl command-line tool that you use to install Anthos Service Mesh is in the bin directory.
    • The Anthos Service Mesh configuration profiles are in the install/kubernetes/operator/profiles directory.

    4. Mac OS

  4. Download the Anthos Service Mesh installation file to your current working directory: 
    curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.7.8-asm.10-osx.tar.gz
  5. Download the signature file and use openssl to verify the signature: 
    curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.7.8-asm.10-osx.tar.gz.1.sig
openssl dgst -sha256 -verify /dev/stdin -signature istio-1.7.8-asm.10-osx.tar.gz.1.sig istio-1.7.8-asm.10-osx.tar.gz <<'EOF'
-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEWZrGCUaJJr1H8a36sG4UUoXvlXvZ
wQfk16sxprI2gOJ2vFFggdq3ixF2h4qNBt0kI7ciDhgpwS8t+/960IsIgw==
-----END PUBLIC KEY-----
EOF

    The expected output is: Verified OK

  6. Extract the contents of the file to any location on your file system. For example, to extract the contents to the current working directory: 
    tar xzf istio-1.7.8-asm.10-osx.tar.gz

    The command creates an installation directory in your current working directory named istio-1.7.8-asm.10 that contains:

    • Sample applications in the samples directory.
    • The istioctl command-line tool that you use to install Anthos Service Mesh is in the bin directory.
    • The Anthos Service Mesh configuration profiles are in the install/kubernetes/operator/profiles directory.

    7. Windows

  7. Download the Anthos Service Mesh installation file to your current working directory: 
    curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.7.8-asm.10-win.zip
  8. Download the signature file and use openssl to verify the signature: 
    curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.7.8-asm.10-win.zip.1.sig
openssl dgst -verify - -signature istio-1.7.8-asm.10-win.zip.1.sig istio-1.7.8-asm.10-win.zip <<'EOF'
-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEWZrGCUaJJr1H8a36sG4UUoXvlXvZ
wQfk16sxprI2gOJ2vFFggdq3ixF2h4qNBt0kI7ciDhgpwS8t+/960IsIgw==
-----END PUBLIC KEY-----
EOF

    The expected output is: Verified OK

  9. Extract the contents of the file to any location on your file system. For example, to extract the contents to the current working directory: 
    tar xzf istio-1.7.8-asm.10-win.zip

    The command creates an installation directory in your current working directory named istio-1.7.8-asm.10 that contains:

    • Sample applications in the samples directory.
    • The istioctl command-line tool that you use to install Anthos Service Mesh is in the bin directory.
    • The Anthos Service Mesh configuration profiles are in the install/kubernetes/operator/profiles directory.

  10. Ensure that you're in the Anthos Service Mesh installation's root directory. 
    cd istio-1.7.8-asm.10
  11. For convenience, add the tools in the /bin directory to your PATH: 
    export PATH=$PWD/bin:$PATH

Create the istio-system namespace

Create a namespace called istio-system for the control plane components: 

kubectl create namespace istio-system

Configure the validating webhook

When you install Anthos Service Mesh, you set a revision label on istiod. You need to set the same revision on the validating webhook.

Copy the following YAML to a file called istiod-service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: istiod
  namespace: istio-system
  labels:
    istio.io/rev: asm-178-10
    app: istiod
    istio: pilot
    release: istio
spec:
  ports:
    - port: 15010
      name: grpc-xds # plaintext
      protocol: TCP
    - port: 15012
      name: https-dns # mTLS with k8s-signed cert
      protocol: TCP
    - port: 443
      name: https-webhook # validation and injection
      targetPort: 15017
      protocol: TCP
    - port: 15014
      name: http-monitoring # prometheus stats
      protocol: TCP
  selector:
    app: istiod
    istio.io/rev: asm-178-10

Installing Anthos Service Mesh

  1. Run the following command to install Anthos Service Mesh using the asm-multicloud profile. If you want to enable a supported optional feature, include -f and the YAML filename on the following command line. See Enabling optional features for more information.

    istioctl install \
  --set profile=asm-multicloud \
  --set revision=asm-178-10

    The --set revision argument adds a revision label in the format istio.io/rev=asm-178-10 to istiod. The revision label is used by the automatic sidecar injector webhook to associate injected sidecars with a particular istiod revision. To enable sidecar auto-injection for a namespace, you must label it with a revision that matches the label on istiod.

  2. Configure the validating webhook so that it can locate the istiod service with the revision label:

    kubectl apply -f istiod-service.yaml

    This command creates a service entry that lets the validating webhook automatically check configurations before they are applied.

Auto mutual TLS (auto mTLS) is enabled by default. With auto mTLS, a client sidecar proxy automatically detects if the server has a sidecar. The client sidecar sends mTLS to workloads with sidecars and sends plain text traffic to workloads without sidecars.

Check the control plane components

Check that the control plane pods in istio-system are running:

kubectl get pod -n istio-system

Expected output is similar to the following:

NAME                                      READY   STATUS      RESTARTS   AGE
istio-ingressgateway-74cc894bfd-786rg     1/1     Running     0          7m19s
istiod-78cdbbbdb-d7tps                    1/1     Running     0          7m36s
promsd-576b8db4d6-lqf64                   2/2     Running     1          7m19s

Injecting sidecar proxies

Anthos Service Mesh uses sidecar proxies to enhance network security, reliability, and observability. With Anthos Service Mesh, these functions are abstracted away from the application's primary container and implemented in a common out-of-process proxy delivered as a separate container in the same Pod. To inject your Pods with the sidecar proxy, you configure automatic sidecar proxy injection (auto-injection) by labeling your namespaces with the same revision label that you set on istiod when you installed Anthos Service Mesh.

  • You need to enable auto-injection on any namespaces with workloads that were running on your cluster before you installed Anthos Service Mesh.

  • Before you deploy new workloads, make sure to configure auto-injection so that Anthos Service Mesh can monitor and secure traffic.

To enable auto-injection:

  1. Use the following command to locate the revision label on istiod:

    kubectl -n istio-system get pods -l app=istiod --show-labels

    The output looks similar to the following:

    NAME                                READY   STATUS    RESTARTS   AGE   LABELS
istiod-asm-178-10-5788d57586-bljj4   1/1     Running   0          23h   app=istiod,istio.io/rev=asm-178-10,istio=istiod,pod-template-hash=5788d57586
istiod-asm-178-10-5788d57586-vsklm   1/1     Running   1          23h   app=istiod,istio.io/rev=asm-178-10,istio=istiod,pod-template-hash=5788d57586

    In the output, under the LABELS column, note the value of the istiod revision label, which follows the prefix istio.io/rev=. In this example, the value is asm-178-10.

  2. Apply the revision label and remove the istio-injection label if it exists. In the following command, NAMESPACE is the name of the namespace where you want to enable auto-injection, and REVISION is the revision label you noted in the previous step.

    kubectl label namespace NAMESPACE istio-injection-istio.io/rev=REVISION --overwrite

    You can ignore the message "istio-injection not found" in the output. That means that the namespace didn't previously have the istio-injection label, which you should expect in new installations of Anthos Service Mesh or new deployments. Because auto-injection fails if a namespace has both the istio-injection and the revision label, all kubectl label commands in the Anthos Service Mesh documentation include removing the istio-injection label.

  3. If workloads were running on your cluster before you installed Anthos Service Mesh, restart the Pods to trigger re-injection.

    How you restart Pods depends on your application and the environment the cluster is in. For example, in your staging environment, you might simply delete all the Pods, which causes them to restart. But in your production environment, you might have a process that implements a blue-green deployment so that you can safely restart Pods to avoid traffic interruption.

    You can use kubectl to perform a rolling restart:

    kubectl rollout restart deployment -n NAMESPACE

  4. Verify that your Pods are configured to point to the new version of istiod.

    kubectl get pods -n NAMESPACE -l istio.io/rev=REVISION

Configuring an external IP address

The default Anthos Service Mesh installation assumes that an external IP address is automatically allocated for LoadBalancer services. This is not true in GKE on VMware. Because of this, you need to allocate an IP address manually for the Anthos Service Mesh ingress Gateway resource.

To configure an external IP address, follow one of the sections below, depending on your cluster's load balancing mode:

Configure integrated load balancing mode

  1. Open the istio-ingressgateway Service's configuration:

    kubectl edit svc -n istio-system istio-ingressgateway

    The configuration for the istio-ingressgateway Service opens in your shell's default text editor.

  2. In the file, add the following line under the specification (spec) block:

    loadBalancerIP: <your static external IP address>

    For example:

    spec:
 loadBalancerIP: 203.0.113.1

  3. Save the file.

Configure manual load balancing mode

To expose a service of type NodePort with a VIP on your selected load balancer, you need to find out the nodePort values first:

  1. View the istio-ingressgateway Service's configuration in your shell:

    kubectl get svc -n istio-system istio-ingressgateway -o yaml

    Each of the ports for Anthos Service Mesh's gateways are displayed. The command output is similar to the following:

     ...
 ports:
 - name: status-port
   nodePort: 30391
   port: 15020
   protocol: TCP
   targetPort: 15020
 - name: http2
   nodePort: 31380
   port: 80
   protocol: TCP
   targetPort: 80
 - name: https
   nodePort: 31390
   port: 443
   protocol: TCP
   targetPort: 443
 - name: tcp
   nodePort: 31400
   port: 31400
   protocol: TCP
   targetPort: 31400
 - name: https-kiali
   nodePort: 31073
   port: 15029
   protocol: TCP
   targetPort: 15029
 - name: https-prometheus
   nodePort: 30253
   port: 15030
   protocol: TCP
   targetPort: 15030
 - name: https-grafana
   nodePort: 30050
   port: 15031
   protocol: TCP
   targetPort: 15031
 - name: https-tracing
   nodePort: 31204
   port: 15032
   protocol: TCP
   targetPort: 15032
 - name: tls
   nodePort: 30158
   port: 15443
   protocol: TCP
   targetPort: 15443
 ...

  2. Expose these ports through your load balancer.

    For example, the service port named http2 has port 80 and nodePort 31380. Suppose the node addresses for your user cluster are 192.168.0.10, 192.168.0.11, and 192.168.0.12, and your load balancer's VIP is 203.0.113.1.

    Configure your load balancer so that traffic sent to 203.0.113.1:80 is forwarded to 192.168.0.10:31380, 192.168.0.11:31380, or 192.168.0.12:31380. You can select the service ports that you want to expose on this given VIP.

What's next?