Installing Anthos Service Mesh on premises

Anthos Service Mesh, is an Istio-compatible framework for connecting, monitoring, and securing services running on Google Kubernetes Engine (GKE) and Anthos GKE on-prem. It lets you create a network of deployed services with load balancing, service-to-service authentication, monitoring, and more, without requiring any changes in service code. Anthos Service Mesh automatically injects a sidecar proxy for each of your applications' pods. The sidecar proxy intercepts all network traffic to and from the pods. Anthos Service Mesh also configures an ingress gateway to manage inbound traffic to the mesh. You can use open source Istio APIs to configure policies that are enforced on sidecars and gateways.

This guide explains how to install Anthos Service Mesh version 1.4.7-asm.0 on an existing Anthos GKE on-prem cluster.

The installation enables the supported core Istio features on your GKE on-prem cluster. Note that the Anthos Service Mesh managed components aren't currently supported on GKE on-prem.

About the control plane components

GKE on-prem comes with the following Istio components preinstalled:

  • Citadel is installed in the kube-system namespace.
  • Pilot and the Istio Ingress Gateway are installed in the gke-system namespace.

GKE on-prem uses these components to enable ingress and to secure communication between Google-controlled components. If you only need ingress functionality, you don't need to install OSS Istio or Anthos Service Mesh. For more information on configuring ingress, see Enabling ingress.

When you install Anthos Service Mesh, its components are installed in the istio-system namespace. Because the Anthos Service Mesh components are in a different namespace, they don't conflict with the GKE on-prem preinstalled Istio components.

Before you begin

Review the following requirements and restrictions before beginning the setup.

Requirements

  • You must have an Anthos trial license and subscription. See the Anthos Pricing guide for details.

  • Make sure the user cluster that you install Anthos Service Mesh on has at least 4 vCPUs, 15 GB memory, and 4 replicas.

  • Review Requirements for Pods and Services before you deploy workloads.

  • Make sure your cluster version is listed in Supported environments. To check your cluster version:

    gkectl version
    

    Output like the following is displayed:

    1.2.0-gke.6 (git-0912663b0)
    

Restrictions

Only one installation of Anthos Service Mesh per Google Cloud project is supported. Multiple mesh deployments in a single project aren't supported.

Setting up your environment

Install the following tools if you don't already have them:

  1. Install and initialize the Cloud SDK (the gcloud command-line tool).

    If you already have the Cloud SDK installed, make sure to update the components:

    gcloud components update
    
  2. Install kubectl:

    gcloud components install kubectl
    

Preparing to install Anthos Service Mesh

Before installing Anthos Service Mesh, you need to:

  • Set required credentials and permissions.
  • Download and extract the Anthos Service Mesh installation file.

Set credentials and permissions

  1. Ensure that you have kubectl for the GKE on-prem user cluster where you want to install Anthos Service Mesh. Note that you can install Anthos Service Mesh only on a GKE on-prem user cluster, not an admin cluster.
  2. Grant cluster admin permissions to the current user. You need these permissions to create the necessary role based access control (RBAC) rules for Anthos Service Mesh:
    kubectl create clusterrolebinding cluster-admin-binding \
      --clusterrole=cluster-admin \
      --user="$(gcloud config get-value core/account)"

    If you see the "cluster-admin-binding" already exists error, you can safely ignore it and continue with the existing cluster-admin-binding.

Download the installation file

Linux

  1. Download the Anthos Service Mesh installation file to your current working directory:
    curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.4.7-asm.0-linux.tar.gz
  2. Download the signature file and use openssl to verify the signature:
    curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.4.7-asm.0-linux.tar.gz.1.sig
    openssl dgst -verify - -signature istio-1.4.7-asm.0-linux.tar.gz.1.sig istio-1.4.7-asm.0-linux.tar.gz <<'EOF'
    -----BEGIN PUBLIC KEY-----
    MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEWZrGCUaJJr1H8a36sG4UUoXvlXvZ
    wQfk16sxprI2gOJ2vFFggdq3ixF2h4qNBt0kI7ciDhgpwS8t+/960IsIgw==
    -----END PUBLIC KEY-----
    EOF

    The expected output is: Verified OK

  3. Extract the contents of the file to any location on your file system. For example, to extract the contents to the current working directory:
    tar xzf istio-1.4.7-asm.0-linux.tar.gz

    The command creates an installation directory in your current working directory named istio-1.4.7-asm.0 that contains:

    • Sample applications in samples
    • The following tools in the bin directory:
      • istioctl: You use istioctl to install Anthos Service Mesh.
      • asmctl: You use asmctl to help validate your security configuration after installing Anthos Service Mesh.

  4. Ensure that you're in the Anthos Service Mesh installation's root directory.
    cd istio-1.4.7-asm.0
  5. For convenience, add the tools in the /bin directory to your PATH:
    export PATH=$PWD/bin:$PATH

Mac OS

  1. Download the Anthos Service Mesh installation file to your current working directory:
    curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.4.7-asm.0-osx.tar.gz
  2. Download the signature file and use openssl to verify the signature:
    curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.4.7-asm.0-osx.tar.gz.1.sig
    openssl dgst -verify - -signature istio-1.4.7-asm.0-osx.tar.gz.1.sig istio-1.4.7-asm.0-osx.tar.gz <<'EOF'
    -----BEGIN PUBLIC KEY-----
    MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEWZrGCUaJJr1H8a36sG4UUoXvlXvZ
    wQfk16sxprI2gOJ2vFFggdq3ixF2h4qNBt0kI7ciDhgpwS8t+/960IsIgw==
    -----END PUBLIC KEY-----
    EOF

    The expected output is: Verified OK

  3. Extract the contents of the file to any location on your file system. For example, to extract the contents to the current working directory:
    tar xzf istio-1.4.7-asm.0-osx.tar.gz

    The command creates an installation directory in your current working directory named istio-1.4.7-asm.0 that contains:

    • Sample applications in samples
    • The istioctl client binary in the bin directory. You can use istioctl to manually inject Envoy as a sidecar proxy and to create routing rules and policies.

  4. Ensure that you're in the Anthos Service Mesh installation's root directory.
    cd istio-1.4.7-asm.0
  5. For convenience, add the istioctl client to your PATH:
    export PATH=$PWD/bin:$PATH

Windows

  1. Download the Anthos Service Mesh installation file to your current working directory:
    curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.4.7-asm.0-win.zip
  2. Download the signature file and use openssl to verify the signature:
    curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.4.7-asm.0-win.zip.1.sig
    openssl dgst -verify - -signature istio-1.4.7-asm.0-win.zip.1.sig istio-1.4.7-asm.0-win.zip <<'EOF'
    -----BEGIN PUBLIC KEY-----
    MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEWZrGCUaJJr1H8a36sG4UUoXvlXvZ
    wQfk16sxprI2gOJ2vFFggdq3ixF2h4qNBt0kI7ciDhgpwS8t+/960IsIgw==
    -----END PUBLIC KEY-----
    EOF

    The expected output is: Verified OK

  3. Extract the contents of the file to any location on your file system. There is a directory named istio-1.4.7-asm.0 that contains:
    • Sample applications in samples
    • The istioctl client binary in the bin directory. You can use istioctl to manually inject Envoy as a sidecar proxy and to create routing rules and policies.

  4. Ensure that you're in the Anthos Service Mesh installation's root directory.
    cd istio-1.4.7-asm.0
  5. For convenience, add the istioctl client to your PATH.

Setup namespace and certificate

  1. Create a namespace called istio-system:

    kubectl create namespace istio-system

  2. Copy the required root certificate into istio-system for Citadel. This is required for Anthos GKE on-prem clusters:

    kubectl get secret istio-ca-secret --namespace=kube-system --export -o yaml | kubectl apply --validate=false --namespace=istio-system -f -

Installing Anthos Service Mesh

This section explains how to install Anthos Service Mesh and enable the Supported default features listed on the Supported features page. For information on enabling the Supported optional features, see Enabling optional features.

To install Anthos Service Mesh:

Choose one of the following commands to install Anthos Service Mesh in PERMISSIVE mutual TLS (mTLS) authentication mode or STRICT mTLS mode.

PERMISSIVE mTLS

istioctl manifest apply --set profile=asm-onprem

STRICT mTLS

istioctl manifest apply --set profile=asm-onprem \
  --set values.global.mtls.enabled=true

If you are upgrading from open source Istio 1.4.x or an earlier version of Anthos Service Mesh, the upgrade requires redeploying the control plane components, and the installation takes about 5 to 10 minutes to complete. The old control plane components are terminated and then deleted as the new components are installed. You can check the progress by looking at the value in the AGE column of the workloads.

kubectl get pod -n istio-system

Example output:

NAME                                     READY   STATUS        RESTARTS   AGE
istio-galley-76d684bf9-jwz65             2/2     Running       0          5m36s
istio-ingressgateway-5bfdf7c586-v6wxx    2/2     Terminating   0          25m
istio-ingressgateway-7b598c5557-b88md    2/2     Running       0          5m44s
istio-pilot-5dc4bc4dbf-ds5dh             2/2     Running       0          5m30s
istio-pilot-74665549c5-7r6qh             2/2     Terminating   0          25m
istio-sidecar-injector-7ddff4b99-b76l7   1/1     Running       0          5m36s
promsd-6d4d5b7c5c-dgnd7                  2/2     Running       0          5m30s

In this example, there are two instances of istio-ingressgateway and istio-pilot. The instances with 25min the AGE column are being terminated. All the other components are newly deployed from the ASM 1.4.7-asm.0 package.

Wait for the deployment to finish:

kubectl wait --for=condition=available --timeout=600s deployment --all -n istio-system

Output:

deployment.extensions/istio-galley condition met
deployment.extensions/istio-ingressgateway condition met
deployment.extensions/istio-pilot condition met
deployment.extensions/istio-sidecar-injector condition met
deployment.extensions/promsd condition met

Verifying the installation

Check that the control plane Pods in istio-system are up:

kubectl get pod -n istio-system

Expect to see output similar to the following:

NAME                                      READY   STATUS      RESTARTS   AGE
istio-citadel-85f4d775cd-dmpj2            1/1     Running     0          18m
istio-galley-5c65896ff7-m2pls             2/2     Running     0          18m
istio-ingressgateway-587cd459f-q6hqt      2/2     Running     0          18m
istio-pilot-9db77b99f-7wfb6               2/2     Running     0          18m
istio-sidecar-injector-69c4d9f875-dt8rn   1/1     Running     0          18m
promsd-55f464d964-lqs7w                   2/2     Running     0          18m

Configuring an external IP address

The default Anthos Service Mesh installation assumes that an external IP address is automatically allocated for LoadBalancer services. This is not true in GKE on-prem clusters. Because of this, you need to allocate an IP address manually for the Anthos Service Mesh ingress Gateway resource.

To configure an external IP address, follow one of the sections below, depending on your cluster's load balancing mode:

Integrated load balancing mode

  1. Open the istio-ingressgateway Service's configuration:

    kubectl edit svc -n istio-system istio-ingressgateway
    

    The configuration for the istio-ingressgateway Service opens in your shell's default text editor.

  2. In the file, add the following line under the specification (spec) block:

    loadBalancerIP: <your static external IP address>
    

    For example:

    spec:
     loadBalancerIP: 203.0.113.1
    
  3. Save the file.

Manual load balancing mode

To expose a service of type NodePort with a VIP on your selected load balancer, you need to find out the nodePort values first:

  1. View the istio-ingressgateway Service's configuration in your shell:

    kubectl get svc -n istio-system istio-ingressgateway -o yaml
    

    Each of the ports for Anthos Service Mesh's gateways are displayed. The command output is similar to the following:

     ...
     ports:
     - name: status-port
       nodePort: 30391
       port: 15020
       protocol: TCP
       targetPort: 15020
     - name: http2
       nodePort: 31380
       port: 80
       protocol: TCP
       targetPort: 80
     - name: https
       nodePort: 31390
       port: 443
       protocol: TCP
       targetPort: 443
     - name: tcp
       nodePort: 31400
       port: 31400
       protocol: TCP
       targetPort: 31400
     - name: https-kiali
       nodePort: 31073
       port: 15029
       protocol: TCP
       targetPort: 15029
     - name: https-prometheus
       nodePort: 30253
       port: 15030
       protocol: TCP
       targetPort: 15030
     - name: https-grafana
       nodePort: 30050
       port: 15031
       protocol: TCP
       targetPort: 15031
     - name: https-tracing
       nodePort: 31204
       port: 15032
       protocol: TCP
       targetPort: 15032
     - name: tls
       nodePort: 30158
       port: 15443
       protocol: TCP
       targetPort: 15443
     ...
    
  2. Expose these ports through your load balancer.

    For example, the service port named http2 has port 80 and nodePort 31380. Suppose the node addresses for your user cluster are 192.168.0.10, 192.168.0.11, and 192.168.0.12, and your load balancer's VIP is 203.0.113.1.

    Configure your load balancer so that traffic sent to 203.0.113.1:80 is forwarded to 192.168.0.10:31380, 192.168.0.11:31380, or 192.168.0.12:31380. You can select the service ports that you want to expose on this given VIP.

Injecting sidecar proxies

Anthos Service Mesh uses sidecar proxies to enhance network security, reliability, and observability. With Anthos Service Mesh, these functions are abstracted away from the application's primary container and implemented in a common out-of-process proxy delivered as a separate container in the same Pod.

Any workloads that were running on your cluster before you installed Anthos Service Mesh need to have the sidecar proxy injected or updated so they have the current Anthos Service Mesh version. Before you deploy new workloads, make sure to configure sidecar proxy injection so that Anthos Service Mesh can monitor and secure traffic. You can enable automatic sidecar injection with one command, for example:

kubectl label namespace NAMESPACE istio-injection=enabled --overwrite

where NAMESPACE is the name of the namespace for your application's services or default if you didn't explicitly create a namespace.

For more information, see Injecting sidecar proxies.

What's next?