Installing on Anthos clusters on AWS

Beta

This product or feature is covered by the Pre-GA Offerings Terms of the Google Cloud Platform Terms of Service. Pre-GA products and features may have limited support, and changes to pre-GA products and features may not be compatible with other pre-GA versions. For more information, see the launch stage descriptions.

This guide explains how to do a clean installation of Anthos Service Mesh version 1.6.14-asm.1 on Anthos clusters on AWS.

With Anthos Service Mesh, the Istio Ingress Gateway comes preinstalled. But if you prefer to use an ingress controller, you can use Anthos Service Mesh to set up a Kubernetes Ingress resource. This guide shows you how to install Anthos Service Mesh and optionally set up a Kubernetes Ingress resource.

Before you begin

Before you begin the Anthos Service Mesh installation, make sure you have performed the following tasks:

• Install a management service.
• Create a user cluster.
• Set up AWS networking.
• Review the following requirements and restrictions before beginning the setup.

Requirements

• Make sure the user cluster that you install Anthos Service Mesh on has at least 4 vCPUs, 15 GB memory, and 4 replicas.

• Review Naming service ports before you deploy workloads.

• Make sure your cluster version is listed in Supported environments.

Restrictions

A Google Cloud project can only have one mesh associated with it.

Set up your environment

You need the following tools on the machine you want to install Anthos Service Mesh from. Note that you can install Anthos Service Mesh only on a user cluster, not an admin cluster.

• The curl command-line tool.
• The Cloud SDK (the gcloud command-line tool).

After installing the Cloud SDK:

1. Authenticate with the Cloud SDK:

   gcloud auth login
   

2. Update the components:

   gcloud components update
   

3. Install kubectl:

   gcloud components install kubectl
   

4. If you want to deploy and test your installation with the Online Boutique sample application, install kpt:

   gcloud components install kpt
   

5. Switch context to your user cluster:

   kubectl config use-context CLUSTER_NAME

6. Grant cluster admin permissions to your user account (your Google Cloud login email address). You need these permissions to create the necessary role based access control (RBAC) rules for Anthos Service Mesh:

   kubectl create clusterrolebinding cluster-admin-binding \
     --clusterrole=cluster-admin \
     --user=USER_ACCOUNT

Download the installation file

1. Download the Anthos Service Mesh installation file to your current working directory:

   curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.6.14-asm.1-linux-amd64.tar.gz

2. Download the signature file and use openssl to verify the signature:

   curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.6.14-asm.1-linux-amd64.tar.gz.1.sig
   openssl dgst -verify - -signature istio-1.6.14-asm.1-linux-amd64.tar.gz.1.sig istio-1.6.14-asm.1-linux-amd64.tar.gz <<'EOF'
   -----BEGIN PUBLIC KEY-----
   MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEWZrGCUaJJr1H8a36sG4UUoXvlXvZ
   wQfk16sxprI2gOJ2vFFggdq3ixF2h4qNBt0kI7ciDhgpwS8t+/960IsIgw==
   -----END PUBLIC KEY-----
   EOF

   The expected output is: Verified OK

3. Extract the contents of the file to any location on your file system. For example, to extract the contents to the current working directory:

   tar xzf istio-1.6.14-asm.1-linux-amd64.tar.gz

   The command creates an installation directory in your current working directory named istio-1.6.14-asm.1. The istio-1.6.14-asm.1/bin directory contains the istioctl command-line tool that you use to install Anthos Service Mesh.

4. Ensure that you're in the Anthos Service Mesh installation's root directory.

   cd istio-1.6.14-asm.1

5. For convenience, add the /bin directory to your PATH:

   export PATH=$PWD/bin:$PATH

Create the istio-system namespace

Create a namespace called istio-system for the control plane components:

kubectl create namespace istio-system

Configure the validating webhook

When you install Anthos Service Mesh, you set a revision label on istiod. You need to set the same revision on the validating webhook.

Copy the following YAML to a file called istiod-service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: istiod
  namespace: istio-system
  labels:
    istio.io/rev: asm-1614-1
    app: istiod
    istio: pilot
    release: istio
spec:
  ports:
    - port: 15010
      name: grpc-xds # plaintext
      protocol: TCP
    - port: 15012
      name: https-dns # mTLS with k8s-signed cert
      protocol: TCP
    - port: 443
      name: https-webhook # validation and injection
      targetPort: 15017
      protocol: TCP
    - port: 15014
      name: http-monitoring # prometheus stats
      protocol: TCP
  selector:
    app: istiod
    istio.io/rev: asm-1614-1

Installing Anthos Service Mesh

This section explains how to install Anthos Service Mesh on your user cluster, which enables the Supported default features listed on the Supported features page for the asm-multicloud profile. You can choose to enable Ingress for the public subnet or the private subnet.

Public

1. Install Anthos Service Mesh:

   istioctl install \
     --set profile=asm-multicloud \
     --set revision=asm-1614-1
   

   The --set revision argument adds a revision label in the format istio.io/rev=asm-1614-1 to istiod. The revision label is used by the automatic sidecar injector webhook to associate injected sidecars with a particular istiod revision. To enable sidecar auto-injection for a namespace, you must label it with a revision that matches the label on istiod.

2. Configure the validating webhook so that it can locate the istiod service with the revision label:

   kubectl apply -f istiod-service.yaml
   

   This command creates a service entry that lets the validating webhook automatically check configurations before they are applied.

Private

In the following, you add the service.beta.kubernetes.io/aws-load-balancer-internal annotation to all services that Anthos Service Mesh creates. When this annotation is present, Anthos clusters on AWS creates private Ingress services.

1. Save the following YAML to a file called istio-operator-internal-lb.yaml:

   apiVersion: install.istio.io/v1alpha1
   kind: IstioOperator
   spec:
     components:
       ingressGateways:
       - enabled: true
         k8s:
           serviceAnnotations:
             service.beta.kubernetes.io/aws-load-balancer-internal: "true"
   

2. Install Anthos Service Mesh:

   istioctl install \
    --set profile=asm-multicloud \
    --set revision=asm-1614-1 \
    -f istio-operator-internal-lb.yaml
   

   The --set revision argument adds a revision label in the format istio.io/rev=asm-1614-1 to istiod. The revision label is used by the automatic sidecar injector webhook to associate injected sidecars with a particular istiod revision. To enable sidecar auto-injection for a namespace, you must label it with a revision that matches the label on istiod.

3. Configure the validating webhook so that it can locate the istiod service with the revision label:

   kubectl apply -f istiod-service.yaml
   

   This command creates a service entry that lets the validating webhook automatically check configurations before they are applied.

Installing without Prometheus and Kiali

By default in Anthos Service Mesh 1.7 and earlier, Anthos Service Mesh installs Prometheus and Kiali from their original package repositories at docker.io and quay.io. If you don't want to install these tools or allow connections to these repositories, pass -set addonComponents.prometheus.enabled=false and --set addonComponents.kiali.enabled=false to istioctl install. For example, use the following command to exclude Prometheus and Kiali from the public subnet installation installation.

  istioctl install \
    --set profile=asm-multicloud \
    --set addonComponents.prometheus.enabled=false \
    --set addonComponents.kiali.enabled=false \
    --revision=asm-1614-1

Check the control plane components

Check that the control plane pods in istio-system are up:

kubectl get pod -n istio-system

Expected output is similar to the following:

NAME                                      READY   STATUS      RESTARTS   AGE
istio-ingressgateway-74cc894bfd-786rg     1/1     Running     0          7m19s
istiod-78cdbbbdb-d7tps                    1/1     Running     0          7m36s
promsd-576b8db4d6-lqf64                   2/2     Running     1          7m19s

Injecting sidecar proxies

Anthos Service Mesh uses sidecar proxies to enhance network security, reliability, and observability. With Anthos Service Mesh, these functions are abstracted away from the application's primary container and implemented in a common out-of-process proxy delivered as a separate container in the same Pod.

Before you deploy workloads, make sure to configure sidecar proxy injection so that Anthos Service Mesh can monitor and secure traffic.

Any workloads that were running on your cluster before you installed Anthos Service Mesh need to have the sidecar proxy injected or updated so they have the current Anthos Service Mesh version. Before you deploy new workloads, make sure to configure sidecar proxy injection so that Anthos Service Mesh can monitor and secure traffic.

To enable sidecar auto-injection, you label your namespaces with the same revision that you set on istiod. Run the following command to show the labels on istiod:

kubectl -n istio-system get pods -l app=istiod --show-labels

Confirm that the revision label, istio.io/rev=asm-1614-1, is in the output. This is the label that you use to enable auto-injection. You can enable auto-injection with one command, for example:

kubectl label namespace NAMESPACE istio-injection- istio.io/rev=asm-1614-1 --overwrite

where NAMESPACE is the name of the namespace for your application's services or default if you didn't explicitly create a namespace.

For more information, see Injecting sidecar proxies.

Accessing the Ingress Gateway

Anthos Service Mesh provides a preconfigured Ingress Gateway, the istio-ingressgateway, that you can use to manage inbound traffic to applications running on your service mesh. To make applications accessible from outside of your cluster, (such as from a browser):

• You need to get the external IP address or host name and port of the external load balancer that the istio-ingressgateway is configured to use.

• Your application must define a Gateway and VirtualService resource, similar to the Online Boutique sample application's frontend-gateway.yaml.

To get the external address of the istio-ingressgateway:

1. Create the INGRESS_HOST environment variable:

   export INGRESS_HOST=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
   

2. Create the INGRESS_PORT environment variable:

   export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].port}')
   

3. To test, deploy a sample application, such as the Online Boutique.

4. To access the application on your browser, use the value of $INGRESS_HOST:$INGRESS_PORT in the URL.

Troubleshooting

Anthos clusters on AWS requires tags on subnets that contain load balancer endpoints. Anthos clusters on AWS automatically tags all subnets specified in the spec.Networking.ServiceLoadBalancerSubnetIDs field of the AWSCluster resource.

If you would like to add additional subnets to your user cluster, or if you need to re-apply tags to existing subnets, perform the following steps.

1. From your anthos-aws directory, use anthos-gke to switch context to your management service.

   cd anthos-aws
   anthos-gke aws management get-credentials

   Note: anthos-gke uses the credentials you specified when you ran aws configure.

2. Get the ID of your cluster's AWS VPC with kubectl and store it as a variable.

    export VPC_ID=$(\
    env HTTP_PROXY=http://localhost:8118 \
    kubectl get awscluster cluster-0 -o jsonpath='{.spec.networking.vpcID}')
   

3. Check the variables content with echo. The output resembles vpc-12345678abcdef0.

   echo $VPC_ID
   

4. Save your cluster ID into an environment variable.

   export CLUSTER_ID=$(\
   env HTTP_PROXY=http://localhost:8118 \
   kubectl get awscluster cluster-0 -o jsonpath='{.status.clusterID}')
   

   You can check the variable with echo:

   echo $CLUSTER_ID
   

   The response includes your cluster ID.

   gke-12345678
   

5. By default, Anthos clusters on AWS sets up a VPC with private and public subnets. To retrieve the subnet ID, use the aws command-line tool.

   Select from the following options:

   • Public, if you want to expose Services on your public subnet.
   • Private, if you want to expose Services on your private subnet.

   • Multiple subnets, if you want to expose Services on multiple subnets.

   Public

    export SUBNET_ID=$(aws ec2 describe-subnets \
    --filters "Name=vpc-id,Values=$VPC_ID" "Name=tag:Name,Values=*public*" \
    --query "Subnets[*].SubnetId" \
    --output text)
   

   The output is an object that contains your subnet ID. It resembles subnet-1234abcdefg. You can check the variable with echo:

   echo $SUBNET_ID
   

   The response includes your subnet ID.

   subnet-012345678abcdef
   

   Private

    export SUBNET_ID=$(aws ec2 describe-subnets \
    --filters "Name=vpc-id,Values=$VPC_ID" "Name=tag:Name,Values=*private*" \
    --query "Subnets[*].SubnetId" \
    --output text)
   

   The output is an object that contains your subnet ID. It resembles subnet-1234abcdefg. You can check the variable with echo:

   echo $SUBNET_ID
   

   The response includes your subnet ID.

   subnet-012345678abcdef
   

   Multiple subnets

   If you are using multiple subnets for your AWSNodePools (for example, if you use multiple availability zones), you need to tag your subnet IDs individually.

   Note: The following commands assume your subnets have public or private in their name. If your subnets do not include this text, change the parameters of the --filters flag. See Using filtering for more information.

   Retrieve your list of subnet IDs with aws ec2 describe-subnets.

   To get a list of all public subnets, run the following command:

   aws ec2 describe-subnets \
    --filters "Name=vpc-id,Values=$VPC_ID" "Name=tag:Name,Values=*public*" \
    --query "Subnets[*].SubnetId" \
    --output text
   

   To get a list of all private subnets, run the following command:

   aws ec2 describe-subnets \
    --filters "Name=vpc-id,Values=$VPC_ID" "Name=tag:Name,Values=*private*" \
    --query "Subnets[*].SubnetId" \
    --output text
   

   The response includes your subnet IDs.

   subnet-012345678abcdef
   subnet-abcdef123456789
   subnet-123456789abcdef
   

6. Tag your subnet with your cluster ID. If you have multiple subnets, select Multiple subnets.

   Single subnet

   aws ec2 create-tags \
   --resources $SUBNET_ID \
   --tags Key=kubernetes.io/cluster/$CLUSTER_ID,Value=shared
   

   Multiple subnets

   For each of your subnets, run the following command:

   aws ec2 create-tags \
   --resources subnet-ids \
   --tags Key=kubernetes.io/cluster/$CLUSTER_ID,Value=shared
   

   Replace subnet-ids with the list of subnet IDs, separated by spaces. For example, subnet-012345678abcdef subnet-abcdef123456789 subnet-123456789abcdef.

What's next

Create an example deployment on Anthos clusters on AWS.