Version 1.9

Add Compute Engine virtual machines to Anthos Service Mesh

Introduction

Anthos Service Mesh allows you to manage, observe and secure services running on Managed Instance Groups (MIGs) together with services running on Google Kubernetes Engine (GKE) clusters in the mesh. This allows you to do the following with the Compute Engine instances in your mesh:

  • Manage traffic.
  • Enforce mTLS.
  • Apply access control to service traffic.
  • Securely access Google Cloud services.
  • Collect metrics, logging and tracing data.
  • Monitor services using the Google Cloud Console.

This enables legacy applications not suitable or ready for containerization to take advantage of Anthos Service Mesh features and allows for integrating those workloads with the rest of your services.

How it works

ASM provides two related Custom Resource Definitions (CRDs) to represent virtual machine workloads:

  • WorkloadGroup represents a logical group of virtual machine workloads that share common properties. This is similar to a Deployment in Kubernetes.
  • WorkloadEntry represents a single instance of a virtual machine workload. This is similar to a Pod in Kubernetes.
  • A Service can then select the WorkloadGroup and have ASM route traffic to the VM instance in a similar way to a Pod. This allows the VM to act like any other workload in the mesh.

You create a Compute Engine instance template for each Compute Engine instance group, which specifies a service proxy agent for each Compute Engine instance in that group. During installation, the agent bootstraps the service proxy, sets up traffic interception and monitors the health of the service proxy during the Compute Engine instance's lifetime. The proxy connects with the Anthos Service Mesh control plane, then automatically registers each Compute Engine instance as a WorkloadEntry for the corresponding WorkloadGroup. This lets the Anthos Service Mesh treat each instance as a service endpoint, like the Kubernetes pods in the cluster. You can also create Kubernetes services for VM workloads, like you would for Kubernetes pods.

To scale up or down the number of workloads on Compute Engine instances, starting from a minimum MIG size of zero, see Autoscaling groups of instances.

The service proxy agent relies on VM manager to ensure the agent is installed in each VM in the MIG. For more information about instance groups and VM management, see Managed instance group (MIG) and VM Manager.

Supported Linux distributions

OS version supported
Debian 10
Debian 9
Centos 8
Centos 7

See Debian support or CentOS support for more information on OS distributions.

Limitations

  • The mesh control plane must be Anthos Service Mesh 1.9 or higher.
  • Only Compute Engine managed instance groups created from a Compute Engine instance template are supported.
  • The cluster and VMs must be on the same network, in the same project, and use a single network interface.
  • You can use this feature without an Anthos subscription, but certain UI elements and features in Google Cloud Console are only available to Anthos subscribers. For information about what is available to subscribers and non-subscribers, see Anthos and Anthos Service Mesh UI differences.

Prerequisites

Before you begin, check the following prerequisites.

Clusters

This page includes the option to install Anthos Service Mesh as part of its steps or update a cluster that already has Anthos Service Mesh installed. In either case, these steps require Anthos Service Mesh 1.9 and a cluster that meets these requirements. In addition, Anthos Service Mesh VM support has further requirements:

  • The control plane must be installed in a cluster that you control. Google-managed control planes are not supported. For more information, see Google-managed control planes.
  • Uses Mesh CA as the certificate authority.
  • Uses Stackdriver for telemetry.
  • Has the Canonical Service deployment enabled, which the Anthos Service Mesh installation process enables automatically.

Your cluster must be registered to an environ. However, if not already registered, the VM installation process can register it for you in your specified project.

If you are an Anthos subscriber, enable the Anthos API.

Enable the API

CLI tools

The installation process requires the following tools, which are already installed if you use the Google Cloud Shell:

  • gcloud
  • kubectl
  • kpt
  • curl
  • jq
  • awk
  • printf
  • tr
  • grep
  • tail

Downloading the script

This section describes how to download the script for VM onboarding.

  1. Download the VM script for Anthos Service Mesh 1.9.3 to the current working directory:

    curl https://storage.googleapis.com/csm-artifacts/asm/asm_vm_1.9 > asm_vm
    
  2. Download the SHA-256 of the file to the current working directory:

    curl https://storage.googleapis.com/csm-artifacts/asm/asm_vm_1.9.sha256 > asm_vm.sha256
    
  3. With both files in the same directory, verify the download:

    sha256sum -c asm_vm.sha256
    

    If the verification is successful, the command outputs: asm_vm: OK

  4. Make the script executable:

    chmod +x asm_vm
    

Getting started

This section explains the steps to add Compute Engine instances to Anthos Service Mesh.

Set up your environment

  1. Use gcloud to configure your kubectl tool to point to the cluster you specified, as some of the steps below require you to make changes directly on the cluster.

    gcloud container clusters get-credentials CLUSTER_NAME --zone CLUSTER_LOCATION --project PROJECT_ID
    

Prepare your cluster

Set up the Anthos Service Mesh cluster for VMs, by preparing the Anthos Service Mesh 1.9 control plane.

Choose your next step based on whether Anthos Service Mesh 1.9 is already installed on your cluster.

Not installed

If Anthos Service Mesh 1.9 has not been installed on your cluster, the following example shows how to add the vm and hub-meshca options to modify the usual Anthos Service Mesh installation steps provided in Installation, migration, and upgrade for GKE. It also explains how to download the install_asm script used in the following example.

After you download the install_asm script, you can install Anthos Service Mesh in your cluster by including the --option hub-meshca, the --option vm and --enable_all flags. For more information, see enable Mesh CA with environ and Enablement flags.

./install_asm --project_id PROJECT_ID \
--cluster_name CLUSTER_NAME \
--cluster_location CLUSTER_LOCATION \
--mode install --option vm --option hub-meshca \
--enable_all

Installed

If Anthos Service Mesh 1.9 has already been installed on your cluster and it has Mesh CA with environ enabled, update the Anthos Service Mesh control plane to support VM based workloads. The script also helps you verify if your Anthos Service Mesh installation in the cluster is ready for VM workloads. Specifically, the prepare_cluster sub-command updates all the revisions in Anthos Service Mesh 1.9 to be ready for VM workloads.

If Mesh CA with environ is not enabled in your Anthos Service Mesh 1.9 installation, reinstall or upgrade your Anthos Service Mesh 1.9 installation by including the --option hub-meshca and the --option vm flags in the install_asm script.

./asm_vm prepare_cluster \
--project_id PROJECT_ID \
--cluster_name CLUSTER_NAME \
--cluster_location CLUSTER_LOCATION

The previous steps performed the following actions for you:

  1. Enable VM Auto Registration: This is done by setting the PILOT_ENABLE_WORKLOAD_ENTRY_AUTOREGISTRATION variable to true. When this is enabled, new VM instances will register with the WorkloadGroup and new WorkloadEntry CRs will be created to route traffic to the VMs. All Anthos Service Mesh 1.9 control planes installed with install_asm will include this by default.

  2. Install an Expansion gateway: This gateway is named the eastwest gateway and is defined in the Anthos Service Mesh config package. This will also expose the control plane to the VMs.

  3. Install the IdentityProvider CRD and register a Google IdentityProvider CR to enable VMs to authenticate to Anthos Service Mesh control plane and securely communicate with the rest of the service mesh.

  4. Register the cluster to an environ and enable workload identity, if you use --enable_all or --enable_registration in the install_asm script.

  5. Enable the Service Mesh feature within the environ. This feature will manage the policies necessary to allow VMs to securely communicate with the mesh.

Add your VMs

In this section, you add Compute Engine instances to your mesh based on the instance template you create with the asm_vm script. The script only generates the necessary configuration for the service proxy agent. To include more configuration in your instance template, create a source instance template and add it to the script.

To add VMs to your mesh, use the following steps:

  1. Set the following environment variables to use in later steps. Set these variables for each VM workload:

    • WORKLOAD_NAME is the name of the workload the VM is part of, which must be a compliant DNS-1123 subdomain consisting of lower case alphanumeric characters.
    • WORKLOAD_VERSION is the version of the workload the VM is part of. Optional.
    • WORKLOAD_SERVICE_ACCOUNT is the service GCP Service Account the VM runs as.
    • WORKLOAD_NAMESPACE is the namespace for the workload.
    • ASM_INSTANCE_TEMPLATE is the name of the instance template to be created. Compute Engine instance template name does not allow underscores.
    • SOURCE_INSTANCE_TEMPLATE is the template name to base the generated template on. Optional.
  2. Create the namespace for the VM workloads if it doesn't already exist:

    kubectl create ns WORKLOAD_NAMESPACE
    
  3. Label the namespace with the control plane revision. For an example of how to find the control plane revision shown as REVISION in the following example, see Deploying and redeploying workloads.

    kubectl label ns WORKLOAD_NAMESPACE istio-injection- istio.io/rev=REVISION --overwrite
    
  4. Create the WorkloadGroup for the VMs to be registered:

    kubectl apply -f - << EOF
    apiVersion: networking.istio.io/v1alpha3
    kind: WorkloadGroup
    metadata:
     name: WORKLOAD_NAME
     namespace: WORKLOAD_NAMESPACE
    spec:
     metadata:
       labels:
         app.kubernetes.io/name: WORKLOAD_NAME
         app.kubernetes.io/version: WORKLOAD_VERSION
       annotations:
         security.cloud.google.com/IdentityProvider: google
     template:
       serviceAccount: WORKLOAD_SERVICE_ACCOUNT
    EOF
    
    Field Description
    name The name of the workload the VM is part of.
    namespace The namespace the workload is a part of.
    app.kubernetes.io/name The recommended labels for Kubernetes applications. You can use your own labels for your VM workloads.
    app.kubernetes.io/version The recommended labels for Kubernetes applications. You can use your own labels for your VM workloads.
    serviceAccount The service account identity used by the VM and project, which will be used as part of the identity of the workload in the SPIFFE format. For more information, see Service accounts.
    security.cloud.google.com/IdentityProvider The identity provider that the VM will use, which should already be registered in your cluster. For Compute Engine VMs it should be set to google. The `IdentityProvider` tells the control plane how to authenticate the VM's credential and where to extract VM's service account.
  5. Run the asm_vm script with the following options and flags to create an instance template for your Anthos Service Mesh Compute Engine instances.

    The script verifies cluster prerequisites, adds VM labels for Anthos Service Mesh, generates the custom metadata configuration for the service proxy agent and creates a new instance template.

    If you have an existing instance template that you would like the script to be based on, you can specify the --source_instance_template option. If you want to add non-default VMs to your Anthos Service Mesh, create an instance template with the intended OS distribution and use that template as the value for the --source_instance_template flag in the asm_vm script. If your existing instance template includes a startup script that requires network connectivity, the script should be resilient to transient network connectivity issues. See the demo application for an example of how to add resiliency against temporary network disruption.

    ./asm_vm create_gce_instance_template \
    ASM_INSTANCE_TEMPLATE \
    --project_id PROJECT_ID \
    --cluster_location CLUSTER_LOCATION \
    --cluster_name CLUSTER_NAME \
    --workload_name WORKLOAD_NAME \
    --workload_namespace WORKLOAD_NAMESPACE \
    --source_instance_template SOURCE_INSTANCE_TEMPLATE
    

    Options

    Option Description
    -p|--project_id PROJECT_ID The project ID that the cluster was created in.
    -n|--cluster_name CLUSTER_NAME The name of the cluster.
    -l|--cluster_location CLUSTER_LOCATION Either the zone (for single-zone clusters) or region (for regional clusters) that the cluster was created in.
    -w|--workload_name WORKLOAD_NAME The name of the workload the Compute Engine instances represent.
    --workload_namespace WORKLOAD_NAMESPACE Optional. The namespace of the workload. Default is `default`.
    -s|--source_instance_template SOURCE_INSTANCE_TEMPLATE_NAME Optional. An existing instance template to use as a base for the Anthos Service Mesh Compute Engine instance template. If not specified, an instance template with default values is created.

    Flags

    Flag Description
    -v|--verbose Print commands before and after execution.
    --dry_run Print commands, but don't execute them.
    --only_validate Run validation but don't create a new Compute Engine instance template.
    -h|--help Show a help message describing the options and flags and exit.
  6. Set the following environment variables for each MIG that you create:

    • INSTANCE_GROUP_NAME is the name of the Compute Engine instance group to create.
    • ASM_INSTANCE_TEMPLATE is the name of the instance template to be created. Compute Engine instance template name does not allow underscores.
    • INSTANCE_GROUP_ZONE is the zone of the Compute Engine instance group to be created.
    • PROJECT_ID is the project ID that the cluster was created in.
    • SIZE is the size of the instance group to be created. It can be changed after the instance group is created.
    • WORKLOAD_NAME is the name of the workload the VM is part of.
    • WORKLOAD_NAMESPACE is the namespace for the workload.
  7. Create a Managed Instance Group for the VM workloads, using the variables created in the previous steps:

    gcloud compute instance-groups managed create \
    INSTANCE_GROUP_NAME \
    --template ASM_INSTANCE_TEMPLATE \
    --zone=INSTANCE_GROUP_ZONE \
    --project=PROJECT_ID \
    --size=SIZE
    

    To scale up or down the number of workloads on Compute Engine instances, starting from a zonal or regional MIG size of zero, see Autoscaling groups of instances. For more information about creating groups, see gcloud compute instance-groups managed create.

    When your instance starts, it will automatically authenticate with the Anthos Service Mesh control plane on your cluster and the control plane will register each VM as a WorkloadEntry.

  8. When the VM instance in the MIG finishes starting up, you can view the registered VMs in the workload namespace by using the following command:

       kubectl get workloadentry -n WORKLOAD_NAMESPACE
    
  9. Add a Kubernetes Service to expose VM workloads added above. Be sure to have the service select the corresponding label on the VM WorkloadGroup registered above for correct traffic routing.

    The following example creates a Kubernetes service named WORKLOAD_NAME in the namespace WORKLOAD_NAMESPACE that exposes VM workloads with the app.kubernetes.io/name: WORKLOAD_NAME label under HTTP port 80.

    kubectl apply -f - << EOF
    apiVersion: v1
    kind: Service
    metadata:
     name: WORKLOAD_NAME
     namespace: WORKLOAD_NAMESPACE
     labels:
       asm_resource_type: VM
    spec:
     ports:
     - port: 80
       name: http
     selector:
       app.kubernetes.io/name: WORKLOAD_NAME
    EOF
    

    For more details on how to create a Kubernetes service, see https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service.

  10. To use a sample application on your VM, see Deploy a sample application.

Anthos Service Mesh control plane upgrade

Upgrade your Anthos Service Mesh control plane to a new version by using the instructions at Upgrading Anthos Service Mesh to the latest version. After installing a new version of the Anthos Service Mesh control plane, redeploy your Kubernetes workloads by using the instructions at Deploying and redeploying workloads.

For VM workloads, create a new instance template and perform a rolling update to the VMs in your MIG:

  1. Use the following command to locate the revision label on istiod:

    kubectl -n istio-system get pods -l app=istiod --show-labels
    

    The output from the command is similar to the following. Note that the output for migrations is slightly different than for upgrades. The following example output is from a migration.

    NAME                                READY   STATUS    RESTARTS   AGE   LABELS
    istiod-7744bc8dd7-qhlss             1/1     Running   0          49m   app=istiod,istio.io/rev=default,istio=pilot,pod-template-hash=7744bc8dd7
    istiod-asm-193-2-85d86774f7-flrt2   1/1     Running   0          26m   app=istiod,istio.io/rev=asm-193-2,istio=istiod,pod-template-hash=85d86774f7
    istiod-asm-193-2-85d86774f7-tcwtn   1/1     Running   0          26m   app=istiod,istio.io/rev=asm-193-2,istio=istiod,pod-template-hash=85d86774f7
    1. In the output, under the LABELS column, note the value in the istiod revision label for the new version, which follows the prefix istio.io/rev=. In this example, the value is asm-193-2.

    2. Also note the value in the revision label for the old istiod version. You need this to delete the old version of istiod when you finish moving workloads to the new version. In the example output, the value in the revision label for the old version of istiod is default.

  2. Add the revision label to a namespace and remove the istio-injection label (if it exists). In the following command, change REVISION to the value that matches the new revision of istiod.

    kubectl label namespace NAMESPACE istio.io/rev=REVISION istio-injection- --overwrite

    If you see "istio-injection not found" in the output, you can ignore it. That means that the namespace didn't previously have the istio-injection label. Because auto-injection fails if a namespace has both the istio-injection and the revision label, all kubectl label commands in the Anthos Service Mesh documentation include removing the istio-injection label.

  3. Create a new instance template using the asm_vm script. Be sure to use a new instance template name and include the same source instance template if you had one for the same workload.

    ./asm_vm create_gce_instance_template \
    NEW_ASM_INSTANCE_TEMPLATE \
    --project_id PROJECT_ID \
    --cluster_location CLUSTER_LOCATION \
    --cluster_name CLUSTER_NAME \
    --workload_name WORKLOAD_NAME \
    --workload_namespace WORKLOAD_NAMESPACE \
    --source_instance_template SOURCE_INSTANCE_TEMPLATE
    
  4. Perform a rolling-update to your existing MIG for the workload.

    For more information, see Starting a basic rolling update.

    gcloud compute instance-groups managed rolling-action start-update INSTANCE_GROUP_NAME \
    --version=template=NEW_ASM_INSTANCE_TEMPLATE \
    --zone=INSTANCE_GROUP_ZONE
    
  5. Test the VM workload to ensure that it is working as expected.

Upgrade VM applications

If you have any updates to your application, including changes to the WorkloadGroup and/or changes to your source instance template, a new instance template is required to update the MIG of your VM workloads.

When the WorkloadGroup change is applied and/or the new source instance template is created, you will create a new instance template for Anthos Service Mesh and perform a rolling update to the VMs in your MIG.

  1. Create a new instance template using the asm_vm script. Be sure to use a new instance template name and include the new source instance template if you created one for your application update.

    ./asm_vm create_gce_instance_template \
    NEW_ASM_INSTANCE_TEMPLATE \
    --project_id PROJECT_ID \
    --cluster_location CLUSTER_LOCATION \
    --cluster_name CLUSTER_NAME \
    --workload_name WORKLOAD_NAME \
    --workload_namespace WORKLOAD_NAMESPACE \
    --source_instance_template NEW_SOURCE_INSTANCE_TEMPLATE
    
  2. Perform a rolling-update to your existing MIG for the workload. For more information about how to use MIG rolling update, see Starting a basic rolling update.

    gcloud compute instance-groups managed rolling-action start-update INSTANCE_GROUP_NAME \
    --version=template=NEW_ASM_INSTANCE_TEMPLATE \
    --zone=INSTANCE_GROUP_ZONE
    
  3. Test the VM workload to ensure it works as expected.

Deploy a sample application

To demonstrate that your new mesh configuration is working correctly, you can install the Bookinfo sample application. This example runs a MySQL database on the VM and the ratings service reads the ratings values from the database.

Install Bookinfo on the cluster

Use the following steps to deploy the BookInfo application's services with the sidecar proxies injected alongside each service. The BookInfo application will be deployed in the default namespace.

  1. On the command line on the computer where you installed Anthos Service Mesh, go to the root of the Anthos Service Mesh installation directory that you created in the Downloading the script step.

  2. To enable automatic sidecar injection, use the following command to locate the label on istiod, which contains the revision label value to use in later steps.

    kubectl -n istio-system get pods -l app=istiod --show-labels
    

    The output looks similar to the following:

    NAME                                READY   STATUS    RESTARTS   AGE   LABELS
    istiod-asm-193-2-5788d57586-bljj4   1/1     Running   0          23h   app=istiod,istio.io/rev=asm-193-2,istio=istiod,pod-template-hash=5788d57586
    istiod-asm-193-2-5788d57586-vsklm   1/1     Running   1          23h   app=istiod,istio.io/rev=asm-193-2,istio=istiod,pod-template-hash=5788d57586
    

    In the output, under the LABELS column, note the value of the istiod revision label, which follows the prefix istio.io/rev=. In this example, the value is asm-193-2.

  3. Apply the revision label to the default namespace. In the following command, REVISION is the value of the istiod revision label that you noted in the previous step.

    kubectl label namespace default istio-injection- istio.io/rev=REVISION --overwrite
    

    You can ignore the message "istio-injection not found" in the output. That means that the namespace didn't previously have the istio-injection label, which you should expect in new installations of Anthos Service Mesh or new deployments. Because auto-injection fails if a namespace has both the istio-injection and the revision label, all kubectl label commands in the Anthos Service Mesh documentation include removing the istio-injection label.

  4. Deploy your application to the default namespace using kubectl:

    kubectl apply -f samples/bookinfo/platform/kube/bookinfo.yaml
    
  5. Confirm that the application has been deployed correctly by running the following commands:

    kubectl get services
    

    Expected output:

    NAME                       CLUSTER-IP   EXTERNAL-IP   PORT(S)              AGE
    details                    10.0.0.31    <none>        9080/TCP             6m
    kubernetes                 10.0.0.1     <none>        443/TCP              7d
    productpage                10.0.0.120   <none>        9080/TCP             6m
    ratings                    10.0.0.15    <none>        9080/TCP             6m
    reviews                    10.0.0.170   <none>        9080/TCP             6m

    and

    kubectl get pod
    

    Expected output:

    NAME                                        READY     STATUS    RESTARTS   AGE
    details-v1-1520924117-48z17                 2/2       Running   0          6m
    productpage-v1-560495357-jk1lz              2/2       Running   0          6m
    ratings-v1-734492171-rnr5l                  2/2       Running   0          6m
    reviews-v1-874083890-f0qf0                  2/2       Running   0          6m
    reviews-v2-1343845940-b34q5                 2/2       Running   0          6m
    reviews-v3-1813607990-8ch52                 2/2       Running   0          6m
  6. Finally, define the ingress gateway routing for the application:

    kubectl apply -f samples/bookinfo/networking/bookinfo-gateway.yaml
    

    Expected output:

    gateway.networking.istio.io/bookinfo-gateway created
    virtualservice.networking.istio.io/bookinfo created
  7. Confirm that the product page is accessible.

    export INGRESS_HOST=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
    export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].port}')
    export GATEWAY_URL="${INGRESS_HOST}:${INGRESS_PORT}"
    curl -s "http://${GATEWAY_URL}/productpage" | grep -o "<title>.*</title>"
    

    Expected output:

    <title>Simple Bookstore App</title>
    

Create Compute Engine instances and install MySQL

In this step, you will create a Compute Engine instance template for the MySQL instance running on the VM. For more detailed steps, see Bookinfo with a Virtual Machine.

  1. Create a Compute Engine instance template that includes a startup script to install MySQL and add a ratings database upon startup. Note that if you are using CentOS it will take up to 10 minutes for the mariadb-server to be ready.

    Debian

    cat << "EOF" > init-mysql
    #!/bin/bash
    
    # Wait until Envoy is ready before installing mysql
    while true; do
      rt=$(curl -s 127.0.0.1:15000/ready)
      if [[ $? -eq 0 ]] && [[ "${rt}" -eq "LIVE" ]]; then
        echo "envoy is ready"
        break
      fi
      sleep 1
    done
    
    sudo apt-get update && sudo apt-get install -y mariadb-server
    
    sudo sed -i '/bind-address/c\bind-address  = 0.0.0.0' /etc/mysql/mariadb.conf.d/50-server.cnf
    
    cat <<EOD | sudo mysql
    # Grant access to root
    GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' IDENTIFIED BY 'password' WITH GRANT OPTION;
    
    # Grant root access to other IPs
    CREATE USER 'root'@'%' IDENTIFIED BY 'password';
    GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' WITH GRANT OPTION;
    FLUSH PRIVILEGES;
    quit;
    EOD
    
    sudo systemctl restart mysql
    
    curl -LO https://raw.githubusercontent.com/istio/istio/release-1.9/samples/bookinfo/src/mysql/mysqldb-init.sql
    
    mysql -u root -ppassword < mysqldb-init.sql
    EOF
    
    gcloud compute \
    --project=PROJECT_ID \
    instance-templates create mysql-instance-template \
    --machine-type=e2-medium \
    --metadata-from-file=startup-script=init-mysql \
    --image=debian-10-buster-v20201014 \
    --image-project=debian-cloud \
    --boot-disk-size=10GB
    

    CentOS

    cat << "EOF" > init-mysql
    #!/bin/bash
    
    # Wait until Envoy is ready before installing mysql
    while true; do
      rt=$(curl -s 127.0.0.1:15000/ready)
      if [[ $? -eq 0 ]] && [[ "${rt}" -eq "LIVE" ]]; then
        echo "envoy is ready"
        break
      fi
      sleep 1
    done
    
    sudo yum update -y && sudo yum install -y mariadb-server
    
    # Wait until mysql is ready
    while true; do
      rt=$(which mysql)
      if [[ ! -z "${rt}" ]]; then
        echo "mysql is ready"
        break
      fi
      sleep 1
    done
    
    sudo sed -i '/bind-address/c\bind-address  = 0.0.0.0' /etc/my.cnf.d/mariadb-server.cnf
    
    sudo systemctl restart mariadb
    
    cat > grantaccess.sql << EOD
    
    GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' IDENTIFIED BY 'password' WITH GRANT OPTION;
    
    CREATE USER 'root'@'%' IDENTIFIED BY 'password';
    GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' WITH GRANT OPTION;
    FLUSH PRIVILEGES;
    EOD
    
    until sudo mysql < grantaccess.sql; do
       sleep 1
    done
    
    sudo systemctl restart mariadb
    
    curl -LO https://raw.githubusercontent.com/istio/istio/release-1.9/samples/bookinfo/src/mysql/mysqldb-init.sql
    
    mysql -u root -ppassword < mysqldb-init.sql
    EOF
    
    gcloud compute \
    --project=PROJECT_ID \
    instance-templates create mysql-instance-template \
    --machine-type=e2-medium \
    --metadata-from-file=startup-script=init-mysql \
    --image-project=centos-cloud \
    --image-family=centos-8 \
    --boot-disk-size=30GB
    
  2. Create a WorkloadGroup for the MySQL workload

    kubectl apply -f - << EOF
    apiVersion: networking.istio.io/v1alpha3
    kind: WorkloadGroup
    metadata:
     name: mysql
     namespace: default
    spec:
     metadata:
       labels:
         app.kubernetes.io/name: mysql
       annotations:
         security.cloud.google.com/IdentityProvider: google
     template:
       serviceAccount: WORKLOAD_SERVICE_ACCOUNT
    EOF
    
  3. Use the VM script above to create a new instance template to prepare the instances for your mesh.

    ./asm_vm create_gce_instance_template \
    asm-mysql-instance-template \
    --project_id PROJECT_ID \
    --cluster_location CLUSTER_LOCATION \
    --cluster_name CLUSTER_NAME \
    --workload_name mysql \
    --source_instance_template mysql-instance-template
    
  4. Create a Compute Engine MIG using the newly created instance template.

    gcloud compute instance-groups managed create mysql-instance \
    --template asm-mysql-instance-template \
    --zone=us-central1-c \
    --project=PROJECT_ID \
    --size=1
    

Create a service

Create a Kubernetes service for the MySQL service.

  1. Create a Kubernetes service by using the following command:

    kubectl apply -f - << EOF
    apiVersion: v1
    kind: Service
    metadata:
     name: mysql
     namespace: default
     labels:
       asm_resource_type: VM
    spec:
     ports:
     - name: mysql
       port: 3306
       protocol: TCP
       targetPort: 3306
     selector:
       app.kubernetes.io/name: mysql
    EOF
    

Use the Anthos UI dashboard

To see the new VM-based service you've created, click Anthos > Service Mesh from the main left navigation bar. It will display a table of the services running in your mesh. The service you added should appear in the table, with a Type value of VM and some high-level metrics. To see more telemetry from your VM-based service, click the service name, which will display the service-level dashboard.

For more information on how to use the Anthos UI dashboard, see Exploring Anthos Service Mesh in the Cloud Console.

Manage traffic to the VM workloads

You can change the networking rules to control how the traffic flows in and out of the VM(s).

Control traffic to a new ratings service (Pod to VM)

Create another ratings service in Bookinfo that will use the MySQL instance created above as the data source and specify a routing rule that forces the review service to use the new rating service.

  1. Create a new rating service to use the MySQL instance.

    kubectl apply -f - << EOF
    apiVersion: apps/v1
    kind: Deployment
    metadata:
     name: ratings-v2-mysql-vm
     labels:
       app: ratings
       version: v2-mysql-vm
    spec:
     replicas: 1
     selector:
       matchLabels:
         app: ratings
         version: v2-mysql-vm
     template:
       metadata:
         labels:
           app: ratings
           version: v2-mysql-vm
       spec:
         serviceAccountName: bookinfo-ratings
         containers:
         - name: ratings
           image: docker.io/istio/examples-bookinfo-ratings-v2:1.16.2
           imagePullPolicy: IfNotPresent
           env:
             - name: DB_TYPE
               value: "mysql"
             - name: MYSQL_DB_HOST
               value: mysql.default.svc.cluster.local
             - name: MYSQL_DB_PORT
               value: "3306"
             - name: MYSQL_DB_USER
               value: root
             - name: MYSQL_DB_PASSWORD
               value: password
           ports:
           - containerPort: 9080
    EOF
    
  2. Create a routing rule.

    kubectl apply -f - << EOF
    apiVersion: networking.istio.io/v1alpha3
    kind: VirtualService
    metadata:
     name: reviews
    spec:
     hosts:
     - reviews
     http:
     - route:
       - destination:
           host: reviews
           subset: v3
    ---
    apiVersion: networking.istio.io/v1alpha3
    kind: VirtualService
    metadata:
     name: ratings
    spec:
     hosts:
     - ratings
     http:
     - route:
       - destination:
           host: ratings
           subset: v2-mysql-vm
    EOF
    
  3. Apply destination rules for the created services.

    kubectl apply -f - << EOF
    apiVersion: networking.istio.io/v1alpha3
    kind: DestinationRule
    metadata:
     name: reviews
    spec:
     host: reviews
     subsets:
     - name: v1
       labels:
         version: v1
     - name: v2
       labels:
         version: v2
     - name: v3
       labels:
         version: v3
    ---
    apiVersion: networking.istio.io/v1alpha3
    kind: DestinationRule
    metadata:
     name: ratings
    spec:
     host: ratings
     subsets:
     - name: v1
       labels:
         version: v1
     - name: v2
       labels:
         version: v2
     - name: v2-mysql
       labels:
         version: v2-mysql
     - name: v2-mysql-vm
       labels:
         version: v2-mysql-vm
    EOF
    

Validating the application deployment

To see if the BookInfo application is working, you need to send traffic to the ingress gateway.

  • If you installed Anthos Service Mesh on GKE, get the external IP address of the ingress gateway that you created in previous steps:

    kubectl get svc istio-ingressgateway -n istio-system
    

    Output:

    NAME                   TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                                      AGE
    istio-ingressgateway   LoadBalancer   10.19.247.233   35.239.7.64   80:31380/TCP,443:31390/TCP,31400:31400/TCP   27m

    In this example, the IP address of the ingress service is 35.239.7.64.

Trying the application

  1. Check that the BookInfo app is running with curl:

    curl -I http://EXTERNAL_IP/productpage
    

    If the response shows 200, it means the application is working properly with Anthos Service Mesh.

  2. To view the BookInfo web page, enter the following address in your browser:

    http://EXTERNAL_IP/productpage
    
  3. Verify on the Bookinfo application homepage that it is showing five stars from Reviewer1 and four stars from Reviewer2.

Enforcing security on the VM workloads

Enforcing security on the VM workloads is the same as enforcing security on the Kubernetes workloads. For more information, see Istio security.

After you complete the previous steps, your Compute Engine VM will have a Google-issued workload certificate. In the certificate, the SubjectAlternativeName value shows the VM's Anthos workload identity in the form spiffe://<workload_identity_pool>/ns/WORKLOAD_NAMESPACE/sa/WORKLOAD_SERVICE_ACCOUNT.

For more information, see workload identity pool.

Enable mTLS strict mode for the mesh

Apply the following YAML to enforce strict mTLS mesh-wide.

kubectl apply -f - << EOF
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: istio-system
spec:
  mtls:
    mode: STRICT
EOF

Authorization for service-to-service traffic

Use AuthorizationPolicy to control access between the applications on your Compute Engine VM and other mesh workloads (e.g., on the GKE cluster).

Example: Deny Kubernetes workloads to access Compute Engine VMs

The following authorization policy denies a Kubernetes workload ratings to access Compute Engine VM workloads that serve the ratings MySQL server.

kubectl apply -f - << EOF
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: mysql-deny
  namespace: default
spec:
  selector:
    matchLabels:
      app: ratings
      app.kubernetes.io/name: mysql
  action: DENY
  rules:
  - from:
    - source:
        principals: ["cluster.local/ns/default/sa/bookinfo-ratings"]
EOF

After applying the example AuthorizationPolicy, you should see a Ratings service is currently unavailable error message in the book reviews section on the product page.

Installing the Cloud Monitoring Agent

You can install the Cloud Monitoring Agent to collect and monitor system and application metrics from your VM instances. This allows you to monitor key metrics, for example CPU and memory utilization on the agent.

For more information, see Cloud Monitoring Agent documentation.

Troubleshooting

For troubleshooting tips, see Troubleshooting VM support.