Add Compute Engine virtual machines to Anthos Service Mesh

Preview

This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA products and features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

Note: Compute Engine virtual machines (VMs) are currently only supported on Anthos Service Mesh 1.11.

Introduction

Anthos Service Mesh allows you to manage, observe and secure services running on Managed Instance Groups (MIGs) together with services running on Google Kubernetes Engine (GKE) clusters in the mesh. This allows you to do the following with the Compute Engine instances in your mesh:

• Manage traffic.
• Enforce mTLS.
• Apply access control to service traffic.
• Securely access Google Cloud services.
• Collect metrics, logging and tracing data.
• Monitor services using the Google Cloud console.

This enables legacy applications not suitable or ready for containerization to take advantage of Anthos Service Mesh features and allows for integrating those workloads with the rest of your services.

How it works

ASM provides two related Custom Resource Definitions (CRDs) to represent virtual machine workloads:

• WorkloadGroup represents a logical group of virtual machine workloads that share common properties. This is similar to a Deployment in Kubernetes.
• WorkloadEntry represents a single instance of a virtual machine workload. This is similar to a Pod in Kubernetes.
• A Service can then select the WorkloadGroup and have ASM route traffic to the VM instance in a similar way to a Pod. This allows the VM to act like any other workload in the mesh.

You create a Compute Engine instance template for each Compute Engine instance group, which specifies a service proxy agent for each Compute Engine instance in that group. During installation, the agent bootstraps the service proxy, sets up traffic interception and monitors the health of the service proxy during the Compute Engine instance's lifetime. The proxy connects with the Anthos Service Mesh control plane, then automatically registers each Compute Engine instance as a WorkloadEntry for the corresponding WorkloadGroup. This lets the Anthos Service Mesh treat each instance as a service endpoint, like the Kubernetes pods in the cluster. You can also create Kubernetes services for VM workloads, like you would for Kubernetes pods.

To scale up or down the number of workloads on Compute Engine instances, starting from a minimum MIG size of zero, see Autoscaling groups of instances.

The service proxy agent relies on VM manager to ensure the agent is installed in each VM in the MIG. For more information about instance groups and VM management, see Managed instance group (MIG) and VM Manager.

Supported Linux distributions

OS version	supported
Debian 10
Debian 9
Centos 8
Centos 7

See Debian support or CentOS support for more information on OS distributions.

Limitations

• The mesh control plane must be Anthos Service Mesh 1.9 or higher.
• Only Compute Engine managed instance groups created from a Compute Engine instance template are supported.
• The cluster and VMs must be on the same network, in the same project, and use a single network interface.
• You can use this feature without an GKE Enterprise subscription, but certain UI elements and features in Google Cloud console are only available to GKE Enterprise subscribers. For information about what is available to subscribers and non-subscribers, see GKE Enterprise and Anthos Service Mesh UI differences.

Prerequisites

Before you begin, check the following prerequisites.

Clusters

This page includes the option to install Anthos Service Mesh as part of its steps or update a cluster that already has Anthos Service Mesh installed. In either case, these steps require Anthos Service Mesh 1.9+ and a cluster that meets these requirements. In addition, Anthos Service Mesh VM support has further requirements:

• The control plane must be installed in a cluster that you control. Google-managed control planes are not supported. For more information, see Google-managed control planes.
• Uses Mesh CA as the certificate authority.
• Uses Stackdriver for telemetry.
• Has the Canonical Service deployment enabled, which the Anthos Service Mesh installation process enables automatically.

Your cluster must be registered to a fleet. However, if not already registered, the VM installation process can register it for you in your specified project.

If you are an GKE Enterprise subscriber, enable the GKE Enterprise API.

Enable the API

CLI tools

The installation process requires the following tools, which are already installed if you use the Google Cloud Shell:

• gcloud
• kubectl
• kpt
• curl
• jq
• awk
• printf
• tr
• grep
• tail

Downloading the script

This section describes how to download the script for VM onboarding.

1. Download the VM script for Anthos Service Mesh 1.9.8 to the current working directory:

   curl https://storage.googleapis.com/csm-artifacts/asm/asm_vm_1.9 > asm_vm
   

2. Download the SHA-256 of the file to the current working directory:

   curl https://storage.googleapis.com/csm-artifacts/asm/asm_vm_1.9.sha256 > asm_vm.sha256
   

3. With both files in the same directory, verify the download:

   sha256sum -c --ignore-missing asm_vm.sha256
   

   If the verification is successful, the command outputs: asm_vm: OK

   For compatibility, the asm_vm.sha256 file includes the checksum twice to allow any version of the script to be renamed to asm_vm. If you get an error that --ignore-missing does not exist, rerun the previous command without the --ignore-missing flag.

4. Make the script executable:

   chmod +x asm_vm
   

Getting started

This section explains the steps to add Compute Engine instances to Anthos Service Mesh.

Set up your environment

1. Use gcloud to configure your kubectl tool to point to the cluster you specified, as some of the steps below require you to make changes directly on the cluster.

   Note: Use --region instead of --zone, if the cluster is a regional cluster.

   gcloud container clusters get-credentials CLUSTER_NAME --zone CLUSTER_LOCATION --project PROJECT_ID
   

Prepare your cluster

Set up the Anthos Service Mesh cluster for VMs, by preparing the Anthos Service Mesh 1.9+ control plane.

Choose your next step based on whether Anthos Service Mesh 1.9+ is already installed on your cluster.

Not installed

If Anthos Service Mesh 1.9+ has not been installed on your cluster, the following example shows how to add the vm and hub-meshca options to modify the usual Anthos Service Mesh installation steps provided in Installation, migration, and upgrade for GKE. It also explains how to download the install_asm script used in the following example.

After you download the install_asm script, you can install Anthos Service Mesh in your cluster by including the --option hub-meshca, the --option vm and --enable_all flags. For more information, see enable Mesh CA with fleet and Enablement flags.

./install_asm --project_id PROJECT_ID \
--cluster_name CLUSTER_NAME \
--cluster_location CLUSTER_LOCATION \
--mode install --option vm --option hub-meshca \
--enable_all

Installed

If Anthos Service Mesh 1.9+ has already been installed on your cluster and it has Mesh CA with fleet enabled, update the Anthos Service Mesh control plane to support VM based workloads. The script also helps you verify if your Anthos Service Mesh installation in the cluster is ready for VM workloads. Specifically, the prepare_cluster sub-command updates all the revisions in Anthos Service Mesh 1.9+ to be ready for VM workloads.

If Mesh CA with fleet is not enabled in your Anthos Service Mesh 1.9+ installation, reinstall or upgrade your Anthos Service Mesh 1.9+ installation by including the --option hub-meshca and the --option vm flags in the install_asm script.

./asm_vm prepare_cluster \
--project_id PROJECT_ID \
--cluster_name CLUSTER_NAME \
--cluster_location CLUSTER_LOCATION

The previous steps performed the following actions for you:

1. Enable VM Auto Registration: This is done by setting the PILOT_ENABLE_WORKLOAD_ENTRY_AUTOREGISTRATION variable to true. When this is enabled, new VM instances will register with the WorkloadGroup and new WorkloadEntry CRs will be created to route traffic to the VMs. All Anthos Service Mesh 1.9+ control planes installed with install_asm will include this by default.

2. Install an Expansion gateway: This gateway is named the eastwest gateway and is defined in the Anthos Service Mesh config package. This will also expose the control plane to the VMs.

3. Install the IdentityProvider CRD and register a Google IdentityProvider CR to enable VMs to authenticate to Anthos Service Mesh control plane and securely communicate with the rest of the service mesh.

4. Register the cluster to a fleet and enable workload identity, if you use --enable_all or --enable_registration in the install_asm script.

5. Enable the Service Mesh feature within the fleet. This feature will manage the policies necessary to allow VMs to securely communicate with the mesh.

Add your VMs

In this section, you add Compute Engine instances to your mesh based on the instance template you create with the asm_vm script. The script only generates the necessary configuration for the service proxy agent. To include more configuration in your instance template, create a source instance template and add it to the script.

To add VMs to your mesh, use the following steps:

1. Set the following environment variables to use in later steps. Set these variables for each VM workload:

   • WORKLOAD_NAME is the name of the workload the VM is part of, which must be a compliant DNS-1123 subdomain consisting of lower case alphanumeric characters.
   • WORKLOAD_VERSION is the version of the workload the VM is part of. Optional.
   • WORKLOAD_SERVICE_ACCOUNT is the service GCP Service Account the VM runs as.
   • WORKLOAD_NAMESPACE is the namespace for the workload.
   • ASM_INSTANCE_TEMPLATE is the name of the instance template to be created. Compute Engine instance template name does not allow underscores.
   • SOURCE_INSTANCE_TEMPLATE is the template name to base the generated template on. Optional.

2. Create the namespace for the VM workloads if it doesn't already exist:

   kubectl create ns WORKLOAD_NAMESPACE
   

3. Label the namespace with the control plane revision. For an example of how to find the control plane revision shown as REVISION in the following example, see Deploying and redeploying workloads.

   kubectl label ns WORKLOAD_NAMESPACE istio-injection- istio.io/rev=REVISION --overwrite
   

4. Create the WorkloadGroup for the VMs to be registered:

   kubectl apply -f - << EOF
   apiVersion: networking.istio.io/v1alpha3
   kind: WorkloadGroup
   metadata:
    name: WORKLOAD_NAME
    namespace: WORKLOAD_NAMESPACE
   spec:
    metadata:
      labels:
        app.kubernetes.io/name: WORKLOAD_NAME
        app.kubernetes.io/version: WORKLOAD_VERSION
      annotations:
        security.cloud.google.com/IdentityProvider: google
    template:
      serviceAccount: WORKLOAD_SERVICE_ACCOUNT
   EOF
   

   Field	Description
   name	The name of the workload the VM is part of.
   namespace	The namespace the workload is a part of.
   app.kubernetes.io/name	The recommended labels for Kubernetes applications. You can use your own labels for your VM workloads.
   app.kubernetes.io/version	The recommended labels for Kubernetes applications. You can use your own labels for your VM workloads.
   serviceAccount	The service account identity used by the VM and project, which will be used as part of the identity of the workload in the SPIFFE format. For more information, see Service accounts.
   security.cloud.google.com/IdentityProvider	The identity provider that the VM will use, which should already be registered in your cluster. For Compute Engine VMs it should be set to google. The `IdentityProvider` tells the control plane how to authenticate the VM's credential and where to extract VM's service account.

5. Run the asm_vm script with the following options and flags to create an instance template for your Anthos Service Mesh Compute Engine instances.

   The script verifies cluster prerequisites, adds VM labels for Anthos Service Mesh, generates the custom metadata configuration for the service proxy agent and creates a new instance template.

   If you have an existing instance template that you would like the script to be based on, you can specify the --source_instance_template option. If you want to add non-default VMs to your Anthos Service Mesh, create an instance template with the intended OS distribution and use that template as the value for the --source_instance_template flag in the asm_vm script. If your existing instance template includes a startup script that requires network connectivity, the script should be resilient to transient network connectivity issues. See the demo application for an example of how to add resiliency against temporary network disruption.

   Note: You cannot modify Compute Engine instance templates after creation, so the script creates a new instance template in this case.

   ./asm_vm create_gce_instance_template \
   ASM_INSTANCE_TEMPLATE \
   --project_id PROJECT_ID \
   --cluster_location CLUSTER_LOCATION \
   --cluster_name CLUSTER_NAME \
   --workload_name WORKLOAD_NAME \
   --workload_namespace WORKLOAD_NAMESPACE \
   --source_instance_template SOURCE_INSTANCE_TEMPLATE
   

   Options

   Option	Description
   -p|--project_id PROJECT_ID	The project ID that the cluster was created in.
   -n|--cluster_name CLUSTER_NAME	The name of the cluster.
   -l|--cluster_location CLUSTER_LOCATION	Either the zone (for single-zone clusters) or region (for regional clusters) that the cluster was created in.
   -w|--workload_name WORKLOAD_NAME	The name of the workload the Compute Engine instances represent.
   --workload_namespace WORKLOAD_NAMESPACE	Optional. The namespace of the workload. Default is `default`.
   -s|--source_instance_template SOURCE_INSTANCE_TEMPLATE_NAME	Optional. An existing instance template to use as a base for the Anthos Service Mesh Compute Engine instance template. If not specified, an instance template with default values is created.

   Flags

   Flag	Description
   -v|--verbose	Print commands before and after execution.
   --dry_run	Print commands, but don't execute them.
   --only_validate	Run validation but don't create a new Compute Engine instance template.
   -h|--help	Show a help message describing the options and flags and exit.

6. Set the following environment variables for each MIG that you create:

   • INSTANCE_GROUP_NAME is the name of the Compute Engine instance group to create.
   • ASM_INSTANCE_TEMPLATE is the name of the instance template to be created. Compute Engine instance template name does not allow underscores.
   • INSTANCE_GROUP_ZONE is the zone of the Compute Engine instance group to be created.
   • PROJECT_ID is the project ID that the cluster was created in.
   • SIZE is the size of the instance group to be created. It can be changed after the instance group is created.
   • WORKLOAD_NAME is the name of the workload the VM is part of.
   • WORKLOAD_NAMESPACE is the namespace for the workload.

7. Create a Managed Instance Group for the VM workloads, using the variables created in the previous steps:

   gcloud compute instance-groups managed create \
   INSTANCE_GROUP_NAME \
   --template ASM_INSTANCE_TEMPLATE \
   --zone=INSTANCE_GROUP_ZONE \
   --project=PROJECT_ID \
   --size=SIZE
   

   To scale up or down the number of workloads on Compute Engine instances, starting from a zonal or regional MIG size of zero, see Autoscaling groups of instances. For more information about creating groups, see gcloud compute instance-groups managed create.

   When your instance starts, it will automatically authenticate with the Anthos Service Mesh control plane on your cluster and the control plane will register each VM as a WorkloadEntry.

8. When the VM instance in the MIG finishes starting up, you can view the registered VMs in the workload namespace by using the following command:

      kubectl get workloadentry -n WORKLOAD_NAMESPACE
   

   Note: If the WorkloadGroup or IdentityProvider Custom Resources are deleted, your VM will be unable to communicate with the mesh.

9. Add a Kubernetes Service to expose VM workloads added above. Be sure to have the service select the corresponding label on the VM WorkloadGroup registered above for correct traffic routing.

   The following example creates a Kubernetes service named WORKLOAD_NAME in the namespace WORKLOAD_NAMESPACE that exposes VM workloads with the app.kubernetes.io/name: WORKLOAD_NAME label under HTTP port 80.

   kubectl apply -f - << EOF
   apiVersion: v1
   kind: Service
   metadata:
    name: WORKLOAD_NAME
    namespace: WORKLOAD_NAMESPACE
    labels:
      asm_resource_type: VM
   spec:
    ports:
    - port: 80
      name: http
    selector:
      app.kubernetes.io/name: WORKLOAD_NAME
   EOF
   

   For more details on how to create a Kubernetes service, see https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service.

10. To use a sample application on your VM, see Deploy a sample application.

Anthos Service Mesh control plane upgrade

Upgrade your Anthos Service Mesh control plane to a new version by using the instructions at Upgrading Anthos Service Mesh to the latest version. After installing a new version of the Anthos Service Mesh control plane, redeploy your Kubernetes workloads by using the instructions at Deploying and redeploying workloads.

For VM workloads, create a new instance template and perform a rolling update to the VMs in your MIG:

1. Use the following command to locate the revision label on istiod:

   kubectl -n istio-system get pods -l app=istiod --show-labels
   

   The output from the command is similar to the following. Note that the output for migrations is slightly different than for upgrades. The following example output is from a migration.

   NAME                                READY   STATUS    RESTARTS   AGE   LABELS
   istiod-7744bc8dd7-qhlss             1/1     Running   0          49m   app=istiod,istio.io/rev=default,istio=pilot,pod-template-hash=7744bc8dd7
   istiod-asm-198-6-85d86774f7-flrt2   1/1     Running   0          26m   app=istiod,istio.io/rev=asm-198-6,istio=istiod,pod-template-hash=85d86774f7
   istiod-asm-198-6-85d86774f7-tcwtn   1/1     Running   0          26m   app=istiod,istio.io/rev=asm-198-6,istio=istiod,pod-template-hash=85d86774f7

   1. In the output, under the LABELS column, note the value in the istiod revision label for the new version, which follows the prefix istio.io/rev=. In this example, the value is asm-198-6.

   2. Also note the value in the revision label for the old istiod version. You need this to delete the old version of istiod when you finish moving workloads to the new version. In the example output, the value in the revision label for the old version of istiod is default.

2. Add the revision label to a namespace and remove the istio-injection label (if it exists). In the following command, change REVISION to the value that matches the new revision of istiod.

   kubectl label namespace NAMESPACE istio.io/rev=REVISION istio-injection- --overwrite

   If you see "istio-injection not found" in the output, you can ignore it. That means that the namespace didn't previously have the istio-injection label. Because auto-injection fails if a namespace has both the istio-injection and the revision label, all kubectl label commands in the Anthos Service Mesh documentation include removing the istio-injection label.

3. Create a new instance template using the asm_vm script. Be sure to use a new instance template name and include the same source instance template if you had one for the same workload.

   Note: The --source_instance_template flag is optional. If you have a source template, you need to include it. If you don't have a source template, then don't include the flag.

   ./asm_vm create_gce_instance_template \
   NEW_ASM_INSTANCE_TEMPLATE \
   --project_id PROJECT_ID \
   --cluster_location CLUSTER_LOCATION \
   --cluster_name CLUSTER_NAME \
   --workload_name WORKLOAD_NAME \
   --workload_namespace WORKLOAD_NAMESPACE \
   --source_instance_template SOURCE_INSTANCE_TEMPLATE
   

4. Perform a rolling-update to your existing MIG for the workload.

   For more information, see Starting a basic rolling update.

   gcloud compute instance-groups managed rolling-action start-update INSTANCE_GROUP_NAME \
   --version=template=NEW_ASM_INSTANCE_TEMPLATE \
   --zone=INSTANCE_GROUP_ZONE
   

5. Test the VM workload to ensure that it is working as expected.

Upgrade VM applications

If you have any updates to your application, including changes to the WorkloadGroup and/or changes to your source instance template, a new instance template is required to update the MIG of your VM workloads.

When the WorkloadGroup change is applied and/or the new source instance template is created, you will create a new instance template for Anthos Service Mesh and perform a rolling update to the VMs in your MIG.

1. Create a new instance template using the asm_vm script. Be sure to use a new instance template name and include the new source instance template if you created one for your application update.

   ./asm_vm create_gce_instance_template \
   NEW_ASM_INSTANCE_TEMPLATE \
   --project_id PROJECT_ID \
   --cluster_location CLUSTER_LOCATION \
   --cluster_name CLUSTER_NAME \
   --workload_name WORKLOAD_NAME \
   --workload_namespace WORKLOAD_NAMESPACE \
   --source_instance_template NEW_SOURCE_INSTANCE_TEMPLATE
   

2. Perform a rolling-update to your existing MIG for the workload. For more information about how to use MIG rolling update, see Starting a basic rolling update.

   gcloud compute instance-groups managed rolling-action start-update INSTANCE_GROUP_NAME \
   --version=template=NEW_ASM_INSTANCE_TEMPLATE \
   --zone=INSTANCE_GROUP_ZONE
   

3. Test the VM workload to ensure it works as expected.

Deploy a sample application

To demonstrate that your new mesh configuration is working correctly, you can install the Bookinfo sample application. This example runs a MySQL database on the VM and the ratings service reads the ratings values from the database.

Install Bookinfo on the cluster

Use the following steps to deploy the BookInfo application's services with the sidecar proxies injected alongside each service. The BookInfo application will be deployed in the default namespace.

1. On the command line on the computer where you installed Anthos Service Mesh, go to the root of the Anthos Service Mesh installation directory that you created in the Downloading the script step.

2. To enable automatic sidecar injection, use the following command to locate the label on istiod, which contains the revision label value to use in later steps.

   kubectl -n istio-system get pods -l app=istiod --show-labels
   

   The output looks similar to the following:

   NAME                                READY   STATUS    RESTARTS   AGE   LABELS
   istiod-asm-198-6-5788d57586-bljj4   1/1     Running   0          23h   app=istiod,istio.io/rev=asm-198-6,istio=istiod,pod-template-hash=5788d57586
   istiod-asm-198-6-5788d57586-vsklm   1/1     Running   1          23h   app=istiod,istio.io/rev=asm-198-6,istio=istiod,pod-template-hash=5788d57586
   

   In the output, under the LABELS column, note the value of the istiod revision label, which follows the prefix istio.io/rev=. In this example, the value is asm-198-6.

3. Apply the revision label to the default namespace. In the following command, REVISION is the value of the istiod revision label that you noted in the previous step.

   kubectl label namespace default istio-injection- istio.io/rev=REVISION --overwrite
   

   You can ignore the message "istio-injection not found" in the output. That means that the namespace didn't previously have the istio-injection label, which you should expect in new installations of Anthos Service Mesh or new deployments. Because auto-injection fails if a namespace has both the istio-injection and the revision label, all kubectl label commands in the Anthos Service Mesh documentation include removing the istio-injection label.

4. Deploy your application to the default namespace using kubectl:

   kubectl apply -f samples/bookinfo/platform/kube/bookinfo.yaml
   

5. Confirm that the application has been deployed correctly by running the following commands:

   kubectl get services
   

   Expected output:

   NAME                       CLUSTER-IP   EXTERNAL-IP   PORT(S)              AGE
   details                    10.0.0.31    <none>        9080/TCP             6m
   kubernetes                 10.0.0.1     <none>        443/TCP              7d
   productpage                10.0.0.120   <none>        9080/TCP             6m
   ratings                    10.0.0.15    <none>        9080/TCP             6m
   reviews                    10.0.0.170   <none>        9080/TCP             6m

   and

   kubectl get pod
   

   Expected output:

   NAME                                        READY     STATUS    RESTARTS   AGE
   details-v1-1520924117-48z17                 2/2       Running   0          6m
   productpage-v1-560495357-jk1lz              2/2       Running   0          6m
   ratings-v1-734492171-rnr5l                  2/2       Running   0          6m
   reviews-v1-874083890-f0qf0                  2/2       Running   0          6m
   reviews-v2-1343845940-b34q5                 2/2       Running   0          6m
   reviews-v3-1813607990-8ch52                 2/2       Running   0          6m

6. Finally, define the ingress gateway routing for the application:

   kubectl apply -f samples/bookinfo/networking/bookinfo-gateway.yaml
   

   Expected output:

   gateway.networking.istio.io/bookinfo-gateway created
   virtualservice.networking.istio.io/bookinfo created

7. Confirm that the product page is accessible.

   export INGRESS_HOST=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
   export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].port}')
   export GATEWAY_URL="${INGRESS_HOST}:${INGRESS_PORT}"
   curl -s "http://${GATEWAY_URL}/productpage" | grep -o "<title>.*</title>"
   

   Expected output:

   <title>Simple Bookstore App</title>
   

Create Compute Engine instances and install MySQL

In this step, you will create a Compute Engine instance template for the MySQL instance running on the VM. For more detailed steps, see Bookinfo with a Virtual Machine.

1. Create a Compute Engine instance template that includes a startup script to install MySQL and add a ratings database upon startup. Note that if you are using CentOS it will take up to 10 minutes for the mariadb-server to be ready.

   Debian

   cat << "EOF" > init-mysql
   #!/bin/bash
   
   # Wait until Envoy is ready before installing mysql
   while true; do
     rt=$(curl -s 127.0.0.1:15000/ready)
     if [[ $? -eq 0 ]] && [[ "${rt}" -eq "LIVE" ]]; then
       echo "envoy is ready"
       break
     fi
     sleep 1
   done
   
   sudo apt-get update && sudo apt-get install -y mariadb-server
   
   sudo sed -i '/bind-address/c\bind-address  = 0.0.0.0' /etc/mysql/mariadb.conf.d/50-server.cnf
   
   cat <<EOD | sudo mysql
   # Grant access to root
   GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' IDENTIFIED BY 'password' WITH GRANT OPTION;
   
   # Grant root access to other IPs
   CREATE USER 'root'@'%' IDENTIFIED BY 'password';
   GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' WITH GRANT OPTION;
   FLUSH PRIVILEGES;
   quit
   EOD
   
   sudo systemctl restart mysql
   
   curl -LO https://raw.githubusercontent.com/istio/istio/release-1.9/samples/bookinfo/src/mysql/mysqldb-init.sql
   
   mysql -u root -ppassword < mysqldb-init.sql
   EOF
   
   gcloud compute \
   --project=PROJECT_ID \
   instance-templates create mysql-instance-template \
   --machine-type=e2-medium \
   --metadata-from-file=startup-script=init-mysql \
   --image=debian-10-buster-v20201014 \
   --image-project=debian-cloud \
   --boot-disk-size=10GB
   

   CentOS

   cat << "EOF" > init-mysql
   #!/bin/bash
   
   # Wait until Envoy is ready before installing mysql
   while true; do
     rt=$(curl -s 127.0.0.1:15000/ready)
     if [[ $? -eq 0 ]] && [[ "${rt}" -eq "LIVE" ]]; then
       echo "envoy is ready"
       break
     fi
     sleep 1
   done
   
   sudo yum update -y && sudo yum install -y mariadb-server
   
   # Wait until mysql is ready
   while true; do
     rt=$(which mysql)
     if [[ ! -z "${rt}" ]]; then
       echo "mysql is ready"
       break
     fi
     sleep 1
   done
   
   sudo sed -i '/bind-address/c\bind-address  = 0.0.0.0' /etc/my.cnf.d/mariadb-server.cnf
   
   sudo systemctl restart mariadb
   
   cat > grantaccess.sql << EOD
   
   GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' IDENTIFIED BY 'password' WITH GRANT OPTION;
   
   CREATE USER 'root'@'%' IDENTIFIED BY 'password';
   GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' WITH GRANT OPTION;
   FLUSH PRIVILEGES;
   EOD
   
   until sudo mysql < grantaccess.sql; do
      sleep 1
   done
   
   sudo systemctl restart mariadb
   
   curl -LO https://raw.githubusercontent.com/istio/istio/release-1.9/samples/bookinfo/src/mysql/mysqldb-init.sql
   
   mysql -u root -ppassword < mysqldb-init.sql
   EOF
   
   gcloud compute \
   --project=PROJECT_ID \
   instance-templates create mysql-instance-template \
   --machine-type=e2-medium \
   --metadata-from-file=startup-script=init-mysql \
   --image-project=centos-cloud \
   --image-family=centos-8 \
   --boot-disk-size=30GB
   

2. Create a WorkloadGroup for the MySQL workload

   kubectl apply -f - << EOF
   apiVersion: networking.istio.io/v1alpha3
   kind: WorkloadGroup
   metadata:
    name: mysql
    namespace: default
   spec:
    metadata:
      labels:
        app.kubernetes.io/name: mysql
      annotations:
        security.cloud.google.com/IdentityProvider: google
    template:
      serviceAccount: WORKLOAD_SERVICE_ACCOUNT
   EOF
   

3. Use the VM script above to create a new instance template to prepare the instances for your mesh.

   ./asm_vm create_gce_instance_template \
   asm-mysql-instance-template \
   --project_id PROJECT_ID \
   --cluster_location CLUSTER_LOCATION \
   --cluster_name CLUSTER_NAME \
   --workload_name mysql \
   --source_instance_template mysql-instance-template
   

4. Create a Compute Engine MIG using the newly created instance template.

   gcloud compute instance-groups managed create mysql-instance \
   --template asm-mysql-instance-template \
   --zone=us-central1-c \
   --project=PROJECT_ID \
   --size=1
   

Create a service

Create a Kubernetes service for the MySQL service.

1. Create a Kubernetes service by using the following command:

   kubectl apply -f - << EOF
   apiVersion: v1
   kind: Service
   metadata:
    name: mysql
    namespace: default
    labels:
      asm_resource_type: VM
   spec:
    ports:
    - name: mysql
      port: 3306
      protocol: TCP
      targetPort: 3306
    selector:
      app.kubernetes.io/name: mysql
   EOF
   

Use the Anthos UI dashboard

To see the new VM-based service you've created, click Anthos > Service Mesh from the main left navigation bar. It will display a table of the services running in your mesh. The service you added should appear in the table, with a Type value of VM and some high-level metrics. To see more telemetry from your VM-based service, click the service name, which will display the service-level dashboard.

For more information on how to use the Anthos UI dashboard, see Exploring Anthos Service Mesh in the Cloud console.

Note: Only HTTP metrics are currently displayed in the GKE Enterprise UI dashboard. For more information, see Protocol support.

Manage traffic to the VM workloads

You can change the networking rules to control how the traffic flows in and out of the VM(s).

Control traffic to a new ratings service (Pod to VM)

Create another ratings service in Bookinfo that will use the MySQL instance created above as the data source and specify a routing rule that forces the review service to use the new rating service.

1. Create a new rating service to use the MySQL instance.

   kubectl apply -f - << EOF
   apiVersion: apps/v1
   kind: Deployment
   metadata:
    name: ratings-v2-mysql-vm
    labels:
      app: ratings
      version: v2-mysql-vm
   spec:
    replicas: 1
    selector:
      matchLabels:
        app: ratings
        version: v2-mysql-vm
    template:
      metadata:
        labels:
          app: ratings
          version: v2-mysql-vm
      spec:
        serviceAccountName: bookinfo-ratings
        containers:
        - name: ratings
          image: docker.io/istio/examples-bookinfo-ratings-v2:1.16.2
          imagePullPolicy: IfNotPresent
          env:
            - name: DB_TYPE
              value: "mysql"
            - name: MYSQL_DB_HOST
              value: mysql.default.svc.cluster.local
            - name: MYSQL_DB_PORT
              value: "3306"
            - name: MYSQL_DB_USER
              value: root
            - name: MYSQL_DB_PASSWORD
              value: password
          ports:
          - containerPort: 9080
   EOF
   

2. Create a routing rule.

   kubectl apply -f - << EOF
   apiVersion: networking.istio.io/v1alpha3
   kind: VirtualService
   metadata:
    name: reviews
   spec:
    hosts:
    - reviews
    http:
    - route:
      - destination:
          host: reviews
          subset: v3
   ---
   apiVersion: networking.istio.io/v1alpha3
   kind: VirtualService
   metadata:
    name: ratings
   spec:
    hosts:
    - ratings
    http:
    - route:
      - destination:
          host: ratings
          subset: v2-mysql-vm
   EOF
   

3. Apply destination rules for the created services.

   kubectl apply -f - << EOF
   apiVersion: networking.istio.io/v1alpha3
   kind: DestinationRule
   metadata:
    name: reviews
   spec:
    host: reviews
    subsets:
    - name: v1
      labels:
        version: v1
    - name: v2
      labels:
        version: v2
    - name: v3
      labels:
        version: v3
   ---
   apiVersion: networking.istio.io/v1alpha3
   kind: DestinationRule
   metadata:
    name: ratings
   spec:
    host: ratings
    subsets:
    - name: v1
      labels:
        version: v1
    - name: v2
      labels:
        version: v2
    - name: v2-mysql
      labels:
        version: v2-mysql
    - name: v2-mysql-vm
      labels:
        version: v2-mysql-vm
   EOF
   

Validating the application deployment

To see if the BookInfo application is working, you need to send traffic to the ingress gateway.

• If you installed Anthos Service Mesh on GKE, get the external IP address of the ingress gateway that you created in previous steps:

  kubectl get svc istio-ingressgateway -n istio-system
  

  Output:

  NAME                   TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                                      AGE
  istio-ingressgateway   LoadBalancer   10.19.247.233   35.239.7.64   80:31380/TCP,443:31390/TCP,31400:31400/TCP   27m

  In this example, the IP address of the ingress service is 35.239.7.64.

Trying the application

1. Check that the BookInfo app is running with curl:

   curl -I http://EXTERNAL_IP/productpage
   

   If the response shows 200, it means the application is working properly with Anthos Service Mesh.

2. To view the BookInfo web page, enter the following address in your browser:

   http://EXTERNAL_IP/productpage
   

3. Verify on the Bookinfo application homepage that it is showing five stars from Reviewer1 and four stars from Reviewer2.

Enforcing security on the VM workloads

Enforcing security on the VM workloads is the same as enforcing security on the Kubernetes workloads. For more information, see Istio security.

After you complete the previous steps, your Compute Engine VM will have a Google-issued workload certificate. In the certificate, the SubjectAlternativeName value shows the VM's Anthos workload identity in the form spiffe://<workload_identity_pool>/ns/WORKLOAD_NAMESPACE/sa/WORKLOAD_SERVICE_ACCOUNT.

For more information, see workload identity pool.

Enable mTLS strict mode for the mesh

Apply the following YAML to enforce strict mTLS mesh-wide.

kubectl apply -f - << EOF
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: istio-system
spec:
  mtls:
    mode: STRICT
EOF

Authorization for service-to-service traffic

Use AuthorizationPolicy to control access between the applications on your Compute Engine VM and other mesh workloads (e.g., on the GKE cluster).

Note: You should use cluster.local in all your authorization policies to refer to the workload identity pool. This allows you to easily update the workload identity pool for your mesh in the case of migration or workload identity pool format changes.

Example: Deny Kubernetes workloads to access Compute Engine VMs

The following authorization policy denies a Kubernetes workload ratings to access Compute Engine VM workloads that serve the ratings MySQL server.

kubectl apply -f - << EOF
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: mysql-deny
  namespace: default
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: mysql
  action: DENY
  rules:
  - from:
    - source:
        principals: ["cluster.local/ns/default/sa/bookinfo-ratings"]
EOF

After applying the example AuthorizationPolicy, you should see a Ratings service is currently unavailable error message in the book reviews section on the product page.

Installing the Cloud Monitoring Agent

You can install the Cloud Monitoring Agent to collect and monitor system and application metrics from your VM instances. This allows you to monitor key metrics, for example CPU and memory utilization on the agent.

For more information, see Cloud Monitoring Agent documentation.

Troubleshooting

For troubleshooting tips, see Troubleshooting VM support.