Configuring the Google-managed control plane

Preview

This product or feature is covered by the Pre-GA Offerings Terms of the Google Cloud Platform Terms of Service. Pre-GA products and features may have limited support, and changes to pre-GA products and features may not be compatible with other pre-GA versions. For more information, see the launch stage descriptions.

Overview

Anthos Service Mesh Google-managed control plane is a Google Cloud managed service that you only need to configure, while Google handles its reliability, upgrades, scaling and security for you in a backward-compatible manner. This guide explains how to set up or migrate applications to Google-managed control plane in a single or multi-cluster configuration.

You can use a single Google-managed control plane for multiple GKE clusters in a single Google Cloud project.

To learn about the supported features and limitations of Anthos Service Mesh managed control plane, see Anthos Service Mesh Google-managed control plane supported features.

Prerequisites

This guide assumes that you have:

• A Cloud project.
• A Cloud Billing account.
• Reviewed the list of Google-managed control plane supported features and limitations.
• Installation script, kpt, and other tools specified in Installing the required tools.
• One or more clusters with GKE 1.16 to 1.20 installed, in one of these available regions:
  • asia-east1
  • asia-southeast1
  • europe-north1
  • europe-west1
  • europe-west4
  • us-central1
  • us-east1
  • us-east4
  • us-west1
  • us-west2
  • us-west3
  • us-west4
• Your clusters must have Workload Identity enabled.

If Workload Identity isn't enabled, you can enable it by using the following command:

Note: This command might take 10-15 minutes to complete, depending on your clusters' configuration and workload.

gcloud container clusters update CLUSTER_NAME --zone LOCATION \
    --workload-pool=PROJECT_ID.svc.id.goog

Download the installation script

1. Download the version of the script that installs Anthos Service Mesh 1.9.3 to the current working directory:

   curl https://storage.googleapis.com/csm-artifacts/asm/install_asm_1.9 > install_asm
   

2. Download the SHA-256 of the file to the current working directory:

   curl https://storage.googleapis.com/csm-artifacts/asm/install_asm_1.9.sha256 > install_asm.sha256
   

3. With both files in the same directory, verify the download:

   sha256sum -c --ignore-missing install_asm.sha256
   

   If the verification is successful, the command outputs: install_asm: OK For compatibility, the preceding command includes the --ignore-missing flag to allow any version of the script to be renamed to install_asm.

4. Make the script executable:

   chmod +x install_asm
   

Configure each cluster

Use the following steps to configure the Google-managed control plane for each cluster in your mesh.

Get cluster credentials

Retrieve the appropriate credentials. The following command will also point the kubectl context to the target cluster.

Note: In the following command, you might replace --zone with --region depending on your cluster configuration.

gcloud container clusters get-credentials  CLUSTER_NAME \
--zone LOCATION \
--project PROJECT_ID

Apply the Google-managed control plane

Run the installation script for each cluster that will use the Google-managed control plane:

./install_asm --mode install --managed -p PROJECT_ID \
    -l LOCATION -n CLUSTER_NAME -v \
    --output_dir CLUSTER_NAME --enable-all

The script will download to the specified --output_dir all the files for configuring the managed control plane, installing an Istio Gateway, along with the istioctl tool and sample applications.

Note: The steps in this guide assume that you run istioctl from the root of the installation directory, with istioctl present in its /bin subdirectory.

Install Istio Gateways (optional)

Optionally, you can install one or more Istio Gateways in your cluster to handle typical ingress traffic, by using the following steps:

1. Choose one of these two options to set up the namespace where you will deploy the gateway in later steps.

   • Enable the namespace for injection:

     kubectl label namespace GATEWAY_NAMESPACE istio-injection- istio.io/rev=asm-managed --overwrite
     

   OR

   • Enable injection only for the gateway pod, but not all pods in the namespace. This command removes all injection labels and later you will enable injection on the pod itself:

     kubectl label namespace GATEWAY_NAMESPACE istio-injection- istio.io/rev-
     

2. Create a Deployment and Service for the gateway, by using the following minimal example:

   apiVersion: v1
   kind: Service
   metadata:
     name: istio-ingressgateway
     namespace: GATEWAY_NAMESPACE
   spec:
     type: LoadBalancer
     selector:
       istio: ingressgateway
     ports:
     - port: 80
       name: http
     - port: 443
       name: https
   ---
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: istio-ingressgateway
     namespace: GATEWAY_NAMESPACE
   spec:
     selector:
       matchLabels:
         istio: ingressgateway
     template:
       metadata:
         annotations:
           # This is required to tell Anthos Service Mesh to inject the gateway with the
           # required configuration.
           inject.istio.io/templates: gateway
         labels:
           istio: ingressgateway
           istio.io/rev: asm-managed # This is required only if the namespace is not labeled.
       spec:
         containers:
         - name: istio-proxy
           image: auto # The image will automatically update each time the pod starts.
   ---
   apiVersion: rbac.authorization.k8s.io/v1
   kind: Role
   metadata:
     name: istio-ingressgateway-sds
     namespace: `GATEWAY_NAMESPACE`
   rules:
   - apiGroups: [""]
     resources: ["secrets"]
     verbs: ["get", "watch", "list"]
   ---
   apiVersion: rbac.authorization.k8s.io/v1
   kind: RoleBinding
   metadata:
     name: istio-ingressgateway-sds
     namespace: `GATEWAY_NAMESPACE`
   roleRef:
     apiGroup: rbac.authorization.k8s.io
     kind: Role
     name: istio-ingressgateway-sds
   subjects:
   - kind: ServiceAccount
     name: default
   

3. After you create the deployment, verify that the new services are working correctly:

   kubectl get pod,service -n GATEWAY_NAMESPACE
   

   Verify the output similar to the following:

   NAME                                      READY   STATUS    RESTARTS   AGE
   pod/istio-ingressgateway-856b7c77-bdb77   1/1     Running   0          3s
   
   NAME                           TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)        AGE
   service/istio-ingressgateway   LoadBalancer   10.24.5.129    34.82.157.6      80:31904/TCP   3s

Configure endpoint discovery (only for multi-cluster installations)

Note: If your mesh has only one cluster, skip these multi-cluster steps and proceed to Deploy applications or Migrate applications.

Anthos Service Mesh managed control plane supports a single-project, single-network, multi-primary configuration, with the difference that the control plane is not installed in the cluster. A multi-primary configuration means that each cluster is configured with secrets to all other clusters, which allows each cluster to discover endpoints from all the other clusters.

Before you continue, you should have already run the install script on each cluster as described in the previous steps. There is no need to indicate that a cluster is a primary cluster, this is the default behavior.

For every cluster, enable endpoints discovery by running the following commands once for every other cluster i=1..N in the mesh:

1. For every cluster, ensure the kubectl config context points to the current cluster:

   export CTX=gke_PROJECT_ID_LOCATION_CLUSTER_NAME
   

2. Enable endpoints discovery by running the following commands once for every other cluster i=1..N in the mesh:

   export CTX_i=gke_PROJECT_ID_LOCATION_i_CLUSTER_NAME_i
   

   ./bin/istioctl x create-remote-secret --context=${CTX_i} --name=CLUSTER_NAME_i | \
     kubectl apply -f - --context=${CTX}
   

3. Verify the secret has been created by running the command:

   kubectl get secret istio-remote-secret-CLUSTER_NAME_i -n istio-system
   

   Verify the expected output:

   NAME                                   TYPE     DATA   AGE
   istio-remote-secret-CLUSTER_NAME_i   Opaque   1      44s
   

4. If the current cluster is added to an existing multi-cluster mesh, let all the other clusters discover its endpoints by creating a secret corresponding to the current cluster in all the other clusters:

   ./bin/istioctl x create-remote-secret --context=${CTX} --name=CLUSTER_NAME | \
     kubectl apply -f - --context=${CTX_i}
   

5. In addition, you can verify the secret for the other cluster also:

   kubectl get secret istio-remote-secret-CLUSTER_NAME -n istio-system --context -CTX_i
   

   Verify the expected output:

   NAME                            TYPE     DATA   AGE
   istio-remote-secret-CLUSTER_NAME   Opaque   1      44s
   

For more details and an example with two clusters, see enable endpoint discovery.

Deploy applications

Before you deploy applications, remove any previous istio-injection labels from their namespaces, and set the istio.io/rev:asm-managed label instead:

Note: You can ignore any label "istio-injection" not found errors.

kubectl label namespace NAMESPACE istio-injection- istio.io/rev=asm-managed --overwrite

At this point, you have successfully configured Anthos Service Mesh managed control plane. You are now ready to deploy your applications or you can deploy the Bookinfo sample application.

If you deploy an application in a multi-cluster setup, replicate the Kubernetes and control plane configuration in all clusters, unless you plan to limit that particular config to a subset of clusters. The configuration applied to a particular cluster is the source of truth for that cluster. In addition, if the cluster also runs Anthos Service Mesh 1.7, 1.8 or higher with Mesh CA in other namespaces, verify the application can communicate with the other applications controlled by the customer-managed control plane.

Verify control plane metrics

To verify that your configuration works correctly:

1. In the Cloud Console, view the control plane metrics:

   Go to Metrics Explorer

2. Choose your workspace and add a custom query using the following parameters:

   • Resource type: Kubernetes Container
   • Metric: Proxy Clients
   • Filter: container_name="cr-asm-managed"
   • Group By: revision label and proxy_version label
   • Aggregator sum
   • Period: 1 minute

   When you run Anthos Service Mesh with both a Google-managed and a customer-managed control plane, you can tell the metrics apart by their container name. For example, managed metrics have container_name="cr-asm-managed", while unmanaged metrics have container_name="discovery". To display metrics from both, remove the Filter on container_name="cr-asm-managed".

3. Verify the control plane version and proxy version by inspecting the following fields in Metrics Explorer:

   • The revision field indicates the control plane version.
   • The proxy_version field indicates the proxy_version.
   • The value field indicates the number of connected proxies.

Migrate applications to Google-managed control plane

The Anthos Service Mesh Google-managed control plane only supports migration from Anthos Service Mesh 1.9 that uses Mesh CA.

To migrate to Google-managed control plane, perform the following steps:

1. Run the script as indicated in the Install the control plane section.

2. Replace the current namespace label with the istio.io/rev:asm-managed label:

   kubectl label namespace NAMESPACE istio-injection- istio.io/rev=asm-managed \
       --overwrite
   

3. Perform a rolling upgrade of deployments in the namespace:

   kubectl rollout restart deployment -n NAMESPACE
   

4. Test your application to verify that the workloads function correctly.

5. If you have workloads in other namespaces, repeat the previous steps for each namespace.

6. If you deployed the application in a multi-cluster setup, replicate the Kubernetes and Istio configuration in all clusters, unless there is a desire to limit that configuration to a subset of clusters only. The configuration applied to a particular cluster is the source of truth for that cluster.

7. Check that the metrics appear as expected by following the steps in Verify control plane metrics.

A cluster can have a mix of revisions, for example Anthos Service Mesh 1.7 and 1.8, and customer-managed Anthos Service Mesh together. You can mix namespaces using different Anthos Service Mesh control plane revisions indefinitely.

If you are satisfied that your application works as expected, you can remove the in-cluster istiod after you switch all namespaces to the customer-managed control plane, or keep them as a backup - istiod will automatically scale down to use fewer resources. To remove, skip to Delete old control plane.

If you encounter problems, you can identify and resolve them by using the information in Resolving managed control plane issues and if necessary, roll back to the previous version.

Delete old control plane

After you install and confirm that all namespaces use the Google-managed control plane, you can delete the old control plane.

kubectl delete Service,Deployment,HorizontalPodAutoscaler,PodDisruptionBudget istiod -n istio-system --ignore-not-found=true

If you used istioctl kube-inject instead of automatic injection, or if you installed additional gateways, check the metrics for the control plane, and verify that the number of connected endpoints is zero.

Roll back

Perform the following steps if you need to roll back to the previous control plane version:

1. Update workloads to be injected with the previous version of the control plane. In the following command, the revision value asm-191-1 is used only as an example. Replace the example value with the revision label of your previous control plane.

   kubectl label namespace NAMESPACE istio-injection- istio.io/rev=asm-191-1 --overwrite
   

2. Restart the Pods to trigger re-injection so the proxies have the previous version:

   kubectl rollout restart deployment -n NAMESPACE
   

The managed control plane will automatically scale to zero and not use any resource when not in use. The mutating webhooks and provisioning will remain and do not affect cluster behavior.

The gateway is now set to the asm-managed revision. To roll back, re-run the Anthos Service Mesh install command, which will re-deploy gateway pointing back to your customer-managed control plane:

kubectl -n istio-system rollout undo deploy istio-ingressgateway

Expect this output on success:

deployment.apps/istio-ingressgateway rolled back

Uninstall

Google-managed control plane will auto-scale to zero when no namespaces are using it, therefore no uninstallation is required.

Troubleshooting

To identify and resolve problems when using managed control plane, see Resolving managed control plane issues.