Configure managed Anthos Service Mesh with fleet API

This page shows you how to use the fleet feature API to set up managed Anthos Service Mesh with automatic control plane management.

When you enable automatic control plane management on a GKE cluster, Google applies the recommended configuration of managed Anthos Service Mesh based on your cluster's release channel and keeps it up to date.

Use this preview onboarding path if you want:

  • To use gcloud to configure managed Anthos Service Mesh using Google Cloud APIs and IAM.
  • To configure Anthos Service Mesh using the same APIs as other fleet features.
  • To automatically get the recommended configuration of Anthos Service Mesh for each of your clusters.

For the GA onboarding experience using the asmcli tool, see Configure managed Anthos Service Mesh.

Prerequisites

As a starting point, this guide assumes that you have:

Requirements

  • One or more clusters with a supported version of GKE, in one of the supported regions.
  • Your clusters must be registered to a fleet. This is included in the instructions, or can be done separately prior to the installation.
  • Your project must have the Service Mesh Feature enabled. This is included in the instructions or can be done separately.
  • All your GKE clusters must be in a single project, on a single network.

Limitations

We recommend that you review the list of managed Anthos Service Mesh supported features and limitations. In particular, note the following:

  • The IstioOperator API isn't supported since its main purpose is to control in-cluster components.

  • Every cluster in the fleet running Anthos Service Mesh must use Mesh CA.

  • The managed data plane is available on the Regular and Rapid release channels.

  • The actual features available to managed Anthos Service Mesh depend on the release channel. For more information, review the full list of managed Anthos Service Mesh supported features and limitations.

  • During the provisioning process for a Google-managed control plane, Istio CRDs corresponding to the selected channel are installed in the specified cluster. If there are existing Istio CRDs in the cluster, they will be overwritten.

Before you begin

  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
  2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  3. Make sure that billing is enabled for your Cloud project. Learn how to check if billing is enabled on a project.

  4. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  5. Make sure that billing is enabled for your Cloud project. Learn how to check if billing is enabled on a project.

  6. Enable the required APIs:

      gcloud services enable mesh.googleapis.com \
          --project=PROJECT_ID
    

Enabling mesh.googleapis.com enables the following APIs:

API
meshconfig.googleapis.com
meshca.googleapis.com
container.googleapis.com
gkehub.googleapis.com
monitoring.googleapis.com
stackdriver.googleapis.com
opsconfigmonitoring.googleapis.com
iam.googleapis.com
iamcredentials.googleapis.com
bigquery.googleapis.com
bigquerystorage.googleapis.com
compute.googleapis.com
oslogin.googleapis.com
containerregistry.googleapis.com
pubsub.googleapis.com
storage-api.googleapis.com
gkeconnect.googleapis.com
multiclustermetering.googleapis.com
logging.googleapis.com
connectgateway.googleapis.com

Configure gcloud

Do the following steps even if you are using Cloud Shell.

  1. Authenticate with the Google Cloud CLI:

    gcloud auth login --project PROJECT_ID
    
  2. Update the components:

    gcloud components update
    
  3. If you are installing Anthos Service Mesh on a GKE cluster, configure kubectl to point to the cluster.

    gcloud container clusters get-credentials CLUSTER_NAME \
         --zone CLUSTER_LOCATION \
         --project PROJECT_ID
    

Enable Anthos Service Mesh

Enable Anthos Service Mesh on the fleet. Note that if you plan to register multiple clusters, enabling Anthos Service Mesh happens at the project-level so you only have to run this command once.

gcloud container fleet mesh enable --project PROJECT_ID

where:

  • PROJECT_ID is the ID of your current project.

Register clusters to a fleet

  1. Register a GKE cluster using Workload Identity to a fleet:

    gcloud container fleet memberships register MEMBERSHIP_NAME \
         --gke-cluster=GKE_CLUSTER \
         --enable-workload-identity \
         --project PROJECT_ID
    

    where:

    • MEMBERSHIP_NAME is the membership name that you choose to uniquely represent the cluster being registered to the fleet.

    • GKE_CLUSTER is the location/name of the GKE cluster from the current project. The location can be a zone or a region, for example: us-central1-a/my-gke-cluster.

  2. Verify your cluster is registered:

    gcloud container fleet memberships list --project PROJECT_ID
    

Configure each cluster

Use the following steps to configure managed Anthos Service Mesh for each cluster in your mesh.

Apply the mesh_id label

Apply the mesh_id label for your GKE cluster:

  gcloud container clusters update CLUSTER_NAME --zone ZONE\
      --update-labels mesh_id=proj-PROJECT_NUMBER

where:

  • CLUSTER_NAME is the name of your cluster.
  • ZONE is the Compute zone for your cluster.
  • PROJECT_NUMBER is the unique identifier for your project.

Enable automatic control plane management

Run the following command to enable automatic control plane management:

  gcloud container fleet mesh update \
     --control-plane automatic \
     --memberships MEMBERSHIP_NAME \
     --project PROJECT_ID

Note that an ingress gateway isn't automatically deployed with the control plane. Decoupling the deployment of the ingress gateway and control plane allows you to more easily manage your gateways in a production environment. If the cluster needs an ingress gateway or an egress gateway, see Deploy gateways. To enable other optional features, see Enabling optional features on managed Anthos Service Mesh.

Verify the control plane has been provisioned

  1. After a few minutes, verify that the control plane status is ACTIVE:

    gcloud container fleet mesh describe --project PROJECT_ID
    

    The output is similar to:

    ...
    membershipSpecs:
      projects/746296320118/locations/global/memberships/demo-cluster-1:
        mesh:
          controlPlane: AUTOMATIC
    membershipStates:
      projects/746296320118/locations/global/memberships/demo-cluster-1:
        servicemesh:
          controlPlaneManagement:
            details:
            - code: REVISION_READY
              details: 'Ready: asm-managed'
            state: ACTIVE
        state:
          code: OK
          description: 'Revision(s) ready for use: asm-managed.'
    ...
    

    Take note of the revision label in the description: field, for example, asm-managed in the provided output. You will need to set this label before you Deploy applications.

Apply the managed data plane (optional)

If you want Google to fully manage upgrades of the proxies, enable the managed data plane. If enabled, the sidecar proxies and injected gateways are automatically updated in conjunction with the managed control plane by restarting workloads to re-inject new versions of the proxy. If disabled, proxy management is driven by the natural lifecycle of the Pods in the cluster and must be manually triggered by the user to control the update rate.

The managed data plane upgrades proxies by evicting Pods that are running older versions of the proxy. The evictions are done gradually, honoring the Pod disruption budget and controlling the rate of change.

Note that the managed data plane requires the Istio Container Network Interface (CNI) plugin, which is enabled by default when you deploy the managed control plane.

This Preview release of managed data plane doesn't manage the following:

  • Uninjected pods
  • Manually injected pods
  • Jobs
  • StatefulSets
  • DaemonSets

The managed data plane is available on both the Rapid and Regular release channels.

To enable the managed data plane:

  1. Enable data plane management:

    kubectl annotate --overwrite namespace NAMESPACE \
    mesh.cloud.google.com/proxy='{"managed":"true"}'
    

    Alternatively, you can enable the managed data plane for a specific Pod by annotating it with the same annotation.

  2. Repeat the previous step for each namespace that you want a managed data plane.

    It could take up to ten minutes for the service to be ready to manage the proxies in the cluster. Run the following command to check the status:

    gcloud alpha container fleet mesh describe --project PROJECT_ID
    

    Expected output

     membershipStates:
       projects/PROJECT_NUMBER/locations/global/memberships/CLUSTER_NAME:
         servicemesh:
           dataPlaneManagement:
             details:
             - code: OK
               details: Service is running.
             state: ACTIVE
         state:
           code: OK
           description: 'Revision(s) ready for use: asm-managed-rapid.'
     ```
    

If the service does not become ready within ten minutes, see Managed data plane status for next steps.

If you want to disable the managed data plane and revert back to managing the sidecar proxies yourself, change the annotation:

kubectl annotate --overwrite namespace NAMESPACE \
  mesh.cloud.google.com/proxy='{"managed":"false"}'

Enable maintenance notifications

You can request to be notified about upcoming maintenance up to week before maintenance is scheduled. Maintenance notifications are not sent by default. You must also Configure a GKE maintenance window before you can receive notifications.

To opt in to maintenance notifications:

  1. Go to the Communication page.

    Go to the Communication page

  2. In the Anthos Service Mesh Upgrade row, under the Email column, select the radio button to turn maintenance notifications ON.

Each user that wants to receive notifications must opt in separately. If you want to set an email filter for these notifications, the subject line is:

Upcoming upgrade for your ASM cluster "CLUSTER_LOCATION/CLUSTER_NAME".

Configure endpoint discovery (only for multi-cluster installations)

Before you continue, you should have already configured managed Anthos Service Mesh on each cluster as described in the previous steps. There is no need to indicate that a cluster is a primary cluster, this is the default behavior. You must finish the Setting the project and cluster variables and Create firewall rule sections before configuring endpoint discovery.

Public clusters

Configure endpoint discovery between public clusters

If you are operating on public clusters (non-private clusters), you can either Configure endpoint discovery between public clusters or more simply Enable endpoint discovery between public clusters.

Private clusters

Configure endpoint discovery between private clusters

When using GKE private clusters, you must configure the cluster control plane endpoint to be the public endpoint instead of the private endpoint. Please refer to Configure endpoint discovery between private clusters.

For an example application with two clusters, see HelloWorld service example.

Deploy applications

To deploy applications, use either the label corresponding to the channel you configured during installation or istio-injection=enabled if you are using default injection labels.

Default injection label

kubectl label namespace NAMESPACE istio-injection=enabled istio.io/rev- --overwrite

Revision label

Before you deploy applications, remove any previous istio-injection labels from their namespaces and set the istio.io/rev=asm-managed-rapid label instead.

This is the revision label you identified when you verified the control plane. If you are using a different revision label, click asm-managed-rapid, and replace it with the applicable label: asm-managed for Regular or asm-managed-stable for Stable.

The revision label corresponds to a release channel:

Revision label Channel
asm-managed Regular
asm-managed-rapid Rapid
asm-managed-stable Stable
kubectl label namespace NAMESPACE istio-injection- istio.io/rev=REVISION_LABEL --overwrite

At this point, you have successfully configured Anthos Service Mesh managed control plane. If you also applied the managed data plane, restart your workloads. If not, perform a rolling update. You are now ready to deploy your applications or you can deploy the Bookinfo sample application.

If you deploy an application in a multi-cluster setup, replicate the Kubernetes and control plane configuration in all clusters, unless you plan to limit that particular config to a subset of clusters. The configuration applied to a particular cluster is the source of truth for that cluster. In addition, if the cluster also runs Anthos Service Mesh or Certificate Authority Service with Mesh CA in other namespaces, verify the application can communicate with the other applications controlled by the in-cluster control plane.

Disable automatic control plane management

Disabling automatic control plane management does not deprovision any resources. All resources are left in the cluster for you to manually manage or remove. To completely offboard, see Uninstall Anthos Service Mesh.

  1. Run the following command to disable automatic control plane management:

    gcloud container fleet mesh update \
       --control-plane manual \
       --memberships MEMBERSHIP_NAME \
       --project PROJECT_ID
    
  2. After a few minutes, verify the status of automatic control plane management is DISABLED:

    gcloud container fleet mesh describe --project PROJECT_ID
    

    The output is similar to:

    ...
    membershipSpecs:
      projects/projectid/locations/global/memberships/cluster-name:
        mesh:
          controlPlane: MANUAL
    membershipStates:
      projects/projectid/locations/global/memberships/cluster-name:
        servicemesh:
          controlPlaneManagement:
            state: DISABLED
        state:
          code: OK
          description: 'Revision(s) ready for use: asm-managed.'
    ...
    

    To completely uninstall Anthos Service Mesh, see Uninstall Anthos Service Mesh.

What's next