Cloud Service Mesh and Traffic Director are now Cloud Service Mesh. For more information, see the Cloud Service Mesh overview.

Managed control plane modernization

Google will gradually update existing fleets which used the ISTIOD managed control plane implementation to use the TRAFFIC_DIRECTOR implementation. By default Google will automatically migrate your fleets, but you may opt to drive the migration yourself. To check which control plane your fleet is using, see Check control plane implementation.

Consider the following when preparing your fleet for modernization:

To prepare for modernization, you have two ways to control the process:
- Google driven automatic migration (default) - You can customize the order that your clusters get modernized using instructions in Google-driven modernization.
- Customer driven migration (optional) - You can choose to drive the modernization of your fleets yourself by labeling them according to instructions in Customer-driven modernization.
Google-driven modernization is the default. With this option, Google determines when your fleets are ready for modernization. Google schedules your fleets for modernization and notifies you before starting the process.
Modernizing a cluster will restart workloads that have proxies. This shouldn't cause downtime if you follow Kubernetes best-practices. In addition, Google will trigger modernization during a maintenance window, if you have configured one. Once started, it runs to completion with an additional 6 days of soak time before being marked as finalized. If your monitoring detects issues, you may request a rollback.
For customer-driven modernization, Google notifies you when your fleets are ready for modernization. You can then choose when to trigger modernization, cluster by cluster. Once complete, you mark each fleet modernization as complete.
Once a fleet is modernized, Google removes all Istiod-based components.
The TRAFFIC_DIRECTOR control plane implementation requires that your cluster be registered to a fleet with the mesh feature enabled. If you onboarded using legacy tooling, Google will automatically register your cluster to its project's fleet, using the gkehub.googleapis.com Membership API. If you have any automation that unregisters a cluster, you must remove it before modernization.

Google-driven modernization

This option is the default if you don't label your fleets for customer-driven modernization. Google will monitor your fleets to determine when they are ready to be safely modernized. When all mesh-enabled fleets in your organization are ready, then your organization will be scheduled for modernization.

Multiple fleets

If your organization has multiple fleets with managed Cloud Service Mesh, you can control the order in which Google modernizes your fleets by setting a project label of mesh-modernization-order to one of the values: early, default, late. Google will complete modernization of each group before beginning the modernization of any fleet in the next group. Fleets that you have opted for customer-driven modernization won't be considered in this ordering.

Use the following command to set the mesh-modernization-order label for a fleet:

gcloud alpha projects update FLEET_PROJECT_ID --update-labels="mesh-modernization-order=VALUE"

For instructions on setting project labels using the Console or REST, see the creating and managing labels document.

If you don't use Google Cloud organizations, then your fleets will be scheduled and modernized independently, and you cannot control the ordering.

Multi-cluster meshes

If a fleet has multiple clusters using managed Cloud Service Mesh, you can control the order in which Google modernizes the clusters by setting the cluster label of mesh-modernization-order to one of the values: early, default, late. Google will begin modernization of each group and wait for automated modernization steps to complete, before beginning the modernization of any cluster in the next group. Note that this ordering will only apply within a fleet, it won't affect other fleets in your organization that may be modernizing in parallel.

Use the following command to set the mesh-modernization-order label for a cluster:

gcloud container clusters update CLUSTER_NAME \
  --location LOCATION \
  --update-labels="mesh-modernization-order=VALUE"

Customer-driven modernization

You can opt to control the exact timing of modernization yourself at the fleet level. To do this, apply a label on the fleet host project using the following command:

gcloud alpha projects update FLEET_PROJECT_ID \
  --update-labels="mesh-modernization-mode=manual"

Note that if your Google Cloud organization has multiple fleets, any fleet not labeled will be scheduled for Google-driven modernization.

Once your fleet is eligible to be modernized, you will get a notification in the fleet-level feature state. You must trigger modernization within 3 months of receiving this notification.

You can check for this notification using Google Cloud CLI:

gcloud container fleet mesh describe --project FLEET_PROJECT_ID

You should see output similar to the following:

name: projects/FLEET_PROJ/locations/global/features/servicemesh
state:
  servicemesh:
    conditions:
      # If eligible, you will see this:
      - code: MODERNIZATION_MANUAL_ELIGIBLE
        details: ...
        documentationLink: ...
        severity: INFO

      # If not yet eligible, you will see this:
      - code: MODERNIZATION_MANUAL_NOT_YET_ELIGIBLE
        details: ...
        documentationLink: ...
        severity: INFO

This documentation will be updated to include details on how to manually trigger modernization. Subscribe to the Cloud Service Mesh release notes feed to get notified.

Active modernization for Google-driven migration

This section describes the steps of Google-driven modernization.

Modernizing fleets

Google will trigger active modernization of each of your organization's fleets. This means for each fleet the following steps are executed:

Modernize all clusters with mesh-modernization-order of early.
Modernize all clusters with mesh-modernization-order of default or unspecified.
Modernize all clusters with mesh-modernization-order of late.
Wait for each cluster's modernization to be marked as finalized. This means waiting at least 6 business days after restarting the last pod in any cluster in this fleet.
Finalize this fleet's modernization, eventually removing Istiod-based components.

Modernizing a cluster

During active modernization of a cluster, both control plane implementations are temporarily run side by side, and in a safe and controlled way the following tasks are processed:

Enable the new control plane implementation. If you configured maintenance windows for your cluster, and are using Google-driven modernization, this step will begin during a maintenance window and continue until complete.
- To enable health checking, the snk daemonset is created in the kube-system namespace of the cluster and a per-cluster firewall rule is created.
  - To enable network endpoint group (NEG) ingestion, the annotation cloud.google.com/neg is added to all Kubernetes services.
  - New Google Cloud resources such as mesh, routes, backend services, and health checks are created in the cluster.
  - Some of the new resources are quota-limited. You can view quotas and request more if necessary.
  - The cluster is monitored during soak time prior to moving to the next step.
Shift traffic to the new control plane implementation. If you configured maintenance windows for your cluster, and are using Google-driven modernization, this step will begin during a maintenance window and continue until complete.
- Pods managed by Kubernetes deployment that have Cloud Service Mesh proxies are restarted, so they reconnect to the new control plane.
- Pods are restarted in progressively-larger waves with soak time after each wave for monitoring.
There is a wait of at least 6 business days of soak time before a cluster's modernization is marked as finalized.

Use the following Google Cloud CLI command to check the status of active modernization:

gcloud container fleet mesh describe --project FLEET_PROJECT_ID

You will see a condition like one of the following in the per-cluster membershipStates field:

...
membershipStates:
  projects/FLEET_PROJ/locations/us-central1/memberships/MEMBERSHIP:
    servicemesh:
      conditions:
      - code: MODERNIZATION_IN_PROGRESS
        details: ...
        documentationLink: ...
        severity: INFO
...
      # If you see this, see instructions provided in the details and documentationLink fields.
      - code: MODERNIZATION_ACTION_REQUIRED
        details: [details about required actions]
        documentationLink: [link to documentation for required actions]
        severity: WARNING
...
      - code: MODERNIZATION_COMPLETED
        details: ...
        documentationLink: ...
        severity: INFO

Scheduling and Notification

You should receive notification that a cluster is scheduled to be updated at least two weeks before the update. Notifications are available in your cluster-level feature state conditions.

Use the following Google Cloud CLI command to check the notification:

gcloud container fleet mesh describe --project=FLEET_PROJECT_ID

You see results similar to the following:

membershipStates:
  projects/656460026795/locations/us-central1/memberships/cluster:
    servicemesh:
      conditions:
      - code: MODERNIZATION_SCHEDULED
        details: This cluster has been scheduled for modernization on or after (date ~ at least 2 weeks).
        documentationLink: 
        severity: INFO