Multi-project installation and migration on GKE

This guide explains how to install or migrate to Anthos Service Mesh from open source Istio for a mesh containing multiple GKE clusters that are in different Google Cloud projects.

You can use this guide for the following onboarding use cases:

  • New installations of Anthos Service Mesh 1.8.6.

  • Migrating from open source Istio 1.7 or 1.8 to Anthos Service Mesh 1.8.6. If you have an earlier version of Istio, you must upgrade first before migrating to Anthos Service Mesh.

Not all features are available for a service mesh with clusters in different projects. In particular, the Anthos Service Mesh dashboards in the Google Cloud console currently aren't available. However, you can still view logs in Cloud Logging and metrics in Cloud Monitoring for each project.

Before you begin

This guide assumes that you already have:

If you are migrating from Istio, be sure to review Preparing to migrate from Istio.

Anthos and Anthos Service Mesh differences

  • GKE Enterprise subscribers, be sure to enable the GKE Enterprise API.

    Enable the API

  • If you aren't an GKE Enterprise subscriber, you can still install Anthos Service Mesh, but certain UI elements and features in Google Cloud console are only available to GKE Enterprise subscribers. For information about what is available to subscribers and non-subscribers, see GKE Enterprise and Anthos Service Mesh UI differences. For information about Anthos Service Mesh pricing for non-subscribers, see Pricing.

Requirements

  • Because your clusters are in different projects, they must be in a Shared Virtual Private Cloud (VPC). For information on configuring your clusters, see Setting up clusters with Shared VPC.

  • Your GKE cluster must meet the following requirements:

    • A machine type that has at least four vCPUs, such as e2-standard-4. If the machine type for your cluster doesn't have at least four vCPUs, change the machine type as described in Migrating workloads to different machine types.

    • The minimum number of nodes depends on your machine type. Anthos Service Mesh requires at least eight vCPUs. If the machine type has four vCPUs, your cluster must have at least two nodes. If the machine type has eight vCPUs, the cluster only needs one node. If you need to add nodes, see Resizing a cluster.

    • To prepare your cluster before installing Anthos Service Mesh, you enable Workload Identity. Workload Identity is the recommended method of calling Google APIs. Enabling Workload Identity changes the way calls from your workloads to Google APIs are secured, as described in Workload Identity limitations.

    • Optionally, but recommended, enroll the cluster in a release channel. We recommend that you enroll in the Regular release channel because other channels might be based on a GKE version that isn't supported with Anthos Service Mesh 1.8.6. For more information, see Supported environments. Follow the instructions in Enrolling an existing cluster in a release channel if you have a static GKE version.

  • To be included in the service mesh, service ports must be named, and the name must include the port's protocol in the following syntax: name: protocol[-suffix] where the square brackets indicate an optional suffix that must start with a dash. For more information, see Naming service ports.

  • If you are installing Anthos Service Mesh on a private cluster, you must open port 15017 in the firewall to get the webhook used with automatic sidecar injection to work properly. For more information, see Opening a port on a private cluster.

  • If you have created a service perimeter in your organization, you might need to add the Mesh CA service to the perimeter. See Adding Mesh CA to a service perimeter for more information.

  • A Google Cloud project can only have one mesh associated with it.

Choosing a certificate authority

For both new installations and migrations, you can use Anthos Service Mesh certificate authority (Mesh CA) or Citadel (now incorporated in istiod) as the certificate authority (CA) for issuing mutual TLS (mTLS) certificates.

We generally recommend that you use Mesh CA for the following reasons:

  • Mesh CA is a highly reliable and scalable service that is optimized for dynamically scaled workloads on Google Cloud.
  • With Mesh CA, Google manages the security and availability of the CA backend.
  • Mesh CA lets you rely on a single root of trust across clusters.

However, there are cases where you might want to consider using Citadel, such as the following:

  • If you have a custom CA.
  • If you're migrating from Istio.

    If you choose Citadel, there's little downtime because mTLS traffic isn't interrupted during the migration. If you choose Mesh CA, you need to schedule downtime for the migration because the root of trust changes from Citadel to Mesh CA. To complete the migration to the Mesh CA root of trust, you need to restart all Pods in all namespaces. During this process, the old Pods cannot establish mTLS connections with the new Pods.

Certificates from Mesh CA include the following data about your application's services:

  • The Google Cloud project ID
  • The GKE namespace
  • The GKE service account name

Registering your cluster

Although not required currently, we recommend that you register your cluster in your project's fleet (formerly known as an environ). A fleet lets you organize clusters to make multi-cluster management easier. By registering your clusters in a fleet, you can group services and other infrastructure as needed to apply consistent policies. If you have clusters in different projects, you need to register the clusters with the fleet host project rather than the project that the cluster was created in. For details on how to register your cluster, see Registering clusters to the fleet.

The concept of a fleet host project is important when you set up your cluster to enable the options that Anthos Service Mesh requires. The service mesh for your cluster is identified with a value that is based on a project number. When you are setting up clusters from different projects, you need to use the project number for the fleet host project.