Installing Istio on GKE

Warning: Istio on GKE is deprecated. After December 31, 2021, the UI no longer supports this feature during the creation of new clusters. After September 30, 2022, Istio on GKE will no longer be supported in existing clusters. You can migrate Istio on GKE to Anthos Service Mesh to continue using your service meshes. For more information, see the migration FAQ.

Beta

This product is covered by the Pre-GA Offerings Terms of the Google Cloud Terms of Service. Pre-GA products might have limited support, and changes to pre-GA products might not be compatible with other pre-GA versions. For more information, see the launch stage descriptions.

This guide shows you how to get started with the Istio on GKE add-on, including installation options for new and existing clusters. You can install the add-on using either the gcloud CLI or the Google Cloud console.

You can find out more about the Istio on GKE add-on and whether it's right for you in the Overview.

Before you start

Take the following steps to enable the Google Kubernetes Engine API:

1. Visit the Kubernetes Engine page in the Google Cloud console.
2. Create or select a project.
3. Wait for the API and related services to be enabled. This can take several minutes.

4. Make sure that billing is enabled for your Cloud project. Learn how to check if billing is enabled on a project.

Ensure you have the following command line tools installed:

• The gcloud CLI is used to create and delete Kubernetes Engine clusters, including creating and updating clusters with the Istio on GKE add-on. The gcloud CLI is included in the Google Cloud CLI: follow the instructions to install and initialize gcloud CLI to work with your GCP projects. If you have an existing Google Cloud CLI installation, verify that it's at least version 208.0.0:

  gcloud version

  Note that you don't have to install gcloud to create a new Istio-enabled cluster, as you can use the Google Cloud console instead, but it's still useful for managing existing clusters and installing other tools like kubectl.
• kubectl is used to manage Kubernetes, the cluster orchestration system used by GKE. You can install kubectl using gcloud:

  gcloud components install kubectl

Set defaults for the gcloud command-line tool

To save time typing your project ID and Compute Engine zone options in the gcloud command-line tool, you can set the defaults:

gcloud config set project project-id
gcloud config set compute/zone compute-zone

Choose a security option

There are two possible default mesh-wide security options to choose from when creating or updating a cluster with Istio on GKE. Which one you choose depends on your initial application needs.

• Strict mTLS: In this security mode, Istio enforces mutual TLS (mTLS) encryption between all services and control plane components in the mesh by default, unless you override it with destination-specific rules. All calls within the mesh are encrypted and services will not accept unencrypted traffic.
• Permissive mTLS: In this security mode, by default Istio allows services in the mesh to accept both encrypted and unencrypted traffic, and all services send unencrypted calls by default. As with strict mTLS, you can override this for specific services. Use this option if you have services that still need to accept unencrypted traffic, for example if you have not fully migrated your services to Istio and have traffic coming from legacy clients outside the mesh. Istio on GKE provides this mode rather than simply installing Istio with no security enabled, as it makes it easier to migrate to strict mTLS later for added security.

You can find out how to update your security defaults and further configure Istio security in Updating security defaults, below.

Installing Istio on GKE

You can install Istio on GKE either in a new cluster or an existing cluster. In both cases, this installs the Istio control plane. To take full advantage of Istio's features, you need to inject Envoy sidecar proxies into the Pods in your service mesh.

Creating a cluster with Istio on GKE

We suggest creating at least a 4 node cluster with the 2 vCPU machine type when using this add-on. You can deploy Istio itself with the default GKE new cluster setup but this may not provide enough resources to explore sample applications.

To create a cluster using Istio on GKE:

Console

1. Go to the Kubernetes page in the console and select Create Cluster.
2. Using the default Standard Cluster dialog, choose your preferred number of nodes and machines, bearing in mind the minimum recommended cluster size for Istio.
3. In the Master Version drop-down, select an Istio on GKE cluster.
4. Select Availability, networking, security, and additional features to display additional configuration options, including Istio on GKE.
5. Select Enable Istio (beta).
6. Select the mTLS security mode you want to use for your cluster from the drop-down.
7. Click Create to create your cluster.

Command line

To create a GKE cluster with Istio enabled and with mutual TLS enforced by default, run this command, replacing CLUSTER_NAME with your chosen cluster name, and CLUSTER_VERSION with a compatible cluster version:

gcloud beta container clusters create CLUSTER_NAME \
    --addons=Istio --istio-config=auth=MTLS_STRICT \
    --cluster-version=CLUSTER_VERSION \
    --machine-type=n1-standard-2 \
    --num-nodes=4

Or to create a GKE cluster with Istio enabled and with mTLS in permissive mode:

gcloud beta container clusters create CLUSTER_NAME \
    --addons=Istio --istio-config=auth=MTLS_PERMISSIVE \
    --cluster-version=CLUSTER_VERSION \
    --machine-type=n1-standard-2 \
    --num-nodes=4

Adding Istio on GKE to an existing cluster

If you want to update a cluster with the add-on, you may need to first resize your cluster to ensure that you have enough resources for Istio. As when creating a new cluster, we suggest at least a 4 node cluster with the 2 vCPU machine type.

To update an existing cluster with the Istio on GKE add-on:

Console

1. Go to the Kubernetes clusters page in the console and select the cluster you want to update
2. Select Edit.
3. Under Istio (beta) select Enabled to display Istio mTLS (beta).
4. Select the mTLS security mode you want to use for your cluster from the drop-down.
5. Click Save to update your cluster.

Command line

To add Istio with mutual TLS enforced by default to an existing cluster, run this command, replacing CLUSTER_NAME with the cluster name:

gcloud beta container clusters update CLUSTER_NAME \
    --update-addons=Istio=ENABLED --istio-config=auth=MTLS_STRICT

Or to add Istio with mTLS in permissive mode to an existing cluster:

gcloud beta container clusters update CLUSTER_NAME \
    --update-addons=Istio=ENABLED --istio-config=auth=MTLS_PERMISSIVE

Note that the clusters update command may require other parameters, depending on your actual cluster configuration.

If you have an existing application on the cluster, you can find out how to migrate it so it's managed by Istio in the Istio documentation.

Verifying installation

To verify that your Istio on GKE installation was successful:

1. If you've just created rather than updated a cluster, check that it's up and running with a GKE version of 1.10.6 or higher:

   gcloud container clusters list
   

   The output is similar to the following:

   NAME        LOCATION       MASTER_VERSION  MASTER_IP      MACHINE_TYPE   NODE_VERSION   NUM_NODES  STATUS
   istio-demo  us-central1-b  1.11.2-gke.15   35.239.252.38  n1-standard-2  1.11.2-gke.15  4          RUNNING
   

2. Get the credentials for your new cluster so you can interact with it with kubectl.

   gcloud container clusters get-credentials CLUSTER_NAME
   

3. Ensure the following Kubernetes services are deployed: istio-citadel, istio-egressgateway, istio-pilot, istio-ingressgateway, istio-policy, istio-sidecar-injector, and istio-telemetry (you'll also see the other deployed services):

   kubectl get service -n istio-system
   

   NAME                       TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                                                                                                                   AGE
   istio-citadel              ClusterIP      10.47.245.92    <none>        8060/TCP,9093/TCP                                                                                                         12s
   istio-egressgateway        ClusterIP      10.47.248.129   <none>        80/TCP,443/TCP                                                                                                            12s
   istio-galley               ClusterIP      10.47.248.109   <none>        443/TCP,9093/TCP                                                                                                          12s
   istio-ingressgateway       LoadBalancer   10.47.248.117   <pending>     80:31380/TCP,443:31390/TCP,31400:31400/TCP,15011:30221/TCP,8060:32445/TCP,853:30663/TCP,15030:32010/TCP,15031:32633/TCP   12s
   istio-pilot                ClusterIP      10.47.251.133   <none>        15010/TCP,15011/TCP,8080/TCP,9093/TCP                                                                                     12s
   istio-policy               ClusterIP      10.47.255.244   <none>        9091/TCP,15004/TCP,9093/TCP                                                                                               12s
   istio-sidecar-injector     ClusterIP      10.47.240.36    <none>        443/TCP                                                                                                                   12s
   istio-statsd-prom-bridge   ClusterIP      10.47.247.135   <none>        9102/TCP,9125/UDP                                                                                                         12s
   istio-telemetry            ClusterIP      10.47.242.73    <none>        9091/TCP,15004/TCP,9093/TCP,42422/TCP                                                                                     12s
   promsd                     ClusterIP      10.47.241.188   <none>        9090/TCP                                                                                                                  12s
   

4. Ensure the corresponding Kubernetes Pods are deployed and all containers are up and running: istio-pilot-*, istio-policy-*, istio-telemetry-*, istio-egressgateway-*, istio-ingressgateway-*, istio-sidecar-injector-*, and istio-citadel-*.

   kubectl get pods -n istio-system
   

   NAME                                        READY   STATUS      RESTARTS   AGE
   istio-citadel-555d845b65-xfdmj              1/1     Running     0          2d
   istio-cleanup-secrets-8x2pl                 0/1     Completed   0          2d
   istio-egressgateway-667d854c49-9q5dl        1/1     Running     0          2d
   istio-galley-6c9cd5b8bb-4j4jk               1/1     Running     0          2d
   istio-ingressgateway-6c796c5594-f972p       1/1     Running     0          2d
   istio-pilot-77f74fc6f-rpbfj                 2/2     Running     0          2d
   istio-policy-655b87fff-4wbwq                2/2     Running     0          2d
   istio-security-post-install-tm2rm           0/1     Completed   1          2d
   istio-sidecar-injector-668c9fb4db-p6lwt     1/1     Running     0          2d
   istio-statsd-prom-bridge-5b645f6f4d-6pbgf   1/1     Running     0          2d
   istio-telemetry-d9848f498-wf6kh             2/2     Running     0          2d
   promsd-6b989699d8-l7jxt                 1/1     Running     0          2d
   

Enabling sidecar injection

To take full advantage of Istio's features, each service in your application needs to have an Envoy sidecar proxy running in its Pod. The Envoy proxy intercepts all inbound and outbound traffic to the service and communicates with the Istio control plane. You can inject an Envoy proxy manually by updating your Pods' Kubernetes configuration, or you can use Istio's webhooks-based automatic sidecar injection.

By default, Istio sidecar auto-injection is disabled for all namespaces. To enable auto-injection, replace NAMESPACE in the following command with the name of the namespace for your application's services or with default:

kubectl label namespace NAMESPACE istio-injection=enabled

Any running Pods must be restarted for the change to take effect, as the sidecar is added at Pod creation time. To disable auto-injection in the namespace, remove the label and restart Pods to remove their sidecars.

To inject Envoy sidecars manually, refer to Installing the sidecar.

Configuring your control plane

Although Istio on GKE manages most of your control plane settings, for production use we recommend that you choose and specify appropriate values for your use case for the following settings. You can change them using kubectl or your Kubernetes tools of choice. These settings are not changed when your installation is upgraded by the add-on.

Horizontal scaling

Configure one of the following to ensure you have enough replicas of your Istio control plane components to handle your mesh's traffic. For production use we recommend a minimum of two replicas for istio-policy, istio-telemetry, istio-pilot and istio-sidecar-injector.

• Horizontal pod autoscaling automatically scales the number of replicas based on observed CPU utilization. We provide default maximum and minimum values for autoscaling control plane components, but you can edit them to suit your needs. For example, here's how you'd specify with kubectl edit that you want to edit the autoscaler settings for Istio-Telemetry:

  kubectl edit -n istio-system HorizontalPodAutoscalers/istio-telemetry
  

• If you're not using autoscaling, you can set the number of replicas for each control plane element (with the exception of Citadel, which is always a singleton) for manual horizontal scaling. For example, here's how you'd specify with kubectl that you want two instances of Pilot:

  kubectl scale -n istio-system --replicas=2 deployment/istio-pilot
  

Resource requests

By default, resource requests are not set, however for production use we recommend setting these to appropriate values to ensure nodes have enough resources to support the pods. You should set resource requests for each container in the pod, otherwise CPU on HPAs show unknown and autoscaling will not work.

You can find recommended starting points for resource settings for each component in the Istio Installation Options guide. For example, resource requests and limits for Mixer are under mixer options.

kubectl edit -n istio-system Deployments/istio-telemetry

Pod disruption budget

PodDisruptionBudgets let you specify the minimum number of available replicas of a given deployment that an application can tolerate and continue to work.

You should configure PodDisruptionBudgets for all deployments that must remain available during Istio on GKE upgrades. At a minimum, we recommend doing this for the add-on's provided Istio ingress gateway (istio-ingressgateway), if you haven't deployed and configured your own ingress gateway, as otherwise external traffic may not be able to reach your application during upgrade.

Pod disruption budget settings should be configured along with horizontal scaling settings, as described above. The number of replicas or autoscaler minimum replicas needs to be greater than your pod disruption budget minimum, because Istio on GKE's upgrade process makes individual replicas unavailable while they are updated. This ensures that the minAvailable setting in PodDisruptionBudget is not violated during upgrades and the upgrade works as intended.

Customizing your installation

While Istio on GKE provides reasonable default behavior for many use cases, you can customize your installation in a number of ways, including configuring telemetry tools and adding gateways. This section describes how to make supported customizations.

Updating security defaults

Switching the default Istio mTLS security mode in a running cluster from Strict to Permissive, or vice versa, uses the same command as adding Istio to a cluster:

Console

1. Go to the Kubernetes clusters page in the console and select the cluster you want to update
2. Select Edit.
3. Under Istio (beta) select Enabled to display Istio mTLS (beta).
4. Select the mTLS security mode you want to use for your cluster from the drop-down.
5. Click Save to update your cluster.

Command line

To change your cluster to use Istio with mutual TLS enforced by default, run this command, replacing CLUSTER_NAME with the cluster name:

$ gcloud beta container clusters update CLUSTER_NAME \
    --update-addons=Istio=ENABLED --istio-config=auth=MTLS_STRICT

Or to change your cluster to mTLS in permissive mode:

$ gcloud beta container clusters update CLUSTER_NAME \
    --update-addons=Istio=ENABLED --istio-config=auth=MTLS_PERMISSIVE

Be aware that if you enable strict mTLS while you still have services that need to send or receive unencrypted traffic, your application may break! You can find out more about migrating to strict mTLS in Mutual TLS Migration. You can also specify more fine-grained destination-specific authentication policies. Destination-specific authentication policies will always override any global default mTLS setting, even if you switch from Strict to Permissive or vice versa.

You can find out much more about configuring and working with Istio security, including setting up role based authorization, in the Istio site.

Tracing and logging

By default, a mesh installed with Istio on GKE can send logging and metrics data to Cloud Logging and Cloud Monitoring, provided you have enabled the relevant features for your project and cluster. Versions of Istio on GKE earlier than 1.1.7 also send trace data by default. You can find out more about this in Google Cloud's operations suite Support.

Both tracing and logging may incur additional costs to use, especially with a large volume of data coming from your mesh. If you would like to disable this feature without disabling the Google Cloud's operations suite APIs entirely for your project, update your Istio on GKE configuration as follows.

To disable Cloud Logging for Istio on GKE

1. Open the stackdriver-log rule for editing:

   kubectl edit -n istio-system rule stackdriver-log
   

2. Replace the match condition (context.protocol == "http" || context.protocol == "grpc") && (context.reporter.kind | "inbound" == "inbound") with "false".

3. Save and close the rule.

4. Open the stackdriver-log-tcp rule.

   kubectl edit -n istio-system rule stackdriver-log-tcp
   

5. Replace the match condition (context.protocol == "tcp") && (context.reporter.kind | "inbound" == "inbound") with "false".

6. Save and close the rule.

To disable Google Cloud's operations suite tracing for Istio on GKE

If you have Istio on GKE version 1.1.3-gke.0 or earlier, or if you have manually enabled Google Cloud's operations suite tracing, you can disable it as follows:

1. Open the stackdriver-tracing-rule rule for editing:

   kubectl edit -n istio-system rule stackdriver-tracing-rule
   

2. Replace the match condition context.protocol == "http" || context.protocol == "grpc" with "false".

3. Save and close the rule.

To enable Google Cloud's operations suite tracing for Istio on GKE

If you are using version 1.1.7 or later and would like to enable Google Cloud's operations suite tracing:

1. Make sure the Cloud Trace API is enabled in your Google Cloud project.

2. Open the stackdriver-tracing-rule rule for editing:

   kubectl edit -n istio-system rule stackdriver-tracing-rule
   

3. Replace the match condition "false" with context.protocol == "http" || context.protocol == "grpc".

4. Save and close the rule.

Adding gateways

An Istio ingress gateway is provided as part of your Istio on GKE installation. The default ingress gateway is suitable for deployments where the installed resources (RBAC, Service, Deployment) don't need much customization. You can add fields to the Istio gateway configuration, and you can modify the following control plane settings:

• Horizontal scaling
• Resource requests
• Pod disruption budgets

When you upgrade GKE, the Istio on GKE add-on and all resources installed by the add-on including the default ingress gateway are upgraded automatically. Don't change any other values in the default ingress gateway configuration because the changes will be reverted back to the default values during the automatic upgrade.

For more complex scenarios where customization is required, you need to create a new ingress gateway. If you add an ingress or egress gateway, they are under your control, and they aren't modified during the automatic upgrade.

Note that an Istio egress gateway isn't installed by default in version 1.1 and later. To add an ingress or egress gateway:

1. Grant cluster admin permissions to the current user:

   1. Set your user account to be the current user.

      gcloud auth login

   2. Grant cluster admin permissions to the current user.

      kubectl create clusterrolebinding cluster-admin-binding \
      --clusterrole=cluster-admin \
      --user="$(gcloud config get-value core/account)"
      

2. Follow the steps in Adding gateways to add the gateway.

Accessing external services

All outbound traffic from an Istio-enabled Pod is redirected to its sidecar proxy. By default, Istio configures the sidecar proxy to passthrough requests for unknown services, but you can configure stricter control. Although you can create ServiceEntries to allow outbound requests to permitted destinations, for security reasons, you might want to add an egress gateway, as described in the Secure Control of Egress Traffic in Istio blogpost.

What's next?

• Try installing and exploring the Bookinfo example to see what Istio can do. To get the sample app and the istioctl tool, go to the Istio release page to download the installation file corresponding to the OS where you're running your commands. Then follow the instructions to deploy and test the application (you don't need to deploy Istio itself) in the GKE Installing Istio tutorial.
• Find out lots more about Istio in the open source documentation.
• If you need to remove the Istio add-on from a cluster, see Uninstalling Istio on GKE.