Installing Istio on GKE

Beta

This product is in a pre-release state and might change or have limited support. For more information, see the product launch stages.

This guide shows you how to get started with the Istio on GKE add-on, including installation options for new and existing clusters. You can install the add-on using either the gcloud command line tool or the Google Cloud Console.

You can find out more about the Istio on GKE add-on and whether it's right for you in the Overview.

Before you start

Take the following steps to enable the Kubernetes Engine API:

1. Visit the Kubernetes Engine page in the Google Cloud Console.
2. Create or select a project.
3. Wait for the API and related services to be enabled. This can take several minutes.

4. Make sure that billing is enabled for your Google Cloud project. Learn how to confirm billing is enabled for your project.

Ensure you have the following command line tools installed:

• gcloud is used to create and delete Kubernetes Engine clusters, including creating and updating clusters with the Istio on GKE add-on. gcloud is included in the Google Cloud SDK: follow the instructions to install and initialize it to work with your GCP projects. If you have an existing gcloud installation, verify that it's at least version 208.0.0:

  gcloud version

  Note that you don't have to install gcloud to create a new Istio-enabled cluster, as you can use the Google Cloud Console instead, but it's still useful for managing existing clusters and installing other tools like kubectl.
• kubectl is used to manage Kubernetes, the cluster orchestration system used by GKE. You can install kubectl using gcloud:

  gcloud components install kubectl

Set defaults for the gcloud command-line tool

To save time typing your project ID and Compute Engine zone options in the gcloud command-line tool, you can set the defaults:

gcloud config set project [PROJECT_ID]
gcloud config set compute/zone [COMPUTE_ENGINE_ZONE]

Choose a security option

There are two possible default mesh-wide security options to choose from when creating or updating a cluster with Istio on GKE. Which one you choose depends on your initial application needs.

• Strict mTLS: In this security mode, Istio enforces mutual TLS (mTLS) encryption between all services and control plane components in the mesh by default, unless you override it with destination-specific rules. All calls within the mesh are encrypted and services will not accept unencrypted traffic.
• Permissive mTLS: In this security mode, by default Istio allows services in the mesh to accept both encrypted and unencrypted traffic, and all services send unencrypted calls by default. As with strict mTLS, you can override this for specific services. Use this option if you have services that still need to accept unencrypted traffic, for example if you have not fully migrated your services to Istio and have traffic coming from legacy clients outside the mesh. Istio on GKE provides this mode rather than simply installing Istio with no security enabled, as it makes it easier to migrate to strict mTLS later for added security.

You can find out how to update your security defaults and further configure Istio security in Updating security defaults, below.

Supported GKE cluster versions

The version of Istio on GKE installed when you create or update a cluster with Istio on GKE depends on the cluster version. We recommend that you use a cluster version with the latest version of Istio (1.2.10-gke.0). If you are using an older version, upgrade your cluster as soon as this Istio on GKE version becomes available.

Other GKE versions that have older but still supported versions of Istio are available (see Istio on GKE versions) but not recommended due to security vulnerabilities. Please read the release notes to understand the vulnerabilities before using these older versions.

GKE versions	Istio version
1.15.4-gke.22+ 1.15.5-gke.6+ 1.15.6-gke.2+ 1.15.4-gke.15 1.15.7 and above - all versions	1.2.10-gke.0 (recommended) See the following for more information: Security Update - ISTIO-SECURITY-2019-007. Notes on upgrading from 1.0 versions of Istio.
1.14.7-gke.10	1.1.16-gke.0 (supported) See the following for more information: Security Update - ISTIO-SECURITY-2019-005. Notes on upgrading from 1.0 versions of Istio.
1.13.11-gke.11	1.1.16-gke.0 (supported) See the following for more information: Security Update - ISTIO-SECURITY-2019-005. Notes on upgrading from 1.0 versions of Istio.

Installing Istio on GKE

You can install Istio on GKE either in a new cluster or an existing cluster. In both cases, this installs the Istio control plane. To take full advantage of Istio's features, you need to inject Envoy sidecar proxies into the Pods in your service mesh.

Creating a cluster with Istio on GKE

We suggest creating at least a 4 node cluster with the 2 vCPU machine type when using this add-on. You can deploy Istio itself with the default GKE new cluster setup but this may not provide enough resources to explore sample applications. Please note that the Istio on GKE add-on is not compatible with Workload Identity.

To create a cluster using Istio on GKE:

Console

1. Go to the Kubernetes page in the Cloud Console and select Create Cluster.
2. Using the default Standard Cluster dialog, choose your preferred number of nodes and machines, bearing in mind the minimum recommended cluster size for Istio.
3. In the Master Version drop-down, select an Istio on GKE recommended cluster version (or a supported version if you can't use the recommended version).
4. Select Availability, networking, security, and additional features to display additional configuration options, including Istio on GKE.
5. Select Enable Istio (beta).
6. Select the mTLS security mode you want to use for your cluster from the drop-down.
7. Click Create to create your cluster.

Command line

To create a GKE cluster with Istio enabled and with mutual TLS enforced by default, run this command, replacing CLUSTER_NAME with your chosen cluster name, and CLUSTER_VERSION with a compatible cluster version:

gcloud beta container clusters create CLUSTER_NAME \
    --addons=Istio --istio-config=auth=MTLS_STRICT \
    --cluster-version=CLUSTER_VERSION \
    --machine-type=n1-standard-2 \
    --num-nodes=4

Or to create a GKE cluster with Istio enabled and with mTLS in permissive mode:

gcloud beta container clusters create CLUSTER_NAME \
    --addons=Istio --istio-config=auth=MTLS_PERMISSIVE \
    --cluster-version=CLUSTER_VERSION \
    --machine-type=n1-standard-2 \
    --num-nodes=4

Adding Istio on GKE to an existing cluster

If you want to update a cluster with the add-on, you may need to first resize your cluster to ensure that you have enough resources for Istio. As when creating a new cluster, we suggest at least a 4 node cluster with the 2 vCPU machine type.

Your cluster must also be running a supported cluster master version to use the add-on. Please note that the Istio on GKE add-on is not compatible with Workload Identity.

To update an existing cluster with the Istio on GKE add-on:

Console

1. Go to the Kubernetes clusters page in the Cloud Console and select the cluster you want to update
2. Select Edit.
3. Under Istio (beta) select Enabled to display Istio mTLS (beta).
4. Select the mTLS security mode you want to use for your cluster from the drop-down.
5. Click Save to update your cluster.

Command line

To add Istio with mutual TLS enforced by default to an existing cluster, run this command, replacing CLUSTER_NAME with the cluster name:

gcloud beta container clusters update CLUSTER_NAME \
    --update-addons=Istio=ENABLED --istio-config=auth=MTLS_STRICT

Or to add Istio with mTLS in permissive mode to an existing cluster:

gcloud beta container clusters update CLUSTER_NAME \
    --update-addons=Istio=ENABLED --istio-config=auth=MTLS_PERMISSIVE

Note that the clusters update command may require other parameters, depending on your actual cluster configuration.

If you have an existing application on the cluster, you can find out how to migrate it so it's managed by Istio in the Istio documentation.

Verifying installation

To verify that your Istio on GKE installation was successful:

1. If you've just created rather than updated a cluster, check that it's up and running with a GKE version of 1.10.6 or higher:

   gcloud container clusters list
   

   The output is similar to the following:

   NAME        LOCATION       MASTER_VERSION  MASTER_IP      MACHINE_TYPE   NODE_VERSION   NUM_NODES  STATUS
   istio-demo  us-central1-b  1.11.2-gke.15   35.239.252.38  n1-standard-2  1.11.2-gke.15  4          RUNNING
   

2. Get the credentials for your new cluster so you can interact with it with kubectl.

   gcloud container clusters get-credentials CLUSTER_NAME
   

3. Ensure the following Kubernetes services are deployed: istio-citadel, istio-egressgateway, istio-pilot, istio-ingressgateway, istio-policy, istio-sidecar-injector, and istio-telemetry (you'll also see the other deployed services):

   kubectl get service -n istio-system
   

   NAME                       TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                                                                                                                   AGE
   istio-citadel              ClusterIP      10.47.245.92    <none>        8060/TCP,9093/TCP                                                                                                         12s
   istio-egressgateway        ClusterIP      10.47.248.129   <none>        80/TCP,443/TCP                                                                                                            12s
   istio-galley               ClusterIP      10.47.248.109   <none>        443/TCP,9093/TCP                                                                                                          12s
   istio-ingressgateway       LoadBalancer   10.47.248.117   <pending>     80:31380/TCP,443:31390/TCP,31400:31400/TCP,15011:30221/TCP,8060:32445/TCP,853:30663/TCP,15030:32010/TCP,15031:32633/TCP   12s
   istio-pilot                ClusterIP      10.47.251.133   <none>        15010/TCP,15011/TCP,8080/TCP,9093/TCP                                                                                     12s
   istio-policy               ClusterIP      10.47.255.244   <none>        9091/TCP,15004/TCP,9093/TCP                                                                                               12s
   istio-sidecar-injector     ClusterIP      10.47.240.36    <none>        443/TCP                                                                                                                   12s
   istio-statsd-prom-bridge   ClusterIP      10.47.247.135   <none>        9102/TCP,9125/UDP                                                                                                         12s
   istio-telemetry            ClusterIP      10.47.242.73    <none>        9091/TCP,15004/TCP,9093/TCP,42422/TCP                                                                                     12s
   promsd                     ClusterIP      10.47.241.188   <none>        9090/TCP                                                                                                                  12s
   

4. Ensure the corresponding Kubernetes Pods are deployed and all containers are up and running: istio-pilot-*, istio-policy-*, istio-telemetry-*, istio-egressgateway-*, istio-ingressgateway-*, istio-sidecar-injector-*, and istio-citadel-*.

   kubectl get pods -n istio-system
   

   NAME                                        READY   STATUS      RESTARTS   AGE
   istio-citadel-555d845b65-xfdmj              1/1     Running     0          2d
   istio-cleanup-secrets-8x2pl                 0/1     Completed   0          2d
   istio-egressgateway-667d854c49-9q5dl        1/1     Running     0          2d
   istio-galley-6c9cd5b8bb-4j4jk               1/1     Running     0          2d
   istio-ingressgateway-6c796c5594-f972p       1/1     Running     0          2d
   istio-pilot-77f74fc6f-rpbfj                 2/2     Running     0          2d
   istio-policy-655b87fff-4wbwq                2/2     Running     0          2d
   istio-security-post-install-tm2rm           0/1     Completed   1          2d
   istio-sidecar-injector-668c9fb4db-p6lwt     1/1     Running     0          2d
   istio-statsd-prom-bridge-5b645f6f4d-6pbgf   1/1     Running     0          2d
   istio-telemetry-d9848f498-wf6kh             2/2     Running     0          2d
   promsd-6b989699d8-l7jxt                 1/1     Running     0          2d
   

Enabling sidecar injection

To take full advantage of Istio's features, each service in your application needs to have an Envoy sidecar proxy running in its Pod. The Envoy proxy intercepts all inbound and outbound traffic to the service and communicates with the Istio control plane. You can inject an Envoy proxy manually by updating your Pods' Kubernetes configuration, or you can use Istio's webhooks-based automatic sidecar injection.

By default, Istio sidecar auto-injection is disabled for all namespaces. To enable auto-injection, replace NAMESPACE in the following command with the name of the namespace for your application's services or with default:

kubectl label namespace NAMESPACE istio-injection=enabled

Any running Pods must be restarted for the change to take effect, as the sidecar is added at Pod creation time. To disable auto-injection in the namespace, remove the label and restart Pods to remove their sidecars.

To inject Envoy sidecars manually, refer to Installing the sidecar.

Configuring your control plane

Although Istio on GKE manages most of your control plane settings, for production use we recommend that you choose and specify appropriate values for your use case for the following settings. You can change them using kubectl or your Kubernetes tools of choice. These settings are not changed when your installation is upgraded by the add-on.

Horizontal scaling

Configure one of the following to ensure you have enough replicas of your Istio control plane components to handle your mesh's traffic. For production use we recommend a minimum of two replicas for istio-policy, istio-telemetry, istio-pilot and istio-sidecar-injector.

• Horizontal pod autoscaling automatically scales the number of replicas based on observed CPU utilization. We provide default maximum and minimum values for autoscaling control plane components, but you can edit them to suit your needs. For example, here's how you'd specify with kubectl edit that you want to edit the autoscaler settings for Istio-Telemetry:

  kubectl edit -n istio-system HorizontalPodAutoscalers/istio-telemetry
  

• If you're not using autoscaling, you can set the number of replicas for each control plane element (with the exception of Citadel, which is always a singleton) for manual horizontal scaling. For example, here's how you'd specify with kubectl that you want two instances of Pilot:

  kubectl scale -n istio-system --replicas=2 deployment/istio-pilot
  

Resource requests

By default, resource requests are not set, however for production use we recommend setting these to appropriate values to ensure nodes have enough resources to support the pods. You should set resource requests for each container in the pod, otherwise CPU on HPAs show unknown and autoscaling will not work.

You can find recommended starting points for resource settings for each component in the Istio Installation Options guide. For example, resource requests and limits for Mixer are under mixer options.

kubectl edit -n istio-system Deployments/istio-telemetry

Pod disruption budget

PodDisruptionBudgets let you specify the minimum number of available replicas of a given deployment that an application can tolerate and continue to work.

You should configure PodDisruptionBudgets for all deployments that must remain available during Istio on GKE upgrades. At a minimum, we recommend doing this for the add-on's provided Istio ingress gateway (istio-ingressgateway), if you haven't deployed and configured your own ingress gateway, as otherwise external traffic may not be able to reach your application during upgrade.

Pod disruption budget settings should be configured along with horizontal scaling settings, as described above. The number of replicas or autoscaler minimum replicas needs to be greater than your pod disruption budget minimum, because Istio on GKE's upgrade process makes individual replicas unavailable while they are updated. This ensures that the minAvailable setting in PodDisruptionBudget is not violated during upgrades and the upgrade works as intended.

Customizing your installation

While Istio on GKE provides reasonable default behavior for many use cases, you can customize your installation in a number of ways, including configuring telemetry tools and adding gateways. This section describes how to make supported customizations.

Updating security defaults

Switching the default Istio mTLS security mode in a running cluster from Strict to Permissive, or vice versa, uses the same command as adding Istio to a cluster:

Console

1. Go to the Kubernetes clusters page in the Cloud Console and select the cluster you want to update
2. Select Edit.
3. Under Istio (beta) select Enabled to display Istio mTLS (beta).
4. Select the mTLS security mode you want to use for your cluster from the drop-down.
5. Click Save to update your cluster.

Command line

To change your cluster to use Istio with mutual TLS enforced by default, run this command, replacing CLUSTER_NAME with the cluster name:

$ gcloud beta container clusters update CLUSTER_NAME \
    --update-addons=Istio=ENABLED --istio-config=auth=MTLS_STRICT

Or to change your cluster to mTLS in permissive mode:

$ gcloud beta container clusters update CLUSTER_NAME \
    --update-addons=Istio=ENABLED --istio-config=auth=MTLS_PERMISSIVE

Be aware that if you enable strict mTLS while you still have services that need to send or receive unencrypted traffic, your application may break! You can find out more about migrating to strict mTLS in Mutual TLS Migration. You can also specify more fine-grained destination-specific authentication policies. Destination-specific authentication policies will always override any global default mTLS setting, even if you switch from Strict to Permissive or vice versa.

You can find out much more about configuring and working with Istio security, including setting up role based authorization, in the Istio site.

Stackdriver tracing and logging

By default, a mesh installed with Istio on GKE can send logging and metrics data to Stackdriver, provided you have enabled the relevant Stackdriver features for your project and cluster. Versions of Istio on GKE earlier than 1.1.7 also send trace data by default. You can find out more about this in Stackdriver Support.

Both tracing and logging may incur additional costs to use, especially with a large volume of data coming from your mesh. If you would like to disable this feature without disabling the Stackdriver APIs entirely for your project, update your Istio on GKE configuration as follows.

To disable Stackdriver logging for Istio on GKE

1. Open the stackdriver-log rule for editing:

   kubectl edit -n istio-system rule stackdriver-log
   

2. Replace the match condition (context.protocol == "http" || context.protocol == "grpc") && (context.reporter.kind | "inbound" == "inbound") with "false".

3. Save and close the rule.

4. Open the stackdriver-log-tcp rule.

   kubectl edit -n istio-system rule stackdriver-log-tcp
   

5. Replace the match condition (context.protocol == "tcp") && (context.reporter.kind | "inbound" == "inbound") with "false".

6. Save and close the rule.

To disable Stackdriver tracing for Istio on GKE

If you have Istio on GKE version 1.1.3-gke.0 or earlier, or if you have manually enabled Stackdriver tracing, you can disable it as follows:

1. Open the stackdriver-tracing-rule rule for editing:

   kubectl edit -n istio-system rule stackdriver-tracing-rule
   

2. Replace the match condition context.protocol == "http" || context.protocol == "grpc" with "false".

3. Save and close the rule.

To enable Stackdriver tracing for Istio on GKE

If you are using version 1.1.7 or later and would like to enable Stackdriver tracing:

1. Make sure the Stackdriver Trace API is enabled in your Google Cloud project.

2. Open the stackdriver-tracing-rule rule for editing:

   kubectl edit -n istio-system rule stackdriver-tracing-rule
   

3. Replace the match condition "false" with context.protocol == "http" || context.protocol == "grpc".

4. Save and close the rule.

Enabling Prometheus and Grafana

Istio provides the following tools to help you observe service behavior in your mesh:

• Prometheus
• Grafana

Although Istio on GKE supports Prometheus and Grafana, you must manage them yourself. Prometheus and Grafana aren't automatically upgraded when Istio on GKE upgrades; a version mismatch with Istio on GKE isn't supported. If Prometheus and Grafana aren't working properly after an upgrade, you need to add them again using the version of the Helm charts that match the Istio on GKE version.

To enable Prometheus

When you enable Prometheus, Istio uses it to collect metrics and store them in the Prometheus time series database.

1. Grant cluster admin permissions to the current user.

2. Get the Helm client and charts.

3. Generate the YAML file with Prometheus disabled.

   helm template --set mixer.adapters.prometheus.enabled=false --namespace istio-system install/kubernetes/helm/istio > off.yaml

4. Optionally, choose the Prometheus installation options that you want to override.

5. Generate the YAML file with Prometheus enabled. If you want to change any default values, include the --set KEY=VALUE option in the Helm command.

   helm template --set mixer.adapters.prometheus.enabled=true --namespace istio-system install/kubernetes/helm/istio > on.yaml

6. Create a unified diff file.

   diff -u off.yaml on.yaml > prometheus.patch

7. Manually edit the unified diff file to resolve the differences and save only the YAML that is applicable to Prometheus in a manifest file.

   Caution: You must delete all the lines that don't apply to Prometheus from the file to avoid service disruption.

8. Apply the file to enable Prometheus. Replace MANIFEST with the name of your YAML file.

   kubectl apply -f MANIFEST

9. Once Prometheus is enabled, you must configure it to collect Istio metrics. We provide a sample Prometheus configuration with the appropriate rules that you can download from https://storage.cloud.google.com/gke_addon/prometheus.yaml.

10. Apply the downloaded file to configure Prometheus, again replacing MANIFEST with the name of your YAML file.

    kubectl apply -f MANIFEST

Installing both these manifests creates a new Prometheus service and deployment to collect Istio metrics. See Querying Metrics from Prometheus for more information.

To enable Grafana

When you enable Grafana, Istio provisions several dashboards that let you monitor performance and metrics for your services and the control plane components. Grafana relies on Prometheus for metrics collection, so you must enable Prometheus to view metrics in Grafana.

1. Grant cluster admin permissions to the current user.

2. Get the Helm client and charts.

3. Generate the YAML file with Grafana disabled.

   helm template --set grafana.enabled=false --namespace istio-system install/kubernetes/helm/istio > off.yaml

4. Optionally, choose the Grafana installation options that you want to set.

5. Generate the YAML file with Grafana enabled. If you want to change any default values, include the --set KEY=VALUE option in the Helm command.

   helm template --set grafana.enabled=true --namespace istio-system install/kubernetes/helm/istio > on.yaml

6. Create a unified diff file.

   diff -u off.yaml on.yaml > adapter.patch

7. Manually edit the unified diff file to resolve the differences and save only the YAML in the manifest file that is applicable to Grafana.

8. Apply the file to enable Grafana. Replace MANIFEST with the name of your manifest file.

   kubectl apply -f MANIFEST

Refer to Visualizing Metrics with Grafana for details on viewing metrics in Grafana.

Adding gateways

An Istio ingress gateway is provided as part of your Istio on GKE installation. When you upgrade GKE, Istio on GKE and all default resources including the default ingress gateway are upgraded automatically. If you add an ingress or egress gateway, they are under your control, and they aren't modified during the automatic upgrade. Note that an Istio egress gateway isn't installed by default in version 1.1 and later.

The default ingress gateway is suitable for deployments where the installed resources (RBAC, Service, Deployment) don't need customization beyond adding fields. Don't change any values in the default ingress gateway configuration because the changes are reverted back to the default values during the automatic upgrade. For more complex scenarios where customization is required, you need to create a new ingress gateway.

To add an ingress or egress gateway:

1. Grant cluster admin permissions to the current user.

2. Follow the steps in the Istio documentation to add the gateway.

Accessing external services

By default, Istio blocks all outbound requests from inside the cluster. Although you can create ServiceEntries to allow outbound requests to permitted destinations, for security reasons, you might want to add an egress gateway, as described in the Secure Control of Egress Traffic in Istio blogpost.

Grant cluster admin permissions

Before you enable Prometheus or Grafana or add a gateway, you need to grant cluster admin permissions to the current user to create the necessary role based access control (RBAC) rules for Istio.

To grant cluster admin permissions:

1. Set your user account to be the current user.

   gcloud auth login

2. Grant cluster admin permissions to the current user.

   kubectl create clusterrolebinding cluster-admin-binding \
   --clusterrole=cluster-admin \
   --user="$(gcloud config get-value core/account)"
   

Get the Helm client and charts

You use Helm to enable Prometheus and Grafana. Because the Helm charts might differ between Istio versions, the version of the Helm charts that you use must match your Istio on GKE version.

To install the Helm client and charts:

1. Install the Helm client.

2. Get the Istio on GKE version.

   1. Find the Pilot Pod id.

      kubectl get pods -n istio-system | grep pilot

      The output is similar to the following:

      istio-pilot-8df95498f-bvnh9    2/2    Running    0    2d23h

   2. Get the name of the container image for Pilot, which contains the Istio on GKE version. Replace PILOT_ID with the ID output from the previous command.

      kubectl get pod istio-pilot-PILOT_ID -n istio-system -oyaml | grep image: | grep pilot

      The output is similar to the following:

      image: gke.gcr.io/istio/pilot:1.1.7-gke.0
      image: gke.gcr.io/istio/pilot:1.1.7-gke.0

3. Download and extract the Istio installation file that matches your version of Istio on GKE. The Istio installation directory contains the Helm charts in install/kubernetes/helm/istio/charts.

   Make sure you download the version of the Istio installation file that matches your Istio on GKE version.

What's next?

• Try installing and exploring the Bookinfo example to see what Istio can do. To get the sample app and the istioctl tool, go to the Istio release page to download the installation file corresponding to the OS where you're running your commands. Then follow the instructions to deploy and test the application (you don't need to deploy Istio itself) in the GKE Installing Istio tutorial.
• Find out lots more about Istio in the open source documentation.
• If you need to remove the Istio add-on from a cluster, see Uninstalling Istio on GKE.