This guide shows you how to get started with the Istio on GKE add-on, including
installation options for new and existing clusters. You can install the add-on
using either the gcloud command line tool or the Google Cloud Console.
You can find out more about the Istio on GKE add-on and whether it's right for you in the Overview.
Before you start
Take the following steps to enable the Kubernetes Engine API:
- Visit the Kubernetes Engine page in the Google Cloud Console.
- Create or select a project.
- Wait for the API and related services to be enabled. This can take several minutes.
-
确保您的 Google Cloud Platform 项目已启用结算功能。 了解如何确认您的项目已启用结算功能。
Ensure you have the following command line tools installed:
-
gcloudis used to create and delete Kubernetes Engine clusters, including creating and updating clusters with the Istio on GKE add-on.gcloudis included in the Google Cloud SDK: follow the instructions to install and initialize it to work with your GCP projects. If you have an existinggcloudinstallation, verify that it's at least version 208.0.0:gcloud version
Note that you don't have to installgcloudto create a new Istio-enabled cluster, as you can use the Google Cloud Console instead, but it's still useful for managing existing clusters and installing other tools likekubectl. -
kubectlis used to manage Kubernetes, the cluster orchestration system used by GKE. You can installkubectlusinggcloud:gcloud components install kubectl
Set defaults for the gcloud command-line tool
To save time typing your project ID
and Compute Engine zone options in the gcloud
command-line tool, you can set the defaults:
gcloud config set project [PROJECT_ID] gcloud config set compute/zone [COMPUTE_ENGINE_ZONE]
Choose a security option
There are two possible default mesh-wide security options to choose from when creating or updating a cluster with Istio on GKE. Which one you choose depends on your initial application needs.
- Strict mTLS: In this security mode, Istio enforces mutual TLS (mTLS) encryption between all services and control plane components in the mesh by default, unless you override it with destination-specific rules. All calls within the mesh are encrypted and services will not accept unencrypted traffic.
- Permissive mTLS: In this security mode, by default Istio allows services in the mesh to accept both encrypted and unencrypted traffic, and all services send unencrypted calls by default. As with strict mTLS, you can override this for specific services. Use this option if you have services that still need to accept unencrypted traffic, for example if you have not fully migrated your services to Istio and have traffic coming from legacy clients outside the mesh. Istio on GKE provides this mode rather than simply installing Istio with no security enabled, as it makes it easier to migrate to strict mTLS later for added security.
You can find out how to update your security defaults and further configure Istio security in Updating security defaults, below.
Supported GKE cluster versions
The version of Istio on GKE installed when you create or update a cluster with Istio on GKE depends on the cluster version. We recommend that you use a cluster version with the latest version of Istio (1.2.10-gke.1). If you are using an older version, upgrade your cluster as soon as this Istio on GKE version becomes available.
Other GKE versions that have older but still supported versions of Istio are available (see Istio on GKE versions) but not recommended due to security vulnerabilities. Please read the release notes to understand the vulnerabilities before using these older versions.
| GKE versions | Istio version |
|---|---|
1.15.9-gke.9+1.14.10-gke.24+
|
1.2.10-gke.1 (recommended)
See the following for more information: |
1.16.4-gke.20+
|
1.2.10-gke.1 (supported)
See the following for more information: |
1.13.11-gke.11
|
1.1.16-gke.0 (supported)
See the following for more information: |
Installing Istio on GKE
You can install Istio on GKE either in a new cluster or an existing cluster. In both cases, this installs the Istio control plane. To take full advantage of Istio's features, you need to inject Envoy sidecar proxies into the Pods in your service mesh.
Creating a cluster with Istio on GKE
We suggest creating at least a 4 node cluster with the 2 vCPU machine type when using this add-on. You can deploy Istio itself with the default GKE new cluster setup but this may not provide enough resources to explore sample applications. Please note that the Istio on GKE add-on is not compatible with Workload Identity.
To create a cluster using Istio on GKE:
Console
- Go to the Kubernetes page in the Cloud Console and select Create Cluster.
- Using the default Standard Cluster dialog, choose your preferred number of nodes and machines, bearing in mind the minimum recommended cluster size for Istio.
- In the Master Version drop-down, select an Istio on GKE recommended cluster version (or a supported version if you can't use the recommended version).
- Select Availability, networking, security, and additional features to display additional configuration options, including Istio on GKE.
- Select Enable Istio (beta).
- Select the mTLS security mode you want to use for your cluster from the drop-down.
- Click Create to create your cluster.
Command line
To create a GKE cluster with Istio enabled and with mutual TLS
enforced by default, run this command, replacing
CLUSTER_NAME with your chosen cluster name,
and CLUSTER_VERSION with a
compatible cluster version:
gcloud beta container clusters create CLUSTER_NAME \
--addons=Istio --istio-config=auth=MTLS_STRICT \
--cluster-version=CLUSTER_VERSION \
--machine-type=n1-standard-2 \
--num-nodes=4
Or to create a GKE cluster with Istio enabled and with mTLS in permissive mode:
gcloud beta container clusters create CLUSTER_NAME \
--addons=Istio --istio-config=auth=MTLS_PERMISSIVE \
--cluster-version=CLUSTER_VERSION \
--machine-type=n1-standard-2 \
--num-nodes=4
Adding Istio on GKE to an existing cluster
If you want to update a cluster with the add-on, you may need to first resize your cluster to ensure that you have enough resources for Istio. As when creating a new cluster, we suggest at least a 4 node cluster with the 2 vCPU machine type.
Your cluster must also be running a supported cluster master version to use the add-on. Please note that the Istio on GKE add-on is not compatible with Workload Identity.
To update an existing cluster with the Istio on GKE add-on:
Console
- Go to the Kubernetes clusters page in the Cloud Console and select the cluster you want to update
- Select Edit.
- Under Istio (beta) select Enabled to display Istio mTLS (beta).
- Select the mTLS security mode you want to use for your cluster from the drop-down.
- Click Save to update your cluster.
Command line
To add Istio with mutual TLS enforced by default to an existing
cluster, run this command, replacing CLUSTER_NAME
with the cluster name:
gcloud beta container clusters update CLUSTER_NAME \
--update-addons=Istio=ENABLED --istio-config=auth=MTLS_STRICT
Or to add Istio with mTLS in permissive mode to an existing cluster:
gcloud beta container clusters update CLUSTER_NAME \
--update-addons=Istio=ENABLED --istio-config=auth=MTLS_PERMISSIVE
Note that the clusters update command may require other parameters, depending
on your actual cluster configuration.
If you have an existing application on the cluster, you can find out how to migrate it so it's managed by Istio in the Istio documentation.
Verifying installation
To verify that your Istio on GKE installation was successful:
- If you've just created rather than updated a cluster, check that it's up and
running with a GKE version of 1.10.6 or higher:
gcloud container clusters list
The output is similar to the following:NAME LOCATION MASTER_VERSION MASTER_IP MACHINE_TYPE NODE_VERSION NUM_NODES STATUS istio-demo us-central1-b 1.11.2-gke.15 35.239.252.38 n1-standard-2 1.11.2-gke.15 4 RUNNING
- Get the credentials for your new cluster so you can interact with it with
kubectl.gcloud container clusters get-credentials CLUSTER_NAME
- Ensure the following Kubernetes services are deployed:
istio-citadel,istio-egressgateway,istio-pilot,istio-ingressgateway,istio-policy,istio-sidecar-injector, andistio-telemetry(you'll also see the other deployed services):kubectl get service -n istio-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE istio-citadel ClusterIP 10.47.245.92 <none> 8060/TCP,9093/TCP 12s istio-egressgateway ClusterIP 10.47.248.129 <none> 80/TCP,443/TCP 12s istio-galley ClusterIP 10.47.248.109 <none> 443/TCP,9093/TCP 12s istio-ingressgateway LoadBalancer 10.47.248.117 <pending> 80:31380/TCP,443:31390/TCP,31400:31400/TCP,15011:30221/TCP,8060:32445/TCP,853:30663/TCP,15030:32010/TCP,15031:32633/TCP 12s istio-pilot ClusterIP 10.47.251.133 <none> 15010/TCP,15011/TCP,8080/TCP,9093/TCP 12s istio-policy ClusterIP 10.47.255.244 <none> 9091/TCP,15004/TCP,9093/TCP 12s istio-sidecar-injector ClusterIP 10.47.240.36 <none> 443/TCP 12s istio-statsd-prom-bridge ClusterIP 10.47.247.135 <none> 9102/TCP,9125/UDP 12s istio-telemetry ClusterIP 10.47.242.73 <none> 9091/TCP,15004/TCP,9093/TCP,42422/TCP 12s promsd ClusterIP 10.47.241.188 <none> 9090/TCP 12s
Ensure the corresponding Kubernetes Pods are deployed and all containers are up and running:
istio-pilot-*,istio-policy-*,istio-telemetry-*,istio-egressgateway-*,istio-ingressgateway-*,istio-sidecar-injector-*, andistio-citadel-*.kubectl get pods -n istio-system
NAME READY STATUS RESTARTS AGE istio-citadel-555d845b65-xfdmj 1/1 Running 0 2d istio-cleanup-secrets-8x2pl 0/1 Completed 0 2d istio-egressgateway-667d854c49-9q5dl 1/1 Running 0 2d istio-galley-6c9cd5b8bb-4j4jk 1/1 Running 0 2d istio-ingressgateway-6c796c5594-f972p 1/1 Running 0 2d istio-pilot-77f74fc6f-rpbfj 2/2 Running 0 2d istio-policy-655b87fff-4wbwq 2/2 Running 0 2d istio-security-post-install-tm2rm 0/1 Completed 1 2d istio-sidecar-injector-668c9fb4db-p6lwt 1/1 Running 0 2d istio-statsd-prom-bridge-5b645f6f4d-6pbgf 1/1 Running 0 2d istio-telemetry-d9848f498-wf6kh 2/2 Running 0 2d promsd-6b989699d8-l7jxt 1/1 Running 0 2d
Enabling sidecar injection
To take full advantage of Istio's features, each service in your application needs to have an Envoy sidecar proxy running in its Pod. The Envoy proxy intercepts all inbound and outbound traffic to the service and communicates with the Istio control plane. You can inject an Envoy proxy manually by updating your Pods' Kubernetes configuration, or you can use Istio's webhooks-based automatic sidecar injection.
By default, Istio sidecar auto-injection is disabled for all namespaces. To
enable auto-injection, replace NAMESPACE in the
following command with the name of the
namespace
for your application's services or with default:
kubectl label namespace NAMESPACE istio-injection=enabled
Any running Pods must be restarted for the change to take effect, as the sidecar is added at Pod creation time. To disable auto-injection in the namespace, remove the label and restart Pods to remove their sidecars.
To inject Envoy sidecars manually, refer to Installing the sidecar.
Configuring your control plane
Although Istio on GKE
manages most of your control plane settings,
for production use we recommend that you choose and specify appropriate values
for your use case for the following settings. You can change them using
kubectl or your Kubernetes tools of choice. These settings are not
changed when your installation is upgraded by the add-on.
Horizontal scaling
Configure one of the following to ensure you have enough replicas
of your Istio control plane components to handle your mesh's traffic. For
production use we recommend a minimum of two replicas for istio-policy,
istio-telemetry, istio-pilot and istio-sidecar-injector.
Horizontal pod autoscaling automatically scales the number of replicas based on observed CPU utilization. We provide default maximum and minimum values for autoscaling control plane components, but you can edit them to suit your needs. For example, here's how you'd specify with
kubectl editthat you want to edit the autoscaler settings for Istio-Telemetry:kubectl edit -n istio-system HorizontalPodAutoscalers/istio-telemetryIf you're not using autoscaling, you can set the number of replicas for each control plane element (with the exception of Citadel, which is always a singleton) for manual horizontal scaling. For example, here's how you'd specify with
kubectlthat you want two instances of Pilot:kubectl scale -n istio-system --replicas=2 deployment/istio-pilot
Resource requests
By default, resource requests
are not set, however for production use we recommend setting these to
appropriate values to ensure nodes have enough resources to support the pods.
You should set resource requests for each container in the pod, otherwise CPU on HPAs
show unknown and autoscaling will not work.
You can find recommended starting points for resource settings for each
component in the Istio Installation Options
guide. For example, resource requests and limits for Mixer are under mixer options.
kubectl edit -n istio-system Deployments/istio-telemetry
Pod disruption budget
PodDisruptionBudgets let you specify the minimum number of available replicas of a given deployment that an application can tolerate and continue to work.
You should configure PodDisruptionBudgets
for all deployments that must remain available during Istio on GKE upgrades. At a minimum,
we recommend doing this for the add-on's provided Istio ingress gateway
(istio-ingressgateway), if you haven't deployed and configured your own
ingress gateway, as otherwise external traffic may not be able to reach your
application during upgrade.
Pod disruption budget settings should be configured along with horizontal scaling settings,
as described above. The number of replicas or autoscaler minimum replicas needs
to be greater than your pod disruption budget minimum, because Istio on
GKE's upgrade process makes individual replicas unavailable while they are
updated. This ensures that the minAvailable setting in PodDisruptionBudget
is not violated during upgrades and the upgrade works as intended.
Customizing your installation
While Istio on GKE provides reasonable default behavior for many use cases, you can customize your installation in a number of ways, including configuring telemetry tools and adding gateways. This section describes how to make supported customizations.
Updating security defaults
Switching the default Istio mTLS security mode in a running cluster from Strict to Permissive, or vice versa, uses the same command as adding Istio to a cluster:
Console
- Go to the Kubernetes clusters page in the Cloud Console and select the cluster you want to update
- Select Edit.
- Under Istio (beta) select Enabled to display Istio mTLS (beta).
- Select the mTLS security mode you want to use for your cluster from the drop-down.
- Click Save to update your cluster.
Command line
To change your cluster to use Istio with mutual TLS enforced by
default, run this command, replacing
CLUSTER_NAME with the cluster name:
$ gcloud beta container clusters update CLUSTER_NAME \
--update-addons=Istio=ENABLED --istio-config=auth=MTLS_STRICT
Or to change your cluster to mTLS in permissive mode:
$ gcloud beta container clusters update CLUSTER_NAME \
--update-addons=Istio=ENABLED --istio-config=auth=MTLS_PERMISSIVE
Be aware that if you enable strict mTLS while you still have services that need to send or receive unencrypted traffic, your application may break! You can find out more about migrating to strict mTLS in Mutual TLS Migration. You can also specify more fine-grained destination-specific authentication policies. Destination-specific authentication policies will always override any global default mTLS setting, even if you switch from Strict to Permissive or vice versa.
You can find out much more about configuring and working with Istio security, including setting up role based authorization, in the Istio site.
the Google Cloud operations suite tracing and logging
By default, a mesh installed with Istio on GKE can send logging and metrics data to the Google Cloud operations suite, provided you have enabled the relevant the Google Cloud operations suite features for your project and cluster. Versions of Istio on GKE earlier than 1.1.7 also send trace data by default. You can find out more about this in the Google Cloud operations suite Support.
Both tracing and logging may incur additional costs to use, especially with a large volume of data coming from your mesh. If you would like to disable this feature without disabling the the Google Cloud operations suite APIs entirely for your project, update your Istio on GKE configuration as follows.
To disable the Google Cloud operations suite logging for Istio on GKE
Open the
stackdriver-logrule for editing:kubectl edit -n istio-system rule stackdriver-log
Replace the
matchcondition(context.protocol == "http" || context.protocol == "grpc") && (context.reporter.kind | "inbound" == "inbound")with"false".Save and close the rule.
Open the
stackdriver-log-tcprule.kubectl edit -n istio-system rule stackdriver-log-tcp
Replace the
matchcondition(context.protocol == "tcp") && (context.reporter.kind | "inbound" == "inbound")with"false".Save and close the rule.
To disable the Google Cloud operations suite tracing for Istio on GKE
If you have Istio on GKE version 1.1.3-gke.0 or earlier, or if you have manually enabled the Google Cloud operations suite tracing, you can disable it as follows:
Open the
stackdriver-tracing-rulerule for editing:kubectl edit -n istio-system rule stackdriver-tracing-rule
Replace the
matchconditioncontext.protocol == "http" || context.protocol == "grpc"with"false".Save and close the rule.
To enable the Google Cloud operations suite tracing for Istio on GKE
If you are using version 1.1.7 or later and would like to enable the Google Cloud operations suite tracing:
Make sure the Cloud Trace API is enabled in your Google Cloud project.
Open the
stackdriver-tracing-rulerule for editing:kubectl edit -n istio-system rule stackdriver-tracing-rule
Replace the
matchcondition"false"withcontext.protocol == "http" || context.protocol == "grpc".Save and close the rule.
Enabling Prometheus and Grafana
Istio provides the following tools to help you observe service behavior in your mesh:
Although Istio on GKE supports Prometheus and Grafana, you must manage them yourself. Prometheus and Grafana aren't automatically upgraded when Istio on GKE upgrades; a version mismatch with Istio on GKE isn't supported. If Prometheus and Grafana aren't working properly after an upgrade, you need to add them again using the version of the Helm charts that match the Istio on GKE version.
To enable Prometheus
When you enable Prometheus, Istio uses it to collect metrics and store them in the Prometheus time series database.
Generate the YAML file with Prometheus disabled.
helm template --set mixer.adapters.prometheus.enabled=false --namespace istio-system install/kubernetes/helm/istio > off.yaml
Optionally, choose the Prometheus installation options that you want to override.
Generate the YAML file with Prometheus enabled. If you want to change any default values, include the
--set KEY=VALUEoption in the Helm command.helm template --set mixer.adapters.prometheus.enabled=true --namespace istio-system install/kubernetes/helm/istio > on.yaml
Create a unified diff file.
diff -u off.yaml on.yaml > prometheus.patch
Manually edit the unified diff file to resolve the differences and save only the YAML that is applicable to Prometheus in a manifest file.
Apply the file to enable Prometheus. Replace
MANIFESTwith the name of your YAML file.kubectl apply -f MANIFEST
Once Prometheus is enabled, you must configure it to collect Istio metrics. We provide a sample Prometheus configuration with the appropriate rules that you can download from https://storage.cloud.google.com/gke_addon/prometheus.yaml.
Apply the downloaded file to configure Prometheus, again replacing
MANIFESTwith the name of your YAML file.kubectl apply -f MANIFEST
Installing both these manifests creates a new Prometheus service and deployment to collect Istio metrics. See Querying Metrics from Prometheus for more information.
To enable Grafana
When you enable Grafana, Istio provisions several dashboards that let you monitor performance and metrics for your services and the control plane components. Grafana relies on Prometheus for metrics collection, so you must enable Prometheus to view metrics in Grafana.
Generate the YAML file with Grafana disabled.
helm template --set grafana.enabled=false --namespace istio-system install/kubernetes/helm/istio > off.yaml
Optionally, choose the Grafana installation options that you want to set.
Generate the YAML file with Grafana enabled. If you want to change any default values, include the
--set KEY=VALUEoption in the Helm command.helm template --set grafana.enabled=true --namespace istio-system install/kubernetes/helm/istio > on.yaml
Create a unified diff file.
diff -u off.yaml on.yaml > adapter.patch
Manually edit the unified diff file to resolve the differences and save only the YAML in the manifest file that is applicable to Grafana.
Apply the file to enable Grafana. Replace
MANIFESTwith the name of your manifest file.kubectl apply -f MANIFEST
Refer to Visualizing Metrics with Grafana for details on viewing metrics in Grafana.
Adding gateways
An Istio ingress gateway is provided as part of your Istio on GKE installation. When you upgrade GKE, Istio on GKE and all default resources including the default ingress gateway are upgraded automatically. If you add an ingress or egress gateway, they are under your control, and they aren't modified during the automatic upgrade. Note that an Istio egress gateway isn't installed by default in version 1.1 and later.
The default ingress gateway is suitable for deployments where the installed resources (RBAC, Service, Deployment) don't need customization beyond adding fields. Don't change any values in the default ingress gateway configuration because the changes are reverted back to the default values during the automatic upgrade. For more complex scenarios where customization is required, you need to create a new ingress gateway.
To add an ingress or egress gateway:
Follow the steps in the Istio documentation to add the gateway.
Accessing external services
By default, Istio blocks all outbound requests from inside the cluster. Although you can create ServiceEntries to allow outbound requests to permitted destinations, for security reasons, you might want to add an egress gateway, as described in the Secure Control of Egress Traffic in Istio blogpost.
Grant cluster admin permissions
Before you enable Prometheus or Grafana or add a gateway, you need to grant cluster admin permissions to the current user to create the necessary role based access control (RBAC) rules for Istio.
To grant cluster admin permissions:
Set your user account to be the current user.
gcloud auth login
Grant cluster admin permissions to the current user.
kubectl create clusterrolebinding cluster-admin-binding \ --clusterrole=cluster-admin \ --user="$(gcloud config get-value core/account)"
Get the Helm client and charts
You use Helm to enable Prometheus and Grafana. Because the Helm charts might differ between Istio versions, the version of the Helm charts that you use must match your Istio on GKE version.
To install the Helm client and charts:
Get the Istio on GKE version.
Find the Pilot Pod id.
kubectl get pods -n istio-system | grep pilot
The output is similar to the following:
istio-pilot-8df95498f-bvnh9 2/2 Running 0 2d23h
Get the name of the container image for Pilot, which contains the Istio on GKE version. Replace
PILOT_IDwith the ID output from the previous command.kubectl get pod istio-pilot-PILOT_ID -n istio-system -oyaml | grep image: | grep pilot
The output is similar to the following:
image: gke.gcr.io/istio/pilot:1.1.7-gke.0 image: gke.gcr.io/istio/pilot:1.1.7-gke.0
Download and extract the Istio installation file that matches your version of Istio on GKE. The Istio installation directory contains the Helm charts in
install/kubernetes/helm/istio/charts.Make sure you download the version of the Istio installation file that matches your Istio on GKE version.
What's next?
- Try installing and exploring the Bookinfo example to see what Istio can do.
To get the sample app and the
istioctltool, go to the Istio release page to download the installation file corresponding to the OS where you're running your commands. Then follow the instructions to deploy and test the application (you don't need to deploy Istio itself) in the GKE Installing Istio tutorial. - Find out lots more about Istio in the open source documentation.
- If you need to remove the Istio add-on from a cluster, see Uninstalling Istio on GKE.