This tutorial describes how to integrate Identity-Aware Proxy (IAP) with Anthos Service Mesh. The IAP integration with Anthos Service Mesh enables you to safely access services based on Google's BeyondCorp principles. IAP verifies user identity and context of the request to determine if a user should be allowed to access an application or resource. The IAP integration with Anthos Service Mesh provides you with the following benefits:
Complete context-aware access control to the workloads running on Anthos Service Mesh. You can set fine-grained access policies based on attributes of the originating request, such as user identity, the IP address, and device type. You can combine your access policies with restrictions based on the hostname and path of a request URL.
Enable support for context-aware claims in Anthos Service Mesh authorization.
Scalable, secure, and highly available access to your application through a Google Cloud load balancer. High performance load balancing provides built-in protection of distributed denial-of-service (DDoS) attacks and support for global anycast IP addressing.
Objectives
- Get set up:
- Set up your Cloud project to grant the permissions and enable the Google APIs required by IAP.
- Reserve an external static IP address and configure a domain name to use the IP address, which the load balancer needs.
- Set up a new Google Kubernetes Engine (GKE) cluster with the options required to integrate IAP with Anthos Service Mesh.
- Install Anthos Service Mesh with the options required for the integration.
- Deploy a sample application.
- Deploy the load balancer.
Enable IAP.
Enable RCToken support on the service mesh.
Costs
This tutorial uses the following billable components of Google Cloud:
To generate a cost estimate based on your projected usage,
use the pricing calculator.
When you finish this tutorial, you can avoid continued billing by deleting the resources you created. For more information, see Clean up.
Before you begin
Requirements
You must have an Anthos trial license or subscription. See the Anthos Pricing guide for details.
Your GKE cluster must meet the following requirements:
- At least four nodes.
- The minimum machine type
is
e2-standard-4
, which has four vCPUs. - Use a release channel rather than a static version of GKE
To be included in the service mesh, service ports must be named, and the name must include the port's protocol in the following syntax:
name: protocol[-suffix]
where the square brackets indicate an optional suffix that must start with a dash. For more information, Naming service ports.If you are installing Anthos Service Mesh on a private cluster, you must add a firewall rule to open port 15017 if you want to use automatic sidecar injection. If you don't add the firewall rule and automatic sidecar injection is enabled, you get an error when you deploy workloads. For details on adding a firewall rule, see Adding firewall rules for specific use cases.
If you have created a service perimeter in your organization, you might need to add the Mesh CA service to the perimeter. See Adding Mesh CA to a service perimeter for more information.
Setting up your environment
For installations on Google Kubernetes Engine, you can follow the installation guides using Cloud Shell, an in-browser command line interface to your Google Cloud resources, or your own computer running Linux or macOS.
Option A: Use Cloud Shell
Cloud Shell provisions a g1-small Compute Engine virtual machine (VM) running a Debian-based Linux operating system. The advantages to using Cloud Shell are:
Cloud Shell includes the
gcloud
,kubectl
andhelm
command-line tools that you need.Your Cloud Shell $HOME directory has 5GB persistent storage space.
You have your choice of text editors:
Code editor, which you access by clicking at the top of the Cloud Shell window.
Emacs, Vim, or Nano, which you access from the command line in Cloud Shell.
To use Cloud Shell:
- Go to the Google Cloud console.
- Select your Cloud project.
Click the Activate Cloud Shell button at the top of the Google Cloud console window.
A Cloud Shell session opens inside a new frame at the bottom of the Google Cloud console and displays a command-line prompt.
Update the components:
gcloud components update
The command responds with output similar to the following:
ERROR: (gcloud.components.update) You cannot perform this action because the gcloud CLI component manager is disabled for this installation. You can run the following command to achieve the same result for this installation: sudo apt-get update && sudo apt-get --only-upgrade install ...
Copy the long command and paste it to update the components.
Install
kubectl
:sudo apt-get install kubectl
Install
kpt
:sudo apt-get install google-cloud-sdk-kpt
Option B: Use command-line tools locally
On your local machine, install and initialize the gcloud CLI.
Authenticate with the Google Cloud CLI:
gcloud auth login
Update the components:
gcloud components update
Install
kubectl
:gcloud components install kubectl
Install
kpt
:gcloud components install kpt
Setting up your project
-
Get the project ID of the project that the cluster will be created in:
gcloud
gcloud projects list
Console
- In the Google Cloud console, go to the Dashboard page:
-
Click the Select from drop-down list at the top of the page. In the
Select from window that appears, select your project.
The project ID is displayed on the project Dashboard Project info card.
- Create an environment variable for the project ID:
export PROJECT_ID=YOUR_PROJECT_ID
-
Set the default project ID for the
gcloud
command-line tool:gcloud config set project ${PROJECT_ID}
- Create an environment variable for the project number:
export PROJECT_NUMBER=$(gcloud projects describe ${PROJECT_ID} --format="value(projectNumber)")
-
Set the required Identity and Access Management (IAM) roles. If you are a
Project Owner, you have all the necessary permissions to complete
the installation and register your cluster with your environ.
If you aren't a Project Owner, you need someone who is to grant you the
following specific IAM roles. In the following command, change
GCP_EMAIL_ADDRESS
to the account that you use to log in to Google Cloud.gcloud projects add-iam-policy-binding ${PROJECT_ID} \ --member user:GCP_EMAIL_ADDRESS \ --role=roles/editor \ --role=roles/compute.admin \ --role=roles/container.admin \ --role=roles/resourcemanager.projectIamAdmin \ --role=roles/iam.serviceAccountAdmin \ --role=roles/iam.serviceAccountKeyAdmin \ --role=roles/gkehub.admin
To learn more about how to grant IAM roles, refer to Granting, changing, and revoking access to resources. For a description of these roles, see Permissions required to install Anthos Service Mesh
- Enable the following APIs:
gcloud services enable \ container.googleapis.com \ compute.googleapis.com \ monitoring.googleapis.com \ logging.googleapis.com \ cloudtrace.googleapis.com \ meshca.googleapis.com \ meshtelemetry.googleapis.com \ meshconfig.googleapis.com \ iamcredentials.googleapis.com \ anthos.googleapis.com \ gkeconnect.googleapis.com \ gkehub.googleapis.com \ cloudresourcemanager.googleapis.com \ iap.googleapis.com
Enabling the APIs can take a minute or more to complete. When the APIs are enabled, you see output similar to the following:
Operation "operations/acf.601db672-88e6-4f98-8ceb-aa3b5725533c" finished successfully.
Reserve a static IP address and configure DNS
To integrate Identity-Aware Proxy with Anthos Service Mesh, you have to set up a Google Cloud HTTP(S) load balancer, which requires a domain name that points to a static IP address. You can reserve a static external IP address, which assigns the address to your project indefinitely until you explicitly release it.
Reserve a static external IP address:
gcloud compute addresses create example-static-ip --global
Get the static IP address:
gcloud compute addresses describe example-static-ip --global
In your domain name registrar, configure a fully qualified domain name (FQDN) with the static IP address. Typically, you add an
A
record to your DNS settings. The configuration steps and terminology for adding anA
record for a FQDN vary depending on your domain name registrar.Set the domain name in an environment variable:
export DOMAIN_NAME=YOUR_DOMAIN_NAME
It can take 24 to 48 hours for the DNS setting to propagate. You can continue setting up everything in this tutorial, but you won't be able to test the setup until the DNS settings propagate.
Setting up a new GKE cluster
To set up a new cluster:
Select a zone, a machine type, and a GKE release channel for the new cluster. The minimum machine type required by Anthos Service Mesh is n1-standard-4. You can use any release channel option.
To get a list of the available GCP zones:
gcloud compute zones list
To get a list of machine types:
gcloud compute machine-types list | more
Create the following environment variables:
Set the cluster name:
export CLUSTER_NAME=YOUR_CLUSTER_NAME
The cluster name must contain only lowercase alphanumerics and '-', must start with a letter and end with an alphanumeric, and must be no longer than 40 characters.
Set the
CLUSTER_LOCATION
to your cluster zone:export CLUSTER_LOCATION=YOUR_ZONE
Set the workload pool:
export WORKLOAD_POOL=${PROJECT_ID}.svc.id.goog
Set the mesh ID:
export MESH_ID="proj-${PROJECT_NUMBER}"
Create the cluster with the options required by Anthos Service Mesh. The following command creates a cluster containing 4 nodes of machine type n1-standard-4, which has 4 vCPUs. This is the minimum machine type and number of nodes required for Anthos Service Mesh. You can specify another machine type as long as it has at least 4 vCPUs, and you can increase the number of nodes as needed for your system requirements.
gcloud beta container clusters create ${CLUSTER_NAME} \ --project=${PROJECT_ID} \ --zone=${CLUSTER_LOCATION} \ --machine-type=n1-standard-4 \ --num-nodes=4 \ --workload-pool=${WORKLOAD_POOL} \ --enable-stackdriver-kubernetes \ --subnetwork=default \ --labels mesh_id=${MESH_ID} \ --addons=HttpLoadBalancing \ --release-channel=regular
The
clusters create
command includes:workload-pool=${WORKLOAD_POOL}
: Enables Workload Identity, which is the recommended way to safely access Google Cloud services from GKE applications.enable-stackdriver-kubernetes
: Enables Cloud Monitoring and Cloud Logging on GKE.subnetwork=default
: Creates a default subnetwork.labels mesh_id=${MESH_ID}
: Sets themesh_id
label on the cluster, which is required for metrics to get displayed on the Anthos Service Mesh dashboard in the Google Cloud console.The
HttpLoadBalancing
add-on, which enables an HTTP (L7) load balancing controller for the cluster.release-channel regular
: Enrolls the cluster in theregular
release channel, though you can choosestable
if you require greater stability orrapid
if you want to try out new (unsupported) GKE features.
Setting credentials and permissions
- Initialize your project to ready it for installation. Among other things,
this command creates a service account to let Istio components, such as the
sidecar proxy, securely access your project's data and resources:
curl --request POST \ --header "Authorization: Bearer $(gcloud auth print-access-token)" \ --data '' \ https://meshconfig.googleapis.com/v1alpha1/projects/${PROJECT_ID}:initialize
The command responds with empty curly braces:
{}
If you install a new version of Anthos Service Mesh on this cluster in the future, you don't need to re-run the command, but running the command again doesn't affect your installation.
-
Get
authentication credentials to interact with the cluster:
gcloud container clusters get-credentials ${CLUSTER_NAME}
-
Grant cluster admin permissions to the current user. You need these
permissions to create the necessary
role based access
control (RBAC) rules for Anthos Service Mesh:
kubectl create clusterrolebinding cluster-admin-binding \ --clusterrole=cluster-admin \ --user="$(gcloud config get-value core/account)"
If you see the
"cluster-admin-binding" already exists
error, you can safely ignore it and continue with the existing cluster-admin-binding.
Registering your cluster
You must register your cluster with the project's fleet to gain access to the unified user interface in the Google Cloud console. A fleet provides a unified way to view and manage the clusters and their workloads, including clusters outside Google Cloud.
Create a Google Cloud service account and key file
A JSON file containing service account credentials is required to register a cluster. To follow the principle of least privilege, we recommend that you create a distinct service account for each cluster that you register.
To create a service account and key file:
Select a name for the service account and create an environment variable for it:
export SERVICE_ACCOUNT_NAME=SERVICE_ACCOUNT_NAME
Create the service account:
gcloud iam service-accounts create ${SERVICE_ACCOUNT_NAME}
List all of a project's service accounts to confirm the service account was created:
gcloud iam service-accounts list
Bind the gkehub.connect IAM role to the service account:
gcloud projects add-iam-policy-binding ${PROJECT_ID} \ --member="serviceAccount:${SERVICE_ACCOUNT_NAME}@${PROJECT_ID}.iam.gserviceaccount.com" \ --role="roles/gkehub.connect"
Create an environment variable for the local filepath where you want to save the JSON file. We recommend that you name the file using the service account name and your project ID, such as:
/tmp/creds/${SERVICE_ACCOUNT_NAME}-${PROJECT_ID}.json
export SERVICE_ACCOUNT_KEY_PATH=LOCAL_KEY_PATH
Download the service account's private key JSON file:
gcloud iam service-accounts keys create ${SERVICE_ACCOUNT_KEY_PATH} \ --iam-account=${SERVICE_ACCOUNT_NAME}@${PROJECT_ID}.iam.gserviceaccount.com
Register the cluster
In the following command, replace MEMBERSHIP_NAME
with a name that uniquely represents the cluster being registered on the Hub.
gcloud container hub memberships register MEMBERSHIP_NAME \ --gke-cluster=${CLUSTER_LOCATION}/${CLUSTER_NAME} \ --service-account-key-file=${SERVICE_ACCOUNT_KEY_PATH}
The command responds with output similar to the following:
kubeconfig entry generated for CLUSTER_NAME. Waiting for membership to be created...done. Created a new membership [projects/PROJECT_ID/locations/global/memberships/MEMBERSHIP_NAME] for the cluster [MEMBERSHIP_NAME] Generating the Connect Agent manifest... Deploying the Connect Agent on cluster [MEMBERSHIP_NAME] in namespace [gke-connect]... Deployed the Connect Agent on cluster [MEMBERSHIP_NAME] in namespace [gke-connect]. Finished registering the cluster [MEMBERSHIP_NAME] with the Hub.
This service account key is stored as a secret named creds-gcp
in the
gke-connect
namespace.
For more information about cluster registration, see Registering a cluster in the Connect documentation.
Downloading the installation file
Before continuing, verify that the ASM Mesh Data Plane Service Account is a member of the project:
gcloud projects get-iam-policy ${PROJECT_ID} | grep -B 1 'roles/meshdataplane.serviceAgent'
If the previous command doesn't output anything, go back to the
Set credentials and permissions
section and run the curl
command.
-
Download the Anthos Service Mesh installation file to your current working
directory:
curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.5.10-asm.2-linux.tar.gz
-
Download the signature file and use
openssl
to verify the signature:curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.5.10-asm.2-linux.tar.gz.1.sig openssl dgst -verify - -signature istio-1.5.10-asm.2-linux.tar.gz.1.sig istio-1.5.10-asm.2-linux.tar.gz <<'EOF' -----BEGIN PUBLIC KEY----- MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEWZrGCUaJJr1H8a36sG4UUoXvlXvZ wQfk16sxprI2gOJ2vFFggdq3ixF2h4qNBt0kI7ciDhgpwS8t+/960IsIgw== -----END PUBLIC KEY----- EOF
The expected output is:
Verified OK
-
Extract the contents of the file to any location on your file system. For
example, to extract the contents to the current working directory:
tar xzf istio-1.5.10-asm.2-linux.tar.gz
The command creates an installation directory in your current working directory named
istio-1.5.10-asm.2
that contains:- Sample applications in
samples
- The following tools in the
bin
directory:istioctl
: You useistioctl
to install Anthos Service Mesh.asmctl
: You useasmctl
to help validate your security configuration after installing Anthos Service Mesh. (Currently,asmctl
isn't supported on Anthos clusters on VMware.)
- Sample applications in
-
Download the Anthos Service Mesh installation file to your current working
directory:
curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.5.10-asm.2-osx.tar.gz
-
Download the signature file and use
openssl
to verify the signature:curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.5.10-asm.2-osx.tar.gz.1.sig openssl dgst -sha256 -verify /dev/stdin -signature istio-1.5.10-asm.2-osx.tar.gz.1.sig istio-1.5.10-asm.2-osx.tar.gz <<'EOF' -----BEGIN PUBLIC KEY----- MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEWZrGCUaJJr1H8a36sG4UUoXvlXvZ wQfk16sxprI2gOJ2vFFggdq3ixF2h4qNBt0kI7ciDhgpwS8t+/960IsIgw== -----END PUBLIC KEY----- EOF
The expected output is:
Verified OK
-
Extract the contents of the file to any location on your file system. For
example, to extract the contents to the current working directory:
tar xzf istio-1.5.10-asm.2-osx.tar.gz
The command creates an installation directory in your current working directory named
istio-1.5.10-asm.2
that contains:- Sample applications in
samples
- The following tools in the
bin
directory:istioctl
: You useistioctl
to install Anthos Service Mesh.asmctl
: You useasmctl
to help validate your security configuration after installing Anthos Service Mesh. (Currently,asmctl
isn't supported on Anthos clusters on VMware.)
- Sample applications in
-
Download the Anthos Service Mesh installation file to your current working
directory:
curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.5.10-asm.2-win.zip
-
Download the signature file and use
openssl
to verify the signature:curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.5.10-asm.2-win.zip.1.sig openssl dgst -verify - -signature istio-1.5.10-asm.2-win.zip.1.sig istio-1.5.10-asm.2-win.zip <<'EOF' -----BEGIN PUBLIC KEY----- MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEWZrGCUaJJr1H8a36sG4UUoXvlXvZ wQfk16sxprI2gOJ2vFFggdq3ixF2h4qNBt0kI7ciDhgpwS8t+/960IsIgw== -----END PUBLIC KEY----- EOF
The expected output is:
Verified OK
-
Extract the contents of the file to any location on your file system. For
example, to extract the contents to the current working directory:
tar xzf istio-1.5.10-asm.2-win.zip
The command creates an installation directory in your current working directory named
istio-1.5.10-asm.2
that contains:- Sample applications in
samples
- The following tools in the
bin
directory:istioctl
: You useistioctl
to install Anthos Service Mesh.asmctl
: You useasmctl
to help validate your security configuration after installing Anthos Service Mesh. (Currently,asmctl
isn't supported on Anthos clusters on VMware.)
- Sample applications in
-
Ensure that you're in the Anthos Service Mesh installation's root directory.
cd istio-1.5.10-asm.2
-
For convenience, add the tools in the
/bin
directory to your PATH:export PATH=$PWD/bin:$PATH
Linux
Mac OS
Windows
Preparing resource configuration files
When you run the istioctl apply command
to install Anthos Service Mesh, you
specify -f istio-operator.yaml
on the command line. This file contains
information about your project and cluster that is needed to enable the Mesh
telemetry and Mesh security features. You need to download the
istio-operator.yaml
and other resource configuration files and set the
project and cluster information.
To prepare the resource configuration files:
If you haven't already, install
kpt
:gcloud components install kpt
Optionally, create a new directory for the Anthos Service Mesh package resource configuration files. If you plan to set up more than one cluster, you might want to use the cluster name as the directory name.
Change to the directory where you want to download the Anthos Service Mesh package.
Download the Anthos Service Mesh package to the current working directory:
kpt pkg get \ https://github.com/GoogleCloudPlatform/anthos-service-mesh-packages.git/asm@release-1.5-asm .
Set the cluster name:
kpt cfg set asm gcloud.container.cluster ${CLUSTER_NAME}
Optionally, customize the resource configuration files by using the
kpt
setters. By default, these setters use the defaults forgcloud config
. If you set thegcloud config
defaults, or if you want to change the values, run the following setters:Set the project ID:
kpt cfg set asm gcloud.core.project ${PROJECT_ID}
Set the default zone or region:
kpt cfg set asm gcloud.compute.location ${CLUSTER_LOCATION}
Optionally, you can check in the resource configuration files to your own source control system, such as Cloud Source Repositories, so that you can track changes to the files.
Installing Anthos Service Mesh
Install Anthos Service Mesh and set the options needed to integrate Anthos Service Mesh with IAP.
PERMISSIVE mTLS
istioctl manifest apply --set profile=asm \ -f asm/cluster/istio-operator.yaml \ --set values.gateways.istio-ingressgateway.type=NodePort
STRICT mTLS
istioctl manifest apply --set profile=asm \ -f asm/cluster/istio-operator.yaml \ --set values.gateways.istio-ingressgateway.type=NodePort \ --set values.global.mtls.enabled=true
You specify NodePort
for the istio-ingressgateway
, which configures
{[mesh_name]} to open a specific port on the service mesh. This allows you to
set up a load balancer, which routes traffic sent to your domain name to this
port. The other options enable Anthos Service Mesh certificate authority (Mesh CA).
Validating the installation
We recommend that you use the asmctl
analysis tool to validate the basic
configuration of your project, cluster, and workloads. If an asmctl
test
fails, asmctl
recommends solutions, if possible. The asmctl validate
command runs basic tests that check:
- That the APIs required by Anthos Service Mesh are enabled on the project.
- That the Istio-Ingressgateway is properly configured to call Mesh CA.
- The general health of Istiod and Istio-Ingressgateway.
If you run the asmctl validate
command with the optional
--with-testing-workloads
flag, in addition to the basic tests, asmctl
runs security tests that check:
- Mutual TLS (mTLS) communication is configured properly.
- Mesh CA can issue certificates.
To run the security tests, asmctl
deploys workloads on your cluster in a
test namespace, runs the mTLS communication tests, outputs the results, and
deletes the test namespace.
To run asmctl
:
Ensure that gcloud application-default credentials are set:
gcloud auth application-default login
If you haven't already, get authentication credentials to interact with the cluster:
gcloud container clusters get-credentials ${CLUSTER_NAME}
To run both the basic and security tests (assuming
istio-1.5.10-asm.2/bin
) is in yourPATH
):asmctl validate --with-testing-workloads
On success, the command responds with output similar to the following:
[asmctl version 0.3.0] Using Kubernetes context: example-project_us-central1-example-cluster To change the context, use the --context flag Validating enabled APIs OK Validating ingressgateway configuration OK Validating istio system OK Validating sample traffic Launching example services... Sent traffic to example service http code: 200 verified mTLS configuration OK Validating issued certs OK
Deploying a sample application
Before you enable IAP, you need an application running on your GKE cluster so you can verify that all requests have an identity. This guide uses the Bookinfo sample to demonstrate how to setup the HTTP(S) load balancer and enable IAP.
Start the application services
Change directory to the root of the Anthos Service Mesh installation.
Label the
default
namespace to use automatic sidecar injection:kubectl label namespace default istio-injection=enabled
Deploy the application:
kubectl apply -f samples/bookinfo/platform/kube/bookinfo.yaml
Confirm all
bookinfo
services are running:kubectl get services
The expected output is similar to:
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE details 10.0.0.31
9080/TCP 6m kubernetes 10.0.0.1 443/TCP 7d productpage 10.0.0.120 9080/TCP 6m ratings 10.0.0.15 9080/TCP 6m reviews 10.0.0.170 9080/TCP 6m Confirm all Pods are running:
kubectl get pods
The expected output is similar to:
NAME READY STATUS RESTARTS AGE details-v1-1520924117-48z17 2/2 Running 0 6m productpage-v1-560495357-jk1lz 2/2 Running 0 6m ratings-v1-734492171-rnr5l 2/2 Running 0 6m reviews-v1-874083890-f0qf0 2/2 Running 0 6m reviews-v2-1343845940-b34q5 2/2 Running 0 6m reviews-v3-1813607990-8ch52 2/2 Running 0 6m
Confirm that the Bookinfo application is running:
kubectl exec -it $(kubectl get pod -l app=ratings -o jsonpath='{.items[0].metadata.name}') -c ratings -- curl productpage:9080/productpage | grep -o "<title>.*</title>"
Expected output:
<title>Simple Bookstore App</title>
Define the ingress gateway and virtual service for the application:
kubectl apply -f samples/bookinfo/networking/bookinfo-gateway.yaml
Confirm that the gateway was created:
kubectl get gateway
The expected output is similar to:
NAME AGE bookinfo-gateway 32s
External requests
Bookinfo's Gateway resource (defined in
samples/bookinfo/networking/bookinfo-gateway.yaml
) uses the preconfigured
istio-ingressgateway
.
Recall that when you deployed Anthos Service Mesh, you specified NodePort
for the
istio-ingressgateway
, which opens a specific port on the service mesh. Until
you set up the load balancer, the Bookinfo application isn't accessible outside
of your GKE cluster (such as from a browser). Although the nodes
in your cluster have external IP addresses, requests coming from outside your
cluster are blocked by Google Cloud firewall rules. With
IAP, the correct way to expose this application to the public
internet is by using a load balancer. Don't expose the node addresses using
firewall rules, which would bypass IAP.
To route requests to Bookinfo, you set up an HTTP(S) load balancer in your Cloud project. Because the load balancer is in your project, it is inside of the firewall and can access the nodes in your cluster. After you configure the load balancer with the static IP address and your domain name, you can send requests to the domain name, and the load balancer forwards the requests to the nodes in the cluster.
Deploying the load balancer
You can use an Ingress resource to create an HTTP(S) load balancer with automatically configured SSL certificates. Google-managed SSL certificates are provisioned, renewed, and managed for your domain.
Create a ManagedCertificate resource. This resource specifies the domain for the SSL certificate. The
spec.domains
list must contain only one domain. Wildcard domains aren't supported.cat <<EOF | kubectl apply -f - apiVersion: networking.gke.io/v1beta1 kind: ManagedCertificate metadata: name: example-certificate namespace: istio-system spec: domains: - ${DOMAIN_NAME} EOF
Create the load balancer by defining the Ingress resource.
Set the
networking.gke.io/managed-certificates
annotation to the name of the certificate you created in the previous step,example-certificate
.Set the
kubernetes.io/ingress.global-static-ip-name
annotation to the name of the static IP address you reserved,example-static-ip
.Set the
serviceName
toistio-ingressgateway
, which is used in the Gateway resource for the Bookinfo sample.
cat <<EOF | kubectl create -f - apiVersion: extensions/v1beta1 kind: Ingress metadata: name: example-ingress namespace: istio-system annotations: kubernetes.io/ingress.global-static-ip-name: example-static-ip networking.gke.io/managed-certificates: example-certificate spec: backend: serviceName: istio-ingressgateway servicePort: 80 EOF
In the Google Cloud console, go to the Kubernetes Engine > Services & Ingress page.
Go to the Services & Ingress page
You should see the "Creating ingress" message in the Status column. Wait for GKE to fully provision the Ingress before continuing. Refresh the page every few minutes to get the most up-to-date status on the Ingress. After the Ingress is provisioned, you might see the "Ok" status, or the error "All backend services are in UNHEALTHY state." One of the resources that GKE provisions is a default health check. If you see the error message, that indicates that the Ingress is provisioned and that the default health check ran. When you see either the "Ok" status or the error, continue with the next section to configure the health checks for the load balancer.
Configure health checks for the load balancer.
To configure the health checks, you need to obtain the ID of the default health check created by the Ingress and then update the health check to use istio-ingress's health check path and port.
Get new user credentials to use for Application Default Credentials:
gcloud auth application-default login
Obtain the ID of the default health check created by the Ingress:
Set the following environment variables:
Backend Service: Bridges various Instance Groups on a given Service NodePort.
BACKEND_SERVICE=$(gcloud compute url-maps list | grep example-ingress | awk '{print $2}' | cut -d'/' -f 2)
Health check: This is the default health check that is created automatically when the Ingress is deployed.
HC=$(gcloud compute backend-services describe ${BACKEND_SERVICE} --global | grep healthChecks | cut -d'/' -f 10 | tail -n 1)
Health check ingress port: This is the health check port of istio-ingress.
export HC_INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="status-port")].nodePort}')
Health check ingress path: This is the health check path of istio-ingress.
export HC_INGRESS_PATH=$(kubectl -n istio-system get deployments istio-ingressgateway -o jsonpath='{.spec.template.spec.containers[?(@.name=="istio-proxy")].readinessProbe.httpGet.path}')
- Health check API: This is the API that you call to configure the health
check.
export HC_API=https://compute.googleapis.com/compute/v1/projects/${PROJECT_ID}/global/healthChecks/${HC}
Get the default health check into a JSON file by calling the
healthChecks
API:curl --request GET --header "Authorization: Bearer $(gcloud auth application-default print-access-token)" ${HC_API} > health_check.json
Update the health check to use istio-ingress's health check path and port:
Update the
health_check.json
file as follows:- Set
httpHealthCheck.port
to the value of${HC_INGRESS_PORT}
. - Set
httpHealthCheck.requestPath
to the value of${HC_INGRESS_PATH}
. - Add the following attribute and set it to an empty string:
httpHealthCheck.portSpecification=""
The easiest way to do this is to use jq, which comes preinstalled on Cloud Shell:
jq ".httpHealthCheck.port=${HC_INGRESS_PORT} | .httpHealthCheck.requestPath=\"${HC_INGRESS_PATH}\" | .httpHealthCheck.portSpecification=\"\"" health_check.json > updated_health_check.json
If you run
cat
on the resultingupdated_health_check.json
file, it looks similar to the following:{ "id": "5062913090021441698", "creationTimestamp": "2019-11-12T10:47:41.934-08:00", "name": "${HC}", "description": "Default kubernetes L7 Loadbalancing health check.", "checkIntervalSec": 60, "timeoutSec": 60, "unhealthyThreshold": 10, "healthyThreshold": 1, "type": "HTTP", "httpHealthCheck": { "port": 32394, "requestPath": "/healthz/ready", "proxyHeader": "NONE", "portSpecification": "" }, "selfLink": "https://www.googleapis.com/compute/v1/projects/${PROJECT_ID}/global/healthChecks/${HC}", "kind": "compute#healthCheck" }
If you edited the JSON file manually instead of using the
jq
command, save the file asupdated_health_check.json
so that it matches the filename in the next command.- Set
Update the health check:
curl --request PATCH --header "Authorization: Bearer $(gcloud auth application-default print-access-token)" --header "Content-Type: application/json" --data @updated_health_check.json ${HC_API}
It takes several minutes for GKE to update the health check. In the Google Cloud console, refresh the Kubernetes Engine > Services & Ingress page every minute or so until the status for the Ingress changes to "Ok."
Test the load balancer. Point your browser to:
http://YOUR_DOMAIN_NAME/productpage
where
YOUR_DOMAIN_NAME
is the domain name that you configured with the external static IP address.You should see the Bookinfo application's
productpage
. If you refresh the page several times, you should see different versions of reviews, presented in a round robin style: red stars, black stars, no stars.You should also test
https
access to Bookinfo.
Enabling IAP
The following steps describe how to enable IAP.
Configuring consent screen
Check if you already have an existing brand by using the list command. You may only have one brand per project.
gcloud iap oauth-brands list
The following is an example gcloud response, if the brand exists:
name: projects/[PROJECT_NUMBER]/brands/[BRAND_ID] applicationTitle: [APPLICATION_TITLE] supportEmail: [SUPPORT_EMAIL] orgInternalOnly: true
If no brand exists, use the create command:
gcloud iap oauth-brands create --application_title=APPLICATION_TITLE --support_email=SUPPORT_EMAIL
The above fields are required when calling this API:
supportEmail
: The support email displayed on the OAuth consent screen. This email address can either be a user's address or a Google Groups alias. While service accounts also have an email address, they are not actual valid email addresses, and cannot be used when creating a brand. However, a service account can be the owner of a Google Group. Either create a new Google Group or configure an existing group and set the desired service account as an owner of the group.applicationTitle
: The application name displayed on OAuth consent screen.
The response contains the following fields:
name: projects/[PROJECT_NUMBER]/brands/[BRAND_ID] applicationTitle: [APPLICATION_TITLE] supportEmail: [SUPPORT_EMAIL] orgInternalOnly: true
Creating an IAP OAuth Client
Use the create command to create a client. Use the brand
name
from previous step.gcloud iap oauth-clients create projects/PROJECT_NUMBER/brands/BRAND-ID --display_name=NAME
The response contains the following fields:
name: projects/[PROJECT_NUMBER]/brands/[BRAND_NAME]/identityAwareProxyClients/[CLIENT_ID] secret: [CLIENT_SECRET] displayName: [NAME]
Turning on IAP for your service
Use the following command to turn on IAP for your service.
Replace CLIENT_ID
and
CLIENT_SECRET
with your OAuth client ID and client
secret from the client you created previously.
gcloud beta iap web enable \ --oauth2-client-id=CLIENT_ID \ --oauth2-client-secret=CLIENT_SECRET \ --resource-type=backend-services \ --service=${BACKEND_SERVICE}
Configure the IAP access list
Add a user to the access policy for IAP:
gcloud beta iap web add-iam-policy-binding \ --member=user:EMAIL_ADDRESS \ --role=roles/iap.httpsResourceAccessor \ --resource-type=backend-services \ --service=$BACKEND_SERVICE
where EMAIL_ADDRESS
is the user's full email address
such as alice@example.com
.
Enable RCToken support on the service mesh
By default, IAP generates a JSON Web Token (JWT) that is scoped to the OAuth client. For Anthos Service Mesh, you can configure IAP to generate a RequestContextToken (RCToken), which is a JWT but with a configurable audience. RCToken lets you configure the audience of the JWT to an arbitrary string, which can be used in the Anthos Service Mesh policies for fine-grained authorization.
To configure the RCToken:
Create an environment variable for your project number. This is the number that was automatically generated and assigned to your project when you created it. (This isn't the same as the project ID.)
export PROJECT_NUMBER=YOUR_PROJECT_NUMBER
Create an environment variable for the RCToken audience. This can be any string that you want.
export RCTOKEN_AUD="your-rctoken-aud"
Fetch the existing IAP settings
gcloud beta iap settings get --format json \ --project=${PROJECT_NUMBER} --resource-type=compute \ --service=${BACKEND_SERVICE} > iapSettings.json
Update
IapSettings
with the RCToken audience.cat iapSettings.json | jq --arg RCTOKEN_AUD_STR $RCTOKEN_AUD \ '. + {applicationSettings: {csmSettings: {rctokenAud: $RCTOKEN_AUD_STR}}}' \ > updatedIapSettings.json
gcloud beta iap settings set updatedIapSettings.json --format json \ --project=${PROJECT_NUMBER} --resource-type=compute --service=${BACKEND_SERVICE}
Enable RCToken authentication on the Istio ingress gateway.
cat <<EOF | kubectl apply -f - apiVersion: "authentication.istio.io/v1alpha1" kind: "Policy" metadata: name: "ingressgateway" namespace: istio-system spec: targets: - name: "istio-ingressgateway" origins: - jwt: issuer: "https://cloud.google.com/iap" jwksUri: "https://www.gstatic.com/iap/verify/public_key-jwk" audiences: - "$RCTOKEN_AUD" jwt_headers: - "ingress-authorization" trigger_rules: - excluded_paths: - exact: /healthz/ready principalBinding: USE_ORIGIN EOF
Make sure requests to the Bookinfo
productpage
are still successful:http://DOMAIN_NAME/productpage
To test the policy:
Create an
IapSettings
request object, but set therctokenAud
to a different string:echo $(cat <<EOF { "name": "projects/${PROJECT_NUMBER}/iap_web/compute/services/${BACKEND_SERVICE}", "applicationSettings": { "csmSettings": { "rctokenAud": "some-other-arbitrary-string" } } } EOF ) > request.txt
Call the
IapSettings
API to set the RCtoken audience.curl --request PATCH --header "Authorization: Bearer $(gcloud beta auth application-default print-access-token)" ${IAP_SETTINGS_API}
Make a request to the Bookinfo
productpage
and it should fail:http://DOMAIN_NAME/productpage
Cleaning up
After completing this tutorial, remove the following resources to prevent unwanted charges incurring on your account:
Delete the managed certificate:
kubectl delete managedcertificates example-certificate
Delete the Ingress, which deallocates the load balancing resources:
kubectl -n istio-system delete ingress example-ingress
Delete the static IP address:
gcloud compute addresses delete example-static-ip --global
If you do this, be sure to delete the IP address from your domain registrar.
Delete the cluster, which deletes the resources that make up the cluster, such as the compute instances, disks and network resources:
gcloud container clusters delete ${CLUSTER_NAME}