Anthos Service Mesh by example: mTLS


In Anthos Service Mesh 1.5 and later, auto mutual TLS (auto mTLS) is enabled by default. With auto mTLS, a client sidecar proxy automatically detects if the server has a sidecar. The client sidecar sends mTLS to workloads with sidecars and sends plaintext to workloads without sidecars. Note, however, services accept both plaintext and mTLS traffic. As you inject sidecar proxies to your Pods, we recommend that you also configure your services to only accept mTLS traffic.

With Anthos Service Mesh, you can enforce mTLS, outside of your application code, by applying a single YAML file. Anthos Service Mesh gives you the flexibility to apply an authentication policy to the entire service mesh, to a namespace, or to an individual workload.

mutual mTLS

Costs

In this document, you use the following billable components of Google Cloud:

To generate a cost estimate based on your projected usage, use the pricing calculator. New Google Cloud users might be eligible for a free trial.

When you finish this tutorial, you can avoid ongoing costs by deleting the resources you created. For more information, see Cleaning up.

Before you begin

This tutorial uses the Online Boutique sample application to demonstrate configuring mTLS on a GKE cluster with Anthos Service Mesh installed.

  • If you need to set up a GKE cluster for this tutorial, see the Anthos Service Mesh quickstart, which walks you through setting up a cluster, installing Anthos Service Mesh, and deploying the Online Boutique sample to the demo namespace.

  • If you have a GKE cluster with Anthos Service Mesh installed, but you need the sample, see Online Boutique, which walks you through deploying the sample to the demo namespace.

Access Online Boutique

  1. Set the current context for kubectl to the cluster where you deployed Online Boutique:

    gcloud container clusters get-credentials CLUSTER_NAME  \
        --project=PROJECT_ID \
        --zone=CLUSTER_LOCATION 
    
  2. Get a list of the Online Boutique services:

    kubectl get services -n demo
    

    Notice that frontend-external is a LoadBalancer, and it has an external IP address. The sample application includes a service that is a load balancer so that it can be deployed on GKE without Anthos Service Mesh.

  3. Visit the application in your browser using the external IP address of the frontend-external service:

    http://FRONTEND_EXTERNAL_IP/
    
  4. Anthos Service Mesh provides a default ingress gateway called the istio-ingressgateway. You can also access the Online Boutique using the external IP address of the istio-ingressgateway. Get the external IP of the istio-ingressgateway:

    kubectl get service istio-ingressgateway -n istio-system
    
  5. Open another tab in your browser and visit the application using the external IP address of the istio-ingressgateway:

    http://INGRESS_GATEWAY_EXTERNAL_IP/
    
  6. Run the following command to curl the frontend service with plain HTTP from another Pod in the demo namespace.

    kubectl exec \
      $(kubectl get pod -l app=productcatalogservice -n demo -o jsonpath={.items..metadata.name}) \
      -c istio-proxy -n demo -- \
      curl http://frontend:80/ -o /dev/null -s -w '%{http_code}\n'
    

    Your request succeeds with status 200, because by default, both TLS and plaintext traffic are accepted.

Enable mutual TLS per namespace

You enforce mTLS by applying a PeerAuthentication policy with kubectl.

  1. Apply the following authentication policy to configure all services in the demo namespace to only accept mTLS:

    kubectl apply -f - <<EOF
    apiVersion: "security.istio.io/v1beta1"
    kind: "PeerAuthentication"
    metadata:
      name: "namespace-policy"
      namespace: "demo"
    spec:
      mtls:
        mode: STRICT
    EOF
    

    Expected output:

    peerauthentication.security.istio.io/namespace-policy created

    The line mode: STRICT in the YAML configures the services to only accept mTLS. By default, the mode is PERMISSIVE, which configures services to accept both plaintext and mTLS.

  2. Go to the tab in your browser that accesses the Online Boutique using the external IP address of the frontend-external service, and refresh the page. The browser displays the following error:

    site can't be reached

    Refreshing the page causes plaintext to be sent to the frontend service. Because of the STRICTauthentication policy, the sidecar proxy blocks the request to the service.

  3. Go to the tab in your browser that accesses the Online Boutique using the external IP address of the istio-ingressgateway, and refresh the page, which displays successfully. When you access Online Boutique using the istio-ingressgateway, the request takes the following path:

    mutual mTLS

    mTLS authentication flow:

    1. The browser sends a plaintext HTTP request to the server.
    2. The istio-ingressgateway proxy container intercepts the request.
    3. The istio-ingressgateway proxy performs a TLS handshake with the server-side proxy (the frontend service in this example). This handshake includes an exchange of certificates. These certs are pre-loaded into the proxy containers by Anthos Service Mesh.
    4. The istio-ingressgateway proxy performs a secure naming check on the server's certificate, verifying that an authorized identity is running the server.
    5. The istio-ingressgateway and server proxies establish a mutual TLS connection, and the server proxy forwards the request to the server application container (the frontend service).
  4. Run the following command to curl the frontend service with plain HTTP from another Pod in the demo namespace.

    kubectl exec \
      $(kubectl get pod -l app=productcatalogservice -n demo -o jsonpath={.items..metadata.name}) \
      -c istio-proxy -n demo -- \
      curl http://frontend:80/ -o /dev/null -s -w '%{http_code}\n'
    

    Your request fails because because all services in the demo namespace are set to STRICT mTLS, and the sidecar proxy blocks the request to the service.

    Expected output:

    000
    command terminated with exit code 56

View mTLS status

You can view the status of Anthos security features, including authentication policies, in the Google Cloud console.

  1. In the Google Cloud console, go to the GKE Enterprise Security page.

    Go to GKE Enterprise Security

  2. Select the Google Cloud project from the drop-down list on the menu bar.

    The Policy Summary displays the status of application security, including mTLS.

  3. Click Policy Audit to view workload policy statuses for each cluster and namespace. The mTLS status card provides an overview. The Workloads list shows the mTLS status of each workload.

    all services strict mtls

Find and delete authentication policies

  1. For a list of all the PeerAuthentication polices in the service mesh:

    kubectl get peerauthentication --all-namespaces
    
  2. Delete the authentication policy:

    kubectl delete peerauthentication -n demo namespace-policy
    

    Expected output:

    peerauthentication.security.istio.io "namespace-policy" deleted
  3. Access the Online Boutique using the external IP address of the frontend-external service, and refresh the page. The page displays as expected.

  4. Run the following command to curl the frontend service with plain HTTP from another Pod in the demo namespace.

    kubectl exec \
      $(kubectl get pod -l app=productcatalogservice -n demo -o jsonpath={.items..metadata.name}) \
      -c istio-proxy -n demo -- \
      curl http://frontend:80/ -o /dev/null -s -w '%{http_code}\n'
    

    Your request succeeds with status 200, because by default, both TLS and plaintext traffic are accepted.

If you refresh the page in the Google Cloud console that displays the Workloads list, it now shows that the mTLS status is Permissive.

Enable mutual TLS per workload

To set a PeerAuthentication policy for a specific workload, you must configure the selector section and specify the labels that match the desired workload. However, Anthos Service Mesh can't aggregate workload-level policies for outbound mTLS traffic to a service. You need to configure a destination rule to manage that behavior.

  1. Apply an authentication policy to a specific workload in the demo namespace. Notice how the following policy uses labels and selectors to target the specific frontend deployment.

    cat <<EOF | kubectl apply -n demo -f -
    apiVersion: "security.istio.io/v1beta1"
    kind: "PeerAuthentication"
    metadata:
      name: "frontend"
      namespace: "demo"
    spec:
      selector:
        matchLabels:
          app: frontend
      mtls:
        mode: STRICT
    EOF
    

    Expected output:

    peerauthentication.security.istio.io/frontend created
  2. Configure a matching destination rule.

    cat <<EOF | kubectl apply -n demo -f -
    apiVersion: "networking.istio.io/v1alpha3"
    kind: "DestinationRule"
    metadata:
      name: "frontend"
    spec:
      host: "frontend.demo.svc.cluster.local"
      trafficPolicy:
        tls:
          mode: ISTIO_MUTUAL
    EOF
    

    Expected output:

    destinationrule.networking.istio.io/frontend created
  3. Access the Online Boutique using the external IP address of the frontend-external service, and refresh the page. The page doesn't display because because the frontend service is set to STRICT mTLS, and the sidecar proxy blocks the request.

  4. Run the following command to curl the frontend service with plain HTTP from another Pod in the demo namespace.

    kubectl exec \
      $(kubectl get pod -l app=productcatalogservice -n demo -o jsonpath={.items..metadata.name}) \
      -c istio-proxy -n demo -- \
      curl http://frontend:80/ -o /dev/null -s -w '%{http_code}\n'
    

    Your request fails with status code 56.

    If you refresh the page in the Google Cloud console that displays the Workloads list, it now shows that the mTLS status for the frontend service is Strict and all the other services are set to Permissive.

    only frontend service is strict mtls

  5. Delete the authentication policy and destination rule:

    kubectl delete peerauthentication -n demo frontend
    kubectl delete destinationrule -n demo frontend
    

Enforcing mesh-wide mTLS

To prevent all your services in the mesh from accepting plaintext traffic, set a mesh-wide PeerAuthentication policy with the mTLS mode set to STRICT. The mesh-wide PeerAuthentication policy shouldn't have a selector and must be applied in the root namespace, istio-system. When you deploy the policy, the control plane automatically provisions TLS certificates so that workloads can authenticate with each other.

  1. Enforce mesh-wide mTLS:

    kubectl apply -f - <<EOF
    apiVersion: "security.istio.io/v1beta1"
    kind: "PeerAuthentication"
    metadata:
      name: "mesh-wide"
      namespace: "istio-system"
    spec:
      mtls:
        mode: STRICT
    EOF
    

    Expected output:

    peerauthentication.security.istio.io/mesh-wide created

  2. Access the Online Boutique using the external IP address of the frontend-external service, and refresh the page. The page doesn't display.

  3. Run the following command to curl the frontend service with plain HTTP from another Pod in the demo namespace.

    kubectl exec \
      $(kubectl get pod -l app=productcatalogservice -n demo -o jsonpath={.items..metadata.name}) \
      -c istio-proxy -n demo -- \
      curl http://frontend:80/ -o /dev/null -s -w '%{http_code}\n'
    

    Your request fails with status code 56.

  4. Delete the mesh-wide policy:

    kubectl delete peerauthentication -n istio-system mesh-wide
    

If you refresh the page in the Google Cloud console, you see that that the mTLS details for all services now display Permissive.

Clean up

To avoid incurring charges to your Google Cloud account for the resources used in this tutorial, either delete the project that contains the resources, or keep the project and delete the individual resources.

  • If you want to prevent additional charges, delete the cluster:

    gcloud container clusters delete  CLUSTER_NAME  \
        --project=PROJECT_ID \
        --zone=CLUSTER_LOCATION 
    
  • If you want to keep your cluster and remove the Online Boutique sample:

    kubectl delete namespaces demo
    

What's next