Version 1.13

Strengthen your app's security with Anthos Service Mesh and Anthos Config Management

This tutorial shows you how to improve your cluster and app's security posture. Imagine you are a platform administrator whose organization is managing the apps for their online store with Anthos Service Mesh, a suite of tools that helps you monitor and manage a reliable service mesh. You are responsible for ensuring that your mesh and apps are secure.

You can prevent misconfiguration and automatically validate your Anthos Service Mesh policies by using Anthos Config Management's Policy Controller and Config Sync. Policy Controller enables the enforcement of fully programmable policies for your clusters. Policy Controller also comes with a default library of constraint templates that you can use with the Anthos Service Mesh security bundle to audit the compliance of your mesh security vulnerabilities and best practices. Config Sync continuously reconciles the state of clusters with a central set of Kubernetes declarative configuration files. Using Policy Controller and Config Sync together enables you to continuously enforce constraints on your Anthos Service Mesh policy configurations.

The following diagram shows you an overview of how Anthos Service Mesh, Policy Controller, and Config Sync work together in this tutorial to manage and protect the Online Boutique sample apps that you use in this tutorial:

A diagram showing the architecture that you create for this tutorial

Objectives

  • Create a Google Kubernetes Engine (GKE) cluster and register the cluster to a fleet.
  • Install Policy Controller, Config Sync, and Anthos Service Mesh on a cluster.
  • Deploy the Online Boutique sample apps and an ingress gateway.
  • Leverage the Anthos Service Mesh policy bundle to enforce the following best practices:
    • Ensure that all workloads in the mesh have automatic sidecar injection.
    • Encrypt all traffic in the mesh.
    • Guarantee that all workloads in the mesh have granular access control.

Costs

This tutorial uses the following billable components of Google Cloud:

  • GKE.
  • Anthos. The billing for Anthos includes billing for the Anthos Service Mesh and the Anthos Config Management components.

To generate a cost estimate based on your projected usage, use the pricing calculator. New Google Cloud users might be eligible for a free trial.

When you finish this tutorial, you can avoid continued billing by deleting the resources you created. For more information, see Clean up.

Before you begin

  1. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  2. Make sure that billing is enabled for your Cloud project. Learn how to check if billing is enabled on a project.

Prepare your environment

In this section, you prepare your environment so that you can install Anthos Service Mesh, Policy Controller, and Config Sync:

  1. Open a Cloud Shell session. To open this session, from the upper-right corner of this page, click Activate Cloud Shell and then click Acknowledge. A Cloud Shell session opens inside a frame lower on the page. Complete the following commands in that Cloud Shell session.
  2. Upgrade to the latest version of the Google Cloud CLI:

    gcloud components update
    
  3. To store the files that you create in this tutorial, create a directory using an environment variable:

    WORK_DIR=~/asm-acm-tutorial-dir
    mkdir $WORK_DIR
    
  4. To simplify the remainder of the tutorial, create the following environment variables:

    PROJECT_ID=PROJECT_ID
    gcloud config set project $PROJECT_ID
    CLUSTER=asm-acm-tutorial
    CLUSTER_ZONE=us-east4-a
    PROJECT_NUMBER=$(gcloud projects describe ${PROJECT_ID} --format='get(projectNumber)')
    

    Replace PROJECT_ID with the project ID that you want to use for this tutorial.

    If you're asked to authorize Cloud Shell, click Authorize to complete the operation.

  5. Enable the APIs that you need for this tutorial:

    gcloud services enable \
        container.googleapis.com \
        gkehub.googleapis.com \
        mesh.googleapis.com \
        anthos.googleapis.com
    

    This operation can take over one minute to complete.

Set up a GKE cluster

In this section, you create a GKE cluster and then register it to a fleet. Fleets are a Google Cloud concept for logically organizing clusters and other resources, letting you use and manage multi-cluster capabilities and apply consistent policies across your systems.

The cluster that you create in this section is the cluster that you install Anthos Service Mesh, Policy Controller, and Config Sync on. It's also the cluster where you deploy the Online Boutique sample apps.

To set up your cluster, complete the following steps:

  1. Create a GKE cluster:

    gcloud container clusters create ${CLUSTER} \
        --zone ${CLUSTER_ZONE} \
        --machine-type=e2-standard-4 \
        --num-nodes 4 \
        --workload-pool ${PROJECT_ID}.svc.id.goog \
        --labels mesh_id=proj-${PROJECT_NUMBER}
    

    This operation can take over five minutes to complete.

    The output is similar to the following:

    kubeconfig entry generated for asm-acm-tutorial.
    NAME: asm-acm-tutorial
    LOCATION: us-east4-a
    MASTER_VERSION: 1.21.10-gke.2000
    MASTER_IP: 34.85.159.120
    MACHINE_TYPE: e2-standard-4
    NODE_VERSION: 1.21.10-gke.2000
    NUM_NODES: 4
    STATUS: RUNNING
    
  2. Register your cluster to a fleet:

    gcloud container fleet memberships register ${CLUSTER} \
        --project=${PROJECT_ID} \
        --gke-cluster=${CLUSTER_ZONE}/${CLUSTER} \
        --enable-workload-identity
    

    The output is similar to the following:

    kubeconfig entry generated for asm-acm-tutorial.
    Waiting for membership to be created...done.
    Created a new membership [projects/PROJECT_ID/locations/global/memberships/asm-acm-tutorial] for the cluster [asm-acm-tutorial]
    Generating the Connect Agent manifest...
    Deploying the Connect Agent on cluster [asm-acm-tutorial] in namespace [gke-connect]...
    Deployed the Connect Agent on cluster [asm-acm-tutorial] in namespace [gke-connect].
    Finished registering the cluster [asm-acm-tutorial] with the Fleet.
    

Explore the repository

In the following installation section, you apply an Anthos Config Management manifest acm-config.yaml This manifest configures your cluster to sync from the asm-acm-tutorial folder of the Anthos Config Management sample repository. This folder contains all the configuration files that you need to complete the remainder of the tutorial.

To simplify the experience throughout this tutorial, you use sed commands to update the acm-config.yaml file in order to deploy the manifests required for each step. Updating a single file helps you focus on the concepts and the flow of securing your clusters, mesh, and applications without repeatedly manipulating the files and running git add|commit|push commands.

Before you apply the acm-config.yaml manifest, it's helpful to understand the overall structure and contents of the repository.

To utilize Config Sync's ability to sync to multiple repositories, the asc-acm-tutorial repository contains two top-level folders; root-sync and online-boutique.

The root-sync folder

The root-sync folder is the root repository. This repository is divided into multiple sub-folders and each of these sub-folders contains resources and policies for a different part of this tutorial.

There are two folders that contain resources for setting up the Online Boutique sample app and the Ingress Gateway:

  • init: resources for setting up the RepoSync for the Online Boutique apps.
  • deployments: resources for deploying the Ingress Gateway and the Online Boutique namespaces and apps.

There are three folders that contain resources for deploying the different Anthos Service Mesh policies:

  • enforce-sidecar-injection: resources for deploying policies to enforce the sidecar injection for Namespace and Pod.
  • enforce-strict-mtls: resources for deploying policies to enforce STRICT mTLS for the entire Mesh and for any PeerAuthentication.
  • enforce-authorization-policies: resources for deploying policies to enforce the default deny AuthorizationPolicy for the entire Mesh as well as enforce granular source principals for any AuthorizationPolicies.

The remaining three folders contain resources for deploying the resources fixing the Anthos Service Mesh policies violations:

  • fix-strict-mtls: resources for deploying the default STRICT mTLS PeerAuthentication in the istio-system namespace.
  • fix-default-deny-authorization-policy: resources for deploying the default deny AuthorizationPolicy in the istio-system namespace.
  • deploy-authorization-policies: resources for deploying the granular AuthorizationPolicy resources to make the Online Boutique sample apps work.

The online-boutique folder

The online-boutique folder is a namespace repository. This repository contains the resources that you need to deploy the Online Boutique sample apps using Kustomize.

Install Policy Controller, Config Sync, and managed Anthos Service Mesh

Now that you have created and registered your cluster, you can install Config Sync, Policy Controller, and Anthos Service Mesh on your cluster and configure your cluster to sync from the configs in the asc-acm-tutorial folder:

  1. Enable Anthos Config Management in your fleet:

    gcloud beta container fleet config-management enable
    
  2. Enable Anthos Service Mesh in your fleet.

    gcloud container fleet mesh enable
    
  3. Enable the Anthos Service Mesh automatic control plane management to let Google apply the recommended configuration of managed Anthos Service Mesh:

    gcloud container fleet mesh update \
        --control-plane automatic \
        --memberships ${CLUSTER}
    
  4. To install and configure Config Sync and Policy Controller, create the following Anthos Config Management manifest:

    cat <<EOF > $WORK_DIR/acm-config.yaml
    applySpecVersion: 1
    spec:
      policyController:
        enabled: true
        templateLibraryInstalled: true
        referentialRulesEnabled: true
      configSync:
        enabled: true
        sourceFormat: unstructured
        syncRepo: https://github.com/GoogleCloudPlatform/anthos-config-management-samples
        syncBranch: main
        secretType: none
        policyDir: asm-acm-tutorial/root-sync/init
    EOF
    

    To learn more about the Anthos Config Management config fields, see gcloud apply spec fields.

  5. Apply the file:

    gcloud beta container fleet config-management apply \
        --membership ${CLUSTER} \
        --config $WORK_DIR/acm-config.yaml
    

    After you apply this file, Policy Controller and Config Sync are installed on your cluster. Next, Config Sync begins syncing all the configs in the asm-acm-tutorial folder to your cluster. These configs install and configure the following key components:

    • The RepoSync object that configures the Online Boutique app is synced:

      apiVersion: configsync.gke.io/v1beta1
      kind: RepoSync
      metadata:
        name: repo-sync
      spec:
        sourceFormat: unstructured
        git:
          repo: https://github.com/GoogleCloudPlatform/anthos-config-management-samples
          revision: HEAD
          branch: main
          dir: asm-acm-tutorial/online-boutique/init
          auth: none
    • Since the RepoSync reconciler needs additional permissions to create Istio resources, the repository also contains cluster roles and cluster role bindings to grant these permissions. This is the manifest for the cluster role that is automatically applied to your cluster:

      apiVersion: rbac.authorization.k8s.io/v1
      kind: ClusterRole
      metadata:
        labels:
          rbac.authorization.k8s.io/aggregate-to-edit: "true"
        name: custom:aggregate-to-edit:istio
      rules:
      - apiGroups:
        - "networking.istio.io"
        - "security.istio.io"
        resources:
        - "virtualservices"
        - "authorizationpolicies"
        verbs:
        - "*"
  6. To ensure the successful installation of Policy Controller and Config Sync, view the status of Anthos Config Management:

    gcloud beta container fleet config-management status
    

    The output is similar to the following:

    Name: asm-acm-tutorial
    Status: SYNCED
    Last_Synced_Token: 4b3384d
    Sync_Branch: main
    Last_Synced_Time: 2022-05-04T21:32:58Z
    Policy_Controller: INSTALLED
    Hierarchy_Controller: PENDING
    

    If you see PENDING or NOT_INSTALLED in the Status or Policy_Controller rows, wait a few minutes and run gcloud beta container fleet config-management status again.

  7. To ensure the successful installation of Anthos Service Mesh, describe its status:

    gcloud container fleet mesh describe
    

    The output is similar to the following:

    createTime: '2022-05-05T23:33:44.041608250Z'
    membershipSpecs:
      projects/841604900168/locations/global/memberships/asm-acm-tutorial:
        mesh:
          controlPlane: AUTOMATIC
    membershipStates:
      projects/841604900168/locations/global/memberships/asm-acm-tutorial:
        servicemesh:
          controlPlaneManagement:
            details:
            - code: REVISION_READY
              details: 'Ready: asm-managed'
            state: ACTIVE
        state:
          code: OK
          description: 'Revision(s) ready for use: asm-managed.'
          updateTime: '2022-05-05T23:45:38.800808838Z'
    name: projects/PROJECT_ID/locations/global/features/servicemesh
    resourceState:
      state: ACTIVE
    spec: {}
    state:
      state: {}
    updateTime: '2022-05-05T23:45:44.848011023Z'
    

    If you see state.code: ERROR instead of state.code: OK, wait a few minutes and run gcloud container fleet mesh describe again. Before moving forward with the tutorial, you need to make sure that the servicemesh.controlPlaneManagement.details[].code field has the REVISION_READY value.

Deploy an ingress gateway and a sample application

In this section, you deploy the Online Boutique sample application and an ingress gateway to manage ingress traffic.

  1. Deploy the Online Boutique sample application and the ingress gateway.

    The following command uses sed to update the acm-config.yaml manifest to get Config Sync deploying the resources you need to deploy the ingress gateway and sample app.

    sed -i "s,root-sync/init,root-sync/deployments,g" $WORK_DIR/acm-config.yaml
    gcloud beta container fleet config-management apply \
        --membership ${CLUSTER} \
        --config $WORK_DIR/acm-config.yaml
    

    Note this step can take a few minutes to complete.

  2. View the Config Sync status for the RootSync:

    gcloud alpha anthos config sync repo describe \
        --sync-name root-sync \
        --sync-namespace config-management-system
    

    The output is similar to:

    getting 1 RepoSync and RootSync from projects/project_id/locations/global/memberships/asm-acm-tutorial
    [
      {
        "clusters": [
          "projects/project_id/locations/global/memberships/asm-acm-tutorial"
        ],
        "commit": "7d15d49af13c44aa531a4565b2277ddcf8b81884",
        "errors": [],
        "source": "https://github.com/GoogleCloudPlatform/anthos-config-management-samples//asm-acm-tutorial/root-sync/deployments@main",
        "status": "SYNCED"
      }
    ]
    

    If you see status: RECONCILING instead of status: SYNCED, wait a few minutes and run gcloud alpha anthos config sync repo describe again.

    To also view managed resources, you can add the --managed-resources flag. For more information, see View Config Sync status across multiple clusters.

  3. View the Config Sync status for the RepoSync:

    gcloud alpha anthos config sync repo describe \
        --sync-name repo-sync \
        --sync-namespace onlineboutique
    

    The output is similar to:

    getting 1 RepoSync and RootSync from projects/project_id/locations/global/memberships/asm-acm-tutorial
    [
      {
        "clusters": [
          "projects/project_id/locations/global/memberships/asm-acm-tutorial"
        ],
        "commit": "7d15d49af13c44aa531a4565b2277ddcf8b81884",
        "errors": [],
        "source": "https://github.com/GoogleCloudPlatform/anthos-config-management-samples//online-boutique/deployments@main:HEAD",
        "status": "SYNCED"
      }
    ]
    

    If you see status: RECONCILING instead of status: SYNCED, wait a few minutes and run gcloud alpha anthos config sync repo describe again.

  4. Wait for the ingress gateway's public IP address to be provisioned:

    until kubectl -n asm-ingress get svc asm-ingressgateway -o jsonpath='{.status.loadBalancer}' | grep "ingress"; do : ; done
    
  5. Get the ingress gateway's public IP address:

    EXTERNAL_IP=$(kubectl get svc asm-ingressgateway -n asm-ingress -o jsonpath="{.status.loadBalancer.ingress[*].ip}")
    
  6. Visit the IP address from your browser to verify that the Online Boutique app has successfully deployed:

    echo http://${EXTERNAL_IP}
    

Enforce policies to secure your mesh

In the following sections, you leverage Policy Controller to enforce policies from the Anthos Service Mesh policy bundle by creating constraints.

Enforce sidecar proxies injection

In this section you enforce policies to ensure that all workloads in the mesh have automatic sidecar injection enabled.

  1. To enforce sidecar proxies injection, apply constraints.

    The following command uses sed to update the acm-config.yaml file to get Config Sync deploying the associated resources.

    sed -i "s,root-sync/deployments,root-sync/enforce-sidecar-injection,g" $WORK_DIR/acm-config.yaml
    gcloud beta container fleet config-management apply \
        --membership ${CLUSTER} \
        --config $WORK_DIR/acm-config.yaml
    

    The preceding command applies the following resources:

  2. View the Config Sync status for the RootSync:

    gcloud alpha anthos config sync repo describe \
        --sync-name root-sync \
        --sync-namespace config-management-system
    

    The output is similar to:

    getting 1 RepoSync and RootSync from projects/project_id/locations/global/memberships/asm-acm-tutorial
    [
      {
        "clusters": [
          "projects/project_id/locations/global/memberships/asm-acm-tutorial"
        ],
        "commit": "7d15d49af13c44aa531a4565b2277ddcf8b81884",
        "errors": [],
        "source": "https://github.com/GoogleCloudPlatform/anthos-config-management-samples//asm-acm-tutorial/root-sync/enforce-sidecar-injection@main",
        "status": "SYNCED"
      }
    ]
    

    If you see status: RECONCILING instead of status: SYNCED, wait a few minutes and run gcloud alpha anthos config sync repo describe again.

  3. Verify the Constraints are created:

    kubectl get constraints
    

    It can take a few minutes for Policy Controller to evaluate these constraints. If you don't see values in the TOTAL-VIOLATIONS column, wait and run kubectl get constraints again.

    The output is similar to:

    NAME                                                                                       ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    podsidecarinjectionannotation.constraints.gatekeeper.sh/pod-sidecar-injection-annotation   deny                 0
    
    NAME                                                                            ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    k8srequiredlabels.constraints.gatekeeper.sh/namespace-sidecar-injection-label   deny                 0
    

    Because we properly set up our Namespaces and Pods, there are 0 TOTAL-VIOLATIONS for these Constraints.

  4. To see these Constraints at work, try to create a Namespace in your cluster with neither a label nor an annotation:

    kubectl create namespace test
    

    The output is similar is the following error:

    Error from server (Forbidden): admission webhook "validation.gatekeeper.sh" denied the request: [namespace-sidecar-injection-label] you must provide labels: {"istio-injection"}
    

Enforce traffic encryption

In this section you enforce policies to ensure that all traffic in the mesh is encrypted.

  1. To enforce traffic encryption, apply constraints.

    The following command uses sed to update the acm-config.yaml file to get Config Sync deploying the associated resources.

    sed -i "s,root-sync/enforce-sidecar-injection,root-sync/enforce-strict-mtls,g" $WORK_DIR/acm-config.yaml
    gcloud beta container fleet config-management apply \
        --membership ${CLUSTER} \
        --config $WORK_DIR/acm-config.yaml
    

    The preceding command applies the following resources:

  2. View the Config Sync status for the RootSync:

    gcloud alpha anthos config sync repo describe \
        --sync-name root-sync \
        --sync-namespace config-management-system
    

    The output is similar to:

    getting 1 RepoSync and RootSync from projects/project_id/locations/global/memberships/asm-acm-tutorial
    [
      {
        "clusters": [
          "projects/project_id/locations/global/memberships/asm-acm-tutorial"
        ],
        "commit": "7d15d49af13c44aa531a4565b2277ddcf8b81884",
        "errors": [],
        "source": "https://github.com/GoogleCloudPlatform/anthos-config-management-samples//asm-acm-tutorial/root-sync/enforce-strict-mtls@main",
        "status": "SYNCED"
      }
    ]
    

    If you see status: RECONCILING instead of status: SYNCED, wait a few minutes and run gcloud alpha anthos config sync repo describe again.

  3. Run the following command to get more information about the PeerAuthentication violation:

    kubectl get asmpeerauthnmeshstrictmtls.constraints.gatekeeper.sh/mesh-level-strict-mtls -ojsonpath='{.status.violations}'  | jq
    

    The output is similar to:

    [
      {
        "enforcementAction": "deny",
        "kind": "AsmPeerAuthnMeshStrictMtls",
        "message": "Root namespace <istio-system> does not have a strict mTLS PeerAuthentication",
        "name": "mesh-level-strict-mtls"
      }
    ]
    
  4. Fix the issue by deploying a PeerAuthentication in the istio-system. To prevent all your services in the mesh from accepting plaintext traffic, set a mesh-wide PeerAuthentication policy with the mTLS mode set to STRICT. When you deploy the policy, the control plane automatically provisions TLS certificates so that workloads can authenticate with each other.

    The following command uses sed to update the acm-config.yaml file to get Config Sync deploying the associated resources.

    sed -i "s,root-sync/enforce-strict-mtls,root-sync/fix-strict-mtls,g" $WORK_DIR/acm-config.yaml
    gcloud beta container hub config-management apply \
        --membership ${CLUSTER} \
        --config $WORK_DIR/acm-config.yaml
    

    The preceding command applies the following STRICT mTLS PeerAuthentication in the istio-system namespace. This applies mTLS STRICT to the entire mesh:

    apiVersion: security.istio.io/v1beta1
    kind: PeerAuthentication
    metadata:
      name: default
    spec:
      mtls:
        mode: STRICT
  5. View the Config Sync status for the RootSync:

    gcloud alpha anthos config sync repo describe \
        --sync-name root-sync \
        --sync-namespace config-management-system
    

    The output is similar to:

    getting 1 RepoSync and RootSync from projects/project_id/locations/global/memberships/asm-acm-tutorial
    [
      {
        "clusters": [
          "projects/project_id/locations/global/memberships/asm-acm-tutorial"
        ],
        "commit": "7d15d49af13c44aa531a4565b2277ddcf8b81884",
        "errors": [],
        "source": "https://github.com/GoogleCloudPlatform/anthos-config-management-samples//asm-acm-tutorial/root-sync/fix-strict-mtls@main",
        "status": "SYNCED"
      }
    ]
    

    If you see status: RECONCILING instead of status: SYNCED, wait a few minutes and run gcloud alpha anthos config sync repo describe again.

  6. Verify the Constraints are created:

    kubectl get constraints
    

    Note this can take a few minutes to get Policy Controller evaluating these Constraints. Wait and run again this kubectl get constraints command until you get values under the TOTAL-VIOLATIONS column for each line.

    The output is similar to:

    NAME                                                                            ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    k8srequiredlabels.constraints.gatekeeper.sh/namespace-sidecar-injection-label   deny                 0
    NAME                                                                          ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    asmpeerauthnmeshstrictmtls.constraints.gatekeeper.sh/mesh-level-strict-mtls   deny                 0
    NAME                                                                               ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    destinationruletlsenabled.constraints.gatekeeper.sh/destination-rule-tls-enabled   deny                 0
    NAME                                                                              ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    asmpeerauthnstrictmtls.constraints.gatekeeper.sh/peerauthentication-strict-mtls   deny                 0
    NAME                                                                             ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    asmsidecarinjection.constraints.gatekeeper.sh/pod-sidecar-injection-annotation   deny                 0
    

Enforce granular access control

In this section you enforce policies to ensure that all workloads in the mesh have granular access control.

  1. To enforce granular access control, apply constraints.

    The following command uses sed to update the acm-config.yaml file to get Config Sync deploying the associated resources.

    sed -i "s,root-sync/fix-strict-mtls,root-sync/enforce-authorization-policies,g" $WORK_DIR/acm-config.yaml
    gcloud beta container fleet config-management apply \
        --membership ${CLUSTER} \
        --config $WORK_DIR/acm-config.yaml
    

    The preceding command applies the following resources:

  2. View the Config Sync status for the RootSync:

    gcloud alpha anthos config sync repo describe \
        --sync-name root-sync \
        --sync-namespace config-management-system
    

    The output is similar to:

    getting 1 RepoSync and RootSync from projects/project_id/locations/global/memberships/asm-acm-tutorial
    [
      {
        "clusters": [
          "projects/project_id/locations/global/memberships/asm-acm-tutorial"
        ],
        "commit": "7d15d49af13c44aa531a4565b2277ddcf8b81884",
        "errors": [],
        "source": "https://github.com/GoogleCloudPlatform/anthos-config-management-samples//asm-acm-tutorial/root-sync/enforce-authorization-policies@main",
        "status": "SYNCED"
      }
    ]
    

    If you see status: RECONCILING instead of status: SYNCED, wait a few minutes and run gcloud alpha anthos config sync repo describe again.

  3. Run the following command to get more information about the associated violation:

    kubectl get asmauthzpolicydefaultdeny.constraints.gatekeeper.sh/default-deny-authorization-policies -ojsonpath='{.status.violations}'  | jq
    

    The output is similar to:

    [
      {
        "enforcementAction": "deny",
        "kind": "AsmAuthzPolicyDefaultDeny",
        "message": "Root namespace <istio-system> does not have a default deny AuthorizationPolicy",
        "name": "default-deny-authorization-policies"
      }
    ]
    
  4. Fix the issue by deploying the AuthorizationPolicy in the istio-system namespace.

    The following command uses sed to update the acm-config.yaml file to get Config Sync deploying the associated resources.

    sed -i "s,root-sync/enforce-authorization-policies,root-sync/fix-default-deny-authorization-policy,g" $WORK_DIR/acm-config.yaml
    gcloud beta container hub config-management apply \
        --membership ${CLUSTER} \
        --config $WORK_DIR/acm-config.yaml
    

    The preceding command applies the following deny-all AuthorizationPolicy in the istio-system namespace:

    apiVersion: security.istio.io/v1beta1
    kind: AuthorizationPolicy
    metadata:
      name: deny-all
    spec:
      {}
  5. View the Config Sync status for the RootSync:

    gcloud alpha anthos config sync repo describe \
        --sync-name root-sync \
        --sync-namespace config-management-system
    

    The output is similar to:

    getting 1 RepoSync and RootSync from projects/project_id/locations/global/memberships/asm-acm-tutorial
    [
      {
        "clusters": [
          "projects/project_id/locations/global/memberships/asm-acm-tutorial"
        ],
        "commit": "7d15d49af13c44aa531a4565b2277ddcf8b81884",
        "errors": [],
        "source": "https://github.com/GoogleCloudPlatform/anthos-config-management-samples//asm-acm-tutorial/root-sync/fix-default-deny-authorization-policy@main",
        "status": "SYNCED"
      }
    ]
    

    If you see status: RECONCILING instead of status: SYNCED, wait a few minutes and run gcloud alpha anthos config sync repo describe again.

  6. Verify the Constraints are created:

    kubectl get constraints
    

    Note this can take a few minutes to get Policy Controller evaluating these Constraints. Wait and run again this kubectl get constraints command until you get values under the TOTAL-VIOLATIONS column for each line.

    The output is similar to:

    NAME                                                                             ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    asmsidecarinjection.constraints.gatekeeper.sh/pod-sidecar-injection-annotation   deny                 0
    NAME                                                                            ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    k8srequiredlabels.constraints.gatekeeper.sh/namespace-sidecar-injection-label   deny                 0
    NAME                                                                                      ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    asmauthzpolicydefaultdeny.constraints.gatekeeper.sh/default-deny-authorization-policies   deny                 0
    NAME                                                                          ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    asmpeerauthnmeshstrictmtls.constraints.gatekeeper.sh/mesh-level-strict-mtls   deny                 0
    NAME                                                                               ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    destinationruletlsenabled.constraints.gatekeeper.sh/destination-rule-tls-enabled   deny                 0
    NAME                                                                              ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    asmpeerauthnstrictmtls.constraints.gatekeeper.sh/peerauthentication-strict-mtls   deny                 0
    NAME                                                                                              ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    asmauthzpolicyenforcesourceprincipals.constraints.gatekeeper.sh/authz-source-principals-not-all   deny                 0
    
  7. Visit the Online Boutique app from your browser:

    echo http://${EXTERNAL_IP}
    

    You should receive the error: RBAC: access denied which confirms that the default deny AuthorizationPolicy applies to the entire mesh.

  8. Fix this issue by deploying more granular AuthorizationPolicies in the asm-ingress and onlineboutique namespaces.

    The following command uses sed to update the acm-config.yaml file to get Config Sync deploying the associated resources.

    sed -i "s,root-sync/fix-default-deny-authorization-policy,root-sync/deploy-authorization-policies,g" $WORK_DIR/acm-config.yaml
    gcloud beta container hub config-management apply \
        --membership ${CLUSTER} \
        --config $WORK_DIR/acm-config.yaml
    

    The preceding command applies the following resources:

  9. View the Config Sync status for the RootSync:

    gcloud alpha anthos config sync repo describe \
        --sync-name root-sync \
        --sync-namespace config-management-system
    

    The output is similar to:

    getting 1 RepoSync and RootSync from projects/project_id/locations/global/memberships/asm-acm-tutorial
    [
      {
        "clusters": [
          "projects/project_id/locations/global/memberships/asm-acm-tutorial"
        ],
        "commit": "7d15d49af13c44aa531a4565b2277ddcf8b81884",
        "errors": [],
        "source": "https://github.com/GoogleCloudPlatform/anthos-config-management-samples//asm-acm-tutorial/root-sync/fix-default-deny-authorization-policy@main",
        "status": "SYNCED"
      }
    ]
    

    If you see status: RECONCILING instead of status: SYNCED, wait a few minutes and run gcloud alpha anthos config sync repo describe again.

  10. View the Config Sync status for the RepoSync:

    gcloud alpha anthos config sync repo describe \
        --sync-name repo-sync \
        --sync-namespace onlineboutique
    

    The output is similar to:

    getting 1 RepoSync and RootSync from projects/project_id/locations/global/memberships/asm-acm-tutorial
    [
      {
        "clusters": [
          "projects/project_id/locations/global/memberships/asm-acm-tutorial"
        ],
        "commit": "7d15d49af13c44aa531a4565b2277ddcf8b81884",
        "errors": [],
        "source": "https://github.com/GoogleCloudPlatform/anthos-config-management-samples//asm-acm-tutorial/online-boutique/authorization-policies@main",
        "status": "SYNCED"
      }
    ]
    

    If you see status: RECONCILING instead of status: SYNCED, wait a few minutes and run gcloud alpha anthos config sync repo describe again.

  11. Visit the Online Boutique app again from your browser:

    echo http://${EXTERNAL_IP}
    

    If you wait a few minutes, you should now see the website working successfully again as expected.

View the status of Anthos security features

You can view the status of Anthos security features, including authentication and authorization policies, in the Google Cloud console.

  1. In the Google Cloud console, go to the Anthos Security page.

    Go to Anthos Security

    The Policy Summary displays the status of application security, including Service access control (AuthorizationPolicies) and mTLS.

  2. Click Policy Audit to view workload policy statuses for the cluster and both namespaces (asm-ingress and onlineboutique).

    The Service access control and mTLS status cards provide a high-level overview.

    High-level overview of service access control and mTLS status

    The Workloads list shows the Service access control and mTLS status of each workload.

    Detailed list of each workload and its service access control and mTLS status

You now have secured your cluster and your mesh with Policy Controller and Config Sync.

Clean up

To avoid incurring charges to your Google Cloud account for the resources used in this tutorial, either delete the project that contains the resources, or keep the project and delete the individual resources.

Delete the project

  1. In the Google Cloud console, go to the Manage resources page.

    Go to Manage resources

  2. In the project list, select the project that you want to delete, and then click Delete.
  3. In the dialog, type the project ID, and then click Shut down to delete the project.

Delete individual resources

To delete the individual resources:

  1. Unregister your cluster from the fleet:

    gcloud container fleet memberships unregister ${CLUSTER} \
        --project=${PROJECT_ID} \
        --gke-cluster=${CLUSTER_ZONE}/${CLUSTER}
    

    The output is similar to the following:

    kubeconfig entry generated for asm-acm-tutorial.
    Waiting for membership to be deleted...done.
    Deleting membership CR in the cluster...done.
    Deleting namespace [gke-connect] in the cluster...done.
    
  2. Delete your cluster:

    gcloud container clusters delete ${CLUSTER} \
        --zone ${CLUSTER_ZONE}
    

    Press y when prompted. This command can take over five minutes to complete.

    The output is similar to the following:

    Deleting cluster asm-acm-tutorial...done.
    Deleted [https://container.googleapis.com/v1/projects/PROJECT_ID/zones/us-east4-a/clusters/asm-acm-tutorial].
    
  3. Delete the files that you created:

    rm -r $WORK_DIR
    

What's next