Configuring audit policies for your services
An audit policy lets you audit data access to your services in Cloud Service Mesh. Auditing your services helps you answer the questions "who did what, when, and possibly why." With an audit policy, you can specify when an audit log is created and the content of the logs. This guide explains how to install Cloud Service Mesh so that you can use an audit policy.
Because you view the audit logs in the Cloud Logging Logs Explorer in the Google Cloud console, audit policies are supported only on the following platforms:
- GKE on Google Cloud
- Google Distributed Cloud
- Google Distributed Cloud
An audit policy extends
AuthorizationPolicy
by adding an AUDIT
action. It takes effect only in its target policy scope
(which can be a workload, a namespace, or the entire mesh). The policies are
ORed
together, that is, a request is logged if any policy says so. If no audit
policy applies to a given workload, no audit log is generated for that workload.
Here is an example audit policy to audit all WRITE access to the
/user/profile/*
path in myapi
:
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
namespace: ns1
name: anyname
spec:
selector:
matchLabels:
app: myapi
action: AUDIT
rules:
- to:
- operation:
methods: ["POST", "UPDATE", "DELETE"]
paths: ["/user/profile/*"]
Limitations
- There is no audit log on the ingress-gateway.
- The audit content isn't configurable.
- Currently, the Cloud Service Mesh audit logs have the same reliability property as normal access logs. For example, if a workload Pod is restarted, some audit logs for the workload, if not persisted, can be lost.
Before you begin
Follow the steps in Install dependent tools and validate cluster to:- Install required tools
- Download
asmcli
- Grant cluster admin permissions
- Validate your project and cluster
Prepare gateway configuration
Cloud Service Mesh gives you the option to deploy and manage gateways as part of your service mesh. A gateway describes a load balancer operating at the edge of the mesh receiving incoming or outgoing HTTP/TCP connections. Gateways are Envoy proxies that provide you with fine-grained control over traffic entering and leaving the mesh.
asmcli
doesn't install the istio-ingressgateway
. We recommend that you
deploy and manage the control plane and gateways separately. For more
information, see Installing and upgrading gateways.
Customizing the Anthos Service Mesh installation
To use an audit policy, customize the Cloud Service Mesh installation:
Installs
Follow the steps in Install Cloud Service Mesh. When you run
asmcli install
, include the following option:--option audit-authorizationpolicy
For example:
./asmcli install \ --project_id PROJECT_ID \ --cluster_name CLUSTER_NAME \ --cluster_location CLUSTER_LOCATION \ --ca mesh_ca \ --output_dir DIR_PATH \ --enable_all \ --option audit-authorizationpolicy
Be sure to specify any other overlay files that you need to configure Cloud Service Mesh.
Complete the Cloud Service Mesh installation to enable automatic sidecar proxy injection on your workloads. See Deploy and redeploy workloads.
Upgrades
Follow the steps in Upgrade Cloud Service Mesh. When you run
asmcli install
, include the following option:--option audit-authorizationpolicy
For example:
./asmcli install \ --project_id PROJECT_ID \ --cluster_name CLUSTER_NAME \ --cluster_location CLUSTER_LOCATION \ --ca mesh_ca \ --output_dir DIR_PATH \ --enable_all \ --option audit-authorizationpolicy
Be sure to specify any other overlay files that you need to configure Cloud Service Mesh.
Complete the Cloud Service Mesh installation to enable automatic sidecar proxy injection on your workloads. For details see Switch to the new control plane
Using audit logging
This section uses the Bookinfo sample to demonstrate how to use audit logging.
Deploy the Bookinfo sample application to the default namespace.
Get the external IP address of the ingress gateway, and send requests to the sample application to generate some traffic.
In the Google Cloud console, go to the navigation menu
and select Logging > Logs Explorer:Select a Google Cloud project.
Because you haven't deployed an audit policy yet, there won't be any audit logs. Note that the audit log is different from the access log. To see
stackdriver
access logs, enter the following query in the Query builder field, and click Run query:logName="projects/PROJECT_ID/logs/server-accesslog-stackdriver"
For more information on using Logs Explorer, see Logs Explorer overview.
Configuring the audit policy and checking audit logs
This section provides several options for auditing the Bookinfo application. After you deploy the audit policy, you can send some requests and then check the audit log in the Logs Explorer.
Enter the following command to get authentication credentials to interact with the cluster. This command also sets the current context for
kubectl
to the cluster.gcloud container clusters get-credentials CLUSTER_NAME \ --project=PROJECT_ID \ --zone=CLUSTER_LOCATION
Apply the following audit policy to audit
GET
requests to the/productpage
path:kubectl apply -f - << EOF apiVersion: "security.istio.io/v1beta1" kind: "AuthorizationPolicy" metadata: name: "audit-productpage" namespace: default spec: action: AUDIT rules: - to: - operation: methods: ["GET"] paths: ["/productpage"] EOF
Send some requests to Bookinfo.
In Logs Explorer, enter the following query in the Query builder field, and click Run query:
logName="projects/PROJECT_ID/logs/server-istio-audit-log"
The query returns logs similar to the following:
Apply the following policy to audit requests to the
bookinfo-ratings
service. The audit policies are additive. After applying the following policy, you see audit logs for requests to both ProductPage and Ratings.kubectl apply -f - << EOF apiVersion: "security.istio.io/v1beta1" kind: "AuthorizationPolicy" metadata: name: "audit-ratings" namespace: default spec: action: AUDIT rules: - from: - source: principals: ["cluster.local/ns/default/sa/bookinfo-ratings"] to: - operation: methods: ["GET"] EOF
The new audit policy has to propagate first before it takes effect.
Send 10 or more requests to Bookinfo to make sure you hit the ratings service, and then check the audit log in Logs Explorer. The audit log looks similar to the following:
Apply the following policy to audit to all services in the default namespace.
kubectl apply -f - << EOF apiVersion: security.istio.io/v1beta1 kind: AuthorizationPolicy metadata: namespace: default name: "audit-all" spec: action: AUDIT rules: - {} EOF
Send some more requests to Bookinfo, and then check the audit log in Logs Explorer. The audit log now logs all requests:
If you want to restrict the audit policy back down to ProductPage and Ratings, you can delete the
audit-all
policy:kubectl delete authorizationpolicy audit-all -n default
Troubleshooting
If you don't see any audit logs after enabling an audit policy, here are a few things you can check:
Make sure there is traffic for the time period specified in Logs Explorer. If you are testing with Bookinfo, you can send requests by running the following command several times:
curl -s http://EXTERNAL_IP/productpage | grep Bookstore
Check if there is an
AuthorizationPolicy
on the ingress gateway that blocks requests to the audited service.Check the
stackdriver
access logs with the following filter in the Logs Explorer to verify if your requests have reached the application:logName="projects/PROJECT_ID/logs/server-accesslog-stackdriver"
To make sure Stackdriver is configured and audit log is enabled, dump the configuration of current the
istiod
state. In theconfig_dump
search forenable_audit_log
and your audit policy's name.istioctl dashboard envoy POD_NAME.NAMESPACE
To make sure your requests are matched with audit policy rules, you can check role-based access control (RBAC) debug logs. Turn on RBAC debug logging with the following command:
kubectl exec POD_NAME -n NAMESPACE -c istio-proxy -- pilot-agent request POST 'logging?rbac=debug'
Send some requests, and then check logs for the Pod with the
kubectl logs
command:kubectl logs POD_NAME -n NAMESPACE -c istio-proxy