Viewing audit logs

This page explains how to view information about deployment status and policy enforcement in Stackdriver Logging.

Overview

When you deploy a container image to Google Kubernetes Engine (GKE) where there is a policy enabled on a cluster, GKE writes details about the deployment to the audit logs in Stackdriver. You can view this information in the Google Cloud Platform Console or at the command line using the gcloud logging read command.

Enforcement status messages

GKE writes messages to the audit log for the following enforcement conditions:

  • Allowed deployment of conformant images
  • Deployment blocked by evaluation mode
  • Deployment blocked by required attestor
  • Allowed deployment of non-conformant images (dryrun mode)
  • Allowed deployment of non-conformant images (break-glass)

Allowed deployment of conformant images

When an image passes the required checks defined in a policy, GKE writes the following to the audit log for the Kubernetes Cluster (k8s_cluster) resource:

jsonPayload.message:"Created pod: POD_NAME"

where POD_NAME is the name of the pod that was created by the deployment.

For example:

{
 insertId: "skv4osf5rrm6a"
 jsonPayload: {
  apiVersion: "v1"
  involvedObject: {...}
  kind: "Event"
  message: "Created pod: hello-server-7695d664dd-knszq"
  metadata: {...}
  reason: "SuccessfulCreate"
  source: {...}
  type:  "Normal"
 }
 logName: "projects/example-project/logs/events"
 receiveTimestamp: "2019-03-15T19:25:56.913756510Z"
 resource: {...}
 severity: "INFO"
 timestamp: "2019-03-15T19:25:52Z"
}

Deployment blocked by evaluation mode

When an image is blocked because the evaluation mode in the policy is Deny All Images, GKE writes the following to the audit log for the Kubernetes Cluster (k8s_cluster) resource:

protoPayload.response.code:403
protoPayload.response.message:pods 'POD_NAME' is forbidden: image policy webhook backend denied one
or more images: Denied by default admission rule. Overridden by evaluation mode

where POD_NAME is the name of the pod that was created by the deployment.

For example:

{
 insertId:  "0330a9d2-4e0f-4554-a3a8-cf7b3ccc5b9e"
 labels: {...}
 logName: "projects/example-project/logs/cloudaudit.googleapis.com%2Factivity"
 operation: {...}
 protoPayload: {
  @type: "type.googleapis.com/google.cloud.audit.AuditLog"
  authenticationInfo: {...}
  authorizationInfo: [1]
  methodName: "io.k8s.core.v1.pods.create"
  request: {...}
  requestMetadata: {...}
  resourceName: "core/v1/namespaces/default/pods"
  response: {
   @type: "core.k8s.io/v1.Status"
   apiVersion: "v1"
   code: 403
   details: {...}
   kind: "Status"
   message: "pods "hello-server-6f5bdf948c-hx6vb" is forbidden: image policy webhook backend denied one or more images: Denied by default admission rule. Overridden by evaluation mode"
   metadata: {...}
   reason:  "Forbidden"
   status:  "Failure"
  }
  serviceName:  "k8s.io"
  status: {...}
 }
 receiveTimestamp:  "2019-03-15T11:14:39.205161199Z"
 resource: {...}
 timestamp:  "2019-03-15T11:14:05.173585Z"
}

Deployment blocked by required attestor

When an image is blocked because it hasn't been verified by a required attestor, GKE writes the following to the audit log for the Kubernetes Cluster (k8s_cluster) resource:

protoPayload.response.code: 403
protoPayload.response.message:pods 'POD_NAME' is forbidden: image policy webhook backend denied one
or more images: Denied by default admission rule. Denied by Attestor. Image
REGISTRY_PATH denied by ATTESTOR

where

  • POD_NAME is the name of the pod that was created by the deployment
  • REGISTRY_PATH is the fully-qualified by to the image in your container image registry
  • ATTESTOR is the full-qualified path to the attestor in Binary Authorization in the format projects/PROJECT_ID/attestors/ATTESTOR_NAME

For example:

{
 insertId: "a48b7804-4b80-4b68-afb6-6d36d829b903"
 labels: {...}
 logName:  "projects/example-project/logs/cloudaudit.googleapis.com%2Factivity"
 operation: {...}
 protoPayload: {
  @type:  "type.googleapis.com/google.cloud.audit.AuditLog"
  authenticationInfo: {...}
  authorizationInfo: [1]
  methodName:  "io.k8s.core.v1.pods.create"
  request: {...}
  requestMetadata: {...}
  resourceName:  "core/v1/namespaces/default/pods"
  response: {
   @type:  "core.k8s.io/v1.Status"
   apiVersion:  "v1"
   code:  403
   details: {...}
   kind:  "Status"
   message:  "pods "hello-server-6f5bdf948c-8lqf5" is forbidden: image policy webhook backend denied one or more images: Denied by default admission rule. Denied by Attestor. Image gcr.io/google-samples/hello-app:1.0 denied by projects/example-project/attestors/test-attestor: Attestor cannot attest to an image deployed by tag"
   metadata: {...}
   reason:  "Forbidden"
   status:  "Failure"
  }
  serviceName:  "k8s.io"
  status: {...}
 }
 receiveTimestamp:  "2019-03-15T11:20:16.424923323Z"
 resource: {...}
 timestamp:  "2019-03-15T11:20:08.899567Z"
}

Allowed deployment (dryrun mode)

Dryrun mode is an enforcement mode in a policy that allows non-conformant images to be deployed, but writes details about the deployment to the audit log. Dryrun mode allows you to test a policy in your production environment before it goes into effect.

When an image fails to pass the required checks in a policy, but is permitted to be deployed by dryrun mode, GKE writes the following to the audit log for the Kubernetes Cluster (k8s_cluster) resource:

imagepolicywebhook.image-policy.k8s.io/dry-run: "true"
imagepolicywebhook.image-policy.k8s.io/overridden-verification-result: "'REGISTRY_PATH': Image REGISTRY_PATH denied by projects/PROJECT_ID/attestors/ATTESTOR: Attestor cannot attest to an image deployed by tag

where:

  • REGISTRY_PATH is the fully-qualified by to the image in your container image registry
  • ATTESTOR is the full-qualified path to the attestor in Binary Authorization in the format projects/PROJECT_ID/attestors/ATTESTOR_NAME

For example:

{
 insertId: "9c29d752-4a82-4ddf-a888-072a2c8ca885"
 labels: {
  authorization.k8s.io/decision: "allow"
  authorization.k8s.io/reason: "RBAC: allowed by ClusterRoleBinding "system:controller:replicaset-controller" of ClusterRole "system:controller:replicaset-controller" to ServiceAccount "replicaset-controller/kube-system""
  imagepolicywebhook.image-policy.k8s.io/dry-run: "true"
  imagepolicywebhook.image-policy.k8s.io/overridden-verification-result: "'gcr.io/google-samples/hello-app:1.0': Image gcr.io/google-samples/hello-app:1.0 denied by projects/example-project/attestors/test-attestor: Attestor cannot attest to an image deployed by tag
'gcr.io/google-samples/hello-app:1.0': Image gcr.io/google-samples/hello-app:1.0 denied by projects/example-project/attestors/attestor: Attestor cannot attest to an image deployed by tag
"
 }
 logName:  "projects/example-project/logs/cloudaudit.googleapis.com%2Factivity"
 operation: {...}
 protoPayload: {...}
 receiveTimestamp:  "2019-03-22T22:25:48.090408823Z"
 resource: {...}
 timestamp:  "2019-03-22T22:25:29.246910Z"
}

Allowed deployment (break-glass)

Binary Authorization supports a feature known as break-glass that lets you override an authorization policy when you deploy a container image.

When an images fails to pass the required checks in a policy, but is permitted to be deployed by a policy override, Binary Authorization writes the following to the audit log for the Kubernetes Cluster (k8s_cluster) resource:

imagepolicywebhook.image-policy.k8s.io/break-glass: "true"
imagepolicywebhook.image-policy.k8s.io/overridden-verification-result: "'REGISTRY_PATH': Image REGISTRY_PATH denied by projects/PROJECT_ID/attestors/ATTESTOR: Attestor cannot attest to an image deployed by tag

where:

  • REGISTRY_PATH is the fully-qualified by to the image in your container image registry
  • ATTESTOR is the full-qualified path to the attestor in Binary Authorization in the format projects/PROJECT_ID/attestors/ATTESTOR_NAME

For example:

{
 insertId: "9c29d752-4a82-4ddf-a888-072a2c8ca885"
 labels: {
  authorization.k8s.io/decision: "allow"
  authorization.k8s.io/reason: "RBAC: allowed by ClusterRoleBinding "system:controller:replicaset-controller" of ClusterRole "system:controller:replicaset-controller" to ServiceAccount "replicaset-controller/kube-system""
  imagepolicywebhook.image-policy.k8s.io/break-glass: "true"
  imagepolicywebhook.image-policy.k8s.io/overridden-verification-result: "'gcr.io/google-samples/hello-app:1.0': Image gcr.io/google-samples/hello-app:1.0 denied by projects/example-project/attestors/test-attestor: Attestor cannot attest to an image deployed by tag
'gcr.io/google-samples/hello-app:1.0': Image gcr.io/google-samples/hello-app:1.0 denied by projects/example-project/attestors/attestor: Attestor cannot attest to an image deployed by tag
"
 }
 logName:  "projects/example-project/logs/cloudaudit.googleapis.com%2Factivity"
 operation: {...}
 protoPayload: {...}
 receiveTimestamp:  "2019-03-22T22:25:48.090408823Z"
 resource: {...}
 timestamp:  "2019-03-22T22:25:29.246910Z"
}

View audit logs in the console

To view the audit logs:

  1. Go to the logs viewer page:

    Go to the Logs Viewer page

  2. Select the type of resource whose logs you want to view (for example, Kubernetes Cluster or GKE Cluster Operations) from the Resources list.

  3. Filter by cluster name, workload name, log level, or other criteria to find the enforcement status message you want to view.

View audit logs from the command line

To view enforcement status messages for a Kubernetes Cluster resource from the command line:

gcloud logging read "resource.type=k8s_cluster AND protoPayload.resourceName=core/v1/namespaces/default/pods/POD_NAME AND protoPayload.methodName=io.k8s.core.v1.pods.create" --limit 1 | head -10

where POD_NAME is the name of the pod associated with the deployment.

Was this page helpful? Let us know how we did:

Send feedback about...

Binary Authorization Documentation