Audit logging

You're viewing Apigee X documentation.
View Apigee Edge documentation.

This page describes the audit logs created by Apigee and Apigee hybrid as part of Cloud Audit Logs.

Overview

Google Cloud Services write audit logs to help you answer the questions, "Who did what, where, and when?" The audit logs for your Apigee and hybrid projects each contain only the log data for resources directly within the project. The audit logs for other entities, such as folders, organizations, and billing accounts, each contain log data for the entity itself.

For a general overview of Cloud Audit Logs, go to Cloud Audit Logs. For a deeper understanding of Cloud Audit Logs, review Understanding audit logs.

Cloud Audit Logs maintains three audit logs for each Google Cloud project, folder, and organization:

  • Admin Activity audit logs
  • Data Access audit logs
  • System Event audit logs

Apigee and hybrid write Admin Activity audit logs, which record operations that modify the configuration or metadata of a resource. You can't disable Admin Activity audit logs.

Only if explicitly enabled, Apigee and hybrid write Data Access audit logs. Data Access audit logs contain API calls that read the configuration or metadata of resources. Data Access audit logs do not record the data-access operations on resources that are publicly shared (available to All Users or All Authenticated Users) or that can be accessed without logging into Google Cloud.

Apigee and hybrid do not write System Event audit logs.

Audited operations

The following summarizes which API operations correspond to each audit log type in Apigee and hybrid:

Audit logs category Operations
DATA_READ GET
LIST
ADMIN CREATE
UPDATE
DELETE

Audit log format

Audit log entries—which can be viewed in Cloud Logging using the Logs Viewer, the Cloud Logging API, or the gcloud command-line tool—include the following objects:

  • The log entry itself, which is an object of type LogEntry. Useful fields include the following:
  • logName contains the project identification and audit log type
  • resource contains the target of the audited operation
  • timeStamp contains the time of the audited operation
  • protoPayload contains the audited information
  • The audit logging data, which is an AuditLog object held in the protoPayload field of the log entry.
  • Optional service-specific audit information, which is a service-specific object held in the serviceData field of the AuditLog object. For details, go to Service-specific audit data.

For other fields in these objects, plus how to interpret them, review Understanding audit logs.

Log name

Cloud Audit Logs resource names indicate the project or other entity that owns the audit logs, and whether the log contains Admin Activity, Data Access, or System Event audit logging data. For example, the following shows log names for a project's Admin Activity audit logs and an organization's Data Access audit logs:

projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Factivity
organizations/ORGANIZATION_ID/logs/cloudaudit.googleapis.com%2Fdata_access

Service name

Apigee and hybrid audit logs use the service name apigee.googleapis.com.

For more details on logging services, go to Mapping services to resources.

Resource types

Audit logs for Apigee and hybrid use the resource type audited_resource for all audit logs.

For a full list, go to Monitored resource types.

Enabling audit logging

Admin Activity audit logs are always enabled; you can't disable them.

Data Access audit logs are disabled by default and aren't written unless explicitly enabled (the exception is Data Access audit logs for BigQuery, which cannot be disabled).

For instructions on enabling some or all of your Data Access audit logs, go to Configuring Data Access logs.

The Data Access audit logs that you configure can affect your logs pricing in Cloud Operations. Review the Pricing section on this page.

Audit log permissions

Cloud Identity and Access Management permissions and roles determine which audit logs you can view or export. Logs reside in projects and in some other entities including organizations, folders, and billing accounts. For more information, go to Understanding roles.

To view Admin Activity audit logs, you must have one of the following IAM roles in the project that contains your audit logs:

  • Project Owner, Project Editor, or Project Viewer.
  • Logging's Logs Viewer role.
  • A custom IAM role with the logging.logEntries.list IAM permission.

To view Data Access audit logs, you must have one of the following roles in the project that contains your audit logs:

If you are using audit logs from a non-project entity, such as an organization, then change the Project roles to suitable organization roles.

Viewing logs

You have the following options for viewing your audit log entries:

Legacy Logs Viewer

You can use the Legacy Logs Viewer interface in the Google Cloud console to retrieve your audit log entries.

Basic viewer

To use the basic viewer, do the following:

  1. In the Google Cloud console, go to the Logging > Legacy Logs Viewer page.

    Go to Legacy Logs Viewer

  2. Select an existing Google Cloud project at the top of the page, or create a new project.
  3. In the first drop-down menu, select the resource type whose audit logs you wish to view.
  4. In the second drop-down menu, select the log type you want to view: activity for Admin Activity audit logs, data_access for Data Access audit logs, and system_events for System Event audit logs.

    If none of these options are displayed, then there are no audit logs of that type available in the project.

For more information on the Legacy Logs Viewer, see Viewing logs in the Legacy Logs Viewer.

Advanced viewer

To use the advanced viewer, do the following:

  1. In the Google Cloud console, go to the Logging > Legacy Logs Viewer page.

    Go to Legacy Logs Viewer

  2. Select an existing Google Cloud project at the top of the page, or create a new project.
  3. In the first drop-down menu, select the resource type whose audit logs you wish to view.
  4. Click the drop-down arrow at the far right of the search-query box and select Convert to advanced filter.

  5. You can also create a query that further specifies the log entries you want to view. For example, to retrieve all audit logs in your project, copy the following query, paste it in the query box, and then click Run query. Replace PROJECT_ID with a valid project ID name.

    logName = ("projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Factivity"
    OR
    "projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Fsystem_events"
    OR
    "projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Fdata_access")
    

For more details about queries, go to Advanced logs queries.

Log Explorer

You can use the Logs Explorer interface in the Google Cloud console to retrieve your audit log entries. Do the following:

  1. In the Google Cloud console, go to the Logging > Legacy Logs Viewer page.

    Go to Legacy Logs Viewer

  2. Select an existing Google Cloud project at the top of the page, or create a new project.
  3. Build a query by selecting from the Resource, Log name, and Severity drop-down menus, and then click Run query.
  4. You can also create a query that further specifies the log entries you want to view. For example, to retrieve all audit logs in your project, copy the following query, paste it in the query box, and then click Run query. Replace PROJECT_ID with a valid project ID name.
    logName = ("projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Factivity"
    OR
    "projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Fsystem_events"
    OR
    "projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Fdata_access")
    

For more information about the Logs Explorer, see Logs Explorer

For more details about queries, go to Advanced logs queries.

API

To look at your audit log entries using the Cloud Logging API:

  1. Use the Try this API option:
    1. Go to the Try this API section in the documentation for the entries.list method.
    2. Put the following into the Request body part of the Try this API form. Replace PROJECT_ID with a valid project ID name.
      {
        "resourceNames": [
          "projects/PROJECT_ID"
        ],
        "pageSize": 5,
        "filter": "logName=(projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Factivity OR projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Fsystem_events OR projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Fdata_access)"
      }
    3. Click Execute.
  2. Or use the following curl command:
  3. curl "https://logging.googleapis.com/v2/entries:list" \
      -X POST \
      -H "Content-type: application/json" \
      -H "Authorization: Bearer $TOKEN" \
      -d  \
      '{
          "resourceNames":["projects/PROJECT_ID"],
           "pageSize":5,
           "filter":"logName=(projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Factivity
           OR projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Fsystem_events
           OR projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Fdata_access)"
      }'
    

    Where $TOKEN is set to your OAuth 2.0 access token, as described in Obtaining an OAuth 2.0 access token. For information about the curl options used in this example, see Using curl. For a description of the environment variables used, see Setting environment variables for Apigee API requests.

    The following shows an example of the response output.

    {
      "entries": [
        {
          "protoPayload": {
            "@type": "type.googleapis.com/google.cloud.audit.AuditLog",
            "authenticationInfo": {
              "principalEmail": "system:serviceaccount:cert-manager:cert-manager-cainjector"
            },
            "authorizationInfo": [
              {
                "granted": true,
                "permission": "io.k8s.core.v1.configmaps.update",
                "resource": "core/v1/namespaces/kube-system/configmaps/cert-manager-cainjector-leader-election-core"
              }
            ],
            "methodName": "io.k8s.core.v1.configmaps.update",
            "requestMetadata": {
              "callerIp": "35.226.36.25",
              "callerSuppliedUserAgent": "cainjector/v0.0.0 (linux/amd64) kubernetes/$Format"
            },
            "resourceName": "core/v1/namespaces/kube-system/configmaps/cert-manager-cainjector-leader-election-core",
            "serviceName": "k8s.io",
            "status": {
              "code": 0
            }
          },
          "insertId": "f87d6dd2-c661-47f7-ad70-699cb322090d",
          "resource": {
            "type": "k8s_cluster",
            "labels": {
              "project_id": "apigee-eda-experiments",
              "cluster_name": "sautest",
              "location": "us-central1"
            }
          },
          "timestamp": "2020-02-04T21:05:29.792617Z",
          "labels": {
            "authorization.k8s.io/decision": "allow",
            "authorization.k8s.io/reason": "RBAC: allowed by RoleBinding \"cert-manager-cainjector:leaderelection/kube-system\" of Role \"cert-manager-cainjector:leaderelection\" to ServiceAccount \"cert-manager-cainjector/cert-manager\""
          },
          "logName": "projects/apigee-eda-experiments/logs/cloudaudit.googleapis.com%2Factivity",
          "operation": {
            "id": "f87d6dd2-c661-47f7-ad70-699cb322090d",
            "producer": "k8s.io",
            "first": true,
            "last": true
          },
          "receiveTimestamp": "2020-02-04T21:05:46.819543686Z"
        },
        {
          "protoPayload": {
            "@type": "type.googleapis.com/google.cloud.audit.AuditLog",
            "authenticationInfo": {
              "principalEmail": "system:serviceaccount:cert-manager:cert-manager-cainjector"
            },
            "authorizationInfo": [
              {
                "granted": true,
                "permission": "io.k8s.core.v1.configmaps.update",
                "resource": "core/v1/namespaces/kube-system/configmaps/cert-manager-cainjector-leader-election"
              }
            ],
            "methodName": "io.k8s.core.v1.configmaps.update",
            "requestMetadata": {
              "callerIp": "35.226.36.25",
              "callerSuppliedUserAgent": "cainjector/v0.0.0 (linux/amd64) kubernetes/$Format"
            },
            "resourceName": "core/v1/namespaces/kube-system/configmaps/cert-manager-cainjector-leader-election",
            "serviceName": "k8s.io",
            "status": {
              "code": 0
            }
          },
          "insertId": "dffd916f-6b4d-464d-b0a2-3cc6c2b3f6c7",
          "resource": {
            "type": "k8s_cluster",
            "labels": {
              "location": "us-central1",
              "project_id": "apigee-eda-experiments",
              "cluster_name": "sautest"
            }
          },
          "timestamp": "2020-02-04T21:05:29.792689Z",
          "labels": {
            "authorization.k8s.io/reason": "RBAC: allowed by RoleBinding \"cert-manager-cainjector:leaderelection/kube-system\" of Role \"cert-manager-cainjector:leaderelection\" to ServiceAccount \"cert-manager-cainjector/cert-manager\"",
            "authorization.k8s.io/decision": "allow"
          },
          "logName": "projects/apigee-eda-experiments/logs/cloudaudit.googleapis.com%2Factivity",
          "operation": {
            "id": "dffd916f-6b4d-464d-b0a2-3cc6c2b3f6c7",
            "producer": "k8s.io",
            "first": true,
            "last": true
          },
          "receiveTimestamp": "2020-02-04T21:05:46.237944230Z"
        },
        {
          "protoPayload": {
            "@type": "type.googleapis.com/google.cloud.audit.AuditLog",
            "authenticationInfo": {
              "principalEmail": "system:serviceaccount:apigee-system:default"
            },
            "authorizationInfo": [
              {
                "granted": true,
                "permission": "io.k8s.core.v1.configmaps.update",
                "resource": "core/v1/namespaces/apigee-system/configmaps/controller-leader-election-helper"
              }
            ],
            "methodName": "io.k8s.core.v1.configmaps.update",
            "requestMetadata": {
              "callerIp": "35.232.81.27",
              "callerSuppliedUserAgent": "manager/v0.0.0 (linux/amd64) kubernetes/$Format"
            },
            "resourceName": "core/v1/namespaces/apigee-system/configmaps/controller-leader-election-helper",
            "serviceName": "k8s.io",
            "status": {
              "code": 0
            }
          },
          "insertId": "79d6f436-c05d-4ecd-bbb8-908caf4515c3",
          "resource": {
            "type": "k8s_cluster",
            "labels": {
              "location": "us-central1",
              "project_id": "apigee-eda-experiments",
              "cluster_name": "russo"
            }
          },
          "timestamp": "2020-02-04T21:05:29.811603Z",
          "labels": {
            "authorization.k8s.io/decision": "allow",
            "authorization.k8s.io/reason": "RBAC: allowed by RoleBinding \"apigee-leader-election-rolebinding/apigee-system\" of Role \"apigee-leader-election-role\" to ServiceAccount \"default/apigee-system\""
          },
          "logName": "projects/apigee-eda-experiments/logs/cloudaudit.googleapis.com%2Factivity",
          "operation": {
            "id": "79d6f436-c05d-4ecd-bbb8-908caf4515c3",
            "producer": "k8s.io",
            "first": true,
            "last": true
          },
          "receiveTimestamp": "2020-02-04T21:05:34.526854453Z"
        },
        {
          "protoPayload": {
            "@type": "type.googleapis.com/google.cloud.audit.AuditLog",
            "authenticationInfo": {
              "principalEmail": "system:serviceaccount:cert-manager:cert-manager-cainjector"
            },
            "authorizationInfo": [
              {
                "granted": true,
                "permission": "io.k8s.core.v1.configmaps.update",
                "resource": "core/v1/namespaces/kube-system/configmaps/cert-manager-cainjector-leader-election"
              }
            ],
            "methodName": "io.k8s.core.v1.configmaps.update",
            "requestMetadata": {
              "callerIp": "35.232.81.27",
              "callerSuppliedUserAgent": "cainjector/v0.0.0 (linux/amd64) kubernetes/$Format"
            },
            "resourceName": "core/v1/namespaces/kube-system/configmaps/cert-manager-cainjector-leader-election",
            "serviceName": "k8s.io",
            "status": {
              "code": 0
            }
          },
          "insertId": "56f901e3-dbc4-445a-82e4-69ff793927ca",
          "resource": {
            "type": "k8s_cluster",
            "labels": {
              "project_id": "apigee-eda-experiments",
              "cluster_name": "russo",
              "location": "us-central1"
            }
          },
          "timestamp": "2020-02-04T21:05:29.987492Z",
          "labels": {
            "authorization.k8s.io/reason": "RBAC: allowed by RoleBinding \"cert-manager-cainjector:leaderelection/kube-system\" of Role \"cert-manager-cainjector:leaderelection\" to ServiceAccount \"cert-manager-cainjector/cert-manager\"",
            "authorization.k8s.io/decision": "allow"
          },
          "logName": "projects/apigee-eda-experiments/logs/cloudaudit.googleapis.com%2Factivity",
          "operation": {
            "id": "56f901e3-dbc4-445a-82e4-69ff793927ca",
            "producer": "k8s.io",
            "first": true,
            "last": true
          },
          "receiveTimestamp": "2020-02-04T21:05:30.501196305Z"
        },
        {
          "protoPayload": {
            "@type": "type.googleapis.com/google.cloud.audit.AuditLog",
            "authenticationInfo": {
              "principalEmail": "system:serviceaccount:cert-manager:cert-manager-cainjector"
            },
            "authorizationInfo": [
              {
                "granted": true,
                "permission": "io.k8s.core.v1.configmaps.update",
                "resource": "core/v1/namespaces/kube-system/configmaps/cert-manager-cainjector-leader-election-core"
              }
            ],
            "methodName": "io.k8s.core.v1.configmaps.update",
            "requestMetadata": {
              "callerIp": "35.232.81.27",
              "callerSuppliedUserAgent": "cainjector/v0.0.0 (linux/amd64) kubernetes/$Format"
            },
            "resourceName": "core/v1/namespaces/kube-system/configmaps/cert-manager-cainjector-leader-election-core",
            "serviceName": "k8s.io",
            "status": {
              "code": 0
            }
          },
          "insertId": "b3a2926d-27e7-414c-afce-de104a4f6cce",
          "resource": {
            "type": "k8s_cluster",
            "labels": {
              "location": "us-central1",
              "cluster_name": "russo",
              "project_id": "apigee-eda-experiments"
            }
          },
          "timestamp": "2020-02-04T21:05:30.011159Z",
          "labels": {
            "authorization.k8s.io/reason": "RBAC: allowed by RoleBinding \"cert-manager-cainjector:leaderelection/kube-system\" of Role \"cert-manager-cainjector:leaderelection\" to ServiceAccount \"cert-manager-cainjector/cert-manager\"",
            "authorization.k8s.io/decision": "allow"
          },
          "logName": "projects/apigee-eda-experiments/logs/cloudaudit.googleapis.com%2Factivity",
          "operation": {
            "id": "b3a2926d-27e7-414c-afce-de104a4f6cce",
            "producer": "k8s.io",
            "first": true,
            "last": true
          },
          "receiveTimestamp": "2020-02-04T21:05:30.642048673Z"
        }
      ],
      "nextPageToken": "EAA43KyVlIDih_pdSoAIIjYiJwolYjNhMjkyNmQtMjdlNy00MTRjLWFmY2UtZGUxMDRhNGY2Y2NlACoLCJq55_EFENiLqQVKxQcKmQdKlgcA0C8gBg0KHKVKT9NI8-QwzATkKBOVymR2R8TBZkfOUGuODDzwuXqNpxcHRu6XC6rCnJ1FifaCeib8nQdWcEjvFOr2WRDBXjTGAV4KG2zn1FBrdDDl_jxWzSBIUIiuMr83AZD4PFYQSpDRdXSJ0XRIe_C1ZiduH231NvBqKf2q1NLqUg9s8okp0sxubdMPSh2NPdmzK_5O6A9wD2DdJN0ClepqsAxru6qrqn9_eyPSrEk9ri2k2LEdT3Vf_Dp-0puBoits_i3AR-_XVJ2i1jmW1fYpiWrXu3IsxwSyOIOep3BSlw6z6B5fQKMxY_e0DnhBF3uNUC-El377Bqg6x7tSu9lg8D8qSXDl77E_1MsqIHQIZJqAVAlOJNE3MW_uaa_gvpwQxGFJ45CtMhgVW8AKPsxZzkbZtBobE-2OlB85D77eb_Z4aTbHyBh0VfDFq9Wsp857UvMxpNZxUKQe233D952Hkca2HrMi4l52xC6USXAx9l9b8pOlB_t5pK8ILKfmyPCqlAtECChavcX0WbYT_9CDgz2wy_yFa_Agw6l6vEJhenyfnmuXY3BQE08C3TmmhHx4AYhI8n-ee2PtBD9OSHzRuRlHRSWrnnDHoAMHtNBL1Lg1-6UUPVs5ROW4BUV1RbW3ZA0oXSRd8XGWlNZKqL1ruj6rGk0EARRC380-63zFQ0hpeeVNDWLhzhmg_UuS-IAE0a9iK_0tjrr0UNMxX2oBNn5yXG9oUFx0sCwVGEBI5z6GyBl-x4-Ck6vWiS1gy1PPl81U5U03D5vFiKHA0t6BS9JHrgx72Fhez0x1LNMnOOk6zSED7-FSZMBa8ERNLYnF6fBxqQS0UG1qN0ZLtRwVwXrdKgDwGlG1s7Cgfr5WkUX9L-aNL9oe4f_FYO5DhZlynfos74ToUtCRcQBDtRHOYGx7cqUNvkcv66EO6pIiePPOjhzstfGNdVMwVyRebTZl9RD4qOPBRAc_uWEkIDpjdiBRDg9uGfofnv6BFTmPNpQ4L-cBy7kF-F-M-KipHif7qzDHzz8lUgHGfbeMdbcb7UBCvoCGA8SLAHUMLw_J2-ny9kWB1aglmJlSVeKyrgLwYLiR85gtOdNgcbuKfeI6yCpIEZNDsyYg_LLR8o98FzaO26MWfCbYFaU9l2z9d_Y4LI1GLoR6pCKzDXsvwgtgWcARoSUDfiJ6rRu9EzFiLIlS6dKcphQVlFPRsNTIDXv5WpMSHBoMCJm55_EFEJW5-asCIgwI_-vp8QUQ_5Pr3ANQ2Neh4Nn0uoSwAVIHCLv_29ORHWC-sv2mmrbe5fMBahsKDAiZ6aSCBhCxjsbNARIHCAIQ1IPULhgFIAA"
    }
    
For more details about queries, go to Advanced logs queries.

gcloud

To read your log entries using the gcloud command-line tool, run the following command. Replace PROJECT_ID with a valid project ID name.

gcloud logging read "logName=(projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Factivity OR projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Fsystem_events OR projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Fdata_access)"

Go to Reading log entries for more information about using the gcloud command-line tool.

For a sample audit log entry and how to find the most important information in it, go to Understanding audit logs.

Exporting audit logs

You can export audit logs in the same way you export other kinds of logs. For details about how to export your logs, go to Overview of logs exports. Here are some applications of exporting audit logs:

  • To keep audit logs for a longer period of time or to use more powerful search capabilities, you can export copies of your audit logs to Cloud Storage, BigQuery, or Pub/Sub. Using Pub/Sub, you can export to other applications, other repositories, and to third parties.
  • To manage your audit logs across an entire organization, you can create aggregated export sinks that can export logs from any or all projects in the organization.
  • If your enabled Data Access audit logs are pushing your projects over their logs allotments, you can export and exclude the Data Access audit logs from Logging. For details, go to Logs exclusions.

Pricing

Cloud Logging does not charge you for audit logs that cannot be disabled, including all Admin Activity audit logs.

Cloud Logging charges you for Data Access audit logs that you explicitly request.