Configuring Data Access Logs

This guide explains how to configure Data Access audit logs in your project or organization. For general information about audit logs, see Cloud Audit Logging.

Admin Activity audit logs are enabled for all services and cannot be configured. On this page, "audit logs" refers to Data Access audit logs.

To skip the explanations, see Example.

Configuration options

Data Access audit logs—except for BigQuery—are disabled by default. You can enable and configure three aspects of audit logs in your project or organization:

  • Services. You can specify the services whose audit logs you want to receive. For example, you might want audit logs from Compute Engine but not from Cloud SQL. For a list of the services that can generate audit logs, see Services producing audit logs.

  • Kinds of information. You can control what kinds of information are in the audit logs. There are three kinds of Data Access audit log information:

    • Admin read: Records operations that read metadata or configuration information.

      Admin Activity audit logs record writes of metadata and configuration information. They cannot be disabled.

    • Data read: Records operations that read user-provided data.

    • Data write: Records operations that write user-provided data.

    For example, you could log only the "data write" operations by default, but log all three kinds of information from Cloud DNS.

  • Exempted users. You can exempt specific users or groups from having their data accesses recorded. For example, you can exempt your internal testing accounts from having their Stackdriver Debugger operations recorded.

IAM policy objects

To configure audit logs, you must edit the IAM policy associated with your project or organization. The audit log configuration is in the auditConfigs section of the policy:

"auditConfigs": [
  {
    object(AuditConfig)
  }
]

For details, see the IAM Policy type.

The following sections describe the AuditConfig object in more detail. For the API and Cloud SDK commands used to change the configuration, see getIamPolicy and setIamPolicy

AuditConfig objects

The audit log configuration consists of a list of AuditConfig objects. Each object configures the logs for one service, or it establishes a default configuration for all services. Each object looks like the following:

{
  "service": [SERVICE],
  "auditLogConfigs": [
    {
      "logType": "ADMIN_READ"
      "exemptedMembers": [ [MEMBER],]
    },
    {
      "logType": "DATA_READ"
      "exemptedMembers": [ [MEMBER],]
    },
    {
      "logType": "DATA_WRITE"
      "exemptedMembers": [ [MEMBER],]
    },
  ]
},

[SERVICE] is service name such as "appengine.googleapis.com", or it is the special value, "allServices". If a configuration doesn't mention a particular service, then the default configuration is used for that service. If there is no default configuration, then Data Access logs are not enabled for that service. For a list of the service names, see Log services.

The auditLogConfigs section of the AuditConfig object is a list of 0 to 3 objects, each of which configures one kind of audit log information. If you omit one of the kinds from the list, then that kind of information is not enabled for the service.

[MEMBER] is a user for whom Data Access audit logs is not collected. You can specify single users, groups, or service accounts. The Binding type describes different kinds of members, but not all of those can be used to configure audit logs.

Following is an example of an audit configuration in both JSON and YAML formats. The YAML format is the default when using the Cloud SDK commands.

JSON

"auditConfigs": [
  {
    "auditLogConfigs": [
      {
        "logType": "ADMIN_READ"
      },
      {
        "logType": "DATA_WRITE"
      },
      {
        "logType": "DATA_READ"
      }
    ],
    "service": "allServices"
  },
  {
    "auditLogConfigs": [
      {
        "exemptedMembers": [
          "499862534253-compute@developer.gserviceaccount.com"
        ],
        "logType": "ADMIN_READ"
      }
    ],
    "service": "cloudsql.googleapis.com"
  }
],

YAML

auditConfigs:
- auditLogConfigs:
  - logType: ADMIN_READ
  - logType: DATA_WRITE
  - logType: DATA_READ
  service: allServices
- auditLogConfigs:
  - exemptedMembers:
    - 499862534253-compute@developer.gserviceaccount.com
    logType: ADMIN_READ
  service: cloudsql.googleapis.com

Default and specific configurations

If there is both a default (allServices) configuration and a configuration for a specific service, then the resulting configuration for the service is the union of the default and specific configurations. In other words:

  • You can enable audit logs for specific services, but you cannot disable services that are enabled in the default configuration.

  • You can add additional kinds of information to a service's audit logs. You cannot remove kinds of information that are specified in the default configuration.

  • You can add members to exemption lists, but you cannot remove members from exemption lists in the default configuration.

Organization and project configurations

You can configure Data Access audit logs organization and projects. If there is a configuration for a service in both a project and the project's organization, then the resulting configuration is the union of the configurations in the project and organization. In other words, at the project level:

  • You can enable logs for a service, but you cannot disable logs for a service that is enabled in the organization.

  • You can enable kinds of information, but you cannot disable kinds of information that are enabled in the organization.

  • You can add members to exemption lists, but you cannot remove members that are on the organization's exemption lists.

Common configurations

Following are some common audit log configurations for projects.

These configurations do not take into account any audit log configuration in the project's organization. An organization can enable data access logs in a project even if the project has no configuration. For more information, see Organization and project configurations.

Enable all access logs

The following auditConfigs section enables Data Access logs for all services and users:

JSON

"auditConfigs": [
      {
        "service": "allServices",
        "auditLogConfigs": [
          { "logType": "ADMIN_READ" },
          { "logType": "DATA_READ"  },
          { "logType": "DATA_WRITE" },
        ]
      },
    ]

YAML

auditConfigs:
- auditLogConfigs:
  - logType: ADMIN_READ
  - logType: DATA_WRITE
  - logType: DATA_READ
  service: allServices

Enable one service and information kind

The following configuration enables Data Access audit logs for Cloud SQL. The logs record only the writes of user-defined data:

JSON

"auditConfigs": [
  {
    "service": "cloudsql.googleapis.com",
    "auditLogConfigs": [
      { "logType": "DATA_WRITE" },
    ]
  },
]

YAML

auditConfigs:
- auditLogConfigs:
  - logType: DATA_WRITE
  service: cloudsql.googleapis.com

Disable all logs

To disable all Data Access audit logs (except BigQuery) in a project, include an empty auditConfigs: section in your new IAM policy:

JSON

"auditConfigs": [],

YAML

auditConfigs:

If you remove the auditConfigs section entirely from your new policy, then setIamPolicy will not change the existing audit logs configuration. For more information, see The setIamPolicy update mask.

BigQuery audit logs cannot be disabled.

Disable BigQuery logs

You cannot presently disable BigQuery Data Access audit logs.

getIamPolicy and setIamPolicy

You use the Cloud Resource Manager's getIamPolicy and setIamPolicy methods to read and write your IAM policy. You have several choices for the specific methods to use:

  • The Cloud Resource Manager API has the following methods:

    projects.getIamPolicy
    projects.setIamPolicy
    organizations.getIamPolicy
    organizations.setIamPolicy
    
  • The Cloud SDK has the following Cloud Resource Manager commands:

    gcloud projects get-iam-policy
    gcloud projects set-iam-policy
    gcloud organizations get-iam-policy
    gcloud organizations set-iam-policy
    

Regardless of your choice, follow these three steps:

  1. Read the current policy using one of the getIamPolicy methods. Save the policy to a temporary file.
  2. Edit the policy in the temporary file. Change (or add) only the auditConfigs section.
  3. Write the edited policy in the temporary file, using one of the setIamPolicy methods.

setIamPolicy will fail if Cloud Resource Manager detects that someone else changed the policy after you read it in the first step. If this happens, then repeat the three steps.

Access control

You need roles/resourcemanager.organizationAdmin to configure your organization, and roles/editor to configure your project.

For more information, see Access control for projects and Access control for organization).

Examples

The following examples demonstrate how to configure your project's Data Access audit logs using the gcloud command and the Cloud Resource Manager API.

To configure organization audit logs, replace the "projects" version of the commands and API methods with the "organizations" version.

Cloud SDK

To configure your audit logs using the gcloud projects command, do the following:

  1. Read your project's IAM policy and store it in a file:

    gcloud projects get-iam-policy [PROJECT_ID] > /tmp/policy.yaml
    

    The returned policy is shown below. This policy does not yet have an auditConfigs section:

    bindings:
    - members:
      - user:colleague@example.com
      role: roles/editor
    - members:
      - user:myself@example.com
      role: roles/owner
    etag: BwVM-FDzeYM=
    version: 1
    
  2. Edit your policy in /tmp/policy.yaml, adding or changing only the audit logs configuration.

    An example of your edited policy, which enables Cloud SQL data-write audit logs, is shown below:

    auditConfigs:
    - auditLogConfigs:
      - logType: DATA_WRITE
      service: cloudsql.googleapis.com
    bindings:
    - members:
      - user:colleague@example.com
      role: roles/editor
    - members:
      - user:myself@example.com
      role: roles/owner
    etag: BwVM-FDzeYM=
    version: 1
    

  3. Write your new IAM policy:

    gcloud projects set-iam-policy [PROJECT_ID] /tmp/policy.yaml
    

    If the preceding command reports a conflict with another change, then repeat these steps, starting with the first step.

JSON

To work with your IAM policy in JSON format instead of YAML, substitute the following gcloud commands into the example:

gcloud projects get-iam-policy [PROJECT_ID] --format=json >/tmp/policy.json
gcloud projects set-iam-policy [PROJECT_ID] /tmp/policy.json

API

To configure your audit logs using the Cloud Resource Manager API, do the following:

  1. Read your project's IAM policy, specifying the following parameters to the getIamPolicy API method:

    • resource: projects/[PROJECT_ID]
    • Request body: empty

    The method returns the current policy object, shown below. This project's policy does not yet have an auditConfigs section:

    {
      "bindings": [
      {
        "members": [
          "user:colleague@example.com"
        ],
        "role": "roles/editor"
      },
      {
        "members": [
          "user:myself@example.com"
        ],
        "role": "roles/owner"
      }
    ],
    "etag": "BwUsv2gimRs=",
    "version": 1
    

    }

  2. Edit the current policy:

    • Change or add the auditConfigs section.

      If you wish to disable your audit logs, then include an empty value for the section: auditConfigs:[].

    • Preserve the value of etag.

    You may, if you wish, remove all other information from the new policy object, as long as you are careful to set updateMask in the next step. The edited policy, which enables Cloud SQL data-write audit logs, is shown below:

    {
      "auditConfigs": [
        {
          "auditLogConfigs": [
            {
              "logType": "DATA_WRITE"
            }
          ],
          "service": "cloudsql.googleapis.com"
        }
      ],
      "etag": "BwVM-FDzeYM="
    }
    
  3. Write the new policy using the setIamPolicy API method, specifying the following parameters:

    • resource: projects/[PROJECT_ID]
    • Request body:
      • updateMask: "auditConfigs,etag"
      • policy: your edited policy object

The setIamPolicy update mask

This section explains the importance of the updateMask parameter in the setIamPolicy method, and explains why you must be careful with the SDK's set-iam-policy command so that you do not cause accidental harm to your project or organization.

The setIamPolicy API method uses an updateMask parameter to control which policy fields are updated. For example, if the mask does not contain bindings, then you cannot accidentally change that policy section. On the other hand, if the mask does contain bindings, then that section is always updated. If you do not include an updated value for bindings, then that section is removed entirely from the policy.

The gcloud projects set-iam-policy command, which calls setIamPolicy, does not let you specify the updateMask parameter. Instead, the command computes a value for updateMask in the following way:

  • The updateMask always contains the fields bindings and etag.
  • If the policy object supplied in set-iam-policy contains any other top-level fields—such as auditConfigs—then those fields are added to updateMask.

As a consequence of these rules, the set-iam-policy command has the following behavior:

  • If you omit the auditConfigs section in your new policy, then the previous value of auditConfigs section (if any) is not changed, because that section is not in the update mask. This is harmless but might be confusing.

  • If you omit bindings or etag in your new policy object, then the bindings or etag sections are removed from your policy, since those sections do appear in the update mask. This is very harmful.

Removing bindings in your policy causes all users to lose access to your project. Removing etag disables the checking for concurrent changes to your policy, and might result in your changes accidentally overwriting someone else's changes.

Monitor your resources on the go

Get the Google Cloud Console app to help you manage your projects.

Send feedback about...

Stackdriver Logging