Configure default settings for organizations and folders

This document describes how to configure default settings for Logging by using the Google Cloud CLI. Default settings, which can be applied to an organization or to a folder, can determine the following:

• Whether a customer-managed encryption key (CMEK) is required for new log buckets.

• The storage location for new _Default and _Required buckets, and for queries run in the Logs Explorer or Log Analytics pages.

• Whether the _Default sink is enabled or disabled.

• The filter that is applied to the _Default sink of new resources.

Overview

The organization resource is at the highest level of the Google Cloud resource hierarchy. The organization resource is the parent of these child resources: Google Cloud projects, folders, billing accounts and, regarding Logging, log buckets.

You can configure Logging to use default settings for a Google Cloud organization and for folders. When you create new resources, those resources inherit the default settings of their parent.

Note: Any default settings for Logging that you change apply only to resources that are created after the configuration is complete; the new settings don't apply to resources that already exist.

Cloud Logging supports the following default settings:

• Whether or not new log buckets in a resource are to be encrypted with a customer-managed key, and if so, the default Cloud KMS key to use for encryption.

  Note: If you configure CMEK for a resource, then you must set a default storage location.

• The storage location for new _Default and _Required log buckets created by child resources, and for queries saved by the Logs Explorer or Log Analytics pages. By setting the storage location, you can control where your logs are stored.

  If you set a default storage location for a resource and don't configure CMEK for that resource, then new log buckets in the resource don't require CMEK.

• Whether the _Default log sink is enabled or disabled for new projects in the resource.

• The inclusion filters or exclusion filters that are applied to all new _Default sinks in the child resources.

Example configurations:

• You configure a default storage location for an organization. For new projects in the organization, the _Default and _Required log buckets are created in the specified location. Also, queries saved by Logs Explorer or Log Analytics pages are stored in the specified location. These queries include the recent queries that are automatically saved after being run, and queries saved by members of the Google Cloud project.

• You configure a default storage location for an organization and you configure a default storage location for each folder in that organization. For new projects that are in a folder, the _Default and _Required buckets are created in the location specified by the folder's settings. For projects that aren't in a folder, their _Default and _Required buckets are created in the location specified by the organization's settings.

• You configure CMEK for an organization, and for the folder named Non-CMEK you only set the default storage location. If you create a project that isn't in the folder named Non-CMEK, then the _Default and _Required buckets are created in the same location as the Cloud Key Management Service key, and these log buckets are encrypted by that key. However, if you create a new project in the folder named Non-CMEK, their log buckets are created in the locations specified by that folder's setting, and those log buckets aren't encrypted by CMEK.

• You configure an exclusion filter that applies to new _Default sinks at an organization level. The filter excludes Data Access audit logs from being routed through the _Default sink in all child resources, which prevents the Data Access audit logs from being stored in the _Default bucket.

Before you begin

This document doesn't contain information about how to configure CMEK as a default setting for Logging. For information about that topic, see Configure CMEK for Logging.

To get started with configuring default settings for Logging, do the following:

1. In the Google Cloud console, activate Cloud Shell.

   Activate Cloud Shell

   At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

2. Ensure that your Identity and Access Management role on the organization or folder whose default settings you want to configure includes the following Cloud Logging permission:

   • logging.settings.get
   • logging.settings.update

3. Identify the location where you want to store your logs and queries. For a list of the supported storage locations, see Supported regions.

View default settings for Logging

To view the default settings for Logging, including the default storage location, use the gcloud logging settings describe command:

FOLDER

 gcloud logging settings describe --folder=FOLDER_ID

Before running the previous command, make the following replacement:

• FOLDER_ID: The unique numeric identifier of the folder. For information about using folders, see Creating and managing folders.

ORGANIZATION

gcloud logging settings describe --organization=ORGANIZATION_ID

Before running the previous command, make the following replacement:

• ORGANIZATION_ID: The unique numeric identifier of the organization. For information about getting this identifier, see Getting your organization ID.

The previous command returns information about the default settings. For example, the following shows the default settings for a particular organization:

name: organizations/ORGANIZATION_ID/settings
kmsKeyName: KMS_KEY_NAME
kmsServiceAccountId: [e-mail]
storageLocation: europe-west1
disableDefaultSink: false

The value of the SERVICE_ACCT_NAME might have the format cmek-12345 or service-12345@.... If you can't use the Google Cloud CLI, then run the Cloud Logging API method getSettings.

Set the default storage location

Note: If you specify a default storage location and also configure CMEK as a default setting for Logging, then the value of the default storage location must match the location of the Cloud KMS key.

Log buckets are the containers in your Google Cloud projects, billing accounts, folders, and organizations that store and organize your log data. For each Google Cloud project, billing account, folder, and organization, Logging automatically creates two log buckets: _Required and _Default, which are automatically stored in the global location.

When you set the default storage location for an organization or folder, you specify where new _Required and _Default log buckets are created and where queries that you run in the Logs Explorer and Log Analytics pages are stored. Setting the default storage location doesn't affect the location of existing log buckets. Similarly, for queries that have been saved, their storage location isn't changed.

After you configure the default storage location for an organization or a folder, the following happens:

• For new child resources created in the organization or folder, their _Required and _Default buckets inherit the default storage location.

• New queries that you run in Logs Explorer or Log Analytics pages are saved in the default storage location. This location also applies to recent queries that are automatically saved.

The default storage location for Cloud Logging doesn't apply to user-defined log buckets or to queries saved by using the Logging API.

Configure the organization policies

Note: You aren't required to configure an organization policy when you want to configure the default storage location for an organization or a folder. For information about organization policies and how to configure them, see Introduction to the Organization Policy Service.

Logging supports organization policies that can restrict where data can be stored. If such a policy exists for your organization, then you can only create log buckets in locations that are allowed by the policy.

When an organization policy that specifies a location constraint exists, the policy values for the constraint must include the location specified in the default settings for Logging. Further, if you plan to modify your default settings, before you update the default settings, review and, if necessary, update the organization policies.

To view or update organization policies, do the following:

1. In the Google Cloud console, go to the Organization Policies page:

   Go to Organization Policies

   If you use the search bar to find this page, then select the result whose subheading is IAM & Admin.

2. Select your organization.

3. View, and if necessary, update the constraint with the ID constraints/gcp.resourceLocations. If this constraint isn't configured, then an update isn't required.

   For information about how to view specific constraints and how to edit these constraints, see Creating and editing policies.

Configure the default storage location for Logging

Note: If you've configured CMEK as a default setting, then you must set the default storage location to the storage location of the Cloud Key Management Service key.

To configure the default storage location for Cloud Logging, run the gcloud logging settings update command and include the --storage-location flag:

FOLDER

gcloud logging settings update --folder=FOLDER_ID--storage-location=LOCATION

Before running the previous command, make the following replacements:

• FOLDER_ID: The unique numeric identifier of the folder. For information about using folders, see Creating and managing folders.
• LOCATION: The location where new _Default and _Required log buckets are created, and where queries are stored. For a list of supported locations, see Supported regions.

ORGANIZATION

gcloud logging settings update --organization=ORGANIZATION_ID --storage-location=LOCATION

Before running the previous command, make the following replacements:

• ORGANIZATION_ID: The unique numeric identifier of the organization. For information about getting this identifier, see Getting your organization ID.
• LOCATION: The location where new _Default and _Required log buckets are created, and where queries are stored. For a list of supported locations, see Supported regions.

If you can't use the Google Cloud CLI, then run the Cloud Logging API method updateSettings.

For information about resolving errors when updating the default storage location, see Troubleshoot setting the default resource location.

Configure the _Default sink

Logging provides a predefined _Default sink for each Google Cloud project, billing account, folder, and organization resource. Any log that is generated in the resource that matches the inclusion filter and that isn't excluded, is routed to the resource's predefined, correspondingly named _Default bucket.

You can configure default settings for the _Default sink for your organization and folders with the following options:

• You can disable the creation of a _Default sink for new child resources.

• You can configure an inclusion filter or several exclusion filters that apply to the _Default sinks of new projects.

Disable the _Default sink

You can disable the _Default sinks for all new resources in an organization or folder; disabling the _Default sinks prevents logs from being stored in the resource's _Default bucket. If you stop storing logs in a resource's _Default bucket, then the logs that would have been routed to that bucket are excluded from storage in Logging, unless those logs are explicitly included in another user-defined sink for that resource.

To disable the _Default sinks for a resource and any of its child resources, run the following gcloud logging settings update command:

FOLDER

gcloud logging settings update --folder=FOLDER_ID--disable-default-sink

Before running the previous command, make the following replacement:

• FOLDER_ID: The unique numeric identifier of the folder. For information about using folders, see Creating and managing folders.

ORGANIZATION

gcloud logging settings update --organization=ORGANIZATION_ID --disable-default-sink

Before running the previous command, make the following replacement:

• ORGANIZATION_ID: The unique numeric identifier of the organization. For information about getting this identifier, see Getting your organization ID.

The disable-default-sink flag applies only to the _Default sink that routes logs into the _Default bucket.

You can re-enable the _Default sinks by running the following gcloud logging settings update command:

FOLDER

gcloud logging settings update --folder=FOLDER_ID--no-disable-default-sink

ORGANIZATION

gcloud logging settings update --organization=ORGANIZATION_ID --no-disable-default-sink

Configure default filter of _Default sinks

Note: Default settings only apply to newly created resources. You can't use the process documented in this section to configure the filter of an existing _Default sink.

The predefined _Default sink routes any log entries that match the sink criteria to the corresponding _Default bucket. You can send an Cloud Logging API command to override the built-in inclusion filter in the _Default sink or to append a filter. The built-in exclusion filter for the _Default sink is empty. However, the API command also lets you add exclusion filters.

Note: If you add exclusion filters and if you want the _Default log sink to be listed on the Logs Router page, then you must specify a name for each exclusion filter. If you omit the name field, then you must use the Cloud Logging API or the Google Cloud CLI to view the details of the sink. In the following example, the exclusion filter is named exclude-data-access.

To specify an inclusion filter or exclusion filter that is applied to all _Default sinks of new resources in an organization or folder, run the Cloud Logging API method updateSettings and specify the defaultSinkConfig object.

You can execute the updateSettings method by using the APIs Explorer widget on the method's reference page. The following example illustrates sample parameters:

• name (URL): organizations/ORGANIZATION_ID/settings
• updateMask: "default_sink_config"

• Request body, which contains an instance of Settings:

  "defaultSinkConfig": {
    {
    "filter": "NOT LOG_ID(\"externalaudit.googleapis.com/activity\") "
    "AND NOT LOG_ID(\"cloudaudit.googleapis.com/system_event\") "
    "AND NOT LOG_ID(\"externalaudit.googleapis.com/system_event\") "
    "AND NOT LOG_ID(\"cloudaudit.googleapis.com/access_transparency\") "
    "AND NOT LOG_ID(\"externalaudit.googleapis.com/access_transparency\") ",
    "exclusions": [
       {
          "name": "exclude-data-access",
          "description": "Prevents Data Access audit logs from being routed",
          "filter": "log_id(\"cloudaudit.googleapis.com/data_access\")",
       }
    ],
    "mode": OVERWRITE
    }
  }
  

The built-in inclusion filter for the _Default sink includes the statement AND NOT LOG_ID("externalaudit.googleapis.com/activity"), which prevents Admin Activity audit logs from being routed to the _Default log bucket. In the previous example, the inclusion filter is changed so that Admin Activity audit logs are routed to the _Default log bucket. The example also adds an exclusion filter that prevents Data Access audit logs from being routed to the _Default bucket. In the previous example, the exclusion filter is named exclude-data-access.

Troubleshoot configuration errors

For troubleshooting information, see Troubleshoot CMEK and default setting errors.