Excluding Logs

The Resource Usage page in Stackdriver Logging tracks the volume of logs in your project. The page also gives you tools to exclude (discard) log entries you're not interested in, so that you can minimize any charges for logs over your monthly allotment. For more information about how excluded log entries are treated, see Overview of exclusions on this page.

Tracking logs usage

To track your project's logs volume, go the Resource Usage page in the Logs Viewer:

Go to the Resource Usage page

At the top of the page, you see a summary of the logs that your projects is receiving:

Resource Usage

Three numbers are reported:

  • Log Volume Received: The amount of logs your project received since the beginning of the month.

  • Log Volume Projected: The estimated amount of logs your project will receive for the entire month. This estimate is based only on the amount of logs received so far during the month.

  • Log Volume Excluded: The amount of logs that you have excluded from your project since the beginning of the month. This number is not included in Log Volume Received. Excluding logs is described later on this page.

The log volumes do not include the default-enabled Cloud Auditing logs—all Admin Activity logs plus BigQuery Data Access logs. Those logs are free and cannot be excluded.

You can also see the breakdown in logs usage by resource type. See Viewing resource type exclusions on this page.

Overview of exclusions

  • Excluded log entries do not count against the Stackdriver Logging logs allotment provided to projects.

  • Excluded log entries are not visible in the Logs Viewer; they are not counted in logs-based metrics; and they are not available to Stackdriver Error Reporting or Stackdriver Debugger.

  • You can export log entries before they are excluded. For more information, see Exporting Logs.

  • You cannot exclude the default Cloud Auditing logs, which are free.

  • You cannot exclude AWS logs or any logs from sources other than GCP.

There are two kinds of exclusions:

  • Resource type exclusions are easy to use. They block all logs from specific resource types. For details, see Using resource type exclusions on this page.

  • Exclusion filters are more flexible. They select log entries for exclusion based on filter expressions. Exclusion filters can also choose a random sample of log entries to exclude. For details, see Using exclusion filters.

When deciding whether or not to exclude a log entry, Stackdriver Logging considers both kinds of exclusions. If any resource type exclusion or any exclusion filter matches the log entry, then that log entry is excluded.

Using resource type exclusions

By default, your project receives all logs from all resource types. To discard all the logs from specific resource types, use resource type exclusions.

Resource type exclusions are a convenience feature in the Logs Viewer. When you create a resource type exclusion, Stackdriver Logging actually creates an exclusion filter that implements the exclusion. For more information, see Using exclusion filters.

Viewing resource type exclusions

To view your logs usage by resource type and to see your resource type exclusions, do the following:

  1. Go to the Resource Usage page of the Logs Viewer.

    Go to the Resource Usage page

  2. Select the Resources tab (the default) under the summary information. You see your logs usage by resource type:

    Resource Usage

The table shows logs usage information for each resource type that has sent logs to your project this month. This information is updated approximately every two hours.

The Status column is an approximate indication of whether there are exclusions related to each resource type. The status can be any of the following:

  • Fully excluded: There are one or more exclusions that exactly target this resource type at a 100% sample rate. That means the exclusion's filter consists of exactly resource.type=[THIS_RESOURCE_TYPE].

  • No exclusions: There have been no log entries from this resource type excluded so far this month, and there are no exclusions that exactly target this resource type.

  • Partially excluded: There are one or more exclusions that target this resource type between 0 percent and 100 percent of the sample rate. If this resource type has had any log entries excluded this month, then this status remains until the end of the month, even if all exclusions are currently removed. See Editing exclusions for more information.

Alternatively, you can inspect resource type exclusions on the Exclusions tab of the Resources Page. Stackdriver Logging implements resource type exclusions by creating exclusion filters. See Viewing exclusion filters.

Creating resource type exclusions

To exclude all logs from a resource type, do the following:

  1. Go to the Resource Usage page of the Logs Viewer.

  2. Select the Resources tab (the default) under the statistics. You see your logs usage by resource type, as shown in the screenshot in the previous section.

  3. Locate the table row for the resource type you want to disable.

  4. Click Disable log source in the menu on the right side of the table row.

    To stop excluding logs from the resource type, click Enable log source in the menu.

Using exclusion filters

By creating exclusion filters you can control exactly which log entries you exclude. For example, you could exclude log entries from a single VM instance rather than from all VM instances.

If you use both exclusion filters and resource type exclusions, they can overlap. A log entry is excluded if it is from a disabled resource type, or if it matches one of the exclusion filters discussed in this section. This is a technical distinction, since as mentioned previously, Stackdriver Logging implements resource type exclusions with exclusion filters.

The per-resource type table on the Resources tab of the Resource Usage page reflects both resource type exclusions and exclusion filters. Even if you do not use resource type exclusions, that table lets you track the effect of your exclusion filters.

Viewing exclusion filters

To view your current exclusion filters—including the filters created by Stackdriver Logging to implement your resource type exclusions—do the following:

  1. Visit the Resource Usage page of the Logs Viewer. Select the project whose logs you want to manage.

    Go to the Resource Usage page

  2. Under the summary statistics, click the Exclusions tab. You see a list of your exclusion filters (if any):

    Exclusion Panel

Creating exclusion filters

To create an exclusion filter using the Logs Viewer, do the following:

  1. Visit the Resource Usage page of the Logs Viewer and select the Exclusions tab.

  2. Click Create Exclusion. You see the Exclusion Editor beside a Logs Viewer panel:

    Create Exclusion

  3. In the Logs Viewer panel, enter a filter expression that matches the log entries you want to exclude. For more information about the panel, see The user interfaces.

  4. In the Exclusion Editor, complete your exclusion filter by filling in the text boxes:

    • Name: A name that identifies the exclusion, such as low-severity.
    • Description: A longer description of the filter, such as Exclude logs whose severity is less than WARNING.
    • Percent to Exclude: Enter an integer from 0 to 100. For example, enter 100 to exclude all matching log entries. Enter 99 to exclude 99% of the matching entries but keep 1% of them in Stackdriver Logging.
  5. Click Create Exclusion to create and start the exclusion.

Tip: If you want to prevent your exclusion filter from being used, then select Disable in the menu next to your exclusion filter.

Stopping exclusions

To stop excluding some or all logs, do one or both of the following:

  • On the Resources tab of the Resource Usage page, click Enable log source in the menu to the right side of a resource type you want to stop excluding.

  • On the Exclusions tab of the Resource Usage page, delete or disable any exclusion filters that target log entries you want to receive.

Tip: Check all your exclusion filters, because the same log entries could be targeted by more than one filter.

Editing exclusions

You can edit your existing exclusion filters to exclude more or fewer log entries.

  1. Go to the Resource Usage page of the Logs Viewer and click the Exclusions tab.

  2. Choose an exclusion filter and select Edit Exclusion from the menu at the right side of the filter listing.

  3. Change the advanced logs filter or change the Percent to Exclude value. If you change the filter, check the preview of matching log entries. You cannot change the name of the exclusion filter.

  4. Click Update Exclusion.

Best practice: Do not edit exclusion filters created by Stackdriver Logging as part of the resource type exclusions. Manage those filters with the Disable log source and Enable log source options on the Resources tab.

Exclusions in the API

To create exclusion filters in the Stackdriver Logging API, use the projects.exclusions.create method. There are also methods to view, delete, and update exclusion filters. There are no API methods specifically for resource type exclusions.

There are also exclusion methods in the API for logs received by organizations, billing accounts, and folders. Those exclusions can only be created in the Stackdriver Logging API; they are not supported in the Logs Viewer.

For examples of logs filters that might be useful in exclusions, see Advanced Logs Filters.

Resource type exclusions in the API

Resource type exclusions are not a separate kind of exclusion in the API. To create an exclusion that discards all log entries from a particular resource type, create an exclusion filter with a logs filter that specifies the resource type:

resource.type = [THE_RESOURCE_TYPE]

Sampled exclusions in the API

To exclude less than 100 percent of the matched log entries, use the sample function in your logs filter.

Exclusion limits

You can have up to 50 exclusion filters in a project. This includes exclusion filters and resource type exclusions created in the Logs Viewer or in the API.

Exporting excluded logs

You can export log entries before you exclude them, so that you don't permanently lose the log entries you exclude.

To start your exclusion and export, do the following:

  1. Create an advanced logs filter that matches the log entries you want to exclude and export.

    Tip: Write the filter so that it does not match any default Cloud Auditing log entries. Matching the audit log entries doesn't matter for exclusions, but it does result in exporting more log entries.

  2. Create an export sink using your logs filter. Start exporting the matching log entries.

  3. Create an exclusion filter using your logs filter. Start excluding the matching log entries.

To stop your exclusions and export, stop the exclusion filter before you stop the export sink.

Monitor your resources on the go

Get the Google Cloud Console app to help you manage your projects.

Send feedback about...

Stackdriver Logging