Control log ingestion

Exclusion filters let you control the volume of Google Kubernetes Engine (GKE) logs ingested by Cloud Logging while still making verbose logging available for debugging. You can use exclusion filters to exclude matching log entries from being ingested by Cloud Logging or from being routed to the destination of a sink.

Create exclusion filters by using the Logging query language.

Log entries are excluded after they are received by the Logging API and therefore these log entries consume entries.write API quota. You can't reduce the number of entries.write API calls by excluding log entries.

Excluded log entries aren't available in the Logs Explorer or in Error Reporting.

For information about viewing ingestion data, see View detailed usage in Metrics Explorer.

Before you begin

Before you start, make sure you have performed the following tasks:

  • Enable the Google Kubernetes Engine API.
    • Enable Google Kubernetes Engine API
  • If you want to use the Google Cloud CLI for this task, install and then initialize the gcloud CLI. If you previously installed the gcloud CLI, get the latest version by running gcloud components update.
  • Verify that you have a Google Cloud project with logs that you can see in the Logs Explorer.

Permissions

Verify that you have one of the following IAM roles for the source Google Cloud project from which you're routing logs.

  • Owner (roles/owner)
  • Logging Admin (roles/logging.admin)
  • Logs Configuration Writer (roles/logging.configWriter)

The permissions contained in these roles let you create, delete, or modify sinks. For information on setting IAM roles, see the Logging Access control guide.

Add an exclusion filter

The following steps describe how to add a Cloud Logging exclusion filter that excludes logs from ingestion into the Default Cloud Logging bucket.

  1. In the Google Cloud console, go to the Logs Router page:

    Go to Logs Router

  2. Find the row with the _Default sink, expand the Actions option, and then click Edit sink.

  3. In Choose logs to filter out of sink, for Build an exclusion filter, click Add exclusion.

  4. Enter a name for your exclusion filter.

  5. In the Build an exclusion filter section, enter the filter criteria. For example, you can exclude log entries for a cluster, or specific log entries for a container.

    Exclude all log entries from a specific cluster:

    resource.labels.cluster_name="CLUSTER_NAME"

    Exclude log entries with the log severity DEFAULT, DEBUG, INFO, or NOTICE for a specific container in a specific cluster:

    resource.type="k8s_container"
resource.labels.container_name="CONTAINER_NAME"
resource.labels.cluster_name="CLUSTER_NAME"
severity=(DEFAULT OR DEBUG OR INFO OR NOTICE)

  6. Click Update sink.

The _Default sink is now configured to exclude logs with the filter you added.

What's next