Regionalizing your project's logs using logs buckets

This page describes how to store your logs in a Cloud Logging bucket in a designated region. To meet regulatory or contractual obligations, organizations might be required to store their logs in specific regions. This guide walks through this process using the example of redirecting all logs to the europe-west1 region.

This process involves the following steps:

1. Creating a logs bucket in the designated region for storing the logs.

2. Redirecting the _Default sink to route the logs to the new logs bucket.

3. Searching logs in the UI.

4. Optionally updating the log retention period.

Before you begin

To complete the steps in this guide, you need to know the following:

• What project do you want to store the logs in? In this example, we use a project called logs-test-project.

• What is the name and location of the logs bucket you want store the logs in? In this example, the bucket name is region-1-logs-bucket and the location is europe-west1.

  When you create your logs bucket, you can choose to store your logs in any of the following regions:

  • asia-east1
  • europe-west1
  • us-central1
  • us-east1
  • us-west1

• Which logs do you want to include? In this example, we include all logs routed by the _Default sink.

Note: Error Reporting is a global service built on Cloud Logging and does not analyze logs stored in regional log buckets or logs routed to other projects.
For more information, see Using Error Reporting with regionalized logs.

Creating the logs bucket

Logs buckets store the logs that are routed from other projects, folders, or organizations. For more information, see Logs buckets.

To create the bucket in the project that you want to store logs in, complete the following steps:

1. Open the Google Cloud Console in the project you're using to store the logs.

   Go to Google Cloud Console

2. In a terminal, run the following command to create a bucket, replacing the parts in bold with your own information:

    gcloud logging buckets create region-1-logs-bucket \
      --location=europe-west1 \
      --project=logs-test-project
   

   Note: Once you create your bucket, you can't change your bucket's region.

3. Verify that the bucket was created:

   gcloud logging buckets list --project=logs-test-project
   

Redirecting the _Default logs sink

You route logs to a logs bucket by creating a logs sink. A sink includes a logs filter, which selects which log entries to export through the sink, and a destination. In this guide, we update the existing _Default sink to route logs to our bucket, region-1-logs-bucket.

Note: Redirecting the _Default sink ensures that all future logs go into the regionalized log bucket rather than the default bucket, which is global. You can't change the _Required sink, with the logs going into a different global bucket.

To update the sink, run the following command, replacing the parts in bold with your own information:

gcloud logging sinks update _Default \
  logging.googleapis.com/projects/logs-test-project/locations/europe-west1/buckets/region-1-logs-bucket \
  --log-filter='NOT LOG_ID("cloudaudit.googleapis.com/activity") AND NOT
    LOG_ID("externalaudit.googleapis.com/activity") AND NOT
    LOG_ID("cloudaudit.googleapis.com/system_event") AND NOT
    LOG_ID("externalaudit.googleapis.com/system_event") AND NOT
    LOG_ID("cloudaudit.googleapis.com/access_transparency") AND NOT
    LOG_ID("externalaudit.googleapis.com/access_transparency")'
  --description="Updated the _Default sink to route logs to the europe-west1 region"

Note: The log-filter shown in the command is the default filter for the _Default sink. The _Required sink routes these logs, so the _Default sink routes all other logs besides these to avoid overlap.

Creating a log entry to assist with sink verification

To verify that you updated the sink properly, complete the following steps:

1. Send a test log message to your regionalized bucket using the gcloud logging write command. For example:

   gcloud logging write TEST_LOG_NAME "Test to route logs to region-1-logs-bucket" --project=logs-test-project
   

2. After a few minutes, view your log entry in Logs Explorer:

   Go to Logs Explorer

3. In the Log field pane, select the Global resource type.

4. Your test log entry displays in the Query results panel.

Searching logs in the UI

After setting the permissions in the previous section, go to the UI and complete the following steps.

1. From the Logging menu for the project you're using to store the logs, select Logs Explorer.

   Go to Logs Explorer

2. Select Refine Scope.

3. On the Refine scope panel, select Scope by storage.

4. Select region-1-logs-bucket.

5. Click Apply.

6. The Logs Explorer refreshes to show logs from your bucket.

   For information on using the Logs Explorer, refer to Using the Logs Explorer.

[Optional] Updating the bucket's log retention period

To change the retention period for your logs in your bucket, run the following command:

gcloud logging buckets update region-1-logs-bucket \
  --location=europe-west1 --project=logs-test-project \
  --retention-days=14

Note: Run into an issue or have feedback or a question? Feel free to engage with us in our forum, [e-mail].