Managing Logs Buckets

This page describes how to create and manage Logs Buckets. Logs Buckets are storage containers in your Google Cloud projects that hold your logs data. You can create logs sinks to route all, or just a subset, of your logs to any Logs Bucket. This flexibility allows you to choose which Google Cloud project your logs are stored in and what other logs are stored with them.

Overview

For each Google Cloud project, Logging automatically creates two Logs Buckets: _Required and _Default. All logs generated in the project are stored in the _Required and _Default Logs Buckets, which live in the project that the logs are generated in. The following describes the role and purpose of the _Required and _Default buckets:

  • _Required: This bucket holds Admin Activity audit logs, System Event audit logs, and Access Transparency logs, and retains them for 400 days. You aren't charged for the logs stored in _Required, and the retention period of the logs stored here cannot be modified. You cannot delete this bucket.

  • _Default: This bucket holds all other ingested logs in a Google Cloud project except for the logs held in the _Required bucket. Standard Cloud Logging pricing applies to these logs. Log entries held in the _Default bucket are retained for 30 days, unless you apply custom retention rules. You can't delete this bucket, but you can disable the _Default log sink that routes logs to this bucket.

For these buckets, Logging automatically creates log sinks named _Required and _Default that route logs to the corresponding buckets.

Logs Buckets only have regional availability, including those created in the global region. Setting location to global means that Logging doesn't specify where it physically stores the logs.

You can configure custom retention periods as needed on each of your custom Logs Buckets and the _Default bucket.

Logging also creates some default views that can be used to access logs in a bucket:

  • The _AllLogs view is available on all buckets and shows all logs in the bucket.

  • The _Default view is only available for the _Default bucket and shows all logs except for Data Access audit logs.

For more information on how Cloud Logging routes and stores your logs data, see Logs Router overview. For information on the Logs Bucket API methods, refer to the LogBucket reference documentation.

Limitations

While Logs Buckets are generally available, some features are only available in the alpha or Preview stages. Be aware of the following limitations:

  • Custom views on a bucket are currently in Preview.

  • Bucket regionalization is currently available for a limited set of regions in the alpha.

  • To sign up for one or both of these features or to get notified when the Preview version is released, fill out this form.

Managing buckets

Using the gcloud command-line tool and the Google Cloud Console, you can create, update, and delete your custom Logs Buckets.

Creating a Logs Bucket

To create a custom Logs Bucket for your project, complete the following steps. You can create a maximum of 10 buckets per project.

gcloud

To create a bucket in your project, run the gcloud alpha logging buckets create command:

gcloud alpha logging buckets create BUCKET_ID --location=LOCATION OPTIONAL_FLAGS

For example:

gcloud alpha logging buckets create my-bucket --location global --description "My first bucket"

Console

To create a bucket in your project, complete the following steps:

  1. From the Logging menu, select Logs Storage.

    Go to Logs Storage

  2. Click Create Logs Bucket.

  3. Enter a Name and Description for your bucket.

  4. Optionally, to set a custom retention period or bucket region, select Next.

  5. In the Retention field, enter the number of days, between 1 to 3650 days, that you want Cloud Logging to retain your logs.

  6. Select your bucket's region by clicking the Select Logs Bucket Region drop-down menu and selecting the region in which you want your bucket.

  7. Click Create bucket. Your new bucket appears in the Logs bucket list.

After creating a bucket, you can configure Logs Views to control who can access the logs in your new bucket and which logs are accessible to them.

Updating a Logs Bucket

To update the attributes of your bucket, complete the following steps.

gcloud

To update your bucket's attributes, run the gcloud alpha logging buckets update command:

gcloud alpha logging buckets update BUCKET_ID --location=LOCATION UPDATED_ATTRIBUTES

For example:

gcloud alpha logging buckets update my-bucket --location=global --description "Updated description"

Console

To update your bucket's attributes, complete the following steps:

  1. From the Logging menu, select Logs Storage.

    Go to Logs Storage

  2. For the bucket you want to update, click More .

  3. Select Edit bucket.

  4. Edit your bucket as needed.

  5. Click Update bucket.

Locking a Logs Bucket

You can lock a bucket to prevent anyone from updating or immediately deleting it. To lock a bucket, complete the following steps.

GCLOUD

To lock your bucket, run the gcloud alpha logging buckets update command with the --locked flag:

gcloud alpha logging buckets update BUCKET_ID --location=LOCATION --locked

For example:

gcloud alpha logging buckets update my-bucket --location=global --locked

Viewing Logs Buckets

To list the Logs Buckets associated with a Google Cloud project, do the following.

gcloud

To list your project's Logs Buckets, run the gcloud alpha logging buckets list command:

gcloud alpha logging buckets list

Logs Buckets have the following attributes:

  • LOCATION
  • BUCKET_ID
  • RETENTION_DAYS
  • LIFECYCLE_STATE
  • LOCKED
  • CREATE_TIME
  • UPDATE_TIME
  • ANALYTICS_ENABLED

To view the details for a specific bucket, run the gcloud alpha logging buckets describe command. For example, to view the details for the _Default Logs Bucket, run this command:

gcloud alpha logging buckets describe _Default --location=global

Console

The Logs Storage page displays your project's Logs Buckets.

Go to Logs Storage

Logs Buckets have the following attributes:

  • Name
  • Description
  • Retention period
  • Region
  • Status

To view the details for a specific bucket, click More and select View bucket details.

Deleting a Logs Bucket

To delete a Logs Bucket, complete the following steps.

gcloud

To delete a Logs Bucket, run the gcloud alpha logging buckets delete command:

gcloud alpha logging buckets delete BUCKET_ID --location=LOCATION

Console

To delete a Logs Bucket, complete the following steps:

  1. From the Logging menu, select Logs Storage.

    Go to Logs Storage

  2. For the bucket you want to delete, click More .

  3. Select Delete bucket.

  4. On the confirmation panel, click Delete.

  5. On the Logs Storage page, your bucket has an indicator that it's pending deletion. The bucket, including all the logs in it, is deleted after 7 days.

Restoring a deleted Logs Bucket

You can restore, or undelete, a bucket that's in the pending deletion state.

GCLOUD

To restore a Logs Bucket that is pending deletion, run the gcloud alpha logging buckets undelete command:

gcloud alpha logging buckets undelete BUCKET_ID --location=LOCATION

CONSOLE

To restore a Logs Bucket that is pending deletion, complete the following steps:

  1. From the Logging menu, select Logs Storage.

    Go to Logs Storage

  2. For the bucket you want to restore, click More .

  3. Select Restore deleted bucket.

  4. On the confirmation panel, click Restore.

  5. On the Logs Storage page, the pending-deletion indicator is removed from your bucket.

Troubleshooting and common questions

If you encounter problems when using Logs Buckets, refer to the following troubleshooting steps and answers to common questions.

Why do I see logs for a project even though I excluded them from my _Default sink?

If you're accessing logs in a centralized project and see logs that you excluded from the _Default sink, you might be viewing the logs under one of the following conditions:

  • Viewing the logs using the Legacy Logs Viewer, which doesn't support viewing centralized logs.

  • Viewing the logs using the Logs Explorer with Scope by project selected in the Refine scope panel, which shows you logs generated by the project regardless of where you store them.

To verify that you correctly excluded the logs, you can select Scope by storage in the Refine scope panel for the Logs Explorer and select the _Default bucket in your project. You shouldn't see the excluded logs anymore.

What's next

For information on addressing common use cases with Logs Buckets, refer to the following documentation: