Cloud Logging storage overview

This page describes how Cloud Logging stores your log entries.

Log buckets

Cloud Logging uses log buckets as containers in your Google Cloud projects to store and organize your logs data. The logs that you store in Cloud Logging are indexed, optimized, and delivered to let you analyze your logs in real time. These are different storage entities than the similarly named Cloud Storage buckets.

For each Cloud project, Logging automatically creates two log buckets: _Required and _Default. Logging automatically creates log sinks named _Required and _Default that route logs to the correspondingly named buckets. Note that you can disable or limit the logs that are routed to the _Default log bucket.

Additionally, you can create user-defined buckets for any Google Cloud project.

For more information on how Cloud Logging routes and stores your logs data, see Log Router overview.

_Required log bucket

Cloud Logging automatically routes the following types of logs to the _Required bucket:

• Admin Activity audit logs
• System Event audit logs
• Access Transparency logs

Cloud Logging retains the logs in this bucket for 400 days; you can't change this retention period.

You can't modify or delete the _Required bucket. You can't disable the _Required sink, which routes logs to the _Required bucket.

Neither ingestion pricing nor storage pricing applies to the logs data stored in the _Required log bucket.

_Default log bucket

Any log entry that isn't ingested by the _Required bucket is routed by the _Default sink to the _Default bucket, unless you disable or otherwise edit the _Default sink. For instructions on modifying sinks, see Managing sinks.

You can't delete the _Default bucket.

Logs held in the _Default bucket are retained for 30 days, unless you configure custom retention for the bucket.

Cloud Logging pricing applies to the logs data held in the _Default bucket.

User-defined log buckets

You can also create user-defined log buckets in any Cloud project. By applying log sinks to your user-defined log buckets, you can route any subset of your logs to any log bucket, letting you choose which Cloud project your logs are stored in and which other logs are stored with them.

For example, for any log generated in Project-A, you can configure a sink to route that log to user-defined buckets in Project-A or Project-B.

Cloud Logging pricing applies to the logs data held in this bucket, regardless of the log type.

You can configure custom retention for the bucket.

For information on managing your user-defined log buckets, including deleting or updating them, see Managing buckets.

Organization policy

You can create an organization policy to ensure that your organization meets your compliance and regulatory needs. Using an organization policy, you can specify in which regions your organization can create new log buckets. You can also restrict your organization from creating new log buckets in specified regions.

Cloud Logging doesn't enforce your newly created organization policy on your existing log bucket; it only enforces the policy on new log buckets.

For information on creating a location-based organization policy, refer to Restricting resource locations.

Logs retention

Note: Effective October 1, 2021, storage costs will apply to logs data retained longer than 30 days. For pricing, see the Cloud Logging pricing summary . Currently, charges for retaining logs past 30 days are not enforced.

Cloud Logging retains logs according to retention rules applying to the log bucket type where the logs are held.

You can configure Cloud Logging to retain logs between 1 day and 3650 days. Custom retention rules apply to all the logs in a bucket, regardless of the log type or whether that log has been copied from another location.

For information on setting retention rules for a log bucket, see Configuring custom retention.

Log views

Log views let you control who has access to the logs within your log buckets. Custom log views provide you with a granular way to control access to the logs in those buckets.

For more information on log views, see Managing log views.

Logs availability

After you create a log sink to route log entries to a log bucket, Logging begins routing log entries within a few minutes.

Routing frequency

Logging routes log entries to log buckets in real time.

Regionalization

When you create your log bucket, you can choose to store your logs in any of the following regions:

Continent	Regions
Asia	asia-east1 asia-northeast1 asia-northeast2 asia-northeast3 asia-south1 asia-southeast1
Australia	australia-southeast1
Europe	europe-north1 europe-west1 europe-west2 europe-west3 europe-west4 europe-west6
North America	northamerica-northeast1 us-central1 us-east1 us-east4 us-west1 us-west2 us-west3
South America	southamerica-east1

In addition to these regions, you also have the option to set the location to global, which means that you don't need to specify where your logs are physically stored.

Note: After you create your log bucket, you can't change your bucket's region. If you need a bucket in a different region, you must create a new bucket in that region, redirect the appropriate sinks to the new bucket, and then delete the old bucket.

Note: Error Reporting is a global service built on Cloud Logging and does not analyze logs stored in regional log buckets or logs routed to other projects.
For more information, see Using Error Reporting with regionalized logs.

Stopping logs ingestion

To learn how to stop ingesting logs into your Google Cloud project, see Stopping logs ingestion.

Limitations

Be aware of the following limitations:

• Custom views on a log bucket are currently in Preview.

• You can't create logs-based metrics at the log bucket-level. These metrics are calculated by the Log Router and apply to both ingested and excluded logs only in the Cloud project in which they're received.

• Error Reporting is a global service built on Cloud Logging and doesn't analyze logs stored in a regional log buckets or logs routed to other Cloud projects.

  For more information, see Using Error Reporting with regionalized logs.

What's next

For information on addressing common use cases with log buckets, refer to the following documentation:

• Aggregate your organization's log into a central logs bucket.

• Regionalize your Google Cloud project's logs using logs buckets.

• Configure multi-tenant logging for Google Kubernetes Engine (GKE) clusters.

• Read the following whitepapers that provide best practices for data governance:

  • Logs data: A step by step guide for overcoming common compliance challenges

  • Data governance: Principles for securing and managing logs