This page describes how Cloud Logging stores your log entries.
Cloud Logging uses log buckets as containers in your Google Cloud projects to store and organize your logs data. The logs that you store in Cloud Logging are indexed, optimized, and delivered to let you analyze your logs in real time. These are different storage entities than the similarly named Cloud Storage buckets.
For each Cloud project, Logging automatically
creates two log buckets:
automatically creates log sinks named
_Default that route
logs to the correspondingly named buckets. Note that you can disable or limit
the logs that are routed to the
_Default log bucket.
Additionally, you can create user-defined buckets for any Google Cloud project.
For more information on how Cloud Logging routes and stores your logs data, see Log Router overview.
_Required log bucket
Cloud Logging automatically routes the following types of logs to the
Cloud Logging retains the logs in this bucket for 400 days; you can't change this retention period.
You can't modify or delete the
_Required bucket. You can't disable the
_Required sink, which routes logs to the
Neither ingestion pricing nor storage pricing applies to the logs data stored in
_Required log bucket.
_Default log bucket
Any log entry that isn't ingested by the
_Required bucket is routed by the
_Default sink to the
_Default bucket, unless you disable or otherwise edit
_Default sink. For instructions on modifying sinks, see
You can't delete the
Logs held in the
_Default bucket are retained for
30 days, unless you
configure custom retention for the
Cloud Logging pricing applies
to the logs data held in the
User-defined log buckets
You can also create user-defined log buckets in any Cloud project. By applying log sinks to your user-defined log buckets, you can route any subset of your logs to any log bucket, letting you choose which Cloud project your logs are stored in and which other logs are stored with them.
For example, for any log generated in Project-A, you can configure a sink to route that log to user-defined buckets in Project-A or Project-B.
Cloud Logging pricing applies to the logs data held in this bucket, regardless of the log type.
You can configure custom retention for the bucket.
For information on managing your user-defined log buckets, including deleting or updating them, see Managing buckets.
You can create an organization policy to ensure that your organization meets your compliance and regulatory needs. Using an organization policy, you can specify in which regions your organization can create new log buckets. You can also restrict your organization from creating new log buckets in specified regions.
Cloud Logging doesn't enforce your newly created organization policy on your existing log bucket; it only enforces the policy on new log buckets.
For information on creating a location-based organization policy, refer to Restricting resource locations.
Cloud Logging retains logs according to retention rules applying to the log bucket type where the logs are held.
You can configure Cloud Logging to retain logs between 1 day and 3650 days. Custom retention rules apply to all the logs in a bucket, regardless of the log type or whether that log has been copied from another location.
For information on setting retention rules for a log bucket, see Configuring custom retention.
Log views let you control who has access to the logs within your log buckets. Custom log views provide you with a granular way to control access to the logs in those buckets.
For more information on log views, see Managing log views.
After you create a log sink to route log entries to a log bucket, Logging begins routing log entries within a few minutes.
Logging routes log entries to log buckets in real time.
When you create your log bucket, you can choose to store your logs in any of the following regions:
In addition to these regions, you also have the option to set the location to
global, which means that you don't need to specify where your logs are
Stopping logs ingestion
To learn how to stop ingesting logs into your Google Cloud project, see Stopping logs ingestion.
Be aware of the following limitations:
Custom views on a log bucket are currently in Preview.
You can't create logs-based metrics at the log bucket-level. These metrics are calculated by the Log Router and apply to both ingested and excluded logs only in the Cloud project in which they're received.
Error Reporting is a global service built on Cloud Logging and doesn't analyze logs stored in a regional log buckets or logs routed to other Cloud projects.
For more information, see Using Error Reporting with regionalized logs.
For information on addressing common use cases with log buckets, refer to the following documentation:
Read the following whitepapers that provide best practices for data governance: