This document describes how you query, view, and analyze log entries by using the Google Cloud console. There are two interfaces available to you, the Logs Explorer and Log Analytics. You can query, view, and analyze logs with both interfaces; however, they use different query languages and they have different capabilities. For troubleshooting and exploration of log data, we recommend using the Logs Explorer. To generate insights and trends, we recommend that you use Log Analytics. You can also query your logs by issuing Logging API commands or by using the Google Cloud CLI.
Logs Explorer
The Logs Explorer is designed to help you troubleshoot and analyze the performance of your services and applications. For example, a histogram displays the rate of errors. If you see a spike in errors or something that is interesting, you can locate and view the corresponding log entries. When a log entry is associated with an error group, the log entry is annotated with a menu of options that let you access more information about the error group.
The same query language is supported by the Cloud Logging API, the Google Cloud CLI, and the Logs Explorer. To simplify query construction when you are using the Logs Explorer, you can build queries by using menus, by entering text, and, in some cases, by using options included with the display of an individual log entry.
The Logs Explorer doesn't support aggregate operations, like counting the number of log entries that contain a specific pattern. To perform aggregate operations, enable analytics on the log bucket and then use Log Analytics.
For details about searching and viewing logs with the Logs Explorer, see View logs by using the Logs Explorer.
Log Analytics
Using Log Analytics, you can run queries that analyze your log data to generate useful insights. For example, suppose that you are troubleshooting a problem and you want to know the average latency for HTTP requests issued to a specific URL over time. When a log bucket is upgraded to use Log Analytics, you can use SQL queries to query logs stored in your log bucket. By grouping and aggregating your logs, you can gain insights into your log data which can help you reduce time spent troubleshooting.
Log Analytics also let you use BigQuery to query your data. For example, suppose that you want to use BigQuery to compare URLs in your logs with a public dataset of known malicious URLs. To make your log data visible to BigQuery, upgrade your bucket to use Log Analytics and then create a linked dataset.
You can continue to troubleshoot issues and view individual log entries in upgraded log buckets by using the Logs Explorer.
Log Analytics feature summary
- Bucket-level configuration: You upgrade your Logging buckets to use Log Analytics.
- A new interface: Query your logs data with the BigQuery standard SQL by using the Log Analytics page of the Google Cloud console. Your query results are displayed in a layout optimized for viewing logs data.
- Easy storage: Your logs data is stored and managed by Logging.
- View logs from BigQuery: You can view your logs data by using BigQuery when you create a linked dataset.
Restrictions
Not all regions are supported for Log Analytics. For more information, see Supported regions.
To upgrade an existing log bucket to use Log Analytics, the following restrictions apply:
- Field-level access control isn't configured.
- The log bucket is unlocked unless it is the
_Requiredbucket. - There aren't pending updates to the bucket.
- Not all regions are supported for Log Analytics. For more information, see Supported regions.
On log buckets that are upgraded to use Log Analytics, you can't do any of the following:
- Configure field-level access control.
- Remove Log Analytics support.
You can delete the link to a linked BigQuery dataset. Deleting the link doesn't change your ability to query views on the log bucket by using the Log Analytics page.
Only log entries written after the upgrade has completed are available for analytics.
Pricing
-
There are no pricing differences between routing to log buckets that don't use Log Analytics and routing to log buckets that have been upgraded to use Log Analytics.
Queries issued through the Logs Explorer page and through the Log Analytics page are free.
-
BigQuery analysis charges apply to SQL queries run from the BigQuery SQL workspace page.
There are no BigQuery ingestion or storage costs when you upgrade a bucket to use Log Analytics and then create a linked dataset. When you create a linked dataset for a log bucket, you don't ingest your log data into BigQuery. Instead, you get read access to the log data stored in your log bucket through the linked dataset.
Blogs
For more information about Log Analytics, see the following blog posts:
- For an overview of Log Analytics, see Log Analytics in Cloud Logging is now GA.
- To learn about analyzing audit logs by using Log Analytics, see Gleaning security insights from audit logs with Log Analytics.
- If you currently route logs to BigQuery and want to understand the difference between that solution and using Log Analytics, then see Moving to Log Analytics for BigQuery export users.
What's next
- Create a log bucket and upgrade it to use Log Analytics
- Upgrade an existing bucket to use Log Analytics
Build queries:
Sample queries: