Query and view logs overview

This document describes how you query, view, and analyze log entries by using the Google Cloud console. There are two interfaces available to you, the Logs Explorer and Log Analytics. You can query, view, and analyze logs with both interfaces; however, they use different query languages and they have different capabilities. For troubleshooting and exploration of log data, we recommend using the Logs Explorer. To generate insights and trends, we recommend that you use Log Analytics. You can also query your logs by issuing Logging API commands or by using the Google Cloud CLI.

Logs Explorer

The Logs Explorer is designed to help you troubleshoot and analyze the performance of your services and applications. For example, a histogram displays the rate of errors. If you see a spike in errors or something that is interesting, you can locate and view the corresponding log entries. When a log entry is associated with an error group, the log entry is annotated with a menu of options that let you access more information about the error group.

The same query language is supported by the Cloud Logging API, the Google Cloud CLI, and the Logs Explorer. To simplify query construction when you are using the Logs Explorer, you can build queries by using menus, by entering text, and, in some cases, by using options included with the display of an individual log entry.

The Logs Explorer doesn't support aggregate operations, like counting the number of log entries that contain a specific pattern. To perform aggregate operations, enable analytics on the log bucket and then use Log Analytics.

For details about searching and viewing logs with the Logs Explorer, see View logs by using the Logs Explorer.

Log Analytics

Using Log Analytics, you can run queries that analyze your log data, and then you can view or chart the query results. Charts, which are in Preview, let you identify patterns and trends in your logs over time. The following screenshot illustrates the charting capabilities in Log Analytics:

User interface for Log Analytics.

For example, suppose that you are troubleshooting a problem and you want to know the average latency for HTTP requests issued to a specific URL over time. When a log bucket is upgraded to use Log Analytics, you can use SQL queries to query logs stored in your log bucket. By grouping and aggregating your logs, you can gain insights into your log data which can help you reduce time spent troubleshooting.

Log Analytics also let you use BigQuery to query your data. For example, suppose that you want to use BigQuery to compare URLs in your logs with a public dataset of known malicious URLs. To make your log data visible to BigQuery, upgrade your bucket to use Log Analytics and then create a linked dataset.

You can continue to troubleshoot issues and view individual log entries in upgraded log buckets by using the Logs Explorer.

Restrictions

  • Not all regions are supported for Log Analytics. For more information, see Supported regions.

  • To upgrade an existing log bucket to use Log Analytics, the following restrictions apply:

    • The log bucket is unlocked unless it is the _Required bucket.
    • There aren't pending updates to the bucket.
    • Not all regions are supported for Log Analytics. For more information, see Supported regions.

  • On log buckets that are upgraded to use Log Analytics, you can't do any of the following:

    • Remove Log Analytics support.

  • You can delete the link to a linked BigQuery dataset. Deleting the link doesn't change your ability to query views on the log bucket by using the Log Analytics page.

  • Only log entries written after the upgrade has completed are available for analytics.

Pricing

  • Logging pricing:

    • There are no charges for queries issued through the Logs Explorer page or through the Log Analytics page.

    • There is no charge to upgrade a log bucket to use Log Analytics.

  • BigQuery pricing:

    • BigQuery analysis charges apply to SQL queries run from the BigQuery Studio page.

    • There are no BigQuery ingestion or storage costs when you upgrade a bucket to use Log Analytics and then create a linked dataset. When you create a linked dataset for a log bucket, you don't ingest your log data into BigQuery. Instead, you get read access to the log data stored in your log bucket through the linked dataset.

Blogs

For more information about Log Analytics, see the following blog posts:

What's next