Log Analytics

Stay organized with collections Save and categorize content based on your preferences.

Using Log Analytics, you can run queries that analyze your log data to generate useful insights. For example, suppose that you are troubleshooting a problem and you want to know the average latency for HTTP requests issued to a specific URL over time. When a log bucket is upgraded to use Log Analytics, you can use SQL queries to query logs stored in your log bucket. By grouping and aggregating your logs, you can gain insights into your log data which can help you reduce time spent troubleshooting.

Log Analytics also let you use BigQuery to query your data. For example, suppose that you want to compare URLs in your logs with a public dataset of known malicious URLs. To perform this analysis, you must analyze your logs with data stored outside of Logging. You can view the data stored in a log bucket when you've upgraded the bucket to use Log Analytics and then create a linked dataset. By using the linked dataset, you can join your log data with other data that is accessible to BigQuery. For example, this data could be a dataset that stores known malicious URLs, or data generated from business intelligence tools like Looker and Looker Studio.

Upgrading a log bucket to use Log Analytics doesn't restrict your access to the Logs Explorer. You can continue to troubleshoot issues and view individual log entries in these buckets by using the Logs Explorer.

If you currently route logs to BigQuery and want to understand the difference between that solution and using Log Analytics, then see the blog post Moving to Log Analytics for BigQuery export users.

Log Analytics feature summary

  • Bucket-level configuration: You upgrade your Logging buckets to use Log Analytics.
  • A new interface: Query your logs data with the BigQuery standard SQL by using the Log Analytics page of the Google Cloud console. Your query results are displayed in a layout optimized for viewing logs data.
  • Easy storage: Your logs data is stored and managed by Logging.
  • View logs from BigQuery: You can view your logs data by using BigQuery when you create a linked dataset.

User interface for Log Analytics.

Pricing

  • Logging pricing:

    • There are no pricing differences between routing to log buckets that don't use Log Analytics and routing to log buckets that have been upgraded to use Log Analytics.

    • SQL queries run from the Log Analytics page are free.

  • BigQuery pricing:

    • BigQuery analysis charges apply to SQL queries run from the BigQuery SQL workspace page.

    • There are no BigQuery ingestion or storage costs when you use a linked dataset. When you create a linked dataset for a log bucket, you don't ingest your log data into BigQuery. Instead, you get read access to the log data stored in your log bucket through the linked dataset.

Restrictions

  • To create a log bucket and upgrade it to use Log Analytics, the following restrictions apply:

    • You must use the Google Cloud console.
    • You must create the log bucket at the Google Cloud project level.
    • The log bucket must use the global region.
    • The retention period for the bucket must be set to the default value.
  • To upgrade an existing log bucket to use Log Analytics, the following restrictions apply:

    • You must use the Google Cloud console.
    • The log bucket must be at the Google Cloud project level.
    • The log bucket must use the global region.
    • The retention period for the bucket must be set to the default value.
    • Customer-managed encryption keys (CMEK) are disabled.
    • Field-level access control isn't configured.
    • The log bucket is unlocked unless it is the _Required bucket.
    • There aren't pending updates to the bucket.
  • On log buckets that are upgraded to use Log Analytics, you can't do any of the following:

    • Change the retention period.
    • Configure field-level access control.
    • Remove Log Analytics support.
  • You can delete the link to a linked BigQuery dataset. Deleting the link doesn't change your ability to query views on the log bucket by using the Log Analytics page.

  • Only log entries written after the upgrade has completed are available for analytics.

What's next