Using the Logs Explorer

Before you begin

You don't need a Workspace to use Logging, unless you're sending logs from Amazon Web Services (AWS) to Logging.

If you're using a Workspace, Logging doesn't combine the logs from the monitored Google Cloud projects. You must select a specific Cloud project to view its logs.

If you're using a Workspace and AWS, select the AWS connector project to see the AWS logs.

Getting started

To navigate to the Logs Explorer, do the following:

  1. Go to the Google Cloud navigation menu and select Logging > Logs Explorer:
    Go to the Logs Explorer
  2. Select a Google Cloud project.
  3. From the Upgrade menu, switch from Legacy Logs Viewer to Logs Explorer.

You're now in the Logs Explorer.

Logs Explorer interface

The Logs Explorer interface lets you retrieve logs, parse and analyze log data, and refine your query parameters.

User interface for the Logs Explorer

  1. Logs Explorer page: Lets you build, analyze, and refine queries.
  2. Organization and project selector: Lets you view logs at an organization or project level.
  3. Query builder: Lets you build queries using either the drop-down menus or the query builder language. It also features tabs for viewing your Saved and Recent queries.
  4. Log fields (beta): Lets you see aggregation-based results for the resource.type, resource.labels, logName, and severity fields, and provides a more efficient way to refine a query.
  5. Histogram: Lets you visualize the frequency of your logs data.
  6. Query results: Lets you view the retrieved logs from your query.
  7. Log entries: Lets you view log entries in the structured JSON format.
  8. Time zone: Lets you change the time zone that logs are displayed in.
  9. Page layout: Lets you enable and disable the Histogram and Logs field explorer panels.
  10. Time-range selector: Lets you restrict results by time range. The default time range is one hour.
  11. Run query: Lets you run your queries after you have built them in the query-builder pane.
  12. Jump to now: Lets you perform a forced refresh to include the current time. If the time-range selector uses a custom range and an end time is set, it runs the query with a default time range of one hour. Otherwise, it refreshes with the current start date or duration, and runs the query.
  13. Actions: Lets you set up a logs-based metric, create a sink destination, or download your logs.
  14. Configure: Lets you add the value of a log field to the summary line at the beginning or end of the log entry. It also lets you choose to show newest logs either first or last.
  15. Hide log summary: Lets you hide the log summary line from the query results.
  16. Expand or collapse nested log fields: Lets you expand or collapse nested fields.
  17. Copy to clipboard: Lets you copy the log entry in its JSON format.
  18. Save: Lets you save queries that can be viewed and run from the Saved tab.
  19. Trace data: Lets you view trace details and refine your query based on the trace.
  20. Expand and collapse query results: Lets you expand the query-results pane to view more log entries.
  21. Adjust time range: Lets you change the time range used for queries by adjusting the handles. After adjusting the handles, click Run to update the time range used in the query.
  22. Refine scope: Lets you scope your search by logs in your current project only or by one or more storage views.

  23. Pin log entry: Lets you pin a log entry to the Query results and Histogram panes. Depending on how your Query results pane is configured, Logging pins the log either to the top or to the bottom of the Query results pane.

  24. Copy link to a log entry: Lets you share a link to a log entry.

  25. Histogram viewport: Lets you see the timeframe of the logs that are currently displayed within the Query results pane.

  26. Share link: Lets you create a shortened URL of the current query and copies it to your clipboard, making it easier to share a query.

Within the query-results pane, you can click the values of a field to choose to do the following:

  1. Show matching entries: Lets you query for matching log entries.
  2. Hide matching entries: Lets you query for log entries that do not match the selected expression.
  3. Add field to summary line: Lets you add the field as a summary line to log entries.

Options after selecting field's value

Refine scope

You can refine the scope of the logs displayed in the Logs Explorer through the Refine scope panel. You have the option to only search logs within the current project or to search logs based on one or more storage views.

To refine the scope of the Logs Explorer, complete the following steps.

  1. From the Logging menu, select Logs Explorer.

    Go to Logs Explorer

  2. Select Refine Scope.

  3. On the Refine scope panel, select a Scope by option.

    The Refine scope panel

    • Scope by project allows you to search logs that the current project generates.

    • Scope by storage allows you to search logs based on one or more storage views. For more information about Logs Views, see Managing Logs Views on your Logs Buckets.

  4. If you select Scope by storage, select one or more buckets you want to view.

    The panel lists storage views that meet the following conditions:

    • The user has access to the storage view.
    • The buckets belong to the selected project, or the selected project has routed logs to the storage buckets.
  5. Click Apply.

Add summary fields

Summary fields help you notice patterns in your logs faster. For example, the following image shows the value for the summary field resource.labels.pod_name added before the logs that contain that value.

The Logs Explorer is showing logs that are preceded with green text displaying
pod names.

Add a summary field from a log entry

To add a summary field to a log entry, complete the following steps:

  1. Expand a log entry by clicking the expand button .

  2. Click a field's value and then select Add field to summary line.

    The summary field now appears before the log entries containing that field.

Add a summary field using the Configure button

To add a summary field using the Configure button, complete the following steps:

  1. Click Configure and select Manage Summary Fields.

    Manage summary fields is selected from the configure drop-down menu

  2. Add fields.

    The summary field selection has the following features:

    • Autocomplete using the logs currently displayed.
    • Field correction for legal characters within quotes.

      For example, if you type jsonPayload.id-field, it gets changed to jsonPayload."id-field".

  3. Click Truncate summary fields to shorten the display of the summary field values. Then choose how many characters to display before the field is truncated and whether the beginning or the end of the field is displayed.

  4. Click Apply.

    The summary field now appears before the log entries containing that field.

Log fields panel

The Log fields panel offers a high-level summary of logs data and provides a more efficient way to refine a query. It shows the count of log entries, sorted by decreasing count, for the given log field. The log field counts correspond to the time range used by the Histogram panel.

The Log fields panel is populated and updated based on an executed query. When there is an empty query, the Log fields panel displays counts of log entries by resource type and log severity fields.

The Log fields panel shows log field data.

Using the Log fields panel

You can add fields from the Log fields panel to the Query builder to narrow down and refine a query. To do so, click on a field value in the Log fields panel. This adds the log field to the Query builder and automatically runs the query by adding it as an expression to the original query using the AND operator.

When a query is run, the log field counts are incrementally loaded as the log entries are progressively scanned. Once the query is complete, which is indicated by the termination of the blue progress bar, you see the total counts for all log fields.

Histogram panel

The histogram panel lets you visualize the distribution of logs over time. This makes it easier to see trends in your logs data and troubleshoot problems.

Enabling the histogram panel

To enable the histogram panel, select Page Layout, and then select the Histogram checkbox. The Histogram panel appears.

Page layout is open and histograms is selected

To disable the histogram panel, clear the Histogram checkbox.

Using the histogram panel

A histogram is generated when you run a query. It displays the frequency of matching log entries for the selected time range.

Analyzing logs

To analyze your log data, hover over a bar in the Histogram panel and select Jump to time to drill into a narrower time range. This runs a new query with that time-range restriction.

Logs histograms showing popup dialog to jump to time

The Histogram panel features a viewport that reflects the time range of the logs displayed in the Query results pane. The viewport helps to orient you to the logs you're currently viewing within the larger timeframe of your query.

The viewport's size is based on the duration between the maximum and minimum timestamp of the log entries displayed in the Query results pane.

Histogram panel is showing the viewport.

Trace data

When a log entry contains both the trace and the latency-related field, both the latency and trace icon appear.

Log entry display that contains trace data.

When a log entry contains only the trace field, then only the trace icon appears.

Log entry display that contains only the trace field has trace icon.

To view the trace data related to the log entry, click the trace icon. You have the following options:

  • View trace details: Shows the parent span and child traces along with details about the trace. To view more details about the trace, navigate to Cloud Trace by clicking View in Trace. For more information about the content in the flyout panel, see Viewing trace details.
  • Show all logs for this trace: Refines and runs the query by adding the trace field set to the identifier of the trace associated with the log entry.

  • Show only traced requests: Refines and runs the query by adding the traceSampled field set to True. For more information on sampling, go to Sampling rate.

Pinning logs

Pinning a log lets you highlight a log entry of interest. To pin a log, hover over the log you want to pin, and then select the pin icon . After you pin a log entry, its background is darkened, and a pin icon is shown.

Once you pin a log and rerun your query, the pinned log appears at either the top or bottom of the Query results pane, depending on how your logs are configured. A pin icon also appears on the Histogram pane based on the pinned log's timestamp.

Logs Explorer shows a pinned log entry in the Query results and Histogram pane.

To unpin the log, select the pin icon, and then select Unpin log entry.

Viewing a pinned log entry in its resource context

You can view the pinned log within its resource context, which lets you examine log entries around the pinned log that have the same resource type as the pinned log.

To view the pinned log within its resource context, select the pin icon and then select Pin and show resource log entries.

Pin and show in resource context is selected.

Logging populates the Query builder with the resource type from the pinned log and runs the query. You can now view your pinned log in relationship with its resource type.

Viewing a pinned log entry in the Histogram pane

Using the Histogram pane, you can select the pinned log, and then select Zoom to log entry to narrow the timeframe the Histogram pane displays. This lets you refine your query to isolate the logs near the pinned log.

Histogram timeframe is narrowed.

To share a link to a log, expand a log entry, and then select Copy link. The link is copied to your clipboard. You can now send the link to users who have access to the project. When a user pastes the link into a browser or selects it, Logging pins the log entry in their Query results pane.

Copy link to share log entry with others.

Downloading logs

You can download your logs in CSV or JSON format. You need the following roles to download logs:

  • Logging Admin (roles/logging.admin)
  • Logs View Accessor (roles/logging.viewAccessor)

To download your logs, do the following:

  1. Select Actions, and then Download Logs.

    Download logs with the Action button.

  2. In the Download logs dialog, select CSV or JSON format, and then select to download the logs either to your computer or to Drive, or to view them in a new tab.

    When you save a CSV and select Drive, you can open the file in Sheets.

Troubleshooting

This section provides instructions for troubleshooting common issues when using the Logs Explorer.

Selecting a Cloud project or organization

To select a Cloud project from anywhere in the Google Cloud Console, including from the Logs Explorer, use the project and organization selector:

A project is selected from the drop-down menu

Getting Cloud project or organization ID

To get a Cloud project or organization ID from anywhere in the Google Cloud Console, expand the list of projects from the project and organization selector and find the project ID in the ID column:

The ID for the project is shown

Cannot see log entries

If you don't see any log entries, check the following:

  • Is the correct project selected? If not, select the correct project from the project and organization selector.

  • Is your project using resources that generate logs and is there activity on those resources? Even if the project is new, it should have audit logs recording the fact that it was created. Verify you are using a resource that generates logs, by going to the "Mapping services to resource types" section in the Monitored resource list page.

  • Is the time range too narrow? Verify the time range in your query is correct.

  • View your current exclusion queries to ensure that the logs you are looking for are not accidentally excluded.

My query is correct but I still don't see log entries

  • You cannot see log entries that are older than the Logging retention period. See Logs retention periods for the logs retention period in effect.

  • During periods of heavy load there could be delays in sending logs to Logging or in receiving and displaying the logs.

  • The Logs Explorer doesn't show log entries that have timestamps in the future until the current time has "caught up" with them. This is an unusual situation, probably caused by a time skew in the application sending the logs.

Getting support

For information on getting support, see Google Cloud's operations suite support page.