Using the Logs Explorer

This guide shows you how to search and view logs with the Logs Explorer.

Before you begin

If you're sending logs from an Amazon Web Services (AWS) account to Logging, select the AWS connector project in the Google Cloud Console project picker and then use the Legacy Logs Viewer. The AWS connector project stores the Amazon Resource Name (ARN) for your AWS account and links your AWS account to Google Cloud services. For more information, see Adding a project or account to a Workspace.

Getting started

To navigate to the Logs Explorer, do the following:

  1. Go to the Google Cloud navigation menu and select Logging > Logs Explorer:
    Go to the Logs Explorer
  2. Select a Google Cloud project.
  3. From the Upgrade menu, switch from Legacy Logs Viewer to Logs Explorer.

You're now in the Logs Explorer.

Logs Explorer interface

The Logs Explorer interface lets you retrieve logs, parse and analyze log data, and refine your query parameters.

User interface for the Logs Explorer

The Logs Explorer contains the following panes:

  1. Action bar
  2. Query builder
  3. Log fields
  4. Histogram
  5. Query results

Action bar

Action bar pane

The Action bar pane offers the following features:

  1. Options: Lets you go to the Legacy Logs Viewer, send feedback, and view a summary of new Logging features.
  2. Refine scope: Lets you scope your search by logs in your current Cloud project only or by one or more storage views. For more information about scoping, see Refining scope.
  3. Share link: Lets you create a shortened URL of the current query and copies it to your clipboard, making it easier to share a query.
  4. Time-range selector: Lets you restrict query results by time range. The default time range is one hour.
  5. Page layout: Lets you enable and disable the Histogram and Logs field explorer panes.
  6. Learn: Lets you view links to relevant documentation.

Refining scope

You can refine the scope of the logs displayed in the Logs Explorer through the Refine scope option. You can search only logs within the current Cloud project or search logs in one or more storage views. To refine the scope of the Logs Explorer, do the following:

  1. From the Logging menu, select Logs Explorer.

    Go to Logs Explorer

  2. Select Refine Scope.

  3. On the Refine scope dialog, select a Scope by option.

    The Refine scope dialog

    • Scope by Cloud project allows you to search logs that the current Cloud project generates.

    • Scope by storage allows you to search logs based on one or more storage views. For more information about log views, see Managing log views on your log buckets.

  4. If you select Scope by storage, select one or more buckets you want to view.

    The dialog lists storage views that meet the following conditions:

    • The user has access to the storage view.
    • The log buckets belong to the selected Cloud project, or the selected Cloud project has previously routed logs to the storage buckets.
  5. Click Apply.

Query builder

Query builder

From the Query builder pane, you can do the following:

  1. Query-builder field: Lets you build queries using the Logging query language.
  2. Query builder drop-down menus: Lets you add query expressions based on Resource, Log name, and Severity. For more information, see Query builder drop-down menus.
  3. Recent: Lets you view your recent queries. For more information, see Recent queries.
  4. Saved: Lets you view your saved queries and queries that other users of the Cloud project have shared with you. For more information, see Saved queries and Shared queries.
  5. Suggested: Lets you view suggested queries based on the resources in your Cloud project. For more information, see Suggested queries.
  6. Save: Lets you save queries that can be viewed and run from the Saved tab.
  7. Stream logs: Lets you view log entries as Logging ingests them. For more information, see Streaming logs.
  8. Run: Lets you run your queries after you have built them in the query-builder field.

Streaming logs

You can stream your logs as Logging ingests them, or you can add a query to stream only those logs that match the query.

To stream logs based on a query, add a query to the query-builder field, and then select Stream logs. As Logging ingests the logs data, only those logs that match the query are shown in the Query result pane. If a query isn't provided, Logging shows each log as it's ingested.

Add a query before selecting to stream logs

To stop streaming, select Stop streaming, or scroll down within the Query results pane.

Log fields

The Log fields pane offers a high-level summary of logs data and provides a more efficient way to refine a query. It shows the count of log entries, sorted by decreasing count, for the given log field, and provides aggregation-based results for the resource.type, resource.labels, logName, and severity fields. The log-field counts correspond to the time range used by the Histogram pane.

Enabling the Log fields pane

To enable the Log fields pane, select Page Layout, and then select the Log fields checkbox. The Log fields pane appears.

Log fields pane selected

To disable the Log fields pane, clear the Log fields checkbox.

Log fields features

The Log fields pane is populated and updated based on an executed query. When there is an empty query, the Log fields pane displays the counts of log entries by the resource.type and severity fields.

log fields pane.

Analyzing logs using the Log fields pane

You can add fields from the Log fields pane to the Query builder to narrow down and refine a query. To do so, click on a field value in the Log fields pane. This adds the log field to the Query builder and automatically runs the query by adding it as an expression to the original query using the AND operator.

When a query is executing, the log entries are scanned and the log-field counts change. When the query is complete, the total counts for all log fields are displayed.

Histogram

The Histogram pane lets you visualize the distribution of logs over time. The histogram is generated when you run a query, making it easier to see trends in your logs data and troubleshoot problems.

Enabling the histogram pane

To enable the histogram pane, select Page Layout, and then select the Histogram checkbox. The Histogram pane appears.

Page layout is open and histograms is selected

To disable the histogram pane, clear the Histogram checkbox.

Histogram features

Histogram pane.

  1. Histogram bars: Each histogram bar represents a time range. Each bar contains a three-color breakdown for the log-severity levels captured in each bar's time range. The colors represent the following log severities:

    • Blue: Low severities such as Default, Debug, Info, and Notice.
    • Yellow: Medium severities such as Warning.
    • Red: High severities such as Error, Critical, Alert, and Emergency.

    You can use the histogram bars to analyze your logs.

  2. Adjust time range: Lets you change the time range used for queries by adjusting the handles. After adjusting the handles, click Run to update the time range used in the query.

    Time range adjusted using handles.

  3. Histogram viewport: Lets you see the time range of the logs, represented by histogram bars, that are currently displayed within the Query results pane. The viewport helps to orient you to the logs you're currently viewing within the larger time range of your query.

    The viewport's size is based on the duration between the maximum and minimum timestamp of the log entries displayed in the Query results pane.

    Histogram pane is showing the viewport.

Analyzing logs using the histogram pane

You can use the histogram to analyze your logs data. For example, perhaps a particular histogram bar is of interest to you based on its severity levels or size relative to the other bars. You can select that histogram bar to adjust the logs data that you see in the Query results pane using either the Scroll to time or Zoom to time features.

The Scroll to time feature lets you browse your logs data without changing the values in the Histogram and Log fields panes. When you select the Scroll to time feature, the following happens:

  • The logs data that you see in the Query results viewport adjusts according to the time range captured by the selected histogram bar.

    The query isn't run, but a partial reload of the data might occur to ensure you're seeing logs in the viewport that correspond with the selected histogram bar's time range.

  • The console URL updates to contain the timestamp of the most recent log captured by the time range of the selected histogram bar.

To select the Scroll to time feature, do the following:

  1. Hover over a bar in the Histogram pane. A pane containing summary information about the logs data for the specified time range appears.

  2. In the pane, select Scroll to time.

    Alternatively, clicking on a histogram bar, instead of hovering over it, is equivalent to selecting Scroll to time.

The Zoom to time feature is similar to Scroll to time, but it runs a query on your logs data based on the time range captured by a selected histogram bar. When you select the Zoom to time feature, the following happens:

  • The logs data that you see in the Query results viewport reloads and narrows according to the time-range restriction of the selected histogram bar.
  • The console URL updates to contain the timestamp of the most recent log captured by the time range of the selected histogram bar.
  • The histogram changes to show only logs that have a timestamp value that falls within the time range of the selected histogram bar.
  • The time-range selector updates to the time range captured by the selected histogram bar.
  • The data in the Log fields pane adjusts according to the time range captured by the selected histogram bar.

To select the Zoom to time feature, do the following:

  1. Hover over a bar in the Histogram pane. A pane containing summary information about the logs data for the specified time range appears.

  2. In the pane, select Zoom to time.

Query results

Query results pane

The Query results pane lets you explore the log entries that match your query expressions.

  1. Query results: Lets you view the retrieved logs from your query.
  2. Log entries: Lets you view log entries in the structured JSON format.
  3. Expand and collapse query results: Lets you expand the query-results pane to view more log entries.
  4. Time zone: Lets you change the time zone that logs are displayed in.
  5. Trace data: Lets you view trace details and refine your query based on the trace. For more information, see Viewing trace data.
  6. Hide log summary: Lets you hide the log summary line from the query results.
  7. Expand or collapse nested log fields: Lets you expand or collapse nested fields.
  8. Copy to clipboard: Lets you copy the log entry in its JSON format.
  9. Copy link to a log entry: Lets you share a link to a log entry. For more information, see Copying a link to a log entry.
  10. Jump to now: Lets you perform a forced refresh to include the current time. If the time-range selector uses a custom range and an end time is set, it runs the query with a default time range of one hour. Otherwise, it refreshes with the current start date or duration, and runs the query.
  11. Actions: Lets you set up a logs-based metric, create a sink destination, or download your logs. For more information on downloading logs, see Downloading logs.
  12. Configure: Lets you add the value of a log field to the summary line at the beginning or end of the log entry. It also lets you choose to show newest logs either first or last. For more information on adding a summary field, see Adding summary fields.
  13. Pin log entry: Lets you pin a log entry to the Query results and Histogram panes. For more information, see Pinning logs.
  14. Cursor scroll: When you scroll the logs in Query results, the URL adjusts to include cursorTimestamp, which indicates the timestamp of the newest log shown in the current Query results viewport.

Within the query-results pane, you can click the values of a field to choose to do the following:

Options after selecting field's value

  1. Show matching entries: Lets you query for matching log entries.
  2. Hide matching entries: Lets you query for log entries that don't match the selected expression.
  3. Add field to summary line: Lets you add the field as a summary line to log entries.

Adding summary fields

Summary fields help you notice patterns in your logs faster. For example, the following image shows the value for the summary field resource.labels.pod_name added before the logs that contain that value.

The Logs Explorer is showing logs that are preceded with green text displaying
pod names.

Add a summary field from a log entry

To add a summary field to a log entry, complete the following steps:

  1. Expand a log entry by clicking the expand button .

  2. Click a field's value and then select Add field to summary line.

    The summary field now appears before the log entries containing that field.

Add a summary field using the Configure button

To add a summary field using the Configure button, complete the following steps:

  1. Click Configure and select Manage Summary Fields.

    Manage summary fields is selected from the configure drop-down menu

  2. Add fields.

    The summary field selection has the following features:

    • Autocomplete using the logs currently displayed.
    • Field correction for legal characters within quotes.

      For example, if you type jsonPayload.id-field, it gets changed to jsonPayload."id-field".

  3. Click Truncate summary fields to shorten the display of the summary field values. Then choose how many characters to display before the field is truncated and whether the beginning or the end of the field is displayed.

  4. Click Apply.

    The summary field now appears before the log entries containing that field.

Pinning log entries

Pinning a log entry lets you highlight a log entry of interest.

To pin a log entry, hover over the log entry you want to pin, and then select the pin icon . After you pin a log entry, its background is darkened, and a pin icon is shown.

If you pin a log entry and rerun your query, the pinned log entry appears at either the top or bottom of the Query results pane, depending on how your logs data is configured. A pin icon also appears on the Histogram pane based on the pinned log entry's timestamp.

Logs Explorer shows a pinned log entry in the Query results and Histogram pane.

To unpin the log entry, select the pin icon, and then select Unpin log entry.

Viewing a pinned log entry in its resource context

To view the pinned log entry within its resource context, select the pin icon and then select Pin and show resource log entries.

Pin and show in resource context is selected.

Logging populates the Query builder with the resource.type field from the pinned log entry and runs the query. You can now view your pinned log entry in relationship with its resource type.

Viewing a pinned log entry in the Histogram pane

You can use the Histogram pane to highlight, scroll to, and further examine a pinned log entry.

Using the Histogram pane, select the pin icon and then choose from the following menu options:

  • Scroll to log entry: This option brings the log entry into the current Query results viewport and lets you view the pinned log entry in the context of nearby logs.
  • Zoom to log entry: This option narrows the time range that the Histogram pane displays and lets you refine your query to isolate the logs near the pinned log.

Histogram timeframe is narrowed.

Viewing trace data

When a log entry contains both the trace and the latency-related field, both the latency and trace icon appear.

Log entry display that contains trace data.

When a log entry contains only the trace field, then only the trace icon appears.

Log entry display that contains only the trace field has trace icon.

To view the trace data related to the log entry, click the trace icon. You have the following options:

  • View trace details: Shows the parent span and child traces along with details about the trace. To view more details about the trace, navigate to Cloud Trace by clicking View in Trace. For more information about the content in the flyout pane, see Viewing trace details.
  • Show all logs for this trace: Refines and runs the query by adding the trace field set to the identifier of the trace associated with the log entry.

  • Show only traced requests: Refines and runs the query by adding the traceSampled field set to True. For more information on sampling, go to Sampling rate.

To share a link to a log, expand a log entry, and then select Copy link. The link is copied to your clipboard. You can now send the link to users who have access to the Cloud project. When a user pastes the link into a browser or selects it, Logging pins the log entry in their Query results pane.

Copy link to share log entry with others.

Downloading logs

You can download your logs in CSV or JSON format. You need one of the following Identity and Access Management roles to download logs:

  • Logging Admin (roles/logging.admin)
  • Logs View Accessor (roles/logging.viewAccessor)

To download your logs, do the following:

  1. Select Actions, and then Download Logs.

    Download logs with the Action button.

  2. In the Download logs dialog, select CSV or JSON format, and then select to download the logs either to your computer or to Drive, or to view them in a new tab.

    When you save a CSV and select Drive, you can open the file in Sheets.

Troubleshooting

This section provides instructions for troubleshooting common issues when using the Logs Explorer.

Selecting a Cloud project or organization

To select a Cloud project from anywhere in the Google Cloud Console, including from the Logs Explorer, use the Cloud project and organization selector:

A project is selected from the drop-down menu

Getting Cloud project or organization ID

To get a Cloud project or organization ID from anywhere in the Google Cloud Console, expand the list of Cloud projects from the Cloud project and organization selector and find the Cloud project ID in the ID column:

The ID for the project is shown

Can't see log entries

If you don't see any log entries, check the following:

  • Is the correct Cloud project selected? If not, select the correct Cloud project from the Cloud project and organization selector.

  • Is your Cloud project using resources that generate logs and is there activity on those resources? Even if the Cloud project is new, it should have audit logs recording the fact that it was created. Verify you're using a resource that generates logs, by going to the "Mapping services to resource types" section in the Monitored resource list page.

  • Is the time range too narrow? Verify the time range in your query is correct.

  • View your current exclusion queries to ensure that the logs you're looking for aren't accidentally excluded.

My query is correct but I still don't see log entries

  • You can't see log entries that are older than the Logging retention period. See Logs retention periods for the logs retention period in effect.

  • During periods of heavy load, there could be delays in sending logs to Logging or in receiving and displaying the logs.

  • The Logs Explorer doesn't show log entries that have timestamps in the future until the current time has "caught up" with them. This is an unusual situation, probably caused by a time skew in the application sending the logs.

Getting support

For information on getting support, see Google Cloud's operations suite support page.