View logs by using the Logs Explorer

This document provides you with an overview of the Logs Explorer in the Google Cloud console, which you can use to retrieve, view, and analyze log data. The Logs Explorer lets you find and view your logs so that you can troubleshoot problems.

The Logs Explorer doesn't support aggregate operations, like counting the number of log entries that contain a specific pattern. To perform aggregate operations, enable analytics on the log bucket and then use the Log Analytics page. You can also use the Logs Explorer to view logs in analytics-enabled buckets. For more information, see Log Analytics overview.

You can also read logs using the Logging API and the Google Cloud CLI.

Get started

To begin using the Logs Explorer, do the following:

1. Ensure that you have the necessary roles to view logs in the Logs Explorer:

   • To view all logs in the _Required bucket, and to view logs in the _Default view on the _Default bucket, you must have the Logs Viewer (roles/logging.viewer) role.

   • To view all logs in the _Required and _Default buckets, including data access logs, you must have the Private Logs Viewer (roles/logging.privateLogViewer) role.

   • To view logs stored in a log view on a user-defined log bucket, you must have the Logs View Accessor (roles/logging.viewAccessor) role with a condition that grants access to the log view. If you have the Logs View Accessor role without any conditions attached, then you can view logs stored in any log view on user-defined log buckets. See Reading logs from a bucket for an example.

   • To view restricted LogEntry fields in a bucket, you must have the Logs Field Accessor (roles/logging.fieldAccessor) role. For more information, see Configure field-level access.

2. In the Google Cloud console, select Logging from the navigation menu, then click Logs Explorer:
   Go to Logs Explorer

3. Select the appropriate Google Cloud project.

   If you use Amazon Web Services Elastic Compute Cloud (AWS EC2), your log entries are located in the AWS connector project that links your AWS account to Google Cloud services.

Logs Explorer interface

The Logs Explorer interface lets you retrieve logs, parse and analyze log data, and refine your query parameters.

The Logs Explorer contains the following sections, which are detailed on this page:

1. Action toolbar
2. Query pane
3. Results toolbar
4. Log fields pane
5. Histogram
6. Query results pane

Action toolbar

Using the Action toolbar features, you can do the following:

• Refine scope: Scope your search by logs in your current Google Cloud project only or by one or more storage views. For more information about scoping, see Refine scope.
• Learn: View links to relevant documentation and topics.
• Share link: Create a shortened URL of the current query and copy it to your clipboard, making it easier to share a query. The copied URL has the corresponding absolute time range represented by the current time range of your query; for example, 7:49:37 PM - 8:49:37 PM.

Note: If your data is managed through an Assured Workloads environment, then this feature might be impacted or restricted. For information, see Restrictions and limitations in Assured Workloads.

Refine scope

If your logs are stored in log buckets, then you can view those logs in the Logs Explorer. Depending on the project you have selected, you might have to adjust the scope of the Logs Explorer to view logs stored in certain log buckets. By default, the Logs Explorer displays logs that are generated in the project or routed directly to the project, but not logs that are routed to log buckets in the project.

For example, suppose the Google Cloud project named Project-A generates logs and routes those logs to a project named Project-B. See the following table for scenarios describing which logs are visible in Project-B's Logs Explorer:

Scenario	Destination of the log sink	Are logs visible in the default view of Project-B's Logs Explorer?	Are logs visible in the log bucket view of Project-B's Logs Explorer?
Scenario 1	Log bucket in Project-A	No	No
Scenario 2	Log bucket in Project-B	No	Yes
Scenario 3	Project-B	Yes	Yes

To customize which logs are displayed in the Logs Explorer, use the Refine scope button in the Action toolbar:

• Scope by project (default): To view logs generated in, or routed directly to, the current project, select Scope by project.

• Scope by storage: To view logs in log buckets that belong to the current project, or to view logs in log buckets that the current project has routed to, select Storage scope. In the dialog, you can select the log views on the buckets whose logs you want to view. For more information about log views, see Configure bucket-level access.

Note: If you route logs to other destinations, such as Pub/Sub, then your logs are not visible in the Logs Explorer.

Query pane

Use the Query pane features to do the following:

• Query tab: Build and refine queries using the following features:

  • Time-range selector: Specify a time range for the logs that you want to view. For more information, see Use the time-range selector.

  • Search-text box: Find log entries that match your search terms or phrases. For details, see Search for text across log fields.

  • Filter menus: Build queries based on Resource, Log name, and Severity. For more information, see Use filter menus.

  • Query-editor field: Build advanced queries using the Logging query language. For details, see Write advanced queries.

• Recent tab: View queries that you have recently run. For more information, see Use recent queries.

• Saved tab: View your saved queries and queries that other users of the Google Cloud project have shared with you. For more information, see Save queries and Share queries.

• Suggested tab: View suggested queries based on the resources in your Google Cloud project. For more information, see Use suggested queries.

• Library tab: View and run Google-provided queries based on your use cases. For more information, see Select queries from the library.

• Save: Save queries that can be viewed and run from the Saved tab.

• Clear query: Clear and reset the selections you made when building a query in the Query pane.

• Stream logs: View log entries as Logging ingests them. For more information, see Stream logs.

• Run query: Run your queries after you have built them in the Query pane.

If you don't see the query-editor field, enable Show query. If you add any search terms in the search-text box, those terms also appear in the query-editor field and are evaluated as part of your query expression.

After you review your query, click Run query. Logs that match you query are listed under the Query results pane. The Histogram and Log fields panes also adjust according to the query expression.

Stream logs

You can stream logs as Logging ingests them, or you can add a query to stream only those logs that match the query.

To stream logs based on a query, add a query in the Query pane, and then select Stream logs. As Logging ingests the logs data, only those logs that match the query are shown in the Query results pane. If a query isn't provided, Logging shows each log as it's ingested.

To stop streaming, click Stop streaming, or scroll within the Query results pane.

Results toolbar

From the Results toolbar, you can do the following:

• Show Logs field pane: Quickly show or hide this pane from your Logs Explorer page layout.
• Show Histogram pane: Quickly show or hide this pane from your Logs Explorer page layout.
• Create metric: Set up a log-based metric based on your current query expression.
• Create alert: Set up a log-based alert based on your current query expression.
• Jump to now: Perform a forced refresh of your query results to include the current time. If the time-range selector uses a custom range and an end time is set, it runs the query with a default time range of one hour. Otherwise, it refreshes with the current start date or duration, and runs the query. Note that the query expression isn't altered when you use this feature.
• More actions: Use these further options to manage log-based alerts or create a sink.

Log fields pane

The Log fields pane offers a high-level summary of logs data and provides a more efficient way to refine a query. It shows log entries broken down by different dimensions, corresponding to fields in these entries. For each field, the Log fields pane shows values and their incidence in descending frequency order. The log-field counts correspond to the time range in the time-range selector.

To show or hide the Log fields pane, click on the Logs fields button in the Results toolbar.

The Log fields pane is populated and updated based on an executed query in the query-editor field.

If the query is empty, the Log fields pane displays the counts of log entries by the Resource type and Severity fields.

If you've selected Scope by storage, you'll also see Project ID and the corresponding counts of log entries.

If you select a resource type from the Log fields pane, a set of relevant fields, based on the resource labels, populate the pane. This lets you investigate logs data for that specific resource type. You can remove them by clicking Clear next to the Resource type field.

Note: If you ingest on-premise and hybrid cloud logs through BindPlane, you can find these logs under the resource type generic_node.

Add fields to Log fields pane

You can add certain LogEntry key-value pairs to the Logs field pane from the log entries populated in the Query results pane. To add a field to the Logs field pane, do the following:

1. In the Query results pane, expand a log entry by clicking the expand button .

2. Click on a field's value. From the menu, select Add field to Log fields pane.

   The custom field appears in the Log fields pane as a list of key-value pairs.

To remove a custom field from the Log fields pane, click Remove next to the field.

Note that the following types of fields can't be added to the Log fields pane:

• Fields related to time; for example, receiveTimestamp and protoPayload.startTime.
• Fields with high cardinality; for example, insertId and protoPayload.latency.
• Fields with array indices in their path; for example, protoPayload.authorizationInfo[0].resource.

Analyze logs using the Log fields pane

To narrow down and refine a query, you can add field-value pairs from the Log fields pane to the expression in the Query pane. To do so, in the Log fields pane, click on a field's value. This adds the field-value pair to the expression using the AND operator. The query then runs.

You can add nested field-value pairs, as well as top-level field-value pairs, to the Log fields pane. For example, suppose you added jsonPayload.message to the Log fields pane. If you select a particular jsonPayload.message value, it populates the query-editor field.

The Histogram and Query results panes also change to reflect the current query.

When a query is executing, the log entries are scanned and the log-field counts change. When the query is complete, the total counts for all log fields are displayed.

Histogram

With the Histogram pane, you can visualize the distribution of logs over time. The histogram regenerates when you run a query, making it easier to see trends in your logs data and troubleshoot problems.

To show or hide the Histogram pane, click on the Histogram button in the Results toolbar.

Histogram features

1. Histogram bars: Each histogram bar represents a time range. Each bar contains a three-color breakdown for the log-severity levels captured in each bar's time range. The colors represent the following log severities:

   • Blue: Low severities such as Default, Debug, Info, and Notice.
   • Yellow: Medium severities such as Warning.
   • Red: High severities such as Error, Critical, Alert, and Emergency.

   Each histogram bar features a menu with options to analyze your logs.

2. Time controls: Let you adjust the time range of the logs you see in the Query results pane. For details on these options, see Analyze logs using time controls.

3. Timeline: Shows you the time range of the logs, represented by histogram bars, that are currently displayed within the Query results pane. The timeline helps to orient you to the logs you're currently viewing within the larger time range of your query.

Analyze logs using time controls

You can use the histogram's time controls to help you investigate and analyze your logs data.

Tip: When you adjust the histogram's time controls, the logs data in the Query results and Log fields panes adjusts according to the time range captured by the histogram timeline. Adjusting the histogram's time controls doesn't alter the query expression in the Query pane.

Adjust time quickly

The histogram provides time controls that let you quickly adjust the data that you see in the Logs Explorer.

• Time handles: Drag the timeline's handles inward to narrow the data or outward to widen the data in the histogram timeline. Click Run.

• Slide the timeline forward and backward: Click the forward arrow to slide the timeline to a later time. Click the backward arrow to slide the timeline to an earlier time.

• Zoom in and out: Click the zoom-out icon to broaden the data shown in the timeline. Click the zoom-in icon to narrow the data shown in the timeline.

Timeline modifications are constrained to be between the current time ("now") and 30 days ago.

Scroll or zoom to time

In addition to the time controls above, the histogram provides the Scroll to time and Zoom to time features to give you more in-depth control of the histogram and the data that you see in other panes in the Logs Explorer.

Perhaps a particular histogram bar interests you based on its relative size or severity levels. You can select that histogram bar to adjust the logs data you see in the Logs Explorer.

You can use the Scroll to time feature to browse your logs data without changing the values in the Histogram and Log fields panes. When you select the Scroll to time feature, the following happens:

• The logs data that you see in the Query results pane adjusts according to the time range captured by the selected histogram bar.

  The query isn't run, but a partial reload of the data might occur to ensure you're seeing logs in the Query results pane that correspond with the selected histogram bar's time range.

• The console URL updates to contain the timestamp of the most recent log captured by the time range of the selected histogram bar.

To select the Scroll to time feature, do the following:

1. Hover over a bar in the histogram timeline. A pane containing summary information about the logs data for the specified time range appears.

2. In the pane, select Scroll to time.

   Alternatively, clicking on a histogram bar, instead of hovering over it, is equivalent to selecting Scroll to time.

The Zoom to time feature is similar to Scroll to time, but it runs a query on your logs data based on the time range captured by a selected histogram bar. When you select the Zoom to time feature, the following happens:

• The logs data that you see in the Query results pane reloads and narrows according to the time-range restriction of the selected histogram bar.
• The console URL updates to contain the timestamp of the most recent log captured by the time range of the selected histogram bar.
• The histogram changes to show only logs that have a timestamp value that falls within the time range of the selected histogram bar. by the selected histogram bar.
• The data in the Log fields pane adjusts according to the time range captured by the selected histogram bar.

To select the Zoom to time feature, do the following:

1. Hover over a bar in the Histogram timeline. A pane containing summary information about the logs data for the specified time range appears.

2. In the pane, select Zoom to time.

Query results

To view the results of your queries, use the Query results pane. To help you troubleshoot your applications, you can view the details of individual log entries, and group and analyze log entries to find patterns in your logs.

Find patterns in your logs by using summary fields

Suppose you're looking through the log entries in your query results and want to quickly skim the results by a certain LogEntry field. Or perhaps you want to group your log entries by a certain field-value pair. You can add summary fields to your results, which appear as chips at the beginning of each log entry line. For example, the following image shows query results with the summary field resource.type added to each log entry line:

The Logs Explorer offers default summary fields and custom summary fields. Default summary fields depend on your current query results, and custom summary fields let you select any field in the LogEntry.

To modify the summary fields, do the following:

1. Click Edit in the Summary column to open the editing menu:

   

2. In the Manage summary fields dialog, you can do the following:

   • To display log entries in raw-text format, turn off Default summary fields by using the toggle switch , and don't add any custom summary fields.

   • Add any custom field names to Custom summary fields.

     The summary field selection has the following features:

     • Autocomplete using the logs currently displayed in your query results.
     • Field correction for legal characters within quotes.

     For example, if you type jsonPayload.id-field, it gets changed to jsonPayload."id-field".

     You can also select any LogEntry field, regardless of whether it is suggested to you with the autocomplete function.

     To remove an existing custom summary field, click the X in its chip.

   • Turn on or off truncation for your custom summary fields. Use the toggle switch by Truncate summary fields to shorten the display of the summary field values. Then choose how many characters to display before the field is truncated, and whether the beginning or the end of the field is displayed.

     Tip: In your query results, hovering over a truncated summary field displays its full value.

3. Click Apply.

   Your summary fields are now updated in your query results.

Search your query results

To search the contents of the Query results, click Find in results in the Query results pane, and then enter your search term. This filter lets you find information in your log entries without building a new query.

Terms that match the search criteria are highlighted in log entries within the Query results pane:

View similar log entries

You can view log entries that are similar to a selected log entry, which lets you focus on logs of interest.

To show similar log entries, do the following:

1. In the Query results pane, expand a log entry.

2. Click Similar entries, and select Show similar entries.

   The query updates with a query similar to the following and reloads the query results:

   --Show similar entries
   protoPayload.methodName="io.k8s.core.v1.configmaps.update"
   --End of show similar entries
   

To see a preview of the similar log entries, do the following:

1. In the Query results pane, expand a log entry.

2. Expand the Similar entries menu, and then select Preview similar entries.

   A separate dialog opens with the following information:

   • The pattern that was found
   • The percentage of log entries that contain the pattern
   • Example log entries that contain the pattern

   In this dialog, you can hide or show log entries:

   

Hide similar log entries

You can hide similar log entries, which lets you remove logs from your query results.

There are two ways to hide similar log entries:

• Hide large amounts of automatically grouped log entries. When you run a query, the query results are analyzed for patterns and log entries are then automatically grouped based on similar log field content. If a significant pattern is detected, a banner appears in the Query results pane showing the percentage of results that can be hidden:

  

  Hide similar entries: This button adds a clause to the query and reloads the query results.

  Preview: A separate window opens which describes the pattern found, and shows examples of the entries.

  When you hide similar logs, no information is saved outside of the Logs Explorer session, and each query produces a new analysis, based only on the logs shown. Different queries analyze different portions of the log entries depending on the types of logs returned.

• Hide log entries similar to a specific log entry. To hide log entries similar to a log entry, do the following:

  1. Click Expand on the log entry, click the Similar entries menu, and then select Hide similar entries.

     The query updates and the Query results pane reloads. Log entries similar to the selected log entry aren't displayed.

View or hide log entries that match a field

You can view or hide log entries that match a field in a log entry, which lets you focus on entries that contain the same field content.

To view or hide log entries that match a specific field in a log entry, do the following:

1. In the Query results pane, click Expand on the log entry.

2. Click a field's value within the log entry, such as compute.googleapis.com, which is a serviceName.

   You see the following menu:

   

3. Select Show matching entries or Hide matching entries.

   The query updates with a query that shows or hides similar entries, and the Query results reload with new results.

Pin log entries

After you run a query, you can highlight a log entry by pinning it. The pinned log entry stays centered in the Query results pane. If you run a new query and the pinned log entry isn't included, then you are prompted to unpin the log entry.

To pin a log entry, do the following:

1. Hover over the log entry that you want to pin.
2. Click on the pin icon .

After you pin a log entry, its background is darkened, and a pin icon is shown. A pin icon also appears on the Histogram pane based on the pinned log entry's timestamp:

To unpin a log entry, click the pin icon again.

Show logs that match the resource of a pinned log entry

After you pin a log entry, you can run a new query that displays log entries that match the resource type or resource labels of the pinned log.

To pin a log entry and display log entries that match the same resource type or resource labels, do the following:

1. Click on the down arrow next to the pinned log to expand the pin menu.

2. Make a selection from the pin menu:

   • To rerun the query with the same resource.type as the pinned log, select Same resource.type.

     For example, suppose you pin a log entry with a resource.type of k8s_node. If you select Same resource.type, then the query is rerun to display all log entries with resource.type="k8s_node".

   • To rerun the query with the same resource.labels as the pinned log, select Same resource.labels.

   • To rerun the query with the same trace as the pinned log, select Same trace.

   • To clear the query and show all log entries, select Show all.

View a pinned log entry in the Histogram pane

You can use the Histogram pane to highlight, scroll to, and further examine a pinned log entry.

Using the Histogram pane, click on the pin icon and then choose from the following menu options:

• Scroll to log entry: Bring the log entry into the current Query results pane and view the pinned log entry in the context of nearby logs.
• Zoom to log entry: Narrow the time range that the Histogram pane displays and refine your query to isolate the logs near the pinned log.

View trace data

When a log entry contains both the trace and the latency-related field, both the latency and trace icon appear.

When a log entry contains only the trace field, then only the trace icon appears.

To view the trace data related to the log entry, click the trace icon. You have the following options:

• View trace details: Shows the parent span and child traces along with details about the trace. To view more details about the trace, navigate to Cloud Trace by clicking View in Trace. For more information about the content in the details panel, see View trace details.

Note: View trace details is only available if traceSampled is true.

• Show all logs for this trace: Refines and runs the query by adding the trace field set to the identifier of the trace associated with the log entry.

• Show only traced requests: Refines and runs the query by adding the traceSampled field set to True. For more information on sampling, see Sampling rate.

View Monitoring data

For certain logs, such as GKE and Compute Engine logs, you can click on the resource type from the log's summary line to display a menu with the following options:

• View monitoring details: opens a details panel for a GKE resource. For information on the details panel, see View resource details.
• View in Monitoring: opens to a Monitoring page for the resource.
• View in GKE or View in Compute Engine: opens the Details page for the resource within the GKE or Compute Engine user interface.

Copy a link to a log entry

To share a link to a log, expand a log entry, and then select Copy link. The link is copied to your clipboard. You can send the link to users who have access to the Google Cloud project. When a user pastes the link into a browser or selects it, Logging pins the log entry in their Query results pane.

Download logs

You can download your logs in CSV or JSON format. You need one of the following Identity and Access Management roles to download logs:

• Logging Admin (roles/logging.admin)
• Logs View Accessor (roles/logging.viewAccessor)

To download your logs, do the following:

1. Click Download in the Query results pane.

2. In the Download logs dialog, select CSV or JSON format.

3. Select what to do with the log data. You have the following options:

   • Download the data to your computer.
   • Download the data to Google Drive.
   • View the data in a new tab.

   When you save a CSV and select Google Drive, you can open the file in Google Sheets.

Note: 10,000 is the maximum number of logs you can download.

Example queries

For suggested queries, arranged by Google Cloud product and use case, see Sample queries using the Logs Explorer. For example, you can run Kubernetes-related queries to find Google Kubernetes Engine logs.

View Compute Engine logs

For certain Compute Engine resource types, such as gce_instance and gce_network, you see the resource name with the resource ID as subtext in several places in the Logs Explorer. For example, for the gce_instance resource type, you see the VM name alongside the VM ID. The resource names help you identify the correct resource ID, on which you can build queries.

You might see Compute Engine resource names in the following places:

• Query pane filter menus: Compute Engine resource types show resource names, with their corresponding resource IDs as subtext.
• Log fields: Compute Engine resource types show the resource name, rather than the resource ID, in the field dimensions.
• Query results: For Compute Engine VM instance logs, the resource.labels field shows metadata with the corresponding resource name.
• Summary fields: For Compute Engine VM instance logs, the chip shows the resource name instead of the resource ID.

Troubleshooting

This section provides instructions for troubleshooting common issues when using the Logs Explorer.

If you're experiencing issues when trying to view logs in sink destinations, see Troubleshoot routing and sinks.

Can't find console logs for a VM instance

Logs written to the console of a Compute Engine instance might might not be available in the Logs Explorer. To view these logs, do the following:

1. Navigate to the list of VM instances for your Google Cloud project:

   Go to VM instances

2. Select the instance, and then select Serial port 1 (console) in the Logs section of the page.

You can configure your instances such that their serial port output is sent to Cloud Logging. For more information, see Enabling and disabling serial port output logging.

Get Google Cloud project or organization ID

To get a Google Cloud project or organization ID from anywhere in the Google Cloud console, expand the list of Google Cloud projects from the Google Cloud project and organization selector and find the Google Cloud project ID in the ID column:

Can't see log entries

If you don't see any log entries, check the following:

• Is the correct Google Cloud project selected? If not, select the correct Google Cloud project from the Google Cloud project and organization selector.

• Is your Google Cloud project using resources that generate logs and is there activity on those resources? Even if the Google Cloud project is new, it should have audit logs recording the fact that it was created. Verify you're using a resource that generates logs, by going to the "Mapping services to resource types" section in the Monitored resource list page.

• Is the time range too narrow? Verify the time range in your query is correct.

• View your current exclusion queries to ensure that the logs you're looking for aren't accidentally excluded.

• Is the correct scope being used to view logs? For instructions on adjusting the scope of your search, see Refine scope.

My query is correct but I still don't see log entries

• You can't see log entries that are older than the Logging retention period. See Log retention periods for the logs retention period in effect.

• During periods of heavy load, there could be delays in sending logs to Logging or in receiving and displaying the logs.

• The Logs Explorer doesn't show log entries that have timestamps in the future until the current time has "caught up" with them. This is an unusual situation, probably caused by a time skew in the application sending the logs.

• The query scope was set too large and couldn't complete within a reasonable amount of time time. You might see this as "deadline expired before operation could complete". Try making your query more specific or reducing the time range.

Query returns an error

If you issue a query over a resource without specifying a bucket, then Cloud Logging uses the history of the sinks in the Google Cloud project to determine where entries might have been written for that resource. If Cloud Logging identifies more than 200 buckets where entries might have been written, then the query fails with the message Error: Invalid query.

To resolve this issue, refine the scope of your query to a subset of the storage. For more information, see Refine scope.

Query results time range doesn't match query

The logs data you see in the Query results and Log fields panes adjusts according to the time range captured by the histogram timeline. You adjust the histogram timeline using the histogram's time controls or the time-range selector. Adjusting these time controls doesn't alter the query expression in the Query pane.

If you have a query with a timestamp, the time-range selector is disabled, and the query uses the timestamp expression as its time-range restriction. If a query doesn't use a timestamp expression, then the query uses the time-range selector as its time-range restriction.

Get support

For information on getting support, see the Google Cloud's operations suite support page.