Display logs and errors on a custom dashboard

This document describes how to display project-scoped log entries and error groups on a custom dashboard. By default, when you add a logs panel widget to a custom dashboard, it displays the most recent log entries in your current Google Cloud project. However, you can configure the logs panel to display log entries stored in Google Cloud projects accessible through the current metrics scope. In contrast, the error-reporting panel displays the most recent error groups for a single Google Cloud project.

When you add a logs panel and an error-reporting panel to a custom dashboard, you can view your logs and errors in the same context as your metrics. Because the logs panel lets you filter your log entries, you can configure it to display only those log entries that are of interest to you. Similarly, the error-reporting panel lets you identify errors and troubleshoot the cause of the failure.

Logs Explorer and logs panels

Logs Explorer is the standard way to investigate logs. Logs panels don't replace Logs Explorer; instead, they provide a way to view log entries in the same context as your metrics.

Logs panels display log entries on rows with columns for severity, the timestamp, and a summary of the log entry. To view the JSON-formatted representation of a log entry, click the View log details button. The structure of each entry summary varies based on the log type. For example, the following picture shows two log entries:

A picture of two log entries in the logs panel.

In the previous picture, the first log entry displays information about a service request. The summary shows information about the service, an email address, and a string-formatted version of the JSON structure for the entry. The second entry displays the results of a request, and shows only a sentence summarizing the action that occurred.

Logs panels are preconfigured to show log entries stored in your current Google Cloud project. However, you can configure a logs panel to show log entries that are stored in any project in the current metrics scope.

Logs panels support the same query language as Logs Explorer.

Logs panels don't support storage views. When your logs are available only through a storage view, use the Logs Explorer. For information about storage views, see Refine scope.

For general information about Cloud Logging, see the following resources:

Before you begin

To get the permissions that you need to add a logs widget to a custom dashboard by using the Google Cloud console, ask your administrator to grant you the following IAM roles on your project:
- Monitoring Editor (roles/monitoring.editor)
- Logging Viewer (roles/logging.viewer) on each project whose log entries you want to view. Dashboards don't display log entries for projects for which you don't have the permissions to view logs. For more information about the roles you need to view log entries, see Cloud Logging Access control with IAM.
For more information about granting roles, see Manage access.

You might also be able to get the required permissions through custom roles or other predefined roles.
You can put up to 40 widgets on a dashboard.

Add and configure logs panel

This section describes how to add and configure a logs panel by using the Google Cloud console. For information about adding a logs panel to a dashboard by using the Cloud Monitoring API, see Dashboard with a LogsPanel widget.

Add a logs panel

To add a logs panel to the dashboard by using the console, do the following:

In the navigation panel of the Google Cloud console, select Monitoring, and then select Dashboards:
Go to Dashboards
Select a dashboard or click Create dashboard.
In the toolbar, click Add widget.
In the Add widget dialog, select Logs.
Optional: Update the default configuration:
1. Add a title.
2. To display logs from multiple Google Cloud projects instead of only the current project, click Projects, and then make your selections.
3. To restrict which logs the logs panel displays, enter a query in the Query Editor field.
For more information about the previous steps, see Configure a logs panel:
To apply your changes to the dashboard, in the toolbar, click Apply. To discard your changes, click Cancel.

The following example shows a logs panel:

$Example of a logs panel configuration pane.$
To save your modified dashboard, in the toolbar, click Save.

By default, the logs panel displays a table of log entries. You can use the Severity menu and filter bar settings to temporarily change which logs are displayed. You can also modify the size and position of the logs panel by using your pointer.

Configure a logs panel

Logs panels are preconfigured to display all log entries stored in the current Google Cloud project. However, you might only be interested in log entries that indicate some type of error condition, or you might want to view logs for multiple projects. This section describes how to modify the default configuration of a logs panel.

To configure the title, the list of projects whose logs are shown, or to enter a query, do the following:

Place your pointer on the logs panel to activate the toolbar, and then select Edit widget.

The following screenshot is an example of the configuration pane:

$Example of a logs panel configuration pane.$
Optional: Add a title, which is shown on the logs panel.
Optional: To display logs from multiple Google Cloud projects, click Projects, and then make your selections.

By default, only log entries from the current Google Cloud project are shown. However, a logs panel can display logs for any Google Cloud project accessible through the current metrics scope.

When you don't have permission to view logs for a Google Cloud project, the Projects menu displays the message "You don't have permission to view logs in this project." To resolve the permission error, contact your system administrator and ask them to grant you the Logs Viewer (roles/logging.viewer) role on the project.
Optional: To restrict which logs the logs panel displays, enter a query in the Query Editor field.

If you've disabled queries from being automatically run, then click Run query after you've entered your query.

To populate the Query Editor field, you can enter text into the field or you can make selections from menus. The Query Editor field uses the same query language as Logs Explorer. For troubleshooting information, see Syntax issues.

The Severity and Log name menus let you select multiple options. The following is an example of a query statement that matches multiple severity levels:
```
 severity=(WARNING OR ERROR OR CRITICAL OR ALERT OR EMERGENCY)
```
The Resource menu activates additional menus and an Add button after you select an individual resource. This menu lets you select a resource type, a service for a specific resource type, or a method for a service for a resource type. For example, to add a clause for the audited resource type, select Audited Resource and then select Add. These steps add the following clause to the Query Editor field:
```
 resource.type="audited_resource"
```
Similarly, to add a clause for a specific service that is audited, select Audited Resource, select the service from the All service menu, and then select Add. The following is an example of a clause that is created by using this approach:
```
 resource.type="audited_resource" resource.labels.service="bigquery.googleapis.com"
```
Optional: Apply template variables to the logs panel. For more information, see Add temporary filters to a custom dashboard and Apply a permanent filter to a widget.
To apply your changes to the dashboard, in the toolbar, click Apply. To discard your changes, click Cancel.
To save your modified dashboard, in the toolbar, click Save.

Filter the log entries

You can modify which log entries a logs panel displays by modifying the values of the following fields or menus:

Dashboard time selector: The query only retrieves log entries that are written in the interval specified by this field.
Filter field on the logs panel: The filter you specify is applied after the query completes. The value of this field is retained only for the current session.
Severity menu on the logs panel: The filter by severity level is applied after the query completes. The value of this field is retained only for the current session.
Query specified by the logs panel: The query field determines which logs are retrieved. If you don't specify a query, then all logs in the selected time interval are retrieved. If you do specify a query, then the logs panel displays a Show query icon, which displays the current query as a tooltip.

To modify the query, do one of the following:
- To permanently change the query, edit the logs panel.
- To temporarily change the query, use the dashboard toolbar to add a filter on a resource label. For example, if you add the dashboard-wide filter cluster_name: mycluster, then the query is updated to include resource.labels."cluster_name"="mycluster". For more information, see Add temporary filters and Add permanent filters.
Google Cloud project specified by the logs panel: To modify this setting, edit the logs panel.

The remainder of this section provides additional information about how to filter your log entries.

Filter log entries by date

To display the most recent logs in a logs panel, use the time fields in the dashboard toolbar. For example, select 1H. Provided you don't select a custom time range, the logs panel always displays the most recent logs, even when they are older than the selected time window.

When you are investigating an incident, you might want to restrict the log entries to those entries that are in a specific time interval. To configure the log entries to on show logs that are in a fixed time interval, do one or both of the following:

Select a time range by using the time-range selector.
Enter a timestamp clause in the Query Editor field of the configuration pane, and then select Apply filter.

When you specify a time interval, the logs panel isn't automatically refreshed.

Filter log entries by severity

When you are monitoring debugging a failure situation, you might only want to view error messages, and hide from the display all other logs entries. To configure a logs panel to display only those log entries that satisfy specific severity levels, do one or both of the following:

To apply a temporary filter, select a value for the Severity menu in the logs panel. This menu specifies a minimum severity level. For example, a value of "Warning" results in log entries with a severity of "Error" being retrieved, but those log entries with a severity of "Notice" aren't retrieved. By default, the value for this field is set to Default, which includes all log entries.

The setting for the Severity menu is discarded when you navigate away from the dashboard.
To apply a permanent filter, enter a severity clause in the Query Editor field of the configuration pane, and then select Apply filter.

Filtering log entries by a specific string or phrase lets you eliminate from view log entries that aren't of interest. For example, to view log entries that include a specific email address, you can filter by that address.

To apply a temporary filter so that you view only those log entries that contain a specific word or phrase, use the Filter field. The contains-test is case insensitive.

If you add multiple filters, then only logs that match all filters are shown because an implicit AND is inserted between adjacent filters. However, you can explicitly insert an OR operator:
- For example, when you add the filters Get and Compute, log entries that contain both words are shown.
- For example, when you add the filters Get, OR, and Compute, log entries that contain Get or Compute are shown.
The value of the Filter field isn't persistent.
To apply a permanent filter, modify the query stored in the Query Editor field, and then select Apply filter. For information about the query syntax, see Logging query language.

For example, to display only those log entries with a MESSAGE field whose value contains the string "lamp", add the clause jsonPayload.MESSAGE:"lamp" to the Query Editor field.

Example configurations

Here are some sample configurations:

To view the most recent logs, set the dashboard toolbar to 1H, leave the Query Editor field empty, and leave the logs panel fields at their default values.
To temporarily display only logs with a severity level of at least "Warning", select Warning in the logs panel Severity menu.

The value of the logs panel Severity menu is reset to Default when you refresh the page or navigate away from the dashboard.
To configure the logs panel to only show logs that have severity level of "Warning", in the Query Editor field, add the clause severity=WARNING.

The query stored in the Query Editor field is saved. Even when you refresh the page or reload the dashboard, the logs panel only shows logs that match the specified warning level.
To configure the logs panel to show only Compute Engine instance logs, in the Query Editor field, add the clause resource.type="gce_instance".

The message "No entries found matching current filter" is shown when no log entries satisfy the current settings. When you configure a query in the Query Editor field and also set the logs panel filters, it's possible that the settings conflict. For more information, see Logs panel displays "No entries found" message.

View log entries in the Logs Explorer

Logs panels display key fields of log entries, they don't display the complete log entry. To examine all fields of a log entry, do the following:

Select View in Logs Explorer in the logs panel.
In the dialog, select the Google Cloud project in which to open Logs Explorer.

Logs Explorer is opened in a new browser tab, and the filters you've selected in the logs panel are automatically applied.
Select the log entry that you want to view.

Add charts generated from a Log Analytics query

You can use the Log Analytics page to generate charts from your log entries by running SQL queries. These charts can be added to custom dashboards. For more information, see Save a chart to a custom dashboard.

This section describes how to add and configure an error-reporting panel by using the Google Cloud console. For information about adding an error-reporting panel to a dashboard by using the Cloud Monitoring API, see Dashboard with a ErrorReportingPanel widget.

To add an error-reporting panel to the dashboard by using the console, do the following:

In the navigation panel of the Google Cloud console, select Monitoring, and then select Dashboards:
Go to Dashboards
Select a dashboard or click Create dashboard.
In the toolbar, click Add widget.
In the Add widget dialog, go to the Visualization section, and then select select Error Reporting panel.
Select the project whose error groups are shown, click Projects, and then make a selection.
Optional: Update the title.
Optional: To restrict the resources for which errors are displayed, do the following:
1. Click All Resources and select a resource.
2. After you select a resource, you can select Apply, or your can use the additional menus to refine the filter criteria:
  - To show errors for App Engine resources, select GAE Services, and then click Apply.
  - To show errors for a specific App Engine service, select GAE Services, select the service, and then click Apply.
To apply your changes to the dashboard, in the toolbar, click Apply. To discard your changes, click Cancel.

The following example shows an error-reporting panel:

$Example of an error-reporting panel configuration pane.$
To save your modified dashboard, in the toolbar, click Save.

Troubleshoot

What's next

For information about how Cloud Monitoring provides visibility to metrics and logs stored in multiple Google Cloud projects, see Metrics scope overview.
For information about how to configure a logs panel by using the Cloud Monitoring API, see Dashboard with a LogsPanel widget.
To learn more about error groups, see Error Reporting overview.