Troubleshoot charts

This page explains some of the behaviors that you might see when using metrics collected by your Google Cloud projects.

Gaps in chart data

Charts occasionally exhibit gaps in the data. For example, the following screenshot illustrates this behavior for the metric loadbalancing.googleapis.com/https/total_latencies:

Example of chart with gaps in values.

Gaps in the data can occur when data is written very infrequently. Cloud Monitoring collects metrics by sampling, and with very sparse data, there might not be any data available when a sample is collected. For example, gaps in load-balancer metrics might occur when the request count is 0.05 queries per second (QPS) or less. When the query rate increases, the gaps disappear.

Dashboard isn't displaying all events

You configure a dashboard to display events; however, only recent events are displayed. Older events aren't shown.

To resolve this problem, do one of the following:

  • Use a shorter time range. For example, display data for the most recent hour instead of for the most recent day or select a custom time range.

  • Add filters.

For performance reasons, dashboards are limited in the number of events that they can display. When these limits are reached, the dashboards display only the most recent events.

Chart doesn't display any data

You configure a chart but the chart displays an error message instead of data:

  • When there isn't data available for the time period specified for your chart, the following message is shown:

    No data is available for the selected time frame.
    

    To resolve the problem, try the following:

    • Increase the timeframe for the chart. Some metric types, such as those that report quota usage, provide only one sample per day.

    • Change the selected metric type for your chart. It is possible that the metric type specified by your chart is inactive. That is, that there isn't data for that metric type.

  • When the metric type or resource type specified by the query isn't recognized, the following message is shown:

    An error occurred requested data. One or more resources could not be found.
    

    To resolve this problem, try the following:

    • Verify that the metric type and resource type specified in the query are defined. You can use the menu-driven interface for charts to search for the metric type.

    • Verify the spelling of the metric type and resource type in the query.

Alert chart doesn't display any data

A dashboard contains an alert chart, but it doesn't display any data.

To resolve this situation, try the following:

  • View the configuration of the alert chart and ensure an alerting policy is selected. If no alerting policy is displayed, then associate the alert chart with an alerting policy or delete the alert chart from the dashboard.

    If an alert chart is added to a dashboard and then the referenced alerting policy is deleted, then the alert chart stops displaying data. Deleting an alerting policy doesn't delete alert charts that reference that policy.

  • View the details of the alerting policy and verify that it contains a single condition.

    If an alert chart is added to a dashboard and then the referenced alerting policy is modified to contain multiple conditions, then the alert chart stops displaying data. To resolve this condition, associate the alert chart with a single-condition alerting policy or delete the alert chart from the dashboard.

  • Increase the period of time range. The time-range selector must be set to a value longer than the sampling period of the data. While most metrics are sampled every 60 seconds, some metrics are sampled infrequently. For example, the serviceruntime.googleapis.com/quota/limit metric is sampled every 86,400 seconds (1 day).

Chart with multiple metrics displays an error message

You are configuring a chart to display multiple metrics and the chart displays the following error message:

    The units for queries using the same Y-axis must be the same for the chart
    to render correctly.

This error message indicates that a chart has at least two metrics mapped to the same Y-axis, but the units of those time series are different. For example, a chart is configured to display a metric that is reported as a percentage and a metric that reports a count.

To resolve this error, do the following:

  1. If your chart contains metrics with more than two unit types, then remove metrics until you have only two unit types.

  2. Assign all metrics with one unit type to the left Y-axis and all metrics with the other unit type to the right Y-axis.

Chart with multiple metrics displays unexpected data

A chart is configured to display multiple metrics, and filters were added to show only specific time series. However, the chart shows time series that don't satisfy the filters.

To resolve this situation, ensure that you apply filters to each metric that is charted.

For example, the following screenshot shows the configuration pane of Metrics Explorer for a chart that displays the number of bytes read, and written, by virtual machine (VM) instances:

Example of Metrics Explorer with two metric types.

In the screenshot, there is one heading for each metric type that is displayed by the chart.

For the example, if you want the chart to display time series only for VMs that are in the us-central1-a zone, then add the filter zone = us-central1-a to each query.

Unable to select filter values for MQL-defined charts

You create charts on a custom dashboard and configure those charts with Monitoring Query Language (MQL). You then create permanent dashboard filters. When you view the dashboard, you attempt to change the default value of the filter, but the menu of label values isn't displayed or the menu of label values isn't correct:

Values for a dashboard-wide filter aren't loaded.

To resolve this issue, enter the filter value, and then select Apply. For example, if the filter key is zone, then you might enter us-central1-a.

Logs panel reports invalid query

You see an error message "Error: Invalid query", and the logs panel doesn't show any log entries.

To resolve this issue, reduce the number of projects whose logs the logs panel is configured to display.

For more information about this failure condition, see Query returns an error.

Logs panel reports no entries found

You see the error message "No entries found matching current filter" in a logs panel. There are multiple reasons why you might see this message. Check the following:

  • For the each project whose log entries you want to view, ensure that one of the following is true:

    • You have the Logs Viewer (roles/logging.viewer) role.
    • For custom roles, you have the following permissions:

      • logging.logs.list
      • logging.logServiceIndexes.list
      • logging.logServices.list
  • Ensure that the clauses in the query editor don't conflict with the settings shown in the logs panel.

    For example, when the Query Editor field contains severity=NOTICE, the logs panel retrieves all logs that have a severity level of exactly NOTICE. If the logs panel Severity menu is set to "Warning", then the logs panel displays "No entries found matching current filter". The message is shown because none of the retrieved log entries have a severity level of at least "Warning".

    In contrast, when the Query Editor field contains severity=WARNING and the logs panel Severity menu is set to "Notice", you might see log entries. With this configuration, the logs panel retrieves logs whose severity level is WARNING. Next, that list of logs is filtered to display only those whose severity is at least "Notice", and all of the retrieved logs have at least that severity level.

Logs panel indicates it displays partial data

You see a warning indicator and the message "Showing partial results" on a logs panel.

When you don't have permission to view logs for a Google Cloud project whose logs you've selected to display, this indicator is shown. To remove the warning indicator, do one of the following:

  • Contact your system administrator and ask them to do one of the following for each Google Cloud project whose logs you want to view::

    • Grant you the role of Logs Viewer (roles/logging.viewer).
    • For custom roles, grant you the following permissions:

      • logging.logs.list
      • logging.logServiceIndexes.list
      • logging.logServices.list
  • Change the configuration of the logs panel to show logs only for Google Cloud projects where you have permission to view log entries.

API call to create dashboard with a logs panel fails

There are multiple reasons why you might see an error message when you create a dashboard with a logs panel:

  • When the logs panel doesn't meet the minimum height or width requirements, the following error is shown:

    Unable to save dashboard: Field mosaicLayout.tiles[0] has an invalid value: LogsPanel must have a minimum width of 3 and height of 3.
    

    To resolve this condition, in the dashboard JSON representation, ensure the height and width fields of the logs panel widget meet the minimum requirements.

  • When the logsPanel.filter field exceeds 2,000 characters, the following error is shown:

    Unable to save dashboard: filter cannot be greater than 2000 characters.
    

    To resolve this condition, shorten the filter.

  • When the resourceNames field is missing, you see the following error message:

    Unable to save dashboard: Dashboard is missing required field mosaicLayout.tiles[0].widget.logsPanel.resourceNames.
    

    To resolve this condition, add a resourceNames field to the dashboard JSON.

  • When the syntax for the resourceNames is incorrect, the following error is similar to the following:

    Unable to save dashboard: Field mosaicLayout.tiles[0].widget.logsPanelWidget.resourceNames has an invalid value of "a-project": resource name must have the format `projects/{project}`."
    

    To resolve this condition, replace a-project with projects/a-project in the dashboard JSON.

  • For information about how to resolve syntax errors in the filter field, see Troubleshooting: syntax issues.

For an example of a correctly formatted JSON file, see Dashboard with a LogsPanel widget.

Importing Grafana dashboards

For information about using and troubleshooting the dashboard importer, see Import Grafana dashboards into Cloud Monitoring.

For information about problems with the conversion of the dashboard contents, see the importer's README file.