Add permanent filters to a custom dashboard

This document describes how you can add permanent, dashboard-level filters to your custom dashboards. Permanent filters—which let you modify the data displayed by a dashboard without having to modify each widget— have a default state and they can apply to the entire dashboard or only to specific widgets. The dashboard toolbar displays each permanent filter, along with a menu, which lets you temporarily change the value of the filter.

When you want to define a default presentation of data on a dashboard along with menu-driven options to modify that display, add permanent filters. For example, if you want to view metrics for all zones and also want to view metrics for a specific zone, then create a permanent filter based on the zone label.

You can create and apply permanent filters by using the Google Cloud console or the Cloud Monitoring API. This document describes how to use the Google Cloud console. For information about using the Cloud Monitoring API, see Create and manage dashboards by API: Dashboard filters.

For information about how to add and modify temporary groupings and filters, see Add temporary filters to a custom dashboard.

How dashboard filters work

A dashboard filter specifies a label and a value. The time series that the dashboard displays determine the labels available for filtering. How a widget supports a dashboard-wide filter depends on the type of data the widget displays, and the configuration of the widget:

For widgets that display time-series data, such as a line charts, the following rules apply:
- If the time series displayed by the widget doesn't support the label in a filter, then the dashboard-wide filter doesn't change the displayed data.
  
  For example, if you add a filter on the label instance_id, but a chart on the dashboard doesn't display data with an instance_id label, then the filter doesn't change the displayed data.
- If the widget specifies a value for a label, then the dashboard-wide filter doesn't change the displayed data.
  
  For example, if you add a filter on the label instance_id, but a chart on the dashboard includes a filter based on the instance_id label, then the dashboard-wide filter doesn't change the displayed data.
- Otherwise, the chart filters the data it displays to only those time series that satisfy the dashboard-wide filter.
For logs panel widgets, dashboard-wide filters that are based on a resource label update the query. For example, if you add the dashboard-wide filter cluster_name: mycluster, then the query is updated to include resource.labels."cluster_name"="mycluster".

You can add multiple dashboard-wide filters. These filters are joined with a logical AND.

Before you begin

To get the permissions that you need to add or modify permanent filters by using the Google Cloud console, ask your administrator to grant you the Monitoring Editor (roles/monitoring.editor) IAM role on your project. For more information about granting roles, see Manage access.

You might also be able to get the required permissions through custom roles or other predefined roles.

For more information about roles, see Control access with Identity and Access Management.

Create a permanent filter

When you design a dashboard, you might identify multiple ways to view the data the dashboard displays. For example, when a dashboard displays metrics for VM instances, you might want to view metrics for all VMs and you might want to view metrics for VMs in a specific zone. You could create a permanent filter for the zone label with the default set to all zones, *. When viewing the dashboard, you can use the menu on the zone filter to select a specific zone.

You can restrict a permanent filter to specific widgets. Consider a scenario where you want a side-by-side view of metrics for VMs in two different zones. You construct two collections of charts and create a filter for each collection. Your default configuration satisfies your original objective. However, when viewing the dashboard, the toolbar lists your variables and includes menus to let you select different values. You can use these menus to view data for different zones.

Permanent filters that apply only to specific widgets are sometimes called template variables. After you create a filter of this type, when you add a widget to your dashboard, you can also apply the filter to that widget. For more information, see Apply a permanent filter to a widget.

To create a permanent filter, do the following:

In the navigation panel of the Google Cloud console, select Monitoring, and then select Dashboards:
Go to Dashboards
In the dashboard toolbar, click Settings, and then select Manage Filters.
Click Label and then select a label. For example, you might select zone.
Click Default value and then select the default value for the label. To match all values, select *.

Note: When dashboards contain MQL-defined charts, the menu of values for a filter might not load or the menu might not contain the correct values. However, you can enter the filter value manually.

After the filter is created, it is added to your dashboard toolbar. To temporarily change the filter value, click on the filter and make a selection from the menu.
Do one of the following:
- To apply the filter to all widgets on the dashboard, select Apply to all, and then select Apply.
- To apply the filter to some widgets on the dashboard:
  1. Clear Apply to all.
  2. In the Variable name text box, enter a display name for the variable.
  3. Expand the Apply to charts menu, select the widgets that the filter applies to, and then select OK.
    
    Note: For log panels and charts configured by using MQL or PromQL, you must edit the widget configuration. You can't select these widgets by using the Apply to charts menu. For more information, see Apply a permanent filter to a widget.
  4. Click Apply.

Apply a permanent filter to a widget

To apply a permanent filter to a widget, you must associate that widget with the variable:

For charts that you configure by using menus, you can perform this association when creating or editing the filter.
For the logs panel, edit the widget and configure the default query. Be sure to follow the guidance in the section titled Syntax to apply a permanent filter.
For MQL- or PromQL-defined charts and tables, edit the chart or table, and then modify the MQL or PromQL. Ensure you follow the guidance in the section titled Syntax to apply a permanent filter.

Syntax to apply a permanent filter

For example, the following shows the partial JSON representation of a dashboard that defines a template variable and a dashboard-wide filter:

{
  "dashboardFilters": [
      {
        "filterType": "RESOURCE_LABEL",
        "labelKey": "instance_id",
        "stringValue": "3133577226154888113",
        "templateVariable": "iid"
      },
      {
        "filterType": "RESOURCE_LABEL",
        "labelKey": "zone"
      }
    ],
  "displayName": "Illustrate Template Variables",
  ...

In the displayed JSON, the first entry in the dashboardFilters structure is for a template variable with the name iid and a dashboard-wide filter with the label key zone. The template variable is an alias of the label instance_id.

The data structure for a template variable doesn't list the widgets to which it applies. Instead, you associate a widget with a template variable by modifying the widget's query to include a reference to the variable. When the widget is displayed on the dashboard, the value of the template variable is resolved.

See the following sections for how to annotate logs panels and charts:

Logs panel
MQL-defined charts and tables
PromQL-defined charts and tables
Charts and tables defined with time-series filters

Logs panel

To configure a logs panel to filter the display based on the value of a template variable, add the variable to the query pane. The following example illustrates a query that filters by the value of the template variable iid:

${iid}

Before the logs panel queries for logs to display, the template variable is resolved. In this example, if the value of the template variable is "12345", then ${iid} is replaced with the statement resource.labels."instance_id"="12345".

You can also include only the value of a template variable in a query. We recommend that you only use the value as part of a filter defined with a regular expression. For example, the following query uses a regular expression to match log entries that have a JSON payload that contains the described string:

jsonPayload.message=~"Connected to instance: ${iid.value}"

If you've configured a query for the logs panel and then select the button to open the Logs Explorer, the template variables are resolved before the Logs Explorer is opened.

The following table shows how the template variable is resolved by the logs panel:

Syntax	Selected Value	Resolved logs panel expression
`${iid}`	`12345`	`resource.labels."instance_id"="12345"`
`${iid}`	`*`	`""`
`${iid.value}`	`12345`	`12345`
`${iid.value}`	`*`	`.*`

MQL-defined charts and tables

When you use Monitoring Query Language (MQL) to configure a chart, append a pipe and the variable to the query string:

fetch gce_instance
| metric 'compute.googleapis.com/instance/cpu/utilization'
| every 1m
| ${iid}

Before the chart queries for the time series to display, the template variable is resolved. In this example, if the value of the template variable is "12345", then ${iid} is replaced with the statement filter (resource.instance_id == '12345'). This filter matches time series that have a label named resource.instance_id, and only when the value of that label is exactly 12345.

When you want to filter time series by using a regular expression, then configure the query to include only the value of the template variable. To illustrate the syntax, the following shows how to use a regular expression to determine if the value of the label resource.instance_id contains the value of the template variable iid:

fetch gce_instance
| metric 'compute.googleapis.com/instance/cpu/utilization'
| filter resource.instance_id=~"${iid.value}"
| group_by 1m, [value_utilization_mean: mean(value.utilization)]
| every 1m

The following table shows how the template variable is resolved for MQL queries:

Syntax	Selected Value	Resolved MQL expression
`${iid}`	`12345`	`filter (resource.instance_id == '12345')`
`${iid}`	`*`	`filter (true)`
`${iid.value}`	`12345`	`12345`
`${iid.value}`	`*`	`.*`

PromQL-defined charts and tables

When you define a chart using PromQL, append to the query string the variable wrapped by braces:

compute_googleapis_com:instance_cpu_utilization {
    project_id="my-project", ${iid}
}

Similar to MQL, when you define a widget with PromQL, the query can extract only the value of the template variable. We recommend that you only use the value as part of a filter defined with a regular expression. To illustrate the syntax, the following shows how to use a regular expression to determine if the value of the label instance_id contains the value of the template variable iid:

compute_googleapis_com:instance_cpu_utilization{
    instance_id=~"${iid.value}"
}

The following table shows how the template variable is resolved for PromQL queries:

Syntax	Selected Value	Resolved PromQL expression
`${iid}`	`12345`	`instance_id == '12345'`
`${iid}`	`*`	`noop_filter=~".*"`
`${iid.value}`	`12345`	`12345`
`${iid.value}`	`*`	`.+`

Charts and tables defined with time-series filters

When you define a chart using time-series filters, append the variable to the filter string:

"filter": "metric.type=\"compute.googleapis.com/instance/cpu/utilization\"
           resource.type=\"gce_instance\" ${iid}"

Unlike MQL- and PromQL-defined charts, you can't use the value of a template variable in a time-series filter.

The following table shows how the template variable is resolved:

Syntax	Selected Value	Resolved filter expression
`${iid}`	`12345`	`resource.instance_id == "12345"`
`${iid}`	`*`	Omitted
`${iid.value}`	`12345`	Not supported
`${iid.value}`	`*`	Not supported

Modify a permanent filter

To temporarily change the value of any filter displayed on the dashboard toolbar, click the filter name, and either enter a value or make a selection from the menu.

To change the configuration of a permanent filter, do the following:

In the navigation panel of the Google Cloud console, select Monitoring, and then select Dashboards:
Go to Dashboards
In the dashboard toolbar, click Settings, and then select Manage Filters.
Find the filter that you want to modify and then use the menus to make your modifications.
Click Apply.

Delete a permanent filter

In the navigation panel of the Google Cloud console, select Monitoring, and then select Dashboards:
Go to Dashboards
In the dashboard toolbar, click Settings, and then select Manage Filters.
Find the filter that you want to remove and then click Delete.
Click Apply.