Create and manage variables and pinned filters

This document describes how you can pin dashboard-level filters so that these filters are permanent. Pinned filters—which let you modify the data displayed by a dashboard without having to modify each widget—have a default state and they can apply to the entire dashboard or only to specific widgets. The dashboard toolbar displays each pinned filter, along with a menu, which lets you temporarily change the value of the filter.

Variables are similar to pinned filters, but they apply only to specific widgets. This document describes how to create variables and how to apply them with a widget. The dashboard toolbar displays each variable, along with a menu, which lets you temporarily change the value of the variable.

This document describes how to use the Google Cloud console to create variables and pinned filters. For information about using the Cloud Monitoring API, see Create and manage dashboards by API: Dashboard filters.

For information about how to add and modify temporary groupings and filters, which apply only to the current session, see Add temporary filters to a custom dashboard.

About pinned filters

A pinned filter contains a label, one or more default values, and a list of all possible values. The list of all possible values is derived dynamically from the time-series data displayed on your dashboard. For information about the structure of a filter, see DashboardFilter.

You don't apply a pinned filter with individual widgets. Instead, the following rules are used to determine whether the pinned filter can change the data displayed by the widget:

  • If the widget displays data that includes the label key specified by a pinned filter and if the widget doesn't specify a value for that label key, then the pinned filter can change the data that is displayed.

    For example, suppose you add the pinned filter cluster_name: mycluster. Then charts automatically display only those time series that have a cluster_name label with a value of mycluster. Similarly, the queries on your logs panel widgets are modified to resource.labels."cluster_name"="mycluster", and your charts automatically display only those time series that include the label in the pinned filter.

  • A pinned filter is ignored by a widget when the widget includes a filter for the same label key, or when the data displayed by the widget doesn't contain the label key specified in the pinned filter.

About variables

A variable has a name and type. A variable is applied to a particular widget by modifying the query that the widget contains. For information about the structure of a filter, see DashboardFilter.

Variables can either be label-based or only have a value:

  • Label-based variables are like pinned filters. These variables contain a label key, one or more default label values, and a list of all possible label values. The list of all possible values is derived dynamically from the time-series data displayed on your dashboard.

  • Value-only variables contain one or more default values, and they might list all possible values. If you don't specify a default value, then the wildcard operator, (*), is selected for the default value. To define the set of all possible values, you either provide an array of values or you write a SQL query.

    There are three types of value-only variables: Custom, Text Input, and SQL Query.

    • Custom: Use when you want to enumerate both the list of all possible values and also set one or more default values. For example, you might set the list of all possible values to prod, staging, dev, and set the default value to prod.

    • Text Input: Use when you don't know all possible values. For example, you might want to filter by URL, but when you create the variable, you don't know the exact list of URLs. Then, when you use the filter menu for the widget, you can enter in new URLs, and then save the changes for the current session or to the variable definition.

    • SQL Query: Use when you want to generate the list of possible values by using a SQL query. These variables are intended to be applied to charts created by using Log Analytics.

      For these variables, ensure that the query result is table with one column, that the data type of that column is string, and that each row in the table contains a unique string value:

      SELECT log_name
FROM `TABLE_NAME_OF_LOG_VIEW`
GROUP BY log_name

      The variable should query the same table as the chart.

To apply a variable to a widget, you update the query contained by the widget. If you use the Google Cloud console, then for label-based variables and some widget types, you can perform this association by using a menu selection. However, you can always edit the query of the widget to apply a variable. For more information, see Apply a variable to a widget.

Before you begin

To get the permissions that you need to add or modify variables or pinned filters by using the Google Cloud console, ask your administrator to grant you the Monitoring Editor (roles/monitoring.editor) IAM role on your project. For more information about granting roles, see Manage access to projects, folders, and organizations.

You might also be able to get the required permissions through custom roles or other predefined roles.

For more information about roles, see Control access with Identity and Access Management.

Create and pin a filter

To create and pin a filter that applies to all dashboard widgets that support filters, do the following:

  1. In the Google Cloud console, go to the  Dashboards page:

    Go to Dashboards

    If you use the search bar to find this page, then select the result whose subheading is Monitoring.

  2. In the dashboard toolbar, click Settings.
  3. In the Filters section, click Add a filter.
  4. Click Label and then select a label. For example, you might select zone.
  5. To let a user select multiple values from the variable's value menu, select Multi-select. When Multi-select isn't enabled, users can only select one option from the value menu.

  6. Click Default value and then select the default value, or values, for the label. To match all values, select *.

    After the filter is created, it is added to your dashboard toolbar. To temporarily change the filter value, select the filter, and then make a selection from the menu.

  7. Click Done and then Apply.

  8. To save your modified dashboard, in the toolbar, click Save.

Create a variable

To create a variable that applies to specific dashboard widgets, do the following:

  1. In the Google Cloud console, go to the  Dashboards page:

    Go to Dashboards

    If you use the search bar to find this page, then select the result whose subheading is Monitoring.

  2. In the dashboard toolbar, click Settings.
  3. In the Variables section, click Add a variable.

  4. Select the type of variable and then complete the dialog.

    To create a label-based variable, select Label. Use this option when you want a variable that has a label key and label value, and that can be used like a filter. The list of all possible values is derived dynamically from the time-series data displayed on your dashboard.

    For value-only variables, you have the following options:

    • SQL: Use this option when you want the results of a SQL query to generate the list of all possible values.

      Enter one or more default values, and enter a SQL query that generates the list of all possible values. Ensure that the query result is table with one column, that the data type of that column is string, and that each row in the table contains a unique string value.

    • Custom: Use this option when you want to specify both the list of possible values and also a default value. For example, you might set the Values field to prod, staging, dev, and set the Default value to prod.

    • Text Input: Use this option when you want to specify a list of values that are applied as defaults.

  5. Enter a name for the variable. The name is displayed on the toolbar of the dashboard.

  6. Click Done and then Apply.

  7. Apply variable to logs panel widgets and to widgets configured with SQL, MQL, or PromQL. For more information, see Apply a variable to a widget.

  8. To save your modified dashboard, in the toolbar, click Save.

Apply a variable to a widget

To apply a variable to a widget, the query used by the widget must be edited:

  • For charts that use SQL, PromQL, or MQL queries, and for logs panel widgets, you must edit the widget and modify the query. This section describes how to make those edits.

  • For charts that you configured by using menus or by using direct-filter mode, Monitoring edits the query when you select the widget by using the Apply to charts menu.

To learn how to apply a label-based variable or value-only variable to a widget, see the following sections:

Create filters and variables

Console

For information about how to use the Google Cloud console to create pinned filters and variables, see the following documents:

API

To define pinned filters and variables, use the dashboardFilters data structure.

  • To create a variable, set the value of the templateVariable field to the name of the variable. Omit this field or set the value to an empty string when you want to create a pinned filter.
  • To create a pinned filter or a label-based variable, you must specify the labelKey field. Omit this field when you want a value-only variable.

  • Set the default value for the filter or variable. The configuration of this field determines whether a user can select exactly one option from the menu of value, or whether they can select multiple values.

    • To set a single default value and to restrict users to selecting exactly one option in the values menu, set the valueType field as STRING and also set the stringValue field:
    "valueType": "STRING",
"stringValue": "my-default-value",
    • To set at least one default value and to let users select multiple options in the values menu, set the valueType field as STRING_ARRAY and also set the stringArrayValue field. In the following example, there are three default values.
    "valueType": "STRING_ARRAY",
"stringArrayValue": {
  "values": [ "a", "b", "c" ]
},

  • Optional: To specify the list of all possible values for a value-only variable, set either the stringArray field or the timeSeriesQuery field. If you specify a query, it must be an analytics query.

For example, consider the following dashboardFilters object:

{
  "dashboardFilters": [
      {
        "labelKey": "zone"
        "stringValue": "us-central1-c",
        "valueType": "STRING",
        "filterType": "RESOURCE_LABEL"
      },
      {
        "labelKey": "instance_id",
        "stringValue": "3133577226154888113",
        "valueType": "STRING",
        "filterType": "RESOURCE_LABEL",
        "templateVariable": "my_label_based_variable"
      },
      {
        "filterType": "VALUE_ONLY",
        "templateVariable": "my_value_only_variable",
        timeSeriesQuery: {
          opsAnalyticsQuery: {
            sql: "
              SELECT log_name
              FROM `MY_TABLE`
              GROUP BY log_name
            ",
          }
        }
      }
    ],
  "displayName": "Illustrate Variables",
  ...
}

The previous JSON defines one pinned filter and two variables:

  • The pinned filter has the label key of zone, which is is displayed on the toolbar. The valueType and stringValue fields specify the single default value. For more information, see the API references page for the dashboardFilters data structure.

  • The label-based variable has the name my_label_based_variable, and its label key is instance_id. The default value for this variable is set to a specific instance ID. You can also configure the default value by using an array. On the toolbar, the filter is displayed with the name my_label_based_variable.

  • The value-only variable is named my_value_only_variable. This entry doesn't specify a default value, so the wildcard operator, (*), is automatically applied. Additionally, this variable uses a SQL query to generate the list of possible values for the variable.

Note that the dashboardFilters object doesn't list the widgets to which the variable applies. To apply a variable to a widget, you modify the query for the widget.

General syntax to dereference a variable

For all widgets, except those that are defined by SQL, use the following syntax to apply a variable to a query:

  • To apply a label-based variable and have the label key and the label value resolved into a valid filter expression for the query language, use ${my_label_based_variable}.

  • To apply only the value of a label-based variable, use ${my_label_based_variable.value}. The comparison must use a regular expression.

  • To apply only the value of a value-only variable, use ${my_value_only_variable}. For value-only variables, don't include a .value clause. The comparison must use a regular expression.

Logs panel widgets

To apply a variable to a logs panel widget, update the queries pane. The syntax for these widgets follows that specified under General syntax.

Console

For example, the following query uses a regular expression to compare the value of the jsonPayload.message field with a string value that includes the value of a label-based variable:

jsonPayload.message=~"Connected to instance: ${my_label_based_variable.value}"

As another example, consider a value-only variable, value_only_severity_variable, and assume that in the menu of values, three values are selected: ERROR, INFO, and NOTICE. Next, you add the following to the query pane of your logs panel widget:

severity =~ "${value_only_severity_variable}"

The following illustrates the rendered form:

severity =~ "^(ERROR|INFO|NOTICE)$"

API

For example, the following JSON illustrates how to apply a label-based variable to the query of a logs panel widget:

"logsPanel": {
  "filter": "${my_label_based_variable}",
  "resourceNames": [
    "projects/1234512345"
  ]
},

For example, the following query uses a regular expression to compare the value of the jsonPayload.message field with a string value that includes the value of a label-based variable:

"logsPanel": {
  "filter": "resource.type=\"gce_instance\"\n
             resource.labels.project_id=~\"${my_label_based_variable.value}\"\n",
  "resourceNames": [
    "projects/012345"
  ]
}

As another example, consider a value-only variable, value_only_severity_variable, and assume that three values are selected in the menu: ERROR, INFO, and NOTICE. Next, you add the following to the query pane of your logs panel widget:

"logsPanel": {
  "filter": "severity =~ \"${value_only_severity_variable}\"\n",
  ...
}

The following illustrates the query as executed by the logs panel widget:

severity =~ "^(ERROR|INFO|NOTICE)$"

If you've configured a query for the logs panel and then select the button to open the Logs Explorer, the variables are resolved before the Logs Explorer is opened.

The following table illustrates how the example variables are are resolved by the logs panel. As previously mentioned, when only the value of a variable is used, you must use a regular expression as the comparison operator:

Syntax Selected
Value		 Resolved logs panel expression
${my_label_based_variable} 12345 resource.labels."instance_id"="12345"

The example variable is based on the resource label instance_id.
${my_label_based_variable} * ""
${my_label_based_variable.value}
${my_value_based_variable}		 12345 12345
${my_label_based_variable.value}
${my_value_based_variable}		 * .*

Charts with PromQL queries

To apply a label-based variable to a chart that has a PromQL query, follow the guidance listed in General syntax.

Console

For example, the following query relies on the label-based variable, my_label_based_variable, being resolved into a filter expression:

compute_googleapis_com:instance_cpu_utilization{
    monitored_resource="gce_instance", ${my_label_based_variable} }

You can also modify query to resolve only the value of a variable. The following example uses a regular expression to compare the value of a label-based query to the instance_id:

compute_googleapis_com:instance_cpu_utilization{
    instance_id=~"${my_label_based_variable.value}"
}

If you have a value-only variable, then omit the .value clause. For example, to filter by zone using a value-only variable, the query would include something like the following:

zone=~"${my_value_only_variable}"

API

For example, the following JSON illustrates a query that relies on the label-based variable, my_label_based_variable, being resolved into a filter expression:

"timeSeriesQuery": {
  "prometheusQuery": "avg_over_time(
    compute_googleapis_com:instance_cpu_utilization{
      monitored_resource=\"gce_instance\",
      ${my_label_based_variable}
      }[${__interval}])",
  "unitOverride": "",
  "outputFullDuration": false
},

You can also modify query to resolve only the value of a variable. The following example uses a regular expression to compare the value of a label-based query to the instance_id:

"timeSeriesQuery": {
  "prometheusQuery": "avg_over_time(
    compute_googleapis_com:instance_cpu_utilization{
    monitored_resource=\"gce_instance\",
    instance_id=~\"${my_label_based_variable.value}\"
    }[${__interval}])",
  "unitOverride": "",
  "outputFullDuration": false
},

If you have a value-only variable, then omit the .value clause. For example, to filter by zone using a value-only variable, the query would include something like the following:

zone=~\"${my_value_only_variable}\"

The following table illustrates how the example variables are are resolved by the PromQL. As previously mentioned, when only the value of a variable is used, you must use a regular expression as the comparison operator:

Syntax Selected
Value		 Resolved PromQL expression
${my_label_based_variable} 12345 instance_id == '12345'

The example variable is based on the resource label instance_id.
${my_label_based_variable} * noop_filter=~".*"
${my_label_based_variable.value}
${my_value_based_variable}		 12345 12345
${my_label_based_variable.value}
${my_value_based_variable}		 * .+

Charts with SQL queries

When you want to apply a variable to a SQL-defined widget, update the WHERE clause to reference the value of the variable. For all variables, prefix the variable name with the "at" sign, for example: @variable_name. For label-based variables, append .value to the variable name, @my_label_based_variabe.value.

For SQL queries, the variable substitution relies on BigQuery, and is SQL-injection safe. For more information, see Running parameterized queries.

Console

Because SQL doesn't interpret the wildcard operator to mean "any value", we recommend that you always use an IF statement when applying variables to a SQL query. The following example illustrates usage for a value-only variable whose data type is a string:

WHERE IF(@my_value_only_variable = "*", TRUE, log_name = @my_value_only_variable)

When the menu option for the variable lets users select multiple values, you must cast the value of the variable to a GoogleSQL data type by using the CAST function. The following query illustrates this syntax:

IF(ARRAY_LENGTH(CAST(@my_value_only_variable)) = 0, TRUE,
   severity IN UNNEST(@my_value_only_variable))

The IF statement shown in the previous examples is recommended because SQL doesn't interpret the wildcard operator to mean "any value". Therefore, if you omit the IF statement and if you select the wildcard operator, then the result of the query is an empty table. In the second example, the UNNEST function converts the array to a table.

To add a properly-formatted WHERE clause, do the following:

  1. Edit the widget.
  2. In the toolbar, select Insert variable filter, and then select the variable that you want to apply to the WHERE clause.
  3. In the dialog that opens, review the generated code and then click Copy and close.

  4. Paste the copied code into the Query pane and make any necessary edits.

    For example, suppose you create a variable named LogName that generates a list of log names and outputs the result in a table with a single column named log_name. Next, you create a chart, select Insert variable filter, and then select the variable LogName. The following code is generated:

    WHERE IF(@LogName = '*', TRUE, LogName = @LogName)

    In this example, you need to edit the generated code and replace LogName = with log_name =, so that the table join can occur:

    WHERE IF(@LogName = '*', TRUE, log_name = @LogName)

  5. Click Run and then Apply.

  6. To save your modified dashboard, in the toolbar, click Save.

API

Because SQL doesn't interpret the wildcard operator to mean "any value", we recommend that you always use an IF statement when applying variables to a SQL query. The following example illustrates usage for a value-only variable whose data type is a string:

WHERE IF(@my_value_only_variable = "*", TRUE, log_name = @my_value_only_variable)

For example, the following shows a partial JSON representation of a chart that displays the results of a SQL query. To support filtering the results by the name of a log, a WHERE clause was added that references the variable named LogName:

"plotType": "STACKED_BAR",
"targetAxis": "Y1",
"timeSeriesQuery": {
  "opsAnalyticsQuery": {
    "queryExecutionRules": {},
    "queryHandle": "",
    "sql": "SELECT\n timestamp, severity, resource.type, log_name, text_payload, proto_payload, json_payload\n
            FROM\n `my-project.global._Default._Default`\n
            WHERE \n IF (@LogName = \"*\", TRUE, log_name=@LogName)\nLIMIT 10000"
  }
}

The variable LogName also issues a query to determine the list of possible log names:

"dashboardFilters": [
  {
    "filterType": "VALUE_ONLY",
    "templateVariable": "LogName",
    "valueType": "STRING",
    "timeSeriesQuery": {
      "opsAnalyticsQuery": {
        "savedQueryId": "",
        "sql": "SELECT log_name FROM `my-project.global._Default._Default` GROUP BY log_name LIMIT 1000",
        "queryHandle": ""
      },
      "unitOverride": "",
      "outputFullDuration": false
    }
  }
],

When the menu option for the variable lets users select multiple values, you must cast the value of the variable to a GoogleSQL data type by using the CAST function. The following query illustrates this syntax:

IF(ARRAY_LENGTH(CAST(@my_value_only_variable)) = 0, TRUE,
   severity IN UNNEST(@my_value_only_variable))

The IF statement shown in the previous examples is recommended because SQL doesn't interpret the wildcard operator to mean "any value". Therefore, if you omit the IF statement and if you select the wildcard operator, then the result of the query is an empty table. In the second example, the UNNEST function converts the array to a table.

Charts with MQL queries

To apply a label-based variable to a chart that has a MQL query, append a pipe, (|), and then follow the guidance listed in General syntax.

When you use the menu-driven interface to create a chart that displays time-series data, your selections are converted into a Monitoring filter

Console

For example, the following query relies on a label-based variable, my_label_based_variable, being resolved into a filter expression:

fetch gce_instance
| metric 'compute.googleapis.com/instance/cpu/utilization'
| every 1m
| ${my_label_based_variable}

You can also modify query to resolve only the value of a variable. The following example uses a regular expression to compare the value of a label-based query to the instance_id:

fetch gce_instance
| metric 'compute.googleapis.com/instance/cpu/utilization'
| filter resource.instance_id=~'${my_label_based_variable.value}'
| group_by 1m, [value_utilization_mean: mean(value.utilization)]
| every 1m

If you have a value-only variable, then omit the .value clause. For example, to filter by zone using a value-only variable, the query would include something like the following:

resource.zone=~'${my_value_only_variable}'

API

For example, the following JSON illustrates a query that relies on a label-based variable, my_label_based_variable, being resolved into a filter expression:

"timeSeriesQuery": {
  "timeSeriesQueryLanguage": "fetch gce_instance\n
    | metric 'compute.googleapis.com/instance/cpu/utilization'\n
    | group_by 1m, [value_utilization_mean: mean(value.utilization)]\n
    | every 1m\n
    | ${my_label_based_variable}",
  "unitOverride": "",
  "outputFullDuration": false
},

You can also modify query to resolve only the value of a variable. The following example uses a regular expression to compare the value of a label-based query to the instance_id:

"timeSeriesQuery": {
  "timeSeriesQueryLanguage": "fetch gce_instance\n
    | metric 'compute.googleapis.com/instance/cpu/utilization'\n
    | filter resource.instance_id=~'${my_label_based_variable.value}'\n
    | group_by 1m, [value_utilization_mean: mean(value.utilization)]\n
    | every 1m\n",
  "unitOverride": "",
  "outputFullDuration": false
},

If you have a value-only variable, then omit the .value clause. For example, to filter by zone using a value-only variable, the query would include something like the following:

resource.zone=~'${my_value_only_variable}'

The following table illustrates how the example variables are are resolved by the MQL. As previously mentioned, when only the value of a variable is used, you must use a regular expression as the comparison operator:

Syntax Selected
Value		 Resolved MQL expression
${my_label_based_variable} 12345 filter (resource.instance_id == '12345')

The example variable is based on the resource label instance_id.
${my_label_based_variable} * filter (true)
${my_label_based_variable.value}
${my_value_based_variable}		 12345 12345
${my_label_based_variable.value}
${my_value_based_variable}		 * .*

Charts with Monitoring filter queries

To apply a label-based variable to a chart that has a query in the form of a Monitoring filter, follow the guidance listed in General syntax.

Console

If you use the Google Cloud console to create your charts, and if you use the menu-driven interface, then you can apply a label-based variable to a chart by using the variable's Apply to charts field or by editing the widget and selecting label-based variable from the Filter menu. The Filter menu lists all label-based variables and all label keys.

To apply a value-based variable to these types of charts, do the following:

  1. Edit the chart.
  2. In the query pane, click Add filter and select a label key. For example, you might select zone.
  3. In the Value menu, select your value-only variable.
  4. Click Apply.
  5. To save your modified dashboard, in the toolbar, click Save.

For example, the following JSON illustrates a query that relies on a label-based variable, my_label_based_variable, being resolved into a filter expression:

metric.type="compute.googleapis.com/instance/cpu/utilization"
resource.type="gce_instance" ${my_label_based_variable}"

Widgets that use a query in the form of a Monitoring filter can't filter the time series by the value in a label-based variables; however, you can filter by value-only variables. For example, the following query shows the value of the Filters field of a query that filters by zone, based on the value of a value-only variable:

metric.type="compute.googleapis.com/instance/cpu/utilization"
resource.type="gce_instance"
resource.label."zone"=monitoring.regex.full_match(${my_value_only_variable})

API

For example, the following JSON illustrates a query that relies on a label-based variable, my_label_based_variable, being resolved into a filter expression:

"timeSeriesQuery": {
  "timeSeriesFilter": {
    "filter": "metric.type=\"compute.googleapis.com/instance/cpu/utilization\"
               resource.type=\"gce_instance\"
               ${my_label_based_variable} ",
    "aggregation": {
      "alignmentPeriod": "60s",
      "perSeriesAligner": "ALIGN_MEAN",
      "groupByFields": []
    }
  },
  "unitOverride": "",
  "outputFullDuration": false
},

Widgets that use a query in the form of a Monitoring filter can't filter the time series by the value in a label-based variables; however, you can filter by value-only variables. For example, the following query shows the "filter" field of a query that filters by zone, based on the value of a value-only variable:

"filter": "metric.type=\"compute.googleapis.com/instance/cpu/utilization\"
          resource.type=\"gce_instance\"
          resource.labels.\"zone\"=monitoring.regex.full_match(${my_value_only_variable})"

The following table illustrates how the example variables are are resolved by the Monitoring filter. As previously mentioned, when only the value of a variable is used, you must use a regular expression as the comparison operator:

Syntax Selected
Value		 Resolved filter expression
${my_label_based_variable} 12345 resource.instance_id == "12345"

The example variable is based on the resource label instance_id.
${my_label_based_variable} * Omitted
${my_label_based_variable.value} 12345 Not supported
${my_label_based_variable.value} * Not supported
${my_value_based_variable} 12345 "12345"
${my_value_based_variable} * ".*"

Modify a pinned filter or variable

To temporarily change the value of any filter displayed on the dashboard toolbar, click the filter name, and either enter a value or make a selection from the menu.

If your dashboard contains a table that displays time-series data, then you can add temporary filters or modify permanent filters and variables by using a cell's Filter dashboard to this value button. This button, when selected, either applies a new temporary filter or it updates the value for an existing filter or variable. Existing filters and variables are updated when the label key for the table column matches the label key for a filter or variable.

To change the configuration of a pinned filter or a variable, do the following:

  1. In the Google Cloud console, go to the  Dashboards page:

    Go to Dashboards

    If you use the search bar to find this page, then select the result whose subheading is Monitoring.

  2. In the dashboard toolbar, click Settings.
  3. Find the filter or variable that you want to modify and expand its definition.
  4. After you complete your modifications, click Apply.
  5. To save your modified dashboard, in the toolbar, click Save.

Delete a pinned filter or variable

To delete a pinned filter or a variable, do the following:

  1. In the Google Cloud console, go to the  Dashboards page:

    Go to Dashboards

    If you use the search bar to find this page, then select the result whose subheading is Monitoring.

  2. In the dashboard toolbar, click Settings.
  3. Find the filter or variable that you want to remove, and then click Delete.
  4. Click Apply.
  5. To save your modified dashboard, in the toolbar, click Save.

What's next