Configuring a Target Metric

This page describes how to use the metric-selection tool to specify a target metric for an alerting policy. The chart next to the Target region gives you visual feedback on the data being captured by the target.

The Target region uses the same metric selector that is used in Metrics Explorer and for creating charts. If you are already familiar with it, you can skip this page.

Selecting a metric

To select a metric, use the Find resource type and metric field to choose one resource type and one metric type. You can specify them in either order. To begin, click in the field. This brings up one or two lists, based on any prior selections. The lists are indicated by headers, Resource types and Metrics, as seen in the following screenshot:

Note: You cannot select metric types with values of type String from the console. String-valued data can't be displayed on charts or used in alerting policies. To retrieve data for these metric types, you must use a method in the API, like timeSeries.list.

You can select an entry in two ways:

• By selecting entries from the lists.

  Because there fewer resource types than metrics, select the Resource type first. If your resource type isn't displayed in the short list, click See all. After you select the resource type, the metric list is automatically filtered to display only relevant metrics.

• By entering a metric filter. To enter a metric filter, do the following:

  1. Next to Find resource type and metric, click Help

  2. Click Direct filter mode in the help pane.

     When Direct filter mode is enabled, the Find resource type and metric option is replaced with an editable text box labeled Resource type, metric, and filter:

     

     If you made selections for a resource type, a metric, or a filter prior to selecting Direct filter mode, then those settings are used to prepopulate the Resource type, metric, and filter text box.

  3. Enter a metric filter in the Resource type, metric, and filter text box. Your filter must include a metric type and a resource type. You can also include label filters. For the filter grammar, see Monitoring filters.

     For example, to display the log entries for all Google Cloud VM instances in the us-east1-b zone, enter the following:

     metric.type="logging.googleapis.com/log_entry_count" resource.type="gce_instance" resource.label."zone"="us-east1-b"
     

     If you've used direct filter mode to configure charts or alerting policies and no data is available, then the chart displays an error message. The exact error message depends on the filter you entered. For example, a typical message is Chart definition invalid. You might also see the message No data is available for the selected time frame.

Hovering over an item on either list brings up a tooltip that displays the information in the item's descriptor. For information on descriptors for metric types or monitored resources, see the metrics list or the monitored resource list.

When at least one resource type and metric pair is selected, the chart shows all the available time series, and additional items appear below the specified metric on the Metric tab. The following screenshot shows the Metric tab after a metric has been specified:

Filtering

You can reduce the amount of data to be charted by specifying filter criteria, applying aggregation, or by using outlier mode. Filters ensure that only time series that meet some set of criteria are used. If you apply filters, then there are fewer lines on the chart, and that can improve the performance of the chart.

If you supply multiple filtering criteria, then the corresponding chart shows only the time series that meet all of the criteria, a logical AND.

In the Google Cloud Console, to add a filter, click the Filter field. This opens a panel containing lists of criteria by which you can filter. For example, you can filter by resource group, by name, by resource label, by zone, and by metric label.

The following screenshot shows the known filter-by labels for a specific metric:

You can select from the lists or type to find matches. Additionally, you can create filters for data that has not yet appeared; such filter criteria won't appear on the selection list, but you can manually specify filters that you know will be valid in the future.

After you choose a label on which to filter, you have to specify the rest of the filter: a value or range of values and a comparison.

Note: If a monitored resource isn't generating data for the selected metric, then a filter's Values menu won't contain information for that monitored resource. For example, assume your project contains one VM instance that is stopped. Assume that you create a chart for the CPU usage of a VM instance. Next, you add a filter and select instance_id. In this case, the Values menu, which lists all of the available instances, is empty. However, you can manually enter the value and then select Apply.

For example, the following screenshot shows a filter on the zone resource label. The Filter field supports a pair of comparison operators for equality, = and =~, and a pair for inequality, != and !=~. The second item in each pair takes a regular-expression as a value. Simple equality, =, is the default.

Below the list of comparison operators is a list of the available values. The following screenshot shows the names of zones in the project:

For the Value field, you can select one of the items on the drop-down list, or you can enter an expression that matches multiple items:

• If you use a direct comparison, = or !=, you can create a filter string like starts_with. For example, the filter string starts_with("us-central") matches any us-central zone:

  

  See Monitoring filters for more information on filter strings.

• If you select =~ or !=~, then enter a RE2 regular expression as the value. For example, the regular expression us-central1-.* matches any us-central1 zones:

  

  Note: You can't use regular expressions to filter the project_id resource label.

  The regular expression ^us.*.a$ matches any US zone that ends with “a”:

  

You can specify multiple filter criteria, and you can use the same label multiple times. This lets you specify a filter for a range of values. To add additional filters, click Add a filter near the bottom of the filter field. Currently, all of the filter criteria must be met; they constitute a logical AND. For example, you can use both starts_with and ends_with filter strings to show only “a” zones in the US:

With a zone="starts_with("asia-east1")" or zone=~"asia-east1.*" filter in place, only the time series with data from one of the asia-east1 zones is displayed:

Aggregation

After the time series are selected, the next step is specify the aggregation options. Aggregation consists of aligning individual time series and then combining the aligned time series. For a detailed explanation of aggregation, see Filtering and aggregation: manipulating time series.

Aligning data

Alignment is the process of converting the time series data received by Monitoring into a new time series with data points spaced by a fixed length of time. The process of alignment consists of collecting all data points received in an interval of a fixed length, applying a function to combine those data points, and assigning a time stamp to the result. That combining function might compute the average of all samples or it might extract the maximum of all samples. For a general discussion of alignment, see Alignment: within-series regularization.

When you create a condition on an alerting policy, you must specify the alignment parameters. If you are using the Google Cloud Console, after you select the resource type and the metric in the alerting condition, several additional fields and a button titled Show advanced options are displayed:

Period: The period is a look-back interval from a particular point in time. For example, if the period is five minutes, then at 1:00 PM, the samples received between 12:55 PM and 1:00 PM are to be aligned. At 1:01 PM, the samples received between 12:56 PM and 1:01 PM are to be aligned. In the context of alerting policies, the alignment period can be viewed as a sliding window that looks to the past. For a more involved discussion about this field, see The alignment period and the duration.

To view the remaining aggregation options, click Show advanced options:

Aligner: The aligner field specifies the function used to combine all the data points in an alignment period. For more information on the available aligners, see Aligner in the API reference. Some aligners both align the data and convert it from one metric kind or type to another. For a detailed explanation, see Kinds, types, and conversions.

Combining time series

You can reduce the amount of data returned for a metric by combining different time series. To combine multiple time series, you typically specify a grouping and a function. Grouping is done by label values. The function defines how all time series within a group are combined into a new time series.

To add a grouping, click the text in the Group by text box, and then make a selection from the menu. The menu is constructed dynamically based on the time series data for the resource and metric you selected. Grouping and filtering use the same set of labels.

When you add the first label, the following occurs:

• An Aggregator is selected. The selected function is determined by the type of data being displayed; however, you can change this function.
• The aggregator determines how the time series that have the same label value are combined into a single time series.
• The chart displays one time series for each value of the label listed in the Group by text box.

If you group by multiple labels, then the aggregator combines those times series that have the same value for the specified labels.

If you don't specify a grouping option and do specify an aggregator, then that function is applied to all of the selected time series and results in a single time series.

The following screenshot shows a grouping by user_labels.version with the aggregator set to the default value of sum:

This selection results in one time series for each value of the user_labels.version. The data points in each time series are computed from the sum of all the values for individual time series for a specific version:

You can group by multiple labels. If you have multiple grouping options, then time series are grouped by each combination of label values, and the aggregator is applied to each group. The resulting chart displays one time series for each combination of label values. The order in which you specify the labels doesn't matter.

For example, the following screenshot illustrates grouping by user_labels.version and system_labels.machine_image:

As illustrated, if you group by both the labels, you get one time series for each pair of values. The fact that you get a time series for each combination of labels means that this technique can easily create more data than you can usefully put on a single chart.

When you specify grouping or if you select an aggregator, the charted time series only contains required labels, such as the project identifier, and the labels specified by the grouping.

To remove a group-by condition, you must:

1. Delete the group-by labels.
2. Set the aggregator to none.

Note: If you only delete the group-by labels, then your chart displays a single line that represents all of the data after the aggregator has been applied. To undo grouping, you must remove the group-by labels and reset the aggregator to none.

Secondary Aggregation

When you have multiple time series that already represent aggregations, like the examples illustrating the Group By option, you can then aggregate across them by choosing a Secondary Aggregator:

Secondary aggregation reduces all the time series on the chart to a single time series.

Legend Template

The Legend Template field lets you customize a description for the time series on your chart. These descriptions appear on the hover card for the chart and on the chart legend in the Name column.

By default, the descriptions in the legend are created for you from the values of different labels in your time series. Because the system selects the labels, the results might not be helpful to you. You can use this field to build a template for the descriptions.

In the Cloud Console, if you expand the aggregation options by clicking Show advanced options, in addition to displaying fields for the aligner, alignment period, and the secondary aggregator, a Legend Template field is displayed.

You can enter plain text and templates in the Legend Template field. When you add a template, you add an expression that is is evaluated when the legend is displayed.

To add a template, do the following:

• Click Insert a template.
• Select an entry from the menu. After you select an entry, a template is automatically added. For example, if you select response_code, then the template ${resource.labels.zone} is added.

For example, the following screenshot shows a legend template that contains plain text and the expression ${resource.labels.zone}:

In the chart legend, the values generated from the template appear in a column with the header Name and in the hover card:

You can configure the legend template to include multiple text strings and templates; however, the display space available on the hover card is limited.