Using quota metrics

This page describes how to create alerting policies and charts to monitor or to view quota usage. For general information about quotas, including information on allocation quotas and rate quotas, see Working with quotas.

For all alerting policies and charts on this page, the monitored resource is consumer_quota. The metrics for this resource are a subset of the serviceruntime metrics.

  • For definitions of the terms gauge, delta, and cumulative, see Kinds of metrics.
  • For details about the fields used for combining time series, see Aligner and Reducer.

Understanding Consumer quotas

To view quota usage for a specific quota, you need to filter the time series by a specific value of the quota_metric label. The example in this section illustrates how to identify the correct value for this label by viewing time-series data. To determine the value of this label for a specific API quota, you can use the following procedure:

  1. In the Google Cloud Console, go to the Identity and Access Management Quotas page:

    Go to Quotas

  2. Locate the quota of interest and then click All quotas.

  3. If a service supports quota metrics, then a Quota Metric field is displayed and it shows the value of the quota_metric label for this API. The following screenshot illustrates that the value of the quota_metric label for the Compute Engine API subnetworks quota is compute.googleapis.com/subnetworks:

    Example of a quota metric details page.

    If a service doesn't support quota metrics, then a Quota Metric field isn't displayed.

Example

The goal is to create a chart that displays, by region, Compute Engine's total disk storage quota.

  1. In the Google Cloud Console, select Cloud Monitoring, and then select Metrics Explorer.

  2. Configure Metrics Explorer to display the allocation quota usage by setting the resource type to Consumer Quota and setting the metric type to Allocation quota usage:

    Select consumer quota.

    The chart displays the allocation quota usage as a bar chart for the period of six weeks. By viewing the legend, you see that the chart displays the quota usage for multiple services.

    If you use the Cloud Monitoring API, then the equivalent filter value is:

    metric.type="serviceruntime.googleapis.com/quota/allocation/usage" resource.type="consumer_quota"
    
  3. To limit the display to the Compute Engine service, add the filter service = "compute.googleapis.com":

    Filter consumer quota by compute service.

    The chart now displays the time series for the allocated quota usage for Compute Engine quotas. The legend displays the value of the quota_metric label for each displayed time series. This value identifies the specific quota. For example, compute.googleapis.com/disks_total_storage identifies that the time series is for Compute Engine's total disk storage quota. The chart only displays quota usage for those quotas that have recorded usage. For example, if a project doesn't have an Compute Engine resources, then this filter results in a chart with no data.

    If you use the Cloud Monitoring API, then the equivalent filter value is:

    metric.type="serviceruntime.googleapis.com/quota/allocation/usage" resource.type="consumer_quota" resource.label.service="compute.googleapis.com"
    
  4. To create chart that displays Compute Engine's total disk storage quota usage, add the filter quota_metric = "compute.googleapis.com/disks_total_storage":

    Filter consumer quota by quota metric.

    If you use the Cloud Monitoring API, then the equivalent filter value is:

    metric.type="serviceruntime.googleapis.com/quota/allocation/usage" resource.type="consumer_quota" metric.label.quota_metric="compute.googleapis.com/disks_total_storage"
    
  5. The previous chart displays time series for the regions us-centra1 and us-east1, and for multiple zones including the zone us-central1-a. To display only the quota usage by region, add a filter based on the location label. For this example, where both regions are prefixed by us- and end with 1, a filter that uses a regular expression, location =~"^us.*1$", works well:

    Filter by quota metric and region.

With consumer quotas, the value of the quota_metric label identifies both the service and the specific quota usage being monitored. You can use that information to create charts or alerting policies that monitor a specific quota metric.

Alerting policy examples

An alerting policy is a collection of conditions and notification information:

  • A condition describes what is being monitored, how the time series data for that resource is to be combined, and when an alert must be generated. An alerting policy must specify at least one condition.
  • The notification channel specifies who is to be notified, and how they are to be notified, when an alert occurs. For example, you can configure the alerting policy to send an email to a specific person or to a group of people.

In the remainder of this section, each subsection includes a JSON representation of an alerting policy. If the policy can be created using the Google Cloud Console, two tables are also included:

  • The first table describes what is being monitored and how the data is to be combined.
  • The second table describes when an alert is to be generated.

The values in these tables can be used to configure an alerting policy condition.

Alerting on the ratio of quota-usage to quota-limit

You can create an alerting policy to notify you if the ratio of quota usage to the quota limit for any service in your Google Cloud project exceeds 80%. You can create this type of policy only by using the Cloud Monitoring API.

Using the Cloud Monitoring API

You can create this alerting policy using the API method alertPolicies.create. You can invoke the Cloud Monitoring API directly, by using the gcloud command-line tool, or by using client libraries. For more information, see Creating policies.

For information on representing alerting policies in JSON or YAML, see Sample policies.

Combining quota limits might not be meaningful if the quotas apply to resources in different locations. Therefore, you want these policies to filter or group the time series by location. This example policy, which monitors illustrates computing the ratio of the Identity and Access Management API quota usage to quota limit, doesn't include filtering by location because this quota only has one location, global.


{
    "combiner": "OR",
    "conditions": [
      {
        "conditionThreshold": {
          "filter": "metric.type=\"serviceruntime.googleapis.com/quota/allocation/usage\" resource.type=\"consumer_quota\" resource.label.\"service\"=\"iam.googleapis.com\"",
          "aggregations": [
            {
              "alignmentPeriod": "86400s",
              "crossSeriesReducer": "REDUCE_SUM",
              "groupByFields": [
                "metric.label.quota_metric"
              ],
              "perSeriesAligner": "ALIGN_NEXT_OLDER"
            }
          ],
          "denominatorFilter": "metric.type=\"serviceruntime.googleapis.com/quota/limit\" resource.type=\"consumer_quota\" resource.label.\"service\"=\"iam.googleapis.com\"",
          "denominatorAggregations" : [
            {
              "alignmentPeriod": "86400s",
              "crossSeriesReducer": "REDUCE_SUM",
              "groupByFields": [
                "metric.label.quota_metric"
              ],
              "perSeriesAligner": "ALIGN_NEXT_OLDER"
            }
          ],
          "comparison": "COMPARISON_GT",
          "duration": "60s",
          "thresholdValue": 0.8,
          "trigger": {
            "count": 1
          }
        },
        "displayName": "Ratio quota usage for iam.googleapis.com by label.quota_metric [SUM]",
      }
    ],
    "displayName": "Ratio quota usage to limit",
    "enabled": true,
  }

Alerting on quota/exceeded errors

You can create an alerting policy to notify you if any service in your Google Cloud project is reporting a quota exceeded error. You can create this type of policy by using Google Cloud Console or by using the Cloud Monitoring API.

Using the Google Cloud Console

The remainder of the content in this subsection is for the conditions dialog of an alerting policy.

Complete the Target dialog by using the settings in the following table. These setting specify that you want to monitor the time series data for the serviceruntime metric /quota/exceeded for all services in your Google Cloud project, and that you want to group the data by the quota limit.

Target pane
Field

Value
Resource type Enter consumer_quota
Metric Select serviceruntime.googleapis.com/quota/exceeded
Filter

Add a filter so the chart only displays data for the service that you want to monitor. For example, to monitor the Identity and Access Management service, add the folliowing filter: service = iam.googleapis.com.

When the filter field is empty, all available metric data is included in the chart.

Group By

Select quota_metric.

This option groups the data by the type of name of the quota_metric.

Aggregator

Select sum.

The aggregation field is automatically set to sum when data is grouped. This setting defines how the data in the individual time series is combined.

Period Select 1 m
Advanced Aggregation Aligner: count true

This metric is of type GAUGE, meaning each data point in the time series is an instantaneous measurement. The value type is BOOL. A value of true indicates that the quota was exceeded.

For this metric, the aligner settings of count true and count are equivalent.

Complete the Configuration dialog by using the following settings. These settings cause the alerting policy to create an alert if the number of quota-exceeded errors exceeds the value of 0 for 1 m. A value of 0 is selected here because quota exceeded errors are unexpected and an indication that a quota needs to be increased or that a service needs to be modified to reduce the API requests. You might want to use a higher threshold.

Configuration pane
Field

Value
Condition triggers if Any time series violates
Condition is above
Threshold 0
For 1 m

Using the Cloud Monitoring API

You can create this alerting policy using the API method alertPolicies.create. You can invoke the Cloud Monitoring API directly, by using the gcloud command-line tool, or by using client libraries. For more information, see Creating policies.

For information on representing alerting policies in JSON or YAML, see Sample policies.

A representation of this alerting policy in JSON format follows.


{
    "combiner": "OR",
    "conditions": [
      {
        "conditionThreshold": {
          "aggregations": [
            {
              "alignmentPeriod": "60s",
              "crossSeriesReducer": "REDUCE_SUM",
              "groupByFields": [
                "metric.label.quota_metric"
              ],
              "perSeriesAligner": "ALIGN_COUNT_TRUE"
            }
          ],
          "comparison": "COMPARISON_GT",
          "duration": "60s",
          "filter": "metric.type=\"serviceruntime.googleapis.com/quota/exceeded\" resource.type=\"consumer_quota\"",
          "trigger": {
            "count": 1
          }
        },
        "displayName": "Quota exceeded error by label.quota_metric SUM",
      }
    ],
    "displayName": "Quota exceeded policy",
  }

Alerting on the absolute quota/allocation/usage

You can create an alerting policy to notify you if the allocation quota usage for a specific service in your Google Cloud project is exceeding a user-specified threshold. You can create this type of policy by using Google Cloud Console or by using the Cloud Monitoring API.

Using the Google Cloud Console

The remainder of the content in this subsection is for the conditions dialog of an alerting policy.

Complete the Target dialog by using the settings in the following table. These setting specify that you want to monitor the time series data for the serviceruntime metric /quota/allocation/usage for one service in your Google Cloud project, and that you want to group the data by the quota limit.

Target pane
Field

Value
Resource type Enter consumer_quota
Metric Select serviceruntime.googleapis.com/quota/allocation/usage
Filter

Add a filter so the chart only displays data for the service that you want to monitor. For example, to monitor the Identity and Access Management service, add the folliowing filter: service = iam.googleapis.com.

When the filter field is empty, all available metric data is included in the chart.

Group By

Select quota_metric.

This option groups the data by the type of name of the quota_metric.

Aggregator

Select sum.

The aggregation field is automatically set to sum when data is grouped. This setting defines how the data in the individual time series is combined.

Period Select 1440 m

The period matches is the sampling interval for this metric.

Advanced Aggregation Aligner: next older

The aligner is set to next older to include the measured value of this GAUGE metric.

Complete the Configuration dialog by using the following settings. These settings cause the alerting policy to create an alert if the value of the time series exceeds 2.5 for 1440 m. The value of 1440 m matches the alignment period. The value of 2.5 was selected for this example, because it is slightly higher than the normal value for the test system. You should select the threshold based on the service you are monitoring in combination with an understanding of the expected range of values for the metric.

Configuration pane
Field

Value
Condition triggers if Any time series violates
Condition is above
Threshold 2.5
For 1440 m

Using the Cloud Monitoring API

You can create this alerting policy using the API method alertPolicies.create. You can invoke the Cloud Monitoring API directly, by using the gcloud command-line tool, or by using client libraries. For more information, see Creating policies.

For information on representing alerting policies in JSON or YAML, see Sample policies.

A representation of this alerting policy in JSON format follows.


{
    "combiner": "OR",
    "conditions": [
      {
        "conditionThreshold": {
          "aggregations": [
            {
              "alignmentPeriod": "86400s",
              "crossSeriesReducer": "REDUCE_SUM",
              "groupByFields": [
                "metric.label.quota_metric"
              ],
              "perSeriesAligner": "ALIGN_NEXT_OLDER"
            }
          ],
          "comparison": "COMPARISON_GT",
          "duration": "84600s",
          "filter": "metric.type=\"serviceruntime.googleapis.com/quota/allocation/usage\" resource.type=\"consumer_quota\" resource.label.\"service\"=\"iam.googleapis.com\"",
          "thresholdValue": 2.5,
          "trigger": {
            "count": 1
          }
        },
        "displayName": "Allocation quota usage for iam.googleapis.com by label.quota_metric SUM",
      }
    ],
    "displayName": "Absolute quota usage policy",
  }

Chart examples

Charts display time series data. You can use Metrics Explorer to create a chart. With Metrics Explorer you can either discard the chart when you no longer need it, or you can save it to a dashboard. In the dashboard view, you can add a chart to the dashboard.

To create a chart, you specify what you want to view and how you want to view that data. The remainder of this section contains settings that can be used to create charts. You can also use the settings for Target dialog of an alerting policy to create a chart.

Maximum usage in interval for quota/allocation/usage

To view the time series data for the serviceruntime metric quota/allocation/usage, for all services in your Google Cloud project, where the data is grouped by the name of the quota metric and the maximum value of the metric in a 25-hour interval is displayed, use the following settings:

Target pane
Field

Value
Resource type Enter consumer_quota
Metric Select serviceruntime.googleapis.com/quota/allocation/usage
Filter

Add a filter so the chart only displays data for the service that you want to monitor. For example, to monitor the Identity and Access Management service, add the folliowing filter: service = iam.googleapis.com.

When the filter field is empty, all available metric data is included in the chart.

Group By

Select quota_metric.

This option groups the data by the type of name of the quota_metric.

Aggregator

Select sum.

The aggregation field is automatically set to sum when data is grouped. This setting defines how the data in the individual time series is combined.

Period Select 1500 m

The period, 25 hours, is slightly longer than the sampling interval for this metric.

Advanced Aggregation Aligner: max
Secondary aggregator:sum

The aligner is set to display the maximum value of the metric measured over the alignment period. The secondary aggregator combines the time series for the different services into a single time series.

Time series for quota/rate/net_usage

To view the time series data for the serviceruntime metric quota/rate/net_usage, for all services in your Google Cloud project, where the data is grouped by the name of the quota metric and the rate of usage is displayed, use the following settings:

Target pane
Field

Value
Resource type Enter consumer_quota
Metric Select serviceruntime.googleapis.com/quota/rate/net_usage
Filter

Add a filter so the chart only displays data for the service that you want to monitor. For example, to monitor the Identity and Access Management service, add the folliowing filter: service = iam.googleapis.com.

When the filter field is empty, all available metric data is included in the chart.

Group By

Select quota_metric.

This option groups the data by the type of name of the quota_metric.

Aggregator

Select sum.

The aggregation field is automatically set to sum when data is grouped. This setting defines how the data in the individual time series is combined.

Period Select 1 m

The period of 1 minute matches the sampling interval for this metric.

Advanced Aggregation Aligner: rate

By setting the aligner to the value of rate, the values stored in the time series for this DELTA metric are converted into a new time series that stores rate data. The y-axis for the chart has the units of quota per second.

Time series for quota/limit

To view the time series data for the serviceruntime metric quota/limit, for all services in your Google Cloud project, where the data is aggregated over 25 hours, use the following settings:

Target pane
Field

Value
Resource type Enter consumer_quota
Metric Select serviceruntime.googleapis.com/quota/limit
Filter

Add a filter so the chart only displays data for the service that you want to monitor. For example, to monitor the Identity and Access Management service, add the folliowing filter: service = iam.googleapis.com.

When the filter field is empty, all available metric data is included in the chart.

Group By Leave empty.
Aggregator none
Period Select 1500 m
Advanced Aggregation Aligner: next older

The aligner is set to next older to display the measured value of this GAUGE metric. The alignment period, 25 hours, is slightly longer than the sampling interval for this metric.