Monitor and manage Data Boost quota usage

Spanner Data Boost is subject to quotas that limit the number of concurrent ExecuteStreamingSQL and StreamingRead calls executing per project and per region. When the number of concurrent requests exceeds the quota, requests fail with RESOURCE EXHAUSTED errors.

Check your quota

To check the concurrency quota in your project, follow these steps:

  1. Go to the Quotas page in the Google Cloud console.

    Go to Quotas

  2. In the Filter field, enter databoost. Then, in the list, select DataBoostQuotaPerProjectPerRegion.

  3. In the resulting table, locate your region and view the limit for that region.

Monitor quota usage

To monitor your quota usage, follow these steps:

  1. In the Google Cloud console, go to Monitoring.
    Go to Monitoring
  2. If Metrics Explorer is shown in the navigation menu, select it. Otherwise, click Resources, and then select Metrics Explorer.
  3. At the top of the page, select a time interval.
  4. In the Metric dropdown list, in the Filter by resource or metric name field, enter consumer and press Enter to narrow the search.
  5. In the list, select Consumer Quota > Quota > Concurrent Quota usage, and then click Apply.
  6. Click + ADD FILTER to create a filter.
  7. In the Label 1 dropdown list, select quota_metric.

  8. In the Value 1 text field, enter or select spanner.googleapis.com/data_boost_quota.

    Metrics explorer shows a line chart of quota usage by region.

  9. Optional: Under Display, for Widget type, select Stacked bar chart.

Monitor quota errors

To monitor quota errors, follow these steps:

  1. In Metrics explorer, select the metric Consumer Quota > Quota > Quota exceeded error.

  2. Add a filter for quota_metric equals spanner.googleapis.com/data_boost_quota.

Set an alert for Data Boost usage

You can create an alert policy that notifies you when the number of concurrent partitioned queries that request Data Boost (concurrent ExecuteStreamingSQL and StreamingRead calls) per project and per region exceeds a particular threshold. To do so, follow these steps:

  1. Follow the instructions in Create metric-threshold alerting policies.
  2. In the Select a metric dropdown list, in the Filter by resource or metric name field, enter data_boost and press Enter to narrow the search.
  3. In the Select a metric list, select Cloud Spanner Instance > Instance > Processing Unit Second, and then click Apply.

  4. Continue with creating an alert policy and set the threshold to some percentage of the quota.

    For example if the default quota is 200 and you want to be notified when the number of concurrent requests reaches 80% of maximum, enter 160 in the Threshold value field.

Handle quota errors

If the rate of quota-exceeded errors is high, when using Dataflow with Data Boost, we recommend that you adjust the maximum number of workers in your Dataflow job to avoid exceeding the Data Boost quota. You can also apply for a higher quota.

Block a principal from consuming Data Boost resources

If one principal is consistently exceeding the amount of Data Boost resources that they can be reasonably expected to consume, you can block the principal from using Data Boost resources by revoking the spanner.databases.useDataBoost Identity and Access Management (IAM) permission from the principal. You can automate revoking the permission by configuring an alert's notification channel as a webhook that invokes a Cloud Function. For more information, see the following topics:

What's next