Creating Custom Cost Controls

If you have multiple BigQuery projects and users, you can manage costs by requesting a custom quota that specifies a limit on the amount of query data processed per day.

Creating a custom quota on query data allows you to control costs at the project-level or at the user-level.

  • Project-level custom quotas limit the aggregate usage of all users in that project.

  • User-level custom quotas are separately applied to each user or service account within a project.

For current on-demand query pricing, see the pricing page.

How to set or modify custom quotas

To request a custom quota or to modify an existing custom quota, complete the BigQuery Custom Quota Request form and click Submit. You can set a custom quota on query data in increments of 10 TBs. You will usually receive a response within 2 to 3 business days. You must be the project owner to request a custom quota.

How custom quotas are enforced

After you set a custom quota, BigQuery returns an error when you exceed it.

If you exceed a project-level custom quota, BigQuery returns the following usageQuotaExceeded error:

Custom quota exceeded: Your project exceeded the custom quota for query
bytes scanned, which is set by your administrator. For more information,
see https://cloud.google.com/bigquery/cost-controls

If a user exceeds a user-level custom quota, BigQuery returns a usageQuotaExceeded error with a different error message:

Custom quota exceeded: Your project_user exceeded the custom quota for query bytes
scanned per user, which is set by your administrator. For more information,
see https://cloud.google.com/bigquery/cost-controls

If you use the _PARTITIONTIME or _TABLE_SUFFIX filter in your query, you might get an unexpected quotaExceeded error. When BigQuery checks your query to see whether you would exceed your custom quota, it currently ignores any _PARTITIONTIME or _TABLE_SUFFIX filters in the query.

Example

Suppose you set the following custom quotas for a project with 10 users, one of which is a service account:

  • Project-level: 50 TB per day
  • User-level: 10 TB per day

The following table describes the remaining quota as the 10 users run queries thoughout the day.

Usage Remaining Quota
Each of the 10 users queries 4 TBs Project-level: 10 TBs remain.
User-level: 6 TBs per user remain, but only up to 10 TBs total.
The service account queries another 6 TBs Project-level: 4 TBs remain.
User-level: The service account can no longer use BigQuery. 6 TBs per user remain for the other users, but only up to 4 TBs total.
One user queries another 4 TBs Project-level: 0 TBs remain.
User-level: Various TBs remain, but no one can use BigQuery because the project-level quota has been exceeded.

With no remaining quota, BigQuery stops working for everyone in that project.

FAQs

Can customers self-provision custom quotas?

No, you must use the BigQuery Custom Quota Request form.

How will I know what my custom quota levels are?

After you submit the form, you will receive an email confirmation that verifies your custom quota preferences.

When is my custom quota refilled?

Daily quotas are replenished at regular intervals throughout the day, reflecting their intent to guide rate limiting behaviors. Intermittent refresh is also done to avoid long disruptions when quota is exhausted. More quota is typically made available within minutes rather than globally replenished once daily.

What if I exceeded my custom quota, but I want to reset it?

You can run your query from another project that has access to your datasets and that doesn't have a custom quota or hasn't yet exceeded it. Alternatively, you can wait approximately 15 minutes when your quota is partially refilled.

Is custom quota proactive or reactive, or in other words, can I run an 11 TB query if I have a 10 TB quota?

Custom quota is proactive, so you won't be able to run an 11 TB query if you have a 10 TB quota.

Monitor your resources on the go

Get the Google Cloud Console app to help you manage your projects.

Send feedback about...