Creating Custom Cost Controls

If you have multiple BigQuery projects and users, you can manage costs by requesting a custom quota that specifies a limit on the amount of query data processed per day.

Creating a custom quota on query data allows you to control costs at the project-level or at the user-level.

  • Project-level custom quotas limit the aggregate usage of all users in that project.

  • User-level custom quotas are separately applied to each user or service account within a project.

For current on-demand query pricing, see the pricing page.

How to set or modify custom quotas

To request a custom quota or to modify an existing custom quota, submit a request from the Google Cloud Platform Console page. You can set a custom quota on query data in increments of 10 TBs. Project-level quota changes should take effect in minutes. User-level quota requests are usually addressed within 2 to 3 business days. You must be the project owner to request a custom quota.

How custom quotas are enforced

After you set a custom quota, BigQuery returns an error when you exceed it.

If you exceed a project-level custom quota, BigQuery returns the following usageQuotaExceeded error:

Custom quota exceeded: Your usage exceeded the custom quota for
QueryUsagePerDay, which is set by your administrator. For more information,
see https://cloud.google.com/bigquery/cost-controls

If a user exceeds a user-level custom quota, BigQuery returns a usageQuotaExceeded error with a different error message:

Custom quota exceeded: Your project_user exceeded the custom quota for query bytes
scanned per user, which is set by your administrator. For more information,
see https://cloud.google.com/bigquery/cost-controls

If you use the _PARTITIONTIME or _TABLE_SUFFIX filter in your query, you might get an unexpected quotaExceeded error. When BigQuery checks your query to see whether you would exceed your custom quota, it currently ignores any _PARTITIONTIME or _TABLE_SUFFIX filters in the query.

Example

Suppose you set the following custom quotas for a project with 10 users, one of which is a service account:

  • Project-level: 50 TB per day
  • User-level: 10 TB per day

The following table describes the remaining quota as the 10 users run queries thoughout the day.

Usage Remaining Quota
Each of the 10 users queries 4 TBs Project-level: 10 TBs remain.
User-level: 6 TBs per user remain, but only up to 10 TBs total.
The service account queries another 6 TBs Project-level: 4 TBs remain.
User-level: The service account can no longer use BigQuery. 6 TBs per user remain for the other users, but only up to 4 TBs total.
One user queries another 4 TBs Project-level: 0 TBs remain.
User-level: Various TBs remain, but no one can use BigQuery because the project-level quota has been exceeded.

With no remaining quota, BigQuery stops working for everyone in that project.

FAQs

Can customers self-provision custom quotas?

Project-level quotas can be self-provisioned from the Google Cloud Platform Console page. User-level quota changes require a request to be submitted, however this is done through the same UI as the project-level quotas.

How will I know what my custom quota levels are?

Quota levels can be viewed on the Google Cloud Platform Console page.

When is my custom quota refilled?

Project-Level daily quotas reset at midnight Pacific Time. User-Level daily quotas replenish continually throughout the day.

What if I exceeded my custom quota, but I want to reset it?

You can run your query from another project that has access to your datasets and that doesn't have a custom quota or hasn't yet exceeded it.

Is custom quota proactive or reactive, or in other words, can I run an 11 TB query if I have a 10 TB quota?

Custom quota is proactive, so you won't be able to run an 11 TB query if you have a 10 TB quota.

Was this page helpful? Let us know how we did:

Send feedback about...