Create custom cost controls
If you have multiple BigQuery projects and users, you can manage costs by requesting a custom quota that specifies a limit on the amount of query data processed per day.
Creating a custom quota on query data lets you control costs at the project-level or at the user-level.
Project-level custom quotas limit the aggregate usage of all users in that project.
User-level custom quotas are separately applied to all users and service accounts within a project.
It is not possible to assign a custom quota to a specific user or service account.
For current on-demand query pricing, see the pricing page.
How to set or modify custom quotas
You can set a custom quota or modify an existing custom quota for any quota displayed on the Quotas page of Google Cloud console. For more information, see Managing your quota using the Cloud console. When you request a lower quota, the change takes effect within a few minutes. (If you request a higher quota, your request goes through an approval process, which can take more time. For more information, see Requesting a higher quota limit.)
In order to change your quotas, you must have
permissions are included in the following predefined IAM roles:
- The Owner and Editor basic roles
- The Quota Administrator (
servicemanagement.quotaAdmin) Service Management role
How custom quotas are enforced
After you set a custom quota, BigQuery returns an error when you exceed it.
If you exceed a project-level custom quota, BigQuery returns the following usageQuotaExceeded error:
Custom quota exceeded: Your usage exceeded the custom quota for QueryUsagePerDay, which is set by your administrator. For more information, see https://cloud.google.com/bigquery/cost-controls
If the user exceeds a user-level custom quota,
BigQuery returns a
error with a different error message:
Custom quota exceeded: Your usage exceeded the custom quota for QueryUsagePerUserPerDay, which is set by your administrator. For more information, see https://cloud.google.com/bigquery/cost-controls
Suppose you set the following custom quotas for a project with 10 users, one of which is a service account:
- Project-level: 50 TB per day
- User-level: 10 TB per day
Project-level custom quotas limit the aggregate usage of all users in that project. User-level custom quotas are separately applied to each user or service account within a project.
The following table describes the remaining quota as the 10 users run queries throughout the day.
|Project-level: 10 TBs remain.
User-level: 6 TBs per user remain, but only up to 10 TBs total.
|The service account queries another 6 TBs||Project-level: 4 TBs remain.
User-level: The service account can no longer use BigQuery. 6 TBs per user remain for the other users, but only up to 4 TBs total.
|One user queries another 4 TBs||Project-level: 0 TBs remain.
User-level: Various TBs remain, but no one can use BigQuery because the project-level quota has been exceeded.
With no remaining quota, BigQuery stops working for everyone in that project.
Can I self-provision custom quotas?
Yes, you can self-provision custom quotas from the Google Cloud console page.
How can I view my custom quota levels?
You can view your custom quota levels on the Google Cloud console page.
When is my custom quota refilled?
Daily quotas reset at midnight Pacific Time.
What if I exceeded my custom quota, but I want to reset it?
You can run your query from another project that has access to your datasets and that doesn't have a custom quota or hasn't yet exceeded it.
Is custom quota proactive or reactive, or in other words, can I run an 11 TB query if I have a 10 TB quota?
Custom quota is proactive, so you can't run an 11 TB query if you have a 10 TB quota.