Introduction to Reservations
BigQuery Reservations enable you to switch between on-demand pricing and flat-rate pricing. With flat-rate pricing, you purchase dedicated query processing capacity. You can allocate this capacity across your organization, by reserving pools of capacity for different projects or different parts of your organization. You can also combine the two billing models, taking advantage of both on-demand and flat-rate pricing.
Overview
BigQuery offers two pricing models for analytics:
On-demand pricing: You pay for the data scanned by your queries. You have a fixed, per-project query processing capacity, and your cost is based on the number of bytes processed.
Flat-rate pricing: You purchase dedicated query processing capacity.
By default, you are billed according to the on-demand pricing model. Using BigQuery Reservations, you can switch to flat-rate pricing by purchasing commitments. Commitments are purchased in units of BigQuery slots. The cost of all bytes processed is included in the flat-rate price.
The benefits of using BigQuery Reservations include:
Predictability. Flat-rate pricing offers predictable and consistent costs. You know up-front what you are spending.
Flexibility. You choose how much capacity to purchase. We will bill you using a flat-rate pricing per second until you delete the capacity commitment. You can combine both billing models. For example, you might run some workloads with on-demand pricing and others with flat-rate pricing.
BigQuery offers flat-rate pricing at a discounted rate, if you purchase a minimum monthly or annual commitment.
Workload management. After you purchase slots, you can allocate them to workloads. That way, a workload has a dedicated pool of BigQuery computational resources available for use. At the same time, if a workload doesn't use all of its allocated slots, any unused slots are shared automatically across your other workloads.
Centralized purchasing: You can purchase and allocate slots for your entire organization. You don't need to purchase slots for each project that uses BigQuery.
Commitments
A capacity commitment is a purchase of BigQuery compute capacity for some minimum duration of time. Commitments are measured in BigQuery slots, which are a unit of computational capacity. A slot represents a virtual CPU used by BigQuery. Generally, if you purchase more slots, you can run more concurrent queries, and complex queries can run faster.
BigQuery offers several commitment plans to choose from. They differ mainly by cost and the minimum duration of your commitment. For current pricing information, see Flat-rate pricing.
Annual commitment. You purchase a 365-day commitment. You can choose whether to renew or convert to a different type of commitment plan after 365 days.
Monthly commitment. You purchase a minimum 30-day commitment. After 30 days, you can delete the plan at any time.
Flex slots. You purchase a 60-second commitment. You can delete it at any time after 60 seconds. Flex slots are a good way to test how your workloads perform with flat-rate billing, before purchasing a longer-term commitment. They are also useful for handling cyclical or seasonal demand, or for high-load events such as tax season.
Whichever plan you select, your slots don't expire at the end of the commitment period. You keep the slots and are billed for them until you delete them. You can also change the plan type after the minimum duration.
Slots are subject to capacity availability. When you attempt to purchase slot commitments, success of this purchase is not guaranteed. However, once your commitment purchase is successful, your capacity is guaranteed until you delete the commitment.
For more details about these plans, see Commitment plans.
Reservations
After you purchase slots, you can assign them to different buckets, called reservations. Reservations let you allocate the slots in ways that make sense for your particular organization.
For example, you might create a reservation named prod
for production
workloads, and a separate reservation named test
for testing. That way, your
test jobs don't compete for resources that your production workloads need. Or,
you might create reservations for different departments in your organization.
A reservation named default
is automatically created when you purchase slots.
There is nothing special about the default
reservation — it's created
as a convenience. You can decide whether you need additional reservations or
just use the default reservation.
To use the slots that you purchase, you must assign a project to a reservation, as described in the next section.
A reservation is the lowest level at which you can specify slot allocation. Slot allocation for assignments within a reservation is handled by the BigQuery scheduler.
Assignments
To use the slots that you purchase, you must assign one or more projects, folders, or organizations to a reservation. Each level in the resource hierarchy inherits the assignment from the level above it. In other words, if a project or folder is not assigned, then that project or folder inherits the assignment of its parent folder or organization, if any. For more information about the resource hierarchy, see Organizing BigQuery resources.
When a job is started from a project that is assigned to a reservation, the job uses that reservation's slots. If a project is not assigned to a reservation (either directly or by inheriting from its parent folder or organization), the jobs in that project use on-demand pricing.
None
assignments represent an absence of an assignment. Projects assigned to
None
use on-demand pricing. The common use case for None
assignments is to
assign an organization to the reservation and to opt out some projects or
folders from that reservation by assigning them to None
. For more information,
see Assign a project to None.
When you create an assignment, you specify the job type for that assignment:
QUERY
: Use this reservation for query jobs, including SQL, DDL, DML, and BigQuery ML queries.PIPELINE
: Use this reservation for load, extract, and other pipeline jobs.By default, load and extract jobs are free and use a shared pool of slots. BigQuery does not make guarantees about the available capacity of this shared pool or the throughput you see. If you are loading large amounts of data, your job might wait as slots become available. In that case, you might want to purchase dedicated slots and assign pipeline jobs to them. We recommend creating an additional dedicated reservation with idle slot sharing disabled.
When load jobs are assigned to a reservation, they lose access to the free pool. Monitor performance to make sure the jobs have enough capacity. Otherwise, performance could actually be worse than using the free pool.
BACKGROUND
: Use this reservation when you choose to use your own reservation to run your index-management jobs. Also use this reservation when you replicate source databases to BigQuery with Datastream's background apply operations.ML_EXTERNAL
: Use this reservation for BigQuery ML queries that use services that are external to BigQuery. For more information, see Assign slots to BigQuery ML workloads.
You can't allocate slots to specific assignments. The BigQuery scheduler handles slot allocation for the assignments in a reservation.
Slot scheduling
Slots are distributed fairly among projects and then within the jobs in the project.
The BigQuery scheduler enforces the equal sharing of slots among projects with running queries within a reservation, and then within jobs of a given project. The scheduler provides eventual fairness. There might be short periods where some jobs get a disproportionate share of slots, but the scheduler eventually corrects this. The goal of the scheduler is to find a medium between being too aggressive with evicting running tasks (which results in wasting slot time) and being too lenient (which results in jobs with long running tasks getting a disproportionate share of the slot time).
If an important job consistently needs more slots than it receives from the scheduler, consider creating an additional reservation with a guaranteed number of slots and assigning the job to that reservation. For more information, see Workload management.
Idle slots
At any given time, some slots might be idle. This can include:
- Slots that are not assigned to any reservation.
- Slots that are assigned to a reservation but aren't currently in use.
By default, queries running in a reservation automatically use idle slots from other reservations. That means a job can always run as long as there's capacity. Queries running in an on-demand pricing model also use idle slots from other reservations. On-demand slots reside in a default reservation in BigQuery.
Idle capacity is immediately preemptible back to the original assigned reservation as needed, regardless of the priority of the query that needs the resources. This happens automatically in real time.
To disable this functionality and force a reservation to use only the slots
provisioned to it, set ignore_idle_slots
to true
. Reservations with
ignore_idle_slots
set to true
receive no idle slots.
As long as ignore_idle_slots
is false, a reservation can have a slot count of
0
and still have access to unused slots. If you are only using the default
reservation, we recommend setting it up this way. You may then assign a project or folder to that reservation and it will only use idle slots.
Assignments of type ML_EXTERNAL
are an exception to the behavior described
earlier. Slots used by BigQuery ML external model creation jobs are not preemptible;
that is, the slots in a reservation with both ml_external and query assignment types are only available for other query jobs when the slots are not occupied by the ML_EXTERNAL
jobs.
Also, these jobs don't use idle slots from other reservations.
Limitations
- Reservations that you buy cannot be shared with other organizations.
- You must create a separate reservation and a separate administration project for each organization.
- Idle capacity cannot be shared between organizations or between different administration projects within a single organization.
- Commitments are a regional resource. Commitments purchased in one region or multi-region cannot be used in any other regions or multi-regions. Commitments cannot be moved between regions or between regions and multi-regions.
- Commitments purchased in one project cannot be moved to a different project.
Quotas
Your slot quota is the maximum number of slots you can purchase in a location. You are not billed for quotas; you are only billed for purchased commitments. For more information, see reservations quotas and limits. For information about increasing your slot quota, see Requesting a quota increase.
Pricing
For information about pricing for reservations, see Flat-rate pricing.
What's next
To get started with BigQuery Reservations, see Get started with reservations
For help in deciding which billing model to use, see Choose a billing model.