Introduction to Reservations
BigQuery Reservations enables you to switch between on-demand pricing and flat-rate pricing. With flat-rate pricing, you purchase dedicated query processing capacity. You can allocate this capacity across your organization, by reserving pools of capacity for different projects or different parts of your organization. You can also combine the two billing models, taking advantage of both on-demand and flat-rate pricing.
BigQuery offers two pricing models for analytics:
Flat-rate pricing: You purchase dedicated query processing capacity.
By default, you are billed according to the on-demand pricing model. Using BigQuery Reservations, you can switch to flat-rate pricing by purchasing commitments. Commitments are purchased in units of BigQuery slots. The cost of all bytes processed is included in the flat-rate price.
The benefits of using BigQuery Reservations include:
Predictability. Flat-rate pricing offers predictable and consistent costs. You know up-front what you are spending.
Flexibility. You choose how much capacity to purchase. You are billed a flat rate per second until you delete the capacity commitment. You can combine both billing models. For example, you might run some workloads with on-demand pricing and others with flat-rate pricing.
BigQuery offers flat-rate pricing at a discounted rate, if you purchase a minimum monthly or annual commitment.
Workload management. After you purchase slots, you can allocate them to workloads. That way, a workload has a dedicated pool of BigQuery computational resources available for use. At the same time, if a workload doesn't use all of its allocated slots, any unused slots are shared automatically across your other workloads.
Centralized purchasing: You can purchase and allocate slots for your entire organization. You don't need to purchase slots for each project that uses BigQuery.
A capacity commitment is a purchase of BigQuery compute capacity for some minimum duration of time. Commitments are measured in BigQuery slots, which are a unit of computational capacity. A slot represents a virtual CPU used by BigQuery. Generally, if you purchase more slots, you can run more concurrent queries, and complex queries can run faster.
BigQuery offers several commitment plans to choose from. They differ mainly by cost and the minimum duration of your commitment. For current pricing information, see Flat-rate pricing.
Annual commitment. You purchase a 365-day commitment. You can choose whether to renew or convert to a different type of commitment plan after 365 days.
Monthly commitment. You purchase a minimum 30-day commitment. After 30 days, you can delete the plan at any time.
Flex slots. You purchase a 60-second commitment. You can delete it at any time after 60 seconds. Flex slots are a good way to test how your workloads perform with flat-rate billing, before purchasing a longer-term commitment. They are also useful for handling cyclical or seasonal demand, or for high-load events such as tax season.
Whichever plan you select, your slots don't expire at the end of the commitment period. You keep the slots and are billed for them until you delete them. You can also change the plan type after the minimum duration.
Slots are subject to capacity availability. When you attempt to purchase slot commitments, success of this purchase is not guaranteed. However, once your commitment purchase is successful, your capacity is guaranteed until you delete the commitment.
For more details about these plans, see Commitment plans.
After you purchase slots, you can assign them to different buckets, called reservations. Reservations let you allocate the slots in ways that make sense for your particular organization.
For example, you might create a reservation named
prod for production
workloads, and a separate reservation named
test for testing. That way, your
test jobs don't compete for resources that your production workloads need. Or,
you might create reservations for different departments in your organization.
A reservation named
default is automatically created when you purchase slots.
There is nothing special about the
default reservation — it's created
as a convenience. You can decide whether you need additional reservations or
just use the default reservation.
To use the slots that you purchase, you must assign a project to a reservation, as described in the next section.
A reservation is the lowest level at which you can specify slot allocation. Slot allocation for assignments within a reservation is handled by the BigQuery scheduler.
To use the slots that you purchase, you must assign one or more projects, folders, or organizations to a reservation. Each level in the resource hierarchy inherits the assignment from the level above it. In other words, if a project or folder is not assigned, then that project or folder inherits the assignment of its parent folder or organization, if any. For more information about the resource hierarchy, see Organizing BigQuery resources.
When a job is started from a project that is assigned to a reservation, the job uses that reservation's slots. If a project is not assigned to a reservation (either directly or by inheriting from its parent folder or organization), the jobs in that project use on-demand pricing.
None assignments represent an absence of an assignment. Projects assigned to
None use on-demand pricing. The common use case for
None assignments is to
assign an organization to the reservation and to opt out some projects or
folders from that reservation by assigning them to
None. For more information,
see Assign a project to None.
When you create an assignment, you specify the job type for that assignment:
QUERY: Use this reservation for query jobs, including SQL, DDL, DML, and BigQuery ML queries.
PIPELINE: Use this reservation for load, extract, and other pipeline jobs.
By default, load and extract jobs are free and use a shared pool of slots. BigQuery does not make guarantees about the available capacity of this shared pool or the throughput you see. If you are loading large amounts of data, your job might wait as slots become available. In that case, you might want to purchase dedicated slots and assign pipeline jobs to them. We recommend creating an additional dedicated reservation with idle slot sharing disabled.
When load jobs are assigned to a reservation, they lose access to the free pool. Monitor performance to make sure the jobs have enough capacity. Otherwise, performance could actually be worse than using the free pool.
ML_EXTERNAL: Use this reservation for BigQuery ML queries that use services that are external to BigQuery.
Certain BigQuery ML queries use services that are external to BigQuery. To use reserved slots with these external services, create an assignment with job type
ML_EXTERNAL. For more information, see Assign slots to BigQuery ML workloads.
You can't allocate slots to specific assignments. The BigQuery scheduler handles slot allocation for the assignments in a reservation.
Slots are distributed fairly among projects and then within the jobs in the project.
The BigQuery scheduler enforces the equal sharing of slots among projects with running queries within a reservation, and then within jobs of a given project. The scheduler provides eventual fairness. There might be short periods where some jobs get a disproportionate share of slots, but the scheduler eventually corrects this. The goal of the scheduler is to find a medium between being too aggressive with evicting running tasks (which results in wasting slot time) and being too lenient (which results in jobs with long running tasks getting a disproportionate share of the slot time).
If an important job consistently needs more slots than it receives from the scheduler, consider creating an additional reservation with a guaranteed number of slots and assigning the job to that reservation. For more information, see Workload management.
At any given time, some slots might be idle. This can include:
- Slots that are not assigned to any reservation.
- Slots that are assigned to a reservation but aren't currently in use.
By default, queries running in a reservation automatically use idle slots from other reservations. That means a job can always run as long as there's capacity. Queries started with on-demand slots will use idle slots from other reservations before making up the difference with on-demand slots. Idle capacity is immediately preemptible back to the original assigned reservation as needed, regardless of the priority of the query that needs the resources. This happens automatically in real time.
To disable this functionality and force a reservation to use only the slots
provisioned to it, set
true. Reservations with
ignore_idle_slots set to
true receive no idle slots.
As long as
ignore_idle_slots is false, a reservation can have a slot count of
0 and still have access to unused slots. If you are only using the
reservation, we recommend setting it up this way.
Assignments of type
ML_EXTERNAL are an exception to the behavior described
earlier. Slots used by BigQuery ML external model creation jobs are not preemptible;
that is, they aren't available for other jobs running in the reservation until they are released.
Also, these jobs don't use idle slots from other reservations.
- Reservations that you buy cannot be shared with other organizations.
- You must create a separate reservation and a separate administration project for each organization.
- There is a limit of 5 administration projects for each organization that can have an active commitment in a given location.
- Idle capacity cannot be shared between organizations or between different administration projects within a single organization.
- Commitments are a regional resource. Commitments purchased in one region or multi-region cannot be used in any other regions or multi-regions. Commitments cannot be moved between regions or between regions and multi-regions.
- Commitments purchased in one project cannot be moved to a different project.
Your slot quota is the maximum number of slots you can purchase in a location. You are not billed for quotas; you are only billed for purchased commitments. For more information, see reservations quotas and limits. For information about increasing your slot quota, see Requesting a quota increase.
For information about pricing for reservations, see Flat-rate pricing.
To get started with BigQuery Reservations, see Get started with reservations
For help in deciding which billing model to use, see Choose a billing model.