BigQuery Reservations enables you to switch between on-demand pricing and flat-rate pricing. With flat-rate pricing, you purchase dedicated query processing capacity. You can allocate this capacity across your organization, by reserving pools of capacity for different projects or different parts of your organization. You can also combine the two billing models, taking advantage of both on-demand and flat-rate pricing.
BigQuery offers two pricing models for analytics:
- On-demand pricing: You pay for the data scanned by your queries. Your cost is based on the number of bytes processed.
- Flat-rate pricing: You purchase dedicated query processing capacity.
By default, you are billed according to the on-demand pricing model. Using BigQuery Reservations, you can switch to flat-rate pricing by purchasing commitments. Commitments are purchased in units of BigQuery slots. The cost of all bytes processed is included in the flat-rate price.
The benefits of using BigQuery Reservations include:
Predictability. Flat-rate pricing offers predictable and consistent costs. You know up-front what you are spending.
Flexibility. You choose how much capacity to purchase. You are billed a flat rate per second until you delete the capacity commitment. You can combine both billing models. For example, you might run some workloads with on-demand pricing and others with flat-rate pricing.
BigQuery offers flat-rate pricing at a discounted rate if you purchase a minimum monthly or annual commitment.
Workload management. After you purchase slots, you can allocate them to workloads. That way, a workload has a dedicated pool of BigQuery computational resources available for use. At the same time, if a workload doesn't use all of its allocated slots, any unused slots are shared automatically across your other workloads.
Centralized purchasing: You can purchase and allocate slots for your entire organization. You don't need to purchase slots for each project that uses BigQuery.
A capacity commitment is a purchase of BigQuery compute capacity for some minimum duration of time. Commitments are measured in BigQuery slots, which are a unit of computational capacity. Slots represent a dynamic amount of CPU, RAM, and distributed memory. Generally, if you purchase more slots, you can run more concurrent queries, and complex queries will run faster.
BigQuery offers several commitment plans to choose from. They differ mainly by cost and the minimum duration of your commitment. For current pricing information, see Flat-rate pricing.
Monthly commitment. You purchase a minimum 30-day commitment. After 30 days, you can cancel the plan at any time.
Annual commitment. You purchase a 365-day commitment. You can choose whether to auto-renew, cancel, or convert to a different plan after 365 days.
Flex slots. You purchase a 60-second commitment. You can cancel any time after 60 seconds. Flex slots are a good way to test how your workloads perform with flat-rate billing, before purchasing a longer-term commitment. They are also useful for handling cyclical or seasonal demand, or for high-load events such as tax season.
Whichever plan you select, your slots don't expire at the end of the commitment period. You keep the slots and are billed for them until you delete them. You can also change the plan type after the minimum duration.
For more details about these plans, see Commitment plans.
After you purchase slots, you can assign them to different buckets, called reservations. Reservations let you allocate the slots in ways that make sense for your particular organization.
For example, you might create a reservation named
prod for production
workloads, and a separate reservation named
test for testing. That way, your
test jobs won't compete for resources that your production workloads need. Or,
you might create reservations for different departments in your organization.
A reservation named
default is automatically created when you purchase slots.
There is nothing special about the
default reservation — it's created
as a convenience. You can decide whether you need additional reservations or
just use the default reservation.
To use the slots that you purchase, you will assign projects, folders, or organizations to reservations. Each level in the resource hierarchy inherits the assignment from the level above it, unless you override. In other words, a project inherits the assignment of its parent folder, and a folder inherits the assignment of its organization. For more information about the Google Cloud resource hierarchy, see Resource hierarchy.
When a job is started from a project that is assigned to a reservation, the job uses that reservation's slots. If a project is not assigned to a reservation (either directly or by inheriting from its parent folder or organization), the jobs in that project use on-demand pricing.
None assignments represent an absence of an assignment. Projects assigned to
None use on-demand pricing. The common use case for
None assignments is to
assign an organization to the reservation and to opt-out some projects or
folders from that reservation by assigning them to
None. For more information,
see Assign a project to None.
When you create an assignment, you specify the job type for that assignment:
QUERY: Use this reservation for query jobs, including SQL, DDL, DML, and BigQuery ML queries.
PIPELINE: Use this reservation for load, export, and other pipeline jobs.
By default, load and export jobs are free and use a shared pool of slots. BigQuery does not make guarantees about the available capacity of this shared pool. If you are loading large amounts of data, your job may wait as slots become available. In that case, you might want to purchase dedicated slots and assign pipeline jobs to them. We recommend creating an additional dedicated reservation with idle slot sharing disabled.
When load jobs are assigned to a reservation, they lose access to the free pool. Monitor performance to make sure the jobs have enough capacity. Otherwise, performance could actually be worse than using the free pool.
ML_EXTERNAL: Use this reservation for BigQuery ML queries that use services that are external to BigQuery.
Certain BigQuery ML queries use services that are external to BigQuery. To use reserved slots with these external services, create an assignment with job type
ML_EXTERNAL. For more information, see Assign slots to BigQuery ML workloads.
Slots are distributed fairly among projects and then within the jobs in the project.
The BigQuery scheduler enforces the equal sharing of slots among projects with running queries within a reservation, and then within jobs of a given project. The scheduler provides eventual fairness. There might be short periods where some jobs get a disproportionate share of slots, but the scheduler will eventually correct this. The goal of the scheduler is to find a medium between being too aggressive with evicting running tasks (which results in wasting slot time) and being too lenient (which results in jobs with long running tasks getting a disproportionate share of the slot time).
If an important job consistently needs more slots than it receives from the scheduler, consider creating an additional reservation with a guaranteed number of slots and assigning the job to that reservation. For more information, see Workload management.
At any given time, some slots might be idle. This can include:
- Slots that are not assigned to any reservation.
- Slots that are assigned to a reservation but aren't currently in use.
By default, queries running in a reservation automatically use idle slots from other reservations. That means a job can always run as long as there's capacity. Idle capacity is immediately preemptible back to the original assigned reservation as needed. This happens automatically in real time.
To disable this functionality and force a reservation to use only the slots
provisioned to it, set
true. Reservations with
ignore_idle_slots set to
true receive no idle slots.
As long as
ignore_idle_slots is false, a reservation can have a slot count of
0 and still have access to unused slots. If you are only using the
reservation, we recommend setting it up this way.
Assignments of type
ML_EXTERNAL are an exception to the behavior described
above. Slots used by these jobs are not preemptible; that is, they aren't
available for other jobs running in the reservation. Also, these jobs don't use
idle slots from other reservations.