Introduction to reservations

BigQuery reservations enable you to switch from on-demand pricing to capacity-based pricing. With capacity-based pricing, you pay for dedicated or autoscaled query processing capacity rather than paying for each query individually.

Reservations enable you to allocate query capacity, measured in slots, to different workloads or different parts of your organization.

Creating a capacity commitment is optional when working with reservations that leverage BigQuery editions, but can save on costs for steady-state workloads.

Overview

BigQuery offers two compute (analysis) pricing models:

  • On-demand pricing: You pay for the data scanned by your queries. You have a fixed, per-project query processing capacity, and your cost is based on the number of bytes processed by each query.

  • Capacity-based pricing: You pay for dedicated or autoscaled query processing capacity, measured in slots, over a period of time. Multiple queries share the same slot capacity.

By default, you are billed according to the on-demand pricing model. Using reservations, you can switch to capacity-based pricing and use slots autoscaling and purchase discounted capacity commitments. There is no charge for bytes processed when using the capacity-based model.

You can combine both billing models. For example, you might run some workloads with on-demand pricing and others with capacity-based pricing. As the billing model is specified per project, this would require that you use multiple projects for your query jobs.

Benefits of reservations

The benefits of using BigQuery reservations include:

  • Predictability. Capacity-based pricing offers predictable and consistent costs. You specify your maximum cost budget up front and can also take advantage of slot commitments, which offer dedicated ongoing capacity at a discounted rate.

  • Flexibility. You choose how much dedicated capacity to allocate to a workload or let BigQuery automatically scale capacity based on your workload requirements. You are billed using for the slots consumed at a minimum of one second increments.

  • Workload management. Each workload has a specified pool of BigQuery computational resources available for use. At the same time, if a workload doesn't use all of its dedicated slots, any unused slots are shared automatically across your other workloads.

  • Centralized purchasing: You can purchase and allocate slots for your entire organization. You don't need to purchase slots for each project that uses BigQuery.

Reservations

BigQuery capacity is measured in slots, which represent virtual CPUs used by queries. Generally, if you provision more slots, you can run more concurrent queries, and complex queries can run faster.

Slots are allocated in pools called reservations. Reservations let you allocate the slots in ways that make sense for your particular organization.

For example, you might create a reservation named prod for production workloads, and a separate reservation named test for testing. That way, your test jobs don't compete for resources that your production workloads need. Or, you might create reservations for different departments in your organization.

Reservations can include baseline slots, which are always allocated, as well as autoscaled slots, which are added or removed dynamically based on the demands of your workload.

A reservation named default is automatically created if you purchase slot commitments before creating a reservation. There is nothing special about the default reservation — it's created as a convenience. You can decide whether you need additional reservations or just use the default reservation.

To use the slots that you allocate, you must assign one or more projects to a reservation, as described in the next section.

A reservation is the lowest level at which you can specify slot allocation. Slot allocation within a reservation is handled by the BigQuery scheduler.

Assignments

To use the slots that you allocate, you must assign one or more projects, folders, or organizations to a reservation. Each level in the resource hierarchy inherits the assignment from the level above it. In other words, if a project or folder is not assigned, then that project or folder inherits the assignment of its parent folder or organization, if any. For more information about the resource hierarchy, see Organizing BigQuery resources.

When a job is started from a project that is assigned to a reservation, the job uses that reservation's slots. If a project is not assigned to a reservation (either directly or by inheriting from its parent folder or organization), the jobs in that project use on-demand pricing.

None assignments represent an absence of an assignment. Projects assigned to None use on-demand pricing. The common use case for None assignments is to assign an organization to the reservation and to opt out some projects or folders from that reservation by assigning them to None. For more information, see Assign a project to None.

When you create an assignment, you specify the job type for that assignment:

  • QUERY: Use this reservation for query jobs, including SQL, DDL, DML, and BigQuery ML queries.

  • PIPELINE: Use this reservation for load and extract jobs.

    By default, load and extract jobs are free and use a shared pool of slots. BigQuery does not make guarantees about the available capacity of this shared pool or the throughput you see. If you are loading large amounts of data, your job might wait as slots become available. In that case, you might want to purchase dedicated slots and assign pipeline jobs to them. We recommend creating an additional dedicated reservation with idle slot sharing disabled.

    When load jobs are assigned to a reservation, they lose access to the free pool. Monitor performance to make sure the jobs have enough capacity. Otherwise, performance could actually be worse than using the free pool.

  • BACKGROUND: Use this reservation when you choose to use your own reservation to run your BigQuery search index management jobs or BigQuery change data capture (CDC) background jobs. Also use this reservation when you replicate source databases to BigQuery with Datastream's background apply operations. BACKGROUND reservations are not available in Standard edition.

  • ML_EXTERNAL: Use this reservation for BigQuery ML queries that use services that are external to BigQuery. For more information, see Assign slots to BigQuery ML workloads. ML_EXTERNAL reservations are not available in Standard edition.

You can't allocate slots to specific assignments. The BigQuery scheduler handles slot allocation for the assignments in a reservation.

Commitments

Reservations concepts.

A capacity commitment is a purchase of a fixed amount of BigQuery compute capacity for some minimum duration of time. Capacity commitments are optional for reservations created with an edition, but can save on costs for steady-state workloads.

BigQuery offers two commitment plans to choose from. They differ mainly by cost and the minimum duration of your commitment. For current pricing information, see capacity commitment pricing.

  • Three-year commitment. You purchase a three year commitment. You can choose whether to renew or convert to a different type of commitment plan after 3*365 days.

  • Annual commitment. You purchase a 365-day commitment. You can choose whether to renew or convert to a different type of commitment plan after 365 days.

Whichever plan you select, your slots don't expire at the end of the commitment period. You keep the slots and are billed for them until you delete them. You can also change the plan type after the minimum duration.

Slots are subject to capacity availability. When you attempt to purchase slot commitments, success of this purchase is not guaranteed. However, once your commitment purchase is successful, your capacity is guaranteed until you delete the commitment.

For more details about these plans, see Commitment plans.

Slot allocation within reservations

BigQuery allocates slot capacity within a single reservation using an algorithm called fair scheduling.

The BigQuery scheduler enforces the equal sharing of slots among projects with running queries within a reservation, and then within jobs of a given project. The scheduler provides eventual fairness. There might be short periods where some jobs get a disproportionate share of slots, but the scheduler eventually corrects this. The goal of the scheduler is to find a medium between being too aggressive with evicting running tasks (which results in wasting slot time) and being too lenient (which results in jobs with long running tasks getting a disproportionate share of the slot time).

If an important job consistently needs more slots than it receives from the scheduler, consider creating an additional reservation with a guaranteed number of slots and assigning the job to that reservation. For more information, see Workload management.

Idle slots

At any given time, some slots might be idle. This can include:

  • Slot commitments that are not allocated to any reservation.
  • Slots that are allocated to a reservation baseline but aren't currently in use.

By default, queries running in a reservation automatically use idle slots from other reservations within the same administration project. That means a job can always run as long as there's capacity. Idle capacity is immediately preemptible back to the original assigned reservation as needed, regardless of the priority of the query that needs the resources. This happens automatically in real time.

To disable this functionality and force a reservation to use only the slots provisioned to it, set ignore_idle_slots to true. Reservations with ignore_idle_slots set to true can, however, share their idle slots with other reservations.

You cannot share idle slots between reservations of different editions. You can share only the baseline slots or committed slots. Autoscaled slots might be temporarily available, but are not shareable as they may scale down.

As long as ignore_idle_slots is false, a reservation can have a slot count of 0 and still have access to unused slots. If you are only using the default reservation, we recommend setting it up this way. You may then assign a project or folder to that reservation and it will only use idle slots.

Assignments of type ML_EXTERNAL are an exception to the behavior described earlier. Slots used by BigQuery ML external model creation jobs are not preemptible; that is, the slots in a reservation with both ml_external and query assignment types are only available for other query jobs when the slots are not occupied by the ML_EXTERNAL jobs. Also, these jobs don't use idle slots from other reservations.

Limitations

  • Reservations that you buy cannot be shared with other organizations.
  • You must create a separate reservation and a separate administration project for each organization.
  • Each organization can have a maximum of 5 administration projects with active commitments in a single location.
  • Idle capacity cannot be shared between organizations or between different administration projects within a single organization.
  • Commitments are a regional resource. Commitments purchased in one region or multi-region cannot be used in any other regions or multi-regions. Commitments cannot be moved between regions or between regions and multi-regions.
  • Commitments purchased in one administration project cannot be moved to a different administration project.
  • Commitments purchased with one edition cannot be used with reservations of another edition.
  • Idle slots are not shared between reservations of different editions.
  • Autoscaled slots are not shareable as they will scale down when no longer required.

Quotas

Your slot quota is the maximum number of slots you can purchase in a location. You are not billed for quotas; you are only billed for reservations and commitments. For more information, see reservations quotas and limits. For information about increasing your slot quota, see Requesting a quota increase.

Pricing

For information about pricing for reservations, see capacity-based pricing.

What's next