This page describes the concepts associated with BigQuery Reservations.
BigQuery Reservations helps you securely manage your entire Google Cloud organization's BigQuery resources centrally. This is done by picking a single Google Cloud project as your administration project.
You elect an administration project by enabling the BigQuery Reservations API on it.
This project creates and maintains ownership of BigQuery Reservations resources, and is the primary source of billing for these resources. These resources can then be shared out to all the other projects in the organization by using assignments. Google recommends that you create a new project with a descriptive name.
Any project using a reservation owned by the admin project isn't billed for analysis (that is, running queries), but continues to be billed separately for storage.
A capacity commitment is a purchase of compute capacity with some minimum committed period of usage (for example, a month). Compute capacity in BigQuery is measured in the form of slots, which is bundle of resources that includes a dynamic amount of CPU, RAM, and distributed memory. A capacity commitment cannot be deleted until the end of the commitment period passes.
Commitments enable flat-rate pricing for your organization. For more information, see BigQuery flat-rate pricing.
- Minimum commitment size is 500 slots, and commitments are available in 500 slot increments, with no maximum.
- Capacity commitments are a regional resource. Commitments purchased in one region cannot be used in any other regions, and commitments cannot be moved between regions.
- Commitments cannot be upgraded. To increase your net committed capacity, you need to purchase an additional commitment.
- You are charged per second from the moment your commitment purchase is successful.
- Slot count (specify on creation):
- The number of slots in this commitment.
- Commitment plan (specify on creation):
- The duration of the commitment.
- State (generated by the system; do not specify on creation):
- Pending: The requested capacity is pending provisioning.
- Active: The requested capacity is provisioned and ready to use.
- Commitment end time (generated by the system based on the commitment plan).
- Auto-renewal (only for Annual plans):
- Extends the commitment end time by one year after this initial end time elapses.
BigQuery offers monthly and annual commitment plans:
- Monthly commitments:
- You cannot cancel for 30 days after your commitment is active.
- After 30 days, you can cancel at any time, and you will be charged only for the seconds your commitment was active.
- For example:
- If you purchase a commitment on 6:00:00 October 5th, you start being charged at that moment.
- You cannot cancel until 6:00:00 November 4th.
- If you cancel on 7:10:10 November 5th, you will be charged for 30 days plus one day, one hour, ten minutes, and ten seconds (6:00:00 October 5th - 7:10:10 November 5th).
- If you do not cancel, you continue to be charged.
- Annual commitments:
- You cannot cancel for 365 days after your commitment is active.
- Annual commitments can be renewed after the commitment end date. Non-renewed commitments roll over to monthly commitments, which you can cancel at any time.
- For example:
- If you purchase a commitment on 6:00:00 October 5th 2019, you start being charged at that moment.
- You cannot cancel or renew until 6:00:00 October 4th 2020 (2020 is a leap year).
- If prior to 6:00:00 October 4th 2020 you choose to auto-renew, at 6:00:00 October 4th 2020 your commitment renews for another year.
- If prior to 6:00:00 October 4th 2020 you choose not to auto-renew, at 6:00:00:00 October 4th 2020 your commitment rolls over to a monthly commitment, you are charged a monthly rate, and you can cancel at any time.
A reservation enables you to split up your capacity commitments to workloads, teams, and departments. Each reservation is an isolated pool of BigQuery slots, with the additional benefit of idle capacity being seamlessly shared across reservations.
After you purchase slots for the first time, the system automatically creates
a reservation called
default. Users can create more reservations for their own
use and specifically allocate slots to them. Reservations automatically use idle
slots from other reservations. A reservation can disable this functionality and
use only the slots explicitly provisioned to it by setting
Projects, folders, and organizations can be assigned to these reservations, allowing them to use the slots for their queries. When multiple projects are allocated to a reservation, the slots are distributed fairly among the projects first and then within the jobs in the project.
Unallocated slots are slots that are committed but not assigned to any
particular reservation. Unallocated slots are not wasted but are rather shared
equally across your reservations, in which case
ignore_idle_slots is not set
Slot sharing within a reservation
The BigQuery scheduler enforces the equal sharing of slots among projects within a reservation and then within jobs of a given project. The scheduler provides eventual fairness, so there might be short periods where some jobs get a disproportionate share of slots, but the scheduler will eventually correct this. The goal is to try and find a medium between being too aggressive with evicting running tasks—which would result in wasting slot time—and being too lenient—which would result in jobs with long running tasks getting a disproportionate share of the slot time.
If an important job is being routinely harmed by this fairness policy, then look into creating an additional reservation with a guaranteed number of slots (see Workload management). This job can then be assigned to the new reservation, which will provide it with the minimum number of slots.
Assignments are used to assign projects, folders, and organizations to a reservation. A project that is assigned to a reservation uses that reservation for processing jobs. A reservation assignment is a child resource of a reservation.
Assignments can be made for two job types:
- Assignments with job type
QUERYdictate a reservation assignment for queries, including SQL, DDL, DML, and BigQuery ML queries.
- Assignments with job type
PIPELINEindicate that batch load and extract jobs should use the reservation.
Reservation assignments can be made at the project, folder, or organization level. Assignments at the folder or organization level dictate that all projects in that folder or organization use a particular reservation. Assignments at the lowest level override those at higher levels, so it's possible for a specific project to override the default organization assignment.
Reservation assignments have the following elements:
- Assignee: A user of the reservation that can run queries using the capacity allocated in the reservation. This can be a project, folder, or organization.
- Job Type: The type of job the assignment is for.