Concepts

This page describes the concepts associated with BigQuery Reservations.

Reservations concepts.

Administration project

BigQuery Reservations helps you securely manage your entire Google Cloud organization's BigQuery resources centrally. This is done by picking a single Google Cloud project as your administration project.

You elect an administration project by enabling the BigQuery Reservations API on it.

This project creates and maintains ownership of BigQuery Reservations resources, and is the primary source of billing for these resources. These resources can then be shared out to all the other projects in the organization by using assignments. Google recommends that you create a new project with a descriptive name.

Any project using a reservation owned by the admin project isn't billed for analysis (that is, running queries), but continues to be billed separately for storage.

Commitments

A capacity commitment is a purchase of compute capacity with some minimum committed period of usage (for example, a month). Compute capacity in BigQuery is measured in the form of slots, which is bundle of resources that includes a dynamic amount of CPU, RAM, and distributed memory. A capacity commitment cannot be deleted until the end of the commitment period passes.

Commitments enable flat-rate pricing for your organization. For more information, see BigQuery flat-rate pricing.

Behaviors

  • Minimum commitment size is 500 slots, and commitments are available in 500 slot increments, with no maximum.
  • There is no limit on the number of commitments you can have.
  • Capacity commitments are a regional resource. Commitments purchased in one region cannot be used in any other regions, and commitments cannot be moved between regions.
  • Commitments cannot be upgraded. To increase your net committed capacity, you need to purchase an additional commitment.
  • Commitments can be split into two. See Splitting commitments for more information.
  • Multiple commitments of the same type can be merged into one. See Merging commitments for more information.
  • You are charged per second from the moment your commitment purchase is successful.
  • You can convert your commitment to a longer-duration commitment type at any time. See Converting commitments to longer-duration commitment types for more information.

Properties

  • Slot count (specify on creation):
    • The number of slots in this commitment.
  • Commitment plan (specify on creation):
    • The duration of the commitment.
  • State (generated by the system; do not specify on creation):
    • Pending: The requested capacity is pending provisioning.
    • Active: The requested capacity is provisioned and ready to use.
  • Commitment end time (generated by the system based on the commitment plan).
  • Renewal plan (only for Annual plans):
    • Indicates the commitment type that the annual plan is converted into when the plan end time lapses.

Commitment plans

BigQuery offers flex, monthly, and annual commitment plans:

  • Flex Slots:
    • You cannot cancel for 60 seconds after your commitment is active.
    • After 60 seconds, you can cancel at any time, and you will be charged only for the seconds your commitment was active.
    • For example:
      • If you purchase a commitment on 6:00:00 October 5th, you start being charged at that moment.
      • You cannot cancel until 6:01:00 October 5th.
      • If you cancel on 6:01:01 October 5th, you will be charged for 61 seconds (6:00:00 October 5th - 6:01:01 October 5th).
    • If you do not cancel, you continue to be charged.
  • Monthly commitments:
    • You cannot cancel for 30 days after your commitment is active.
    • After 30 days, you can cancel at any time, and you will be charged only for the seconds your commitment was active.
    • For example:
      • If you purchase a commitment on 6:00:00 October 5th, you start being charged at that moment.
      • You cannot cancel until 6:00:00 November 4th.
      • If you cancel on 7:10:10 November 5th, you will be charged for 30 days plus one day, one hour, ten minutes, and ten seconds (6:00:00 October 5th - 7:10:10 November 5th).
    • If you do not cancel, you continue to be charged.
  • Annual commitments:
    • You cannot cancel for 365 days after your commitment is active.
    • Prior to the commitment end date, you can choose to renew the annual commitment, or convert it to a monthly or flex commitment. For more information, see Renewing commitments.
    • For example:
      • If you purchase a commitment on 6:00:00 October 5th 2019, you start being charged at that moment.
      • You cannot cancel or renew until 6:00:00 October 4th 2020 (2020 is a leap year).
      • If prior to 6:00:00 October 4th 2020 you choose to auto-renew, at 6:00:00 October 4th 2020 your commitment renews for another year.
      • If prior to 6:00:00 October 4th 2020 you choose to renew to a flex commitment, at 6:00:00:00 October 4th 2020 your commitment rolls over to a flex commitment, you are charged the flex slots rate, and you can cancel at any time.
      • If prior to 6:00:00 October 4th 2020 you choose to renew to a monthly commitment, at 6:00:00:00 October 4th 2020 your commitment rolls over to a monthly commitment, you are charged the monthly commitment rate, and you cannot cancel for 30 days.
  • Trial Slots:

    • You cannot cancel for 182 days after your commitment is active.
    • After 182 days, your commitment will be converted to flex slots, you will be able to cancel at any time, and you will be charged only for the seconds your commitment was active. You will start getting charged flex slots rates.
    • For example:
      • If you purchase a commitment on 6:00:00 October 5th, 2020, you start being charged at that moment.
      • You cannot cancel until 6:00:00 April 5th, 2021
      • On 6:00:00 April 5th, 2021, your commitment will be converted to flex slots, you are charged the flex slots rates, and you can cancel at any time.

Renewing commitments

For annual commitments, prior to the commitment end date, you can choose what happens to the commitment after the commitment ends. The following options are available:

  • Renew for another year.
  • Decline to renew. Upon commitment end date, convert to a monthly commitment. You will be charged the monthly rate, and you will be able to cancel at any time.
  • Decline to renew. Upon commitment end date, convert to a flex commitment. You will be charged the flex slots rate, and you will be able to cancel at any time.

For more information about how to choose your renewal option, see Renewing commitments.

Cancelling commitments

After you create a commitment, it is not possible to cancel it until the commitment end time.

If you accidentally purchased a commitment or made a mistake when you configured your commitment, contact Cloud Billing Support for help.

Reservations

A reservation enables you to split up your capacity commitments into workloads, teams, and departments. Each reservation is an isolated pool of BigQuery slots, with the additional benefit of idle capacity being seamlessly shared across reservations.

After you purchase slots for the first time, the system automatically creates a reservation called default. There is nothing special about the default reservation—it is created for convenience to the user. Users can create more reservations for their own use and specifically allocate slots to them.

You assign projects, folders, and organizations to reservations. Jobs initiated from a project that's assigned to a reservation will run from that reservation. When multiple projects are allocated to a reservation, the slots are distributed fairly among the projects first and then within the jobs in the project.

Reservations.

Idle slots

Queries running in a reservation automatically use idle slots from other reservations. Idle capacity is immediately preemptible back to the parent reservation as needed. This causes no disruptions to any queries that are using idle capacity. This happens automatically and in real time.

To disable this functionality and force a reservation to use only the slots provisioned to it, set ignore_idle_slots to true.

Unallocated slots

Unallocated slots are slots that are committed but not assigned to any particular reservation. Unallocated slots are not wasted; they are shared equally across your projects and queries according to rules explained in Slot sharing across reservations.

Slot sharing within a reservation

The BigQuery scheduler enforces the equal sharing of slots among projects with running queries within a reservation, and then within jobs of a given project. The scheduler provides eventual fairness; there might be short periods where some jobs get a disproportionate share of slots, but the scheduler will eventually correct this. The goal of the scheduler is to find a medium between being too aggressive with evicting running tasks (which results in wasting slot time) and being too lenient (which results in jobs with long running tasks getting a disproportionate share of the slot time).

If an important job is being routinely harmed by this fairness policy, then look into creating an additional reservation with a guaranteed number of slots (see Workload management). This job can then be assigned to the new reservation, which will provide it with the minimum number of slots.

Slot sharing across reservations

Sharing idle or unallocated slots works very similarly to sharing within a reservation. Slots are shared equally across all projects with running queries, then across queries in those projects. Reservations with ignore_idle_slots set to true receive no idle slots and are ignored by this process.

Idle or unallocated slots are only shared across reservations created in the same admin project. If you used separate admin projects to create reservations, there is no idle slot sharing across reservations in those admin projects.

Assignments

Assignments are used to assign projects, folders, and organizations to a reservation. A project that is assigned to a reservation uses that reservation for processing jobs.

Assignments can be made for the following job types:

  • Assignments with job type QUERY dictate a reservation assignment for queries, including SQL, DDL, DML, and BigQuery ML queries.
  • Assignments with job type PIPELINE indicate that batch load and extract jobs should use the reservation.
  • Assignments with job type ML_EXTERNAL dictate a reservation assignment for BigQuery ML queries that use services that are external to BigQuery and used for model training. These external services use slots from the specified reservation. Slots used by these jobs are not preemptible; that is, they are not available for other jobs running in the reservation. These jobs do not use idle slots from other reservations.

Reservation assignments can be made at the project, folder, or organization level. Assignments at the folder or organization level dictate that all projects in that folder or organization use a particular reservation. Assignments at the lowest level override those at higher levels, so it's possible for a specific project to override the default organization assignment.

Reservation assignments have the following elements:

  • Assignee: A user of the reservation that can run queries using the capacity allocated in the reservation. This can be a project, folder, or organization.
  • Job Type: The type of job the assignment is for.

What's next