Achieve peace of mind with BigQuery pricing and control
Tino Tereshko
Product Manager, Google BigQuery
For many companies, data analytics has evolved from an occasional task to something that’s mission-critical to their business. When you’re doing data analytics at scale, predictable spending is key—something we often hear from our enterprise customers, like HSBC and Sky.To that end, we launched BigQuery’s flat-rate pricing a few years ago, a fixed-rate pricing model that makes it easy for you to predict and control your monthly BigQuery bill.
We’re happy to announce BigQuery Reservations, an easy and flexible self-service way to take advantage of BigQuery flat-rate pricing, available in beta in coming days. Reservations makes it even simpler to plan your spending and add flexibility and visibility to your data analytics use cases. You’ll see this feature in cloud console in the next two weeks.
BigQuery Reservations lets you:
Control your transparent, predictable BigQuery analysis spending.
Purchase BigQuery slots in BigQuery’s web UI in seconds.
Seamlessly manage your enterprise workloads in BigQuery.
Avoid compute silos by easily sharing idle capacity across your entire organization.
BigQuery Reservations solves enterprise customers’ largest problems
"BigQuery Reservations will bring a new kind of flexibility and predictability to enterprises doing large-scale data analytics. For a serverless, cloud-native data warehouse like BigQuery, the ability to predict costs for customers is huge. And with our research showing that 42% of organizations plan to use or are exploring serverless analytics over the next 12 months, pricing and consumption flexibility will serve as a key differentiator for GCP," says Mike Leone, Senior Analyst at Enterprise Strategy Group. "This announcement opens up more possibilities for workload management, and adds higher levels of efficiency with idle slot capacity being available for reuse."
We’ve heard that you need more power and flexibility with your resources, and the ability to buy and manage BigQuery slots on your own.
“Reservations were instrumental in helping us incrementally ramp up slot capacity as we migrated over from another data warehouse, greatly increasing our cost performance,” says Jingsong Wang, engineering manager, Discord. “The ability to share idle slot capacity across projects, workloads and users helps ensure our business-critical workstreams stay online, while giving users the flexibility to run more complex workloads.”
You can get a full demo here on how BigQuery Reservations work:
Reservations can bring solutions to common issues:
Cost predictability and conformism to budgets. While cloud-native services offer unparalleled scalability and efficiency, it’s often at the expense of cost predictability, resulting in budget overruns. Pay-per-use pricing models are especially hard to manage. BigQuery Reservations offers a predictable flat-rate pricing model—no surprises on your monthly bill.
Immediate access to capacity. With BigQuery Reservations, purchasing slot commitments merely takes seconds. There is no need to wait for your data warehouse to spin up, and you no longer need to warm up your data warehouse’s disk-driven adaptive cache to get optimal performance.
Enterprise-grade workload management. Your data science group may run high-priority, high-demand workloads, and need to have isolated and guaranteed analytics capacity, whereas your test workloads need access to only a small amount of capacity. Reservations lets you dynamically and programmatically partition your BigQuery slots into pockets of resources dedicated to departments or workloads.
Efficiency. Partitioning analytics capacity can create compute silos, in which capacity is wasted. BigQuery Reservations distribute any unused BigQuery slots in real time to workloads with high-capacity demands, so even the largest and most complex environments can take advantage of every single BigQuery slot at any time. It’s time to say no to compute silos!
We’ve heard from media company Sky that they’ve found this pricing useful. "Sky has been using BigQuery's flat-rate for some time now,” says Vince Marco, enterprise infrastructure architecture manager at Sky. “Taking advantage of BigQuery's flat-rate pricing has given Sky peace of mind when it comes to performance and our BigQuery bill. Reservations helped Sky rethink how to protect business-critical workloads, while isolating lower-priority development projects and making sure we get the most of BigQuery's performance."
Adding cost predictability to your data warehouse
BigQuery Reservations is a flexible platform for administering resources and workloads. It involves a three-step process to manage your environment:
Commitments, which let you purchase slot capacity.
Reservations (optional), which give you the ability to partition your capacity.
- Assignments, which gives you the ability to assign projects, folders, or your entire organization to Reservations.
As an example, you may need 1,000 BigQuery slots for your organization. Your BigQuery users include a data science team, a high-priority ELT workload, and BI dashboards.
With BigQuery Reservations, you can:
Purchase a 1,000-slot commitment
Create reservations “ds” with 500 slots, “elt” with 300 slots, and “bi” with 200 slots
Assign the data science team’s Google Cloud projects to “ds” reservation
Assign your ELT projects to “elt” reservation
Assign the project that runs your BI dashboard to “bi” reservation
Now each of your workloads has dedicated capacity. In addition, any single unused BigQuery slot is automatically and immediately available to other workloads in your organization.
You can perform these actions right in the BigQuery UI, or programmatically in the BigQuery command-line tool.
We’ve heard from customers that BigQuery Reservations can streamline workload management and add efficiency. "The Slot Reservation API strikes a good balance between control and flexibility for managing BigQuery workloads,” says Henry Lin, engineering manager, Reddit. “We're able to isolate expensive queries from each other without fearing that we're underutilizing BigQuery resources. The API has been remarkably easy to use and, in turn, has empowered us to optimize our workflows without needing to micromanage them."
Getting started with BigQuery Reservations
BigQuery’s flat-rate pricing starts at 500 slots, and is generally a good fit for production usage and when customers are looking for price predictability. Customers can still take advantage of on-demand, serverless pricing for proof-of-concepts (POCs) and ad-hoc analysis. Here’s a look at when you might use one or the other:
To get started with Reservations, head over to the Reservations getting started documentation.
What’s next
BigQuery is a serverless enterprise data warehouse. As such, we strive to reduce our users’ administrative overhead, and to automate the day-to-day toil associated with managing a typical data warehouse. Reservations continues that trend by introducing powerful features that give you more control over your BigQuery environment. We are looking forward to hearing your feedback!
We're putting the final touches on BigQuery Reservations. Check back in soon.
Learn more about:
