Develop and deploy apps more easily with Cloud Spanner and Cloud Bigtable updates

One of our goals at Google Cloud is to offer all the database choices that you want as managed services in order to remove operational complexity and toil from your day. We’ve got a full range of managed database services to address a variety of your workload needs. We’re always working to add features to our databases to make your work building apps easier.

Today at Next London ‘18, we’re excited to announce the general availability of two important new features for two of our cloud-native database offerings: Cloud Spanner and Cloud Bigtable.

• Cloud Spanner: We’re adding enhancements to Cloud Spanner’s SQL capabilities to make it easier to read and write data in Cloud Spanner databases using SQL, and use off-the-shelf drivers and tooling. Cloud Spanner’s API now supports INSERT, UPDATE, and DELETE SQL statements.

• Cloud Bigtable: We’re offering Key Visualizer so you can see key access patterns in heatmap format to optimize your Cloud Bigtable schemas for improved performance. We’re adding this feature so you can build optimized data models to get the most out of your database. Key Visualizer is now generally available.

To build the databases that you want, our process is simple. We listen to you, and then we build features and products to address your needs. If you have something you want us to build, let us know. Read on for more details on these useful new database capabilities, which add compatibility and ease of use for your cloud workloads.

Cloud Spanner DML

Cloud Spanner is a relational database built for ease of use at any scale, and relied on by leading enterprise, SaaS, and gaming applications. It offers massive scalability when you need it, along with relational semantics such as strong consistency and transactions across regions and continents. The standard way to manipulate data in relational databases is the SQL data manipulation language (DML). Cloud Spanner already supports SELECT queries, and today’s announcement adds support for INSERT, UPDATE, and DELETE statements in SQL as well. Here’s what you’ll see with this new query support:

We’ve gotten great user feedback on the benefits of using DML in Cloud Spanner, including from cryptocurrency leader Blockchain. “Since we implemented Cloud Spanner for our teams, we've been impressed by its ease of use and simple integration with our existing tooling,” says Lewis Tuff, Engineering Lead at Blockchain. “One thing that has been missing for us is DML in SQL to support easier ad hoc querying, and now that we have it, we're excited to ship code even faster.”

Along with DML support in Cloud Spanner, we are also releasing a new version of our JDBC driver that now supports writes, in addition to SELECTs. This update makes it easier to reuse existing code and tool chains, so you can integrate with popular object-relational mappers (ORMs). This reduces the learning curve for new Cloud Spanner users and makes porting existing applications easier. To get started, see the documentation.

Key Visualizer for Cloud Bigtable

The Key Visualizer feature, now generally available, helps visualize key access patterns to enable schema and performance optimization in Cloud Bigtable, our fully managed NoSQL key-value and wide-column database service. Cloud Bigtable is built for high throughput and low latency at any scale and is relied on for time-series, graph, and IoT workloads. We've heard from developers the desire for more visibility into their application behavior to optimize database schemas and get the most performance from their database—and we've built Key Visualizer to address this need. Key Visualizer has helped many teams here at Google and Google Cloud customers by providing them with insights into their database usage patterns, without impacting the serving capabilities of the database itself. This means you can design and refine your data model to best take advantage of the horizontal scalability of our GCP services.This functionality hasn’t been previously available for databases, but it’s an essential part of maximizing the performance of your applications.

Key Visualizer helps debug performance issues that are caused by unbalanced access patterns across the key space, or single rows that are either too large or receiving too much read or write activity. With Key Visualizer, you get a heat map visualization of access patterns over time, along with the ability to zoom into specific key or time ranges to find more details about the hotspotting keys. This tool provides clear insights into the access patterns of a table, helping you resolve performance problems due to schema design quickly. Key Visualizer is free and works at any scale, without any impact to Cloud Bigtable cluster performance.

The Key Visualizer documentation provides guidance on what to do once a problem is detected. Key Visualizer creates scans for all Cloud Bigtable tables that meet at least one of the minimum criteria for size or activity. You can see more on how it works here:

We’ve heard great feedback from customers who have been using Key Visualizer in beta to achieve their performance objectives.

“Key Visualizer for Google Cloud Bigtable is an amazing tool to quickly and visually identify hotspots in Bigtable,” says Joeri De Turck, CTO, OTA Insight, which provides revenue management tools for the hotel industry. “It helped us greatly to compare the performance of multiple possible key schemas and evaluate which one is the most efficient for our workload.”

Retailer eBay has also used Key Visualizer in beta. “With close to 200TB of data, 15 billion rows, and 10GB/s throughput, we are always working to better understand our data access patterns at the storage level,” says Neeraj Jain, Data Architect, New Product Development at eBay. “One area we commonly look for is high read/write pressure in our Bigtable queries. Using Key Visualizer for Cloud Bigtable, we were able to get more details at the table level, which helped us perform faster diagnostics and make further optimizations to how we store our data.”

With this launch, we are also adding a new feature: Multimetric views. This lets you view multiple metrics at the same time when you want to compare metrics or find correlations between them.

Here's what a multi-metric display looks like, with miniature heatmaps on the right:

For more information on using Key Visualizer, check out this Google Cloud Next ‘18 presentation by Steve Niemitz, Staff Software Engineer at Twitter, together with Misha Brukman and Wim De Pauw from our team here at Google.

These features continue to bring a better development experience to our managed database offerings so you can build apps faster. We look forward to seeing what you will build next!