Overview of Key Visualizer

This page provides an overview of the Key Visualizer diagnostic tool for Bigtable.

What is Key Visualizer?

Key Visualizer is a tool that helps you analyze your Bigtable usage patterns. It generates visual reports for your tables that break down your usage based on the row keys that you access.

Key Visualizer can provide insights into usage patterns at scale that are difficult to understand otherwise. Uses for Key Visualizer include the following:

  • Iteratively designing a schema or improving the design of an existing schema. In each iteration, you check Key Visualizer to spot problems your schema may be causing, then tweak your schema and check again.
  • Troubleshooting performance issues.
  • Getting a better understanding of how you access the data that you store in Bigtable.

To accomplish these goals, Key Visualizer can help you complete the following tasks:

  • Check whether your reads or writes are creating hotspots on specific rows
  • Find rows that contain too much data
  • Look at whether your access patterns are balanced across all of the rows in a table

Although Key Visualizer shows a variety of metrics, it doesn't display every single metric that can affect the performance of Bigtable. For example, if there are network issues between your application and Google Cloud, those network issues might not be visible, or they might appear as vertical bands across all keys in Key Visualizer, which only gives you hints about when the problems occurred. If you can't identify the cause of a performance issue by looking at your Key Visualizer scans, you'll need to do additional troubleshooting.

Key Visualizer scans

The following image shows a Key Visualizer scan. Each scan includes a few different types of information:

  • A large heatmap, which shows access patterns for a group of row keys over time.
  • Aggregate values along each axis of the heatmap, including average values and either total or maximum values.

Example of a Key Visualizer scan

Key Visualizer also provides tools to help you understand the data in each scan. If you haven't used Key Visualizer before, see Getting Started with Key Visualizer for instructions. If you're a more experienced user, see Exploring Heatmaps for details.

Heatmaps

The core of a Key Visualizer scan is the heatmap, which shows the value of a metric over time, broken down into contiguous ranges of row keys. The x-axis of the heatmap represents time, and the y-axis represents row keys. If the metric had a low value for a group of row keys at a point in time, the metric is "cold," and it appears in a dark color. A high value is "hot," and it appears in a bright color; the highest values appear in white.

Different types of usage result in different visual patterns within the heatmap, which can make it possible to diagnose issues at a glance. See Heatmap Patterns for examples of some common patterns.

By default, a Key Visualizer heatmap shows the Ops metric, which represents the combined number of reads and writes. You can switch to the heatmap for a different metric at any time. See Switching metrics for details.

You can also view more than one metric at a time, which can help you find connections between different metrics.

Hierarchical row keys

Row keys are often composed of a hierarchy of values, with each value separated by a delimiter. For example, the row key memusage#1423523569918 contains an identifier for all rows that capture memory usage, followed by a timestamp that identifies a specific set of data within that group.

Key Visualizer automatically recognizes this type of row key and breaks it down into a hierarchy of tabs, as shown on the left side of the example above. This feature helps you understand how your data and access patterns are distributed across the table's rows. It also enables you to drill down into the data for specific ranges of row keys more quickly.

If your row keys are not composed of multiple values, Key Visualizer still displays tabs on the left side of the scan, but the tabs might split up your row keys in unexpected ways rather than presenting a clear hierarchy.

Key buckets

A Bigtable table can have trillions of rows, so it's not always practical to report metrics for each individual row. Instead, Key Visualizer divides all of the row keys into 1,000 contiguous ranges, with roughly the same number of row keys in each range. These ranges are known as key buckets.

Key Visualizer reports most metrics as averages over each key bucket, or as maximum values within each key bucket. For Warnings metrics and Performance metrics, Key Visualizer provides higher precision by reporting metrics for individual row keys or for specific key ranges within the key bucket.

Behind the scenes, Key Visualizer generates new scan data every 15 minutes. In order to be able to concatenate data in contiguous time periods, Key Visualizer tries to keep the same key buckets as the previous 15-minute period. It is also possible that keys that are deleted still show up as bucket boundaries. Because tables change over time as data is added and removed, key buckets are sometimes recalculated so that they are balanced.

Key Visualizer also generates a new set of key buckets every two weeks, even if the table has not changed a lot, just to remove any residual bucket boundaries from deleted keys. As a result, even if your table is stable, you will notice that your key buckets are updated occasionally.

Aggregate values

In addition to the heatmap, a Key Visualizer scan includes aggregate values in bar charts along the bottom and right sides of the heatmap. When you hover over the aggregate values, Key Visualizer highlights a narrow area in the heatmap and shows the following information:

  • For the x-axis, Key Visualizer shows the average value for the current metric, along with either the total or maximum value. These values appear for all of the visible key ranges in 15-minute intervals.
  • For the y-axis, Key Visualizer shows the average value for the current metric across the visible time range, broken down into key ranges.

Limits

Minimum table size for scans

Key Visualizer is available for tables that contain at least 1 GB of data per cluster. It can take up to an hour after a table reaches that size for scans to be available.

Time spans

  • Minimum: The shortest time span you can view in Key Visualizer is 15 minutes.
  • Maximum: The longest time span you can view is 14 days.
  • Increment size: Key Visualizer data is available in 15-minute increments.

Data duration

Key Visualizer data is available for the last 14 days. For example, if you launch Key Visualizer on December 19 at 12:45 PM, the earliest start time you can choose for a time span is December 5 at 12:45 PM.

This limit also means that if you bookmark or share the URL for a Key Visualizer scan, the URL has a maximum life of 14 days.

What's next