Storage utilization metrics

This page describes the storage utilization metrics that Cloud Spanner provides.

Storage metrics

Spanner provides the following storage metrics:

  • Total database storage: The amount of data that is stored in the database or the databases in the instance. This is subject to the storage limit.
  • Total backup storage: The amount of data that is stored by the backups associated with the instance or database. Backup storage is stored and billed separately and there is no limit on the amount you can store.

You can view charts for these metrics in the Google Cloud console or in the Cloud Monitoring console.

Also, database storage utilization is shown in the Instances and Instance details pages in the Cloud console.

Go to the instances page

A note about data retention and storage sizes

If you use the above storage metrics to check the size of your data frequently, you might encounter results contrary to your expectations. For example, you might see the reported total storage of your database decrease by a noticeable amount, even though you hadn't recently removed any data. Conversely, you might see its size remain relatively unchanged right after performing a significant deletion.

These effects stem from Spanner's support for multi-version storage. Multi-version storage keeps all deleted or overwritten data in-storage and available for a limited time to enable features that let you read previous data values, such as stale reads and point-in-time recovery. Performing a large data deletion doesn't immediately get reflected in your database's storage metrics. Similarly, an apparently unprompted drop in a database's total size likely means that Spanner's regular data-compaction process recently cleaned up a large set of data that was deleted or overwritten as far back as several days ago.

Spanner guarantees the continued availability of deleted or overwritten data for the interval defined by the version_retention_period option (one hour, by default). It automatically runs a background process every several days that permanently removes all obsolete data older than this version-retention interval.

Create storage alerts

You can create storage alerts in the Cloud Monitoring console. We also provide an easy way to create a database storage alert directly from the Google Cloud console. The Create alerting policy link in the chart (see screenshot) takes you to the create alert page in Cloud Monitoring console and automatically prefills the relevant fields.

We recommend keeping your total database storage below the storage limit. This ensures that Cloud Spanner has enough headroom to operate normally and perform routine maintenance on the data.

If you are approaching the limit, Spanner may prevent you from performing operations that put you over the limit, such as:

  • Restoring a database from a backup.
  • Modifying the database's schema (for example, adding an index).
  • Reducing the compute capacity of your instance.

If you are over the storage limit, Spanner will attempt to operate normally, but you may experience degraded performance or failure due to resource pressure. If you do approach or exceed the recommended maximum, Google Cloud console displays a warning reading "The instance has reached its maximum storage capacity and may experience degraded activity" when displaying the affected instance.

You can also create alerts in Cloud Monitoring to notify you.

Reduce database storage utilization

To reduce an instance's database storage utilization, you can:

  • Add more compute capacity.
  • Delete a database.
  • Delete data from a database. Note that even though data deletion takes effect immediately from a visibility perspective, it does not affect the storage utilization metric until Spanner compacts the data (typically within 12 hours, but it can take longer in certain cases). Therefore, you might notice a delay from when data is deleted to when the changes appears in the metric.

In general, we recommend that you add compute capacity to your instance as a starting point. After you add compute capacity, you can investigate and address the root causes of high storage utilization.

If you want to automate this process, you can create an application that monitors database storage utilization, then adds and removes compute capacity as needed, using the UpdateInstance method.

What's next