This page describes the storage utilization metrics that Cloud Spanner provides.
Cloud Spanner provides the following storage metrics:
- Total database storage: The amount of data that is stored in the database or the databases in the instance. This is subject to the storage limit.
- Total backup storage: The amount of data that is stored by the backups associated with the instance or database. Backup storage is stored and billed separately and there is no limit on the amount you can store.
Also, database storage utilization is shown in the Instances and Instance details pages in the Cloud console.
A note about data retention and storage sizes
If you use the above storage metrics to check the size of your data frequently, you might encounter results contrary to your expectations. For example, you might see the reported total storage of your database decrease by a noticeable amount, even though you hadn't recently removed any data. Conversely, you might see its size remain relatively unchanged right after performing a significant deletion.
These effects stem from Cloud Spanner's support for multi-version storage. This keeps all deleted or overwitten data in-storage and available for a limited time to enable features that let you read previous data values, such as stale reads and point-in-time recovery.
Cloud Spanner guarantees the continued availability of deleted or overwritten data for the interval defined by the
version_retention_period option (one hour, by default). It automatically runs a background process every several days that permanently removes all obsolete data older than this version-retention interval.
Because of multi-version storage, performing a large data deletion doesn't immediately get reflected in your database's storage metrics. Similarly, an apparently unprompted drop in a database's total size likely means that Cloud Spanner's regular data-compaction process recently cleaned up a large set of data that was deleted or overwitten as far back as several days ago.
You can create alerts on any metric in the Cloud Monitoring console. We also provide an easy way to create a database storage alert directly from the Cloud console. The Create alerting policy link in the chart (see screenshot) takes you to the create alert page in Cloud Monitoring console and automatically prefills the relevant fields.
Recommendations for database storage utilization
We recommend keeping your total database storage below the storage limit. This ensures that Cloud Spanner has enough headroom to operate normally and perform routine maintenence on the data.
If you are approaching the limit, Cloud Spanner may prevent you from performing operations that put you over the limit, such as:
- restoring a database from a backup
- modifying the database's schema (for example, adding an index)
- reducing the compute capacity of your instance
If you are over the storage limit, Cloud Spanner will attempt to operate normally, but you may experience degraded performance or failure due to resource pressure. If you do approach or exceed the recommended maximum, Cloud console displays a warning reading "The instance has reached its maximum storage capacity and may experience degraded activity" when displaying the affected instance.
You can also create alerts in Cloud Monitoring to notify you.
Reducing database storage utilization
To reduce an instance's database storage utilization, you can:
- add more compute capacity
- delete a database
- delete data from a database. Note that even though data deletion takes effect immediately from a visibility perspective, it does not affect the storage utilization metric until Cloud Spanner compacts the data (typically within 12 hours, but can take longer in certain cases). Therefore, you may notice a delay from when data is deleted to when the changes appears in the metric.
In general, we recommend that you add compute capacity to your instance as a starting point. After you add compute capacity, you can investigate and address the root causes of high storage utilization.
If you want to automate this process, you can create an application
that monitors database storage utilization, then adds and removes compute
capacity as needed, using the
- Monitor your instance with the Cloud console or the Cloud Monitoring console.
- Create alerts for Cloud Spanner.
- Find out how to change the compute capacity of a Cloud Spanner instance.