Stay organized with collections Save and categorize content based on your preferences.

Manage data retention with TTL policies

This page describes how to use the Google Cloud console and the Google Cloud CLI to configure time-to-live (TTL) policies. Before you read this page, you should understand the Firestore data model.

Time-to-live overview

Use time-to-live (TTL) policies to automatically remove stale data from your databases. A TTL policy designates a given field as the expiration time for documents in a given collection group. With TTL, you can decrease storage costs by cleaning out obsolete data. Data is typically deleted within 72 hours after its expiration date.

Pricing

TTL delete operations count towards your document delete costs. For pricing of delete operations, see Firestore pricing.

Limits and constraints

  • Only one field per collection group can be marked as a TTL field.
  • A total of 200 field level configurations are allowed. One field configuration can contain multiple configurations for the same field. For example, a single-field indexing exemption and a TTL policy on the same field count as one field configuration towards the limit.

TTL deletion

Note the following key behaviors of TTL-driven deletion:

  • Deletion through TTL is not an instantaneous process. Expired documents continue to appear in queries and lookup requests until the TTL process actually deletes them. TTL trades deletion timeliness for the benefit of reduced total cost of ownership for deletions. Data is typically deleted within 72 hours after its expiration date.

  • Applying a TTL policy on an existing collection group results in a bulk deletion of all data that's expired according to the new TTL policy. Note that this bulk deletion is also not instantaneous and depends on how much data exists for that collection group.

  • TTL does not necessarily delete documents in the same order as their expiration timestamps.

  • Deletions are not done transactionally. Documents with the same expiration time are not necessarily deleted at the same time. If you require this behavior, perform the deletions using a client library.

  • Firestore will always honor the latest TTL field to determine the expiration. For example, if an expired but not-yet-deleted document has its TTL field updated to a later date, the document will not be expired and the new date will be used.

  • TTL is designed to minimize impact on other database activities. Deletions driven by TTL are treated with a lower priority. Other strategies are also in place to smooth out traffic spikes from TTL-driven deletes.

  • Deletion through TTL calls all active snapshot listeners and triggers Cloud Functions Firestore triggers.

TTL fields and indexes

A TTL field can be indexed or un-indexed. However, because a TTL field is a timestamp, indexing the field can affect performance at higher traffic rates. Indexing a timestamp field can create hotspots which is against best practices. Hotspots are high read, write, and delete rates to a narrow document range.

By default, Firestore creates a single-field index for all fields. You can create a single-field index exemption to disable indexes on a TTL field.

Permissions

The principal configuring a TTL policy requires the following permission in the project:

  • Viewing TTL policies requires the datastore.indexes.list and datastore.indexes.get permissions.
  • Modifying TTL policies requires the datastore.indexes.update permission.
  • Checking the status of TTL operations requires datastore.operations.list and datastore.operations.get.

For roles that assign these permissions, see Firestore Identity and Access Management roles.

Before you begin

Before you use the gcloud CLI to manage TTL policies, use the gcloud components update command to update components to the latest available version:

gcloud components update

Create a TTL policy

When you create a TTL policy, you designate a document field as the expiration time for documents in a collection group.

TTL uses a specified field to identify documents that are eligible for deletion. This TTL field must be of type Date and time. You can select a field that already exists or you can designate a field that you plan to add later. The TTL field value can be a time in the future, now, or in the past. If the value is time in the past, the document is immediately eligible for deletion.

For example, you might create a TTL policy with the field expireAt which you then add to existing documents.

Follow the steps below to create a TTL policy:

Google Cloud Console

  1. Go to the Firestore Time-to-live page in the Google Cloud console.

    Go to the Time-to-live page

  2. Click Create Policy.

  3. Enter a collection group name and a timestamp field name.

  4. Click Create.

The console returns to the Time-to-live page. If the operation successfully starts, the page adds an entry to the TTL policies table. On failure, the page displays an error message.

gcloud

Use the firestore fields ttls update command to configure a TTL policy. Add the --async flag to prevent the gcloud CLI from waiting for the operation to complete.

gcloud beta firestore fields ttls update ttl_field --collection-group=collection_group_name --enable-ttl

Even on an empty database, it can take ten minutes or more to enable a TTL policy. Once you start an operation, closing the terminal does not cancel the operation.

View TTL policies

Follow the steps below to view TTL policies and their statuses.

Google Cloud Console

Go to the Firestore Time-to-live page in the Google Cloud console.

Go to the Time-to-live page

The console lists TTL policies for your database and includes each policy's status.

gcloud

Use the firestore fields ttls list command to configure a TTL policy. The following command lists all TTL policies.

gcloud beta firestore fields ttls list 

To list TTL policies under a specific collection group, use the following:

gcloud beta firestore fields ttls list  --collection-group=collection_group_name

View operation details

You can use the gcloud CLI to view more details about a TTL policy that is in the CREATING state.

Use the operations list command to see all running and recently completed operations:

gcloud firestore operations list

The response includes an estimate of the operation's progress.

Disable a TTL policy

Follow the steps below to disable a TTL policy.

Google Cloud Console

  1. Go to the Firestore Time-to-live page in the Google Cloud console.

    Go to the Time-to-live page

  2. In the TTL policy table, find the row for the TTL policy. Within this table row, click the Delete (trashcan) button.

  3. Confirm by clicking Delete.

The console returns to the Time-to-live page. On success, Firestore removes the TTL policy from the table.

gcloud

Use the firestore fields ttls update command to configure a TTL policy. Add the --async flag to prevent the gcloud CLI from waiting for the operation to complete.

gcloud beta firestore fields ttls update ttl_field --collection-group=collection_group_name --disable-ttl

Monitor TTL deletions

You can use Cloud Monitoring to view metrics about TTL-driven deletions. Firestore provides the following metrics for TTL:

firestore.googleapis.com/document/ttl_deletion_count Time-to-live deletion count

Total count of documents deleted by Time-to-live (TTL) policies.

firestore.googleapis.com/document/ttl_expiration_to_deletion_delays Time-to-live expiration to deletion delays

Time elapsed between when a document expired under a Time-to-live (TTL) policy and when it was actually deleted.

To set up a dashboard with Firestore metrics, see manage custom dashboard and add dashboard widgets.