Managing indexes

Managing indexes

Firestore ensures query performance by requiring an index for every query. The indexes required for the most basic queries are automatically created for you. As you use and test your app, Cloud Firestore generates error messages that help you create additional indexes your app requires. This page describes how to manage your single-field and composite indexes.

Create a missing index through an error message

If you attempt a compound query with a range clause that doesn't map to an existing index, you receive an error. The error message includes a direct link to create the missing index in the Firebase console.

Note: You can manage Firestore through the Firebase console or the Google Cloud Platform Console, but these links will always open in the Firebase console.

Follow the generated link to the Firebase console, review the automatically populated info, and click Create.

Note: For non-array and non-map fields, you must select ascending or descending ordering, even if you don't need the field for ordering. Your choice doesn't impact the behavior of equalities in the query.

Roles and permissions

Before you can create an index in Firestore, make sure that you are assigned either of the following roles:

Note: The following roles are managed through Identity and Access Management (IAM). For more information about roles and associated permissions, see Predefined roles.

• roles/datastore.owner
• roles/datastore.indexAdmin
• roles/editor
• roles/owner

If you have defined custom roles, assign all of the following permissions to create indexes:

• datastore.indexes.create
• datastore.indexes.delete
• datastore.indexes.get
• datastore.indexes.list
• datastore.indexes.update

Use the Google Cloud Platform Console

From the Google Cloud Platform Console, you can manage single-field indexing exemptions and composite indexes.

Create a composite index

To manually create a new composite index from the GCP console:

1. In the Google Cloud console, go to the Databases page.

   Go to Databases

2. Select the required database from the list of databases.

3. In the navigation menu, click Indexes, and then click the Composite tab.

4. Click Create Index.

5. Enter a Collection ID. Add the names of the fields you want to index and an index mode for each field. Click Save Index.

Your new index will show up in the list of composite indexes and Firestore will begin creating your index. When your index is done creating, you will see a green check mark next to the index.

Delete a composite index

To delete a composite index:

1. In the Google Cloud console, go to the Databases page.

   Go to Databases

2. Select the required database from the list of databases.

3. In the navigation menu, click Indexes, and then click the Composite tab.

4. In the list of your composite indexes, click the More button for the index you want to delete. Click Delete.

5. Confirm that you want to delete this index by clicking Delete Index from the alert.

Add a single-field index exemption

Single-field index exemptions allow you to override automatic index settings for specific fields in a collection. You can add a single-field exemptions from the console:

1. In the Google Cloud console, go to the Databases page.

   Go to Databases

2. Select the required database from the list of databases.

3. In the navigation menu, click Indexes, and then click the Single Field tab.

4. Click Add Exemption.

5. Enter a Collection ID and Field path.

6. Select new indexing settings for this field. Enable or disable automatically updated ascending, descending, and array-contains single-field indexes for this field.

7. Click Save Exemption.

Delete a single-field index exemption

To delete a single-field index exemption, do the following:

1. In the Google Cloud console, go to the Databases page.

   Go to Databases

2. Select the required database from the list of databases.

3. In the navigation menu, click Indexes, and then click the Single Field tab.

4. In the list of your single-field index exemptions, click the More button for the exemption you want to delete. Click Delete.

5. Confirm that you want to delete this exemption by clicking Delete from the alert.

When you delete a single-field exemption, the specified field or sub-field will use inherited indexing settings. Document fields revert to your database's automatic index settings. Sub-fields in a map inherit any exemptions on parent fields before inheriting automatic index settings.

Use the Firebase CLI

You can also deploy indexes with the Firebase CLI. To get started, run firebase init firestore in your project directory. During setup, the Firebase CLI generates a JSON file with the default indexes in the correct format. Edit the file to add more indexes and deploy it with the firebase deploy command.

To deploy Firestore indexes and rules only, add the --only firestore flag.

If you make edits to the indexes using the Firebase console, make sure you also update your local indexes file. Refer to the JSON index definition reference.

Index build time

To build an index, Firestore must set up the index and then backfill the index with existing data. Index build time is the sum of setup time and backfill time:

• Setting up an index takes a few minutes. The minimum build time for an index is a few minutes, even for an empty database.

• Backfill time depends on how much existing data belongs in the new index. The more field values that match the index definition, the longer it takes to backfill the index.

Index builds are long-running operations.

Key Term: Firestore supports several administrative operations that can take a long time to complete. These operations are called long-running operations. Firestore includes features to execute and manage long- running operations. Supported long-running operations include index builds and export operations.

After you start an index build, Firestore assigns the operation a unique name. Operation names are prefixed with projects/[PROJECT_ID]/databases/(default)/operations/, for example:

projects/project-id/databases/(default)/operations/ASA1MTAwNDQxNAgadGx1YWZlZAcSeWx0aGdpbi1zYm9qLW5pbWRhEgopEg

However, you can leave out the prefix when specifying an operation name for the describe command.

Listing all long-running operations

To list long-running operations, use the gcloud firestore operations list command. This command lists ongoing and recently completed operations. Operations are listed for a few days after completion:

gcloud firestore operations list

Check operation status

Instead of listing all long-running operations, you can list the details of a single operation:

gcloud firestore operations describe operation-name

Estimating the completion time

As your operation runs, see the value of the state field for the overall status of the operation.

A request for the status of a long-running operation also returns the metrics workEstimated and workCompleted. These metrics are returned for the number of documents. workEstimated shows the estimated total number of documents an operation will process. workCompleted shows the number of documents processed so far. After the operation completes, workCompleted reflects the total number of documents that were actually processed, which might be different than the value of workEstimated.

Divide workCompleted by workEstimated for a rough progress estimate. The estimate might be inaccurate because it depends on delayed statistics collection.

For example, here is the progress status of an index build:

{
  "operations": [
    {
      "name": "projects/project-id/operations/AyAyMDBiM2U5NTgwZDAtZGIyYi0zYjc0LTIzYWEtZjg1ZGdWFmZWQHEjF0c2Flc3UtcmV4ZWRuaS1uaW1kYRUKSBI",
      "metadata": {
        "@type": "type.googleapis.com/google.firestore.admin.v1.IndexOperationMetadata",
        "common": {
          "operationType": "CREATE_INDEX",
          "startTime": "2020-06-23T16:52:25.697539Z",
          "state": "PROCESSING"
        },
        "progressDocuments": {
          "workCompleted": "219327",
          "workEstimated": "2198182"
        }
       },
    },
    ...

When an operation is done, the operation description will contain "done": true. See the value of the state field for the result of the operation. If the done field is not set in the response, then its value is false. Do not depend on the existence of the done value for in-progress operations.

Index building errors

You might encounter index building errors when managing composite indexes and single-field index exemptions. An indexing operation can fail if Firestore encounters a problem with the data it's indexing. Most commonly, this means you hit an index limit. For example, the operation may have reached the maximum number of index entries per document.

If index creation fails, you see the error message in the console. After you verify that you are not hitting any index limits, re-try your index operation.