Google Cloud Datastore uses indexes for every query your application makes.
These indexes are updated whenever an entity changes, so the results can be
returned quickly when the application makes a query. Cloud Datastore provides built-in indexes automatically, but needs to
know in advance which composite indexes the application will require. You specify which
composite indexes your application needs in a configuration file. The Cloud Datastore emulator can
generate the Cloud Datastore index configuration automatically as you test your
gcloud command-line tool provides commands to update the
indexes that are available to your production Cloud Datastore instance.
To use the
gcloud tool, you must have installed the Google Cloud SDK.
Every Cloud Datastore query made by an application needs a corresponding index.
Indexes for simple queries, such as queries over a single property, are created
automatically. Indexes for complex queries must be defined in a configuration
index.yaml. This file is uploaded with the application to create
indexes in Cloud Datastore.
The Cloud Datastore emulator automatically adds items to this file when the
application tries to execute a query that needs an index that does not have an
appropriate entry in the configuration file. You can adjust indexes or create
new ones manually by editing the file. The
index.yaml is located in the
<project-directory>/WEB-INF/ folder. By default, the data directory that
Cloud Datastore emulator project directories for additional details.
The following is an example of an
indexes: - kind: Task ancestor: no properties: - name: done - name: priority direction: desc - kind: Task properties: - name: collaborators direction: asc - name: created direction: desc - kind: TaskList ancestor: yes properties: - name: percent_complete direction: asc - name: type direction: asc
The syntax of
index.yaml is the YAML format. For more information about this
syntax, see the YAML website.
index.yaml has a single list element called
indexes. Each element in the
list represents an index for the application.
An index element can have the following elements:
- The kind of the entity for the query. This element is required.
A list of properties to include as columns of the index, in the order to be sorted: properties used in equality filters first, followed by the property used in inequality filters, then the sort orders and their directions.
Each element in this list has the following elements:
- The Cloud Datastore name of the property.
- The direction to sort, either
ascfor ascending or
descfor descending. This is only required for properties used in sort orders of the query, and must match the direction used by the query. The default is
yesif the query has an ancestor clause. The default is
Automatic and manual indexes
When the Cloud Datastore emulator adds a generated index definition to
index.yaml, it does so below the following line, inserting it if necessary:
The emulator considers all index definitions below this line to be automatic, and it may update existing definitions below this line as the application makes queries.
All index definitions above this line are considered to be under manual control,
and are not updated by the emulator. The emulator will only
make changes below the line, and will only do so if the complete
file does not describe an index that accounts for a query executed by the
application. To take control of an automatic index definition, move it above
datastore create-indexes command looks at your local Datastore index
index.yaml file), and if the index configuration defines an
index that doesn't exist yet in your production Cloud Datastore instance, Cloud Datastore
creates the new index. See the development workflow using the
for an example of how to use
Depending on how much data is already in Cloud Datastore that belongs in the new index, the process of creating the index may take a while. If the application performs a query that requires an index that hasn't finished building yet, the query will raise an exception. To prevent this, you must be careful about deploying a new version of your application that requires a new index before the new index finishes building.
Deleting unused indexes
When you change or remove an index from the index configuration, the original index is not deleted from Cloud Datastore automatically. This gives you the opportunity to leave an older version of the application running while new indexes are being built, or to revert to the older version immediately if a problem is discovered with a newer version.
When you are sure that old indexes are no longer needed, you can delete them
from Cloud Datastore using the
datastore cleanup-indexes command. This command
deletes all indexes for the production Cloud Datastore instance that are not mentioned
in the local version of
the development workflow using the
gcloud tool for an example of how to
For details on command-line arguments for creating and cleaning indexes, see
datastore create-indexes and
respectively. For details on command-line arguments for the
gcloud tool, see
gcloud tool reference.