datastore-indexes.xml reference

Python 2.7/3.7 | Java 8 |PHP |Go 1.9/1.11 |Node.js

Note: This reference uses an XML formatted configuration file used by tooling based on the App Engine SDK for Java; for example the appcfg tool. If you are using Cloud SDK-based tooling, such as the gcloud command line, or Cloud SDK-based Maven, Gradle, Eclipse, or IntelliJ plugins, refer to the YAML reference.

App Engine predefines a simple index on each property of an entity. An App Engine application can define further custom indexes in an index configuration file named datastore-indexes.xml. The development server can generate the index configuration needed by your application when it performs these queries while you are testing your app. You can tune indexes manually by editing the datastore-indexes.xml file before uploading the application. See Configuring Datastore Indexes for information about how to deploy and manage index configurations.

Example

The following is an example of a datastore-indexes.xml file:

<?xml version="1.0" encoding="utf-8"?>
<datastore-indexes
  autoGenerate="true">
    <datastore-index kind="Employee" ancestor="false">
        <property name="lastName" direction="asc" />
        <property name="hireDate" direction="desc" />
    </datastore-index>
    <datastore-index kind="Project" ancestor="false">
        <property name="dueDate" direction="asc" />
        <property name="cost" direction="desc" />
    </datastore-index>
</datastore-indexes>

Syntax

The <datastore-index> element can contain the following elements:

Element	Description
<kind>	Required. The kind of the entity for the query.
<properties>	A list of properties to include as columns of the index, in the order to be sorted: properties used in equality filters first, followed by the property used in inequality filters, then the sort orders and their directions. Each element in this list has the following elements: <name> The datastore name of the property. <direction> The direction to sort, either asc for ascending or desc for descending. This is only required for properties used in sort orders of the query, and must match the direction used by the query. The default is asc.
<ancestor>	yes if the query has an ancestor clause (). The default is no.

Automatic and manual indexes

Determining the indexes required by your application's queries manually can be tedious and error-prone. Thankfully, the development server can determine the index configuration for you. To use automatic index configuration, add the attribute autoGenerate="true" to your WEB-INF/datastore-indexes.xml file's <datastore-indexes> element. Automatic index configuration is also used if your app does not have a datastore-indexes.xml file.

With automatic index configuration enabled, the development server maintains a file named WEB-INF/appengine-generated/datastore-indexes-auto.xml in your app's war/ directory. When your app, running in the development server, attempts a datastore query for which there is no corresponding index in either datastore-indexes.xml or datastore-indexes-auto.xml, the server adds the appropriate configuration to datastore-indexes-auto.xml.

If automatic index configuration is enabled when you upload your application, the appcfg command uses both datastore-indexes.xml and datastore-indexes-auto.xml to determine which indexes need to be built for your app in production.

If autoGenerate="false" is in your datastore-indexes.xml, the development server and AppCfg ignore the contents of datastore-indexes-auto.xml. If the app running locally performs a query whose index is not specified in datastore-indexes.xml, the development server throws an exception, just as the production Datastore would.

It's a good idea to occasionally move index configuration from datastore-indexes-auto.xml to datastore-indexes.xml, then disable automatic index configuration and test your app in the development server. This makes it easy to maintain indexes without having to manage two files, and ensures that your testing will reproduce errors caused by missing index configuration.

Deploying the index configuration file

The datastore-indexes.xml file should reside in your application's /WEB-INF/ directory of the default service.

To deploy the datastore indexes configuration file without otherwise altering the currently serving version, use the command:

appcfg.sh update_indexes <application root path>

Deleting unused indexes

When you change or remove an index from the index configuration, the original index is not deleted from App Engine automatically. This gives you the opportunity to leave an older version of the app running while new indexes are being built, or to revert to the older version immediately if a problem is discovered with a newer version.

When you are sure that old indexes are no longer needed, you can delete them from App Engine as follows:

./appengine-java-sdk/bin/appcfg.sh vacuum_indexes myapp/war