The App Engine datastore uses indexes for every query your application makes. These indexes are updated whenever an entity changes, so the results can be returned quickly when the app makes a query. To do this, the datastore needs to know in advance which queries the application will make. You specify which indexes your app needs in a configuration file. The development server can generate the datastore index configuration automatically as you test your app.
This guides provides instructions for creating your indexes and managing them. For more background information, see Datastore Indexes.
Creating Datastore indexes
Every datastore query made by an application needs a corresponding index.
Indexes for simple queries, such as queries over a single property, are created
automatically. Indexes for complex queries must be defined in a configuration
This file is uploaded with the application to create indexes in the datastore.
The following is an example of an
indexes: - kind: Cat ancestor: no properties: - name: name - name: age direction: desc - kind: Cat properties: - name: name direction: asc - name: whiskers direction: desc - kind: Store ancestor: yes properties: - name: business direction: asc - name: owner direction: asc
Creating indexes using the development server
The development web server (dev_appserver.py) automatically adds items to this file when the application tries to execute a query that needs an index that does not have an appropriate entry in the configuration file.
In the development server, if you exercise every query that your app will make,
the development server will generate a complete list of entries in the
When the development web server adds a generated index definition to
index.yaml, it does so below the following line, inserting it if necessary:
The development web server considers all index definitions below this line to be automatic, and it might update existing definitions below this line as the application makes queries.
Creating indexes manually
You can add indexes manually to the
index.yaml file or you can edit existing
entries. For manually controlled queries, you must add them above the line that
is commented with
All index definitions above this line are considered to be under manual control,
and are not updated by the development web server. The development server will
only make changes below the line, and will only do so if the complete
index.yaml file does not describe an index that accounts for a query executed
by the application. To take control of an automatic index definition, move it
above this line.
index.yaml reference for
When you upload an application using
appcfg.py update, the update includes the
index.yaml file. If the
index.yaml file defines an index that doesn't
exist yet on App Engine, App Engine creates the new index. Depending on how
much data is already in the datastore that needs to be mentioned in the new
index, the process of creating the index may take a while. If the app performs
a query that requires an index that hasn't finished building yet, the query will
raise an exception.
To prevent this, you must ensure that the new version of the app that requires a
new index is not the live version of the application until the indexes finish
building. One way to do this is to give the app a new version number in
app.yaml whenever you add or change an index in
app is uploaded as a new version, and does not become the default version
automatically. When your indexes have finished building, you can change the
default version in the
Cloud Platform Console.
Another way to ensure that new indexes are built before the new app goes live is
to upload the
index.yaml configuration separately before uploading the app. To
upload only the index configuration for an app, use the following command:
appcfg.py update_indexes myapp/
In the Cloud Platform Console, you can check the status of the app's indexes.
Deleting unused indexes
When you change or remove an index from
index.yaml, the original index is
not deleted from App Engine automatically. This gives you the opportunity to
leave an older version of the app running while new indexes are being built, or
to revert to the older version immediately if a problem is discovered with a
When you are sure that old indexes are no longer needed, you can delete them from App Engine using the following command:
appcfg.py vacuum_indexes myapp/
This command deletes all indexes for the app that are not mentioned in the local