App Engine uses indexes in Datastore for every query your application makes. These indexes are updated whenever an entity changes, so the results can be returned quickly when the app makes a query. To do this, Datastore needs to know in advance which queries the application will make. You specify which indexes your app needs in a configuration file. The local development server can automatically generate the index configuration file as you test your app.
This guides provides instructions for creating your indexes and managing them. For more background information, see Datastore Indexes.
Creating Datastore indexes
Every Datastore query made by an application needs a
corresponding index. Indexes for simple queries, such as queries over a single
property, are created automatically. Indexes for complex queries must be defined
in a configuration file named
index.yaml
.
This file is deployed with your application to create the indexes in
Datastore.
The following is an example of an index.yaml
file:
indexes:
- kind: Cat
ancestor: no
properties:
- name: name
- name: age
direction: desc
- kind: Cat
properties:
- name: name
direction: asc
- name: whiskers
direction: desc
- kind: Store
ancestor: yes
properties:
- name: business
direction: asc
- name: owner
direction: asc
Creating indexes using the development server
The development web server (dev_appserver.py) automatically adds items to this file when the application tries to execute a query that needs an index that does not have an appropriate entry in the configuration file.
In the development server, if you exercise every query that your app will make,
the development server will generate a complete list of entries in the
index.yaml
file.
When the development web server adds a generated index definition to
index.yaml
, it does so below the following line, inserting it if necessary:
# AUTOGENERATED
The development web server considers all index definitions below this line to be automatic, and it might update existing definitions below this line as the application makes queries.
Creating indexes manually
You can manually add indexes to the index.yaml
file or you can edit existing
entries. For manually controlled queries, you must add them above the line that
is commented with AUTOGENERATED
.
All index definitions above this line are considered to be under manual control,
and are not updated by the development web server. The development server will
only make changes below the line, and will only do so if the complete
index.yaml
file does not describe an index that accounts for a query
executed by the application. To take control of an automatic index definition,
move it above this line.
See the index.yaml
reference for syntax
information.
Updating indexes
You upload your index.yaml
configuration file to Datastore
with the gcloud
command. If
the index.yaml
file defines any indexes that don't exist in
Datastore, those new indexes are built.
It can take a while for Datastore to create all the indexes and therefore, those indexes won't be immediately available to App Engine. If your app is already configured to receive traffic, then exceptions can occur for queries that require an index that is still in the process of being built.
To avoid exceptions, you must allow time for all the indexes to build. For more information and examples about creating indexes, see Deploying a Python 2 App.
To upload your index configuration to Datastore, run the following command from the directory where your index.yaml is located:
gcloud
gcloud datastore indexes create index.yaml
For information, see the gcloud datastore
reference.
appcfg
appcfg.py update_indexes [YOUR_APP_DIRECTORY]
You can use the Cloud Console, to check the status of your indexes.
Deleting unused indexes
When you change or remove an index from index.yaml
, the original index is
not automatically deleted from Datastore. This gives you the
opportunity to leave an older version of the app running while new indexes are
being built, or to revert to the older version immediately if a problem is
discovered with a newer version.
When you are sure that the old indexes remaining in Datastore are
no longer needed, you can run the following command to delete any index that is
not defined in your index.yaml
file:
gcloud
gcloud datastore cleanup-indexes index.yaml
For information, see the gcloud datastore
reference.
appcfg
appcfg.py vacuum_indexes [YOUR_APP_DIRECTORY]