Getting Information about Your Search Indexes

Amy Unruh, Oct 2012
Google Developer Relations


After you've created some documents and added them to some indexes, you'll often find it useful to inspect the structure and contents of the indexes. This lesson teaches you how.


Learn about the Search API's indexes


Index schemas

Each index has an associated schema that is used to interpret queries on that index. The schema is created dynamically from the field names and types present in all of the documents in the index; so adding documents to an index can cause updates to the index's schema as well.

It can be useful to view an index's schema, since it allows you to see the structure of the data in that index. A simple schema for an index might look like the following:

{'comment': ['TEXT'], 'date': ['DATE'], 'author': ['TEXT'], 'count': ['NUMBER']}

This shows that the documents in the index use the four fields comment, date, author, and count, with the indicated field types. (A given individual document need not have used all of these fields.) The schema takes the form of a dictionary in which each key is the name of a document field that has been added to the index, and the corresponding value is a list of the field types used with that field name. If you use the same field name with different field types (in different documents, for instance), the schema will list more than one field type for that name, like this:

{'field1': ['TEXT', 'NUMBER']}

As a more complex example, here is the schema for the product document index in the example application, once the sample data has been added to it. (This output shows the raw field names, not the class variables we've been using in our code examples).

{u'category': ['ATOM'], u'publisher': ['TEXT'], u'isbn': ['TEXT'],
 u'description': ['TEXT'], u'name': ['TEXT'], u'author': ['TEXT'],
 u'price': ['NUMBER'], u'title': ['TEXT'], u'pid': ['TEXT'], u'modified': ['DATE'],
 u'tv_type': ['TEXT'], u'ar': ['NUMBER'],
 u'brand': ['TEXT'], u'pages': ['NUMBER'], u'size': ['NUMBER']}

In the sample app, this index is called productsearch1 by default. The schema reflects both categories of product documents used by the application (books and hd_televisions), showing not only the core product fields shared by both categories, but also those specific to one category or the other, such as author (used only by books), tv_type (used only by hd_televisions), and so on.

Viewing indexes and documents

Use the Cloud Platform Console Cloud Datastore section to view information about your application's indexes, and the documents they contain.

Accessing index schemas programatically

It's also possible to access all of your application's indexes programmatically, along with the schemas for each:

from google.appengine.api import search
import logging

for index in search.get_indexes(fetch_schema=True):"index %s","schema: %s", index.schema)

The listed schema information will look like the dictionaries shown above.

There is currently no way to delete a schema or remove fields from it.

Summary and review

In this lesson, we've looked at ways to get information about an application's indexes and their schemas.

Try this yourself with the example application: Examine the application's indexes in the Cloud Platform Console, then try adding the code of the previous section (which lists your indexes and schemas) to the get() method of the IndexHandler class in

class IndexHandler(BaseHandler):
  """Displays the 'home' page."""

  def get(self):
    cat_info = models.Category.getCategoryInfo()
    sort_info = docs.Product.getSortMenu()
    # add this code temporarily to look at your indexes
    for index in search.get_indexes(fetch_schema=True):"index %s","schema: %s", index.schema)
    template_values = {
        'cat_info': cat_info,
        'sort_info': sort_info,
    self.render_template('index.html', template_values)

You should only add this code temporarily, because it adds expense to the request. When you redeploy the application and look at its logs when you load the index page, you'll see your indexes and schemas listed out for you.

Now that we've worked through the basics of indexing and querying content, you already have the tools to add full-text search to your App Engine applications. However, the Search API supports much more sophisticated querying than we've shown so far. In the follow-on class, A Deeper Look at the Python Search API, we'll delve deeper into search query construction and learn how to control and sort the results that are returned.

Send feedback about...

App Engine standard environment for Python