Storing Data in Datastore

---

This part of the Python Guestbook code walkthrough shows how to store structured data in Datastore. With App Engine and Datastore, you don't have to worry about distribution, replication, and load balancing of data. That is done for you behind a simple API—and you get a powerful query engine and transactions as well.

Note: Datastore is only one option you have for storing application data. If you prefer to use relational data and SQL instead of data structures, try walking through the instructions for using Google Cloud SQL, which also use the Guestbook example application.

This page is part of a multi-page tutorial. To start from the beginning and see instructions for setting up, go to Creating a Guestbook.

Storing the submitted greetings

Data is written to Datastore in objects known as entities. Each entity has a key that uniquely identifies it. An entity can optionally designate another entity as its parent; the first entity is a child of the parent entity. The entities in the data store thus form a hierarchically-structured space similar to the directory structure of a file system. For detailed information, see Structuring Data for Strong Consistency.

App Engine includes a data modeling API for Python. To use the data modeling API, the sample app imports the google.appengine.ext.ndb module. Each greeting includes the author's name, the message content, and the date and time the message was posted. The app displays messages in chronological order. The following code defines the data model:

guestbook.py

View on GitHub

class Author(ndb.Model):
    """Sub model for representing an author."""
    identity = ndb.StringProperty(indexed=False)
    email = ndb.StringProperty(indexed=False)


class Greeting(ndb.Model):
    """A main model for representing an individual Guestbook entry."""
    author = ndb.StructuredProperty(Author)
    content = ndb.StringProperty(indexed=False)
    date = ndb.DateTimeProperty(auto_now_add=True)

The code defines a Greeting model with three properties: author whose value is an Author object with the email address and the author's identity, content whose value is a string, and date whose value is a datetime.datetime.

Some property constructors take parameters to further configure their behavior. Passing the ndb.StringProperty constructor the indexed=False parameter says that values for this property will not be indexed. This saves writes which aren't needed because the app never uses that property in a query. Passing the ndb.DateTimeProperty constructor an auto_now_add=True parameter configures the model to automatically give new objects a datetime stamp of the time the object is created, if the application doesn't otherwise provide a value. For a complete list of property types and their options, see NDB Properties.

The application uses the data model to create new Greeting objects and put them into Datastore. The Guestbook handler creates new greetings and saves them to the data store:

guestbook.py

View on GitHub

class Guestbook(webapp2.RequestHandler):

    def post(self):
        # We set the same parent key on the 'Greeting' to ensure each
        # Greeting is in the same entity group. Queries across the
        # single entity group will be consistent. However, the write
        # rate to a single entity group should be limited to
        # ~1/second.
        guestbook_name = self.request.get('guestbook_name',
                                          DEFAULT_GUESTBOOK_NAME)
        greeting = Greeting(parent=guestbook_key(guestbook_name))

        if users.get_current_user():
            greeting.author = Author(
                    identity=users.get_current_user().user_id(),
                    email=users.get_current_user().email())

        greeting.content = self.request.get('content')
        greeting.put()

        query_params = {'guestbook_name': guestbook_name}
        self.redirect('/?' + urllib.urlencode(query_params))

This Guestbook handler creates a new Greeting object, then sets its author and content properties with the data posted by the user. The parent of Greeting is a Guestbook entity. There's no need to create the Guestbook entity before setting it to be the parent of another entity. In this example, the parent is used as a placeholder for transaction and consistency purposes. See the Transactions page for more information. Objects that share a common ancestor belong to the same entity group. The code does not set the date property, so date is automatically set to the present, using auto_now_add=True.

Finally, greeting.put() saves the new object to the data store. If we had acquired this object from a query, put() would have updated the existing object. Because we created this object with the model constructor, put() adds the new object to the data store.

Because querying in Datastore is strongly consistent only within entity groups, the code assigns all of one book's greetings to the same entity group by setting the same parent for each greeting. This means the user always sees a greeting immediately after it is written. However, the rate at which you can write to the same entity group is limited to one write to the entity group per second. When you design a real application you'll need to keep this fact in mind. Note that by using services such as Memcache, you can lower the chance that a user sees stale results when querying across entity groups after a write.

Retrieving submitted greetings

Datastore has a sophisticated query engine for data models. Because Datastore is not a traditional relational database, queries are not specified using SQL. Instead, data is queried one of two ways: either by using Datastore queries, or by using an SQL-like query language called GQL. To access the full range of Datastore's query capabilities, we recommend using queries over GQL.

The MainPage handler retrieves and displays previously submitted greetings. The greetings_query.fetch(10) call performs the query.

More about Datastore indexes

Every query in Datastore is computed from one or more indexes—tables that map ordered property values to entity keys. This is how App Engine is able to serve results quickly regardless of the size of your application's data store. Many queries can be computed from the built-in indexes, but for queries that are more complex, Datastore requires a custom index. Without a custom index, Datastore can't execute these queries efficiently.

For example, the Guestbook application filters by guestbook, and orders by date, using an ancestor query and a sort order. This requires a custom index to be specified in the application's index.yaml file. You can edit this file manually, or you can take care of it automatically by running the queries in the application locally. After the index is defined in index.yaml, deploying the application will also deploy the custom index information.

The definition for the query in index.yaml looks like this:

index.yaml

View on GitHub

indexes:
- kind: Greeting
  ancestor: yes
  properties:
  - name: date
    direction: desc

You can read all about Datastore indexes in the Datastore Indexes page. You can read about the proper specification for index.yaml files in Python Datastore Index Configuration.

< Prev Next >