Storing Data in Cloud Datastore

This part of the Python Guestbook code walkthrough shows how to store structured data in Google Cloud Datastore. With App Engine and Cloud Datastore, you don't have to worry about distribution, replication, and load balancing of data. That is done for you behind a simple API—and you get a powerful query engine and transactions as well.

This page is part of a multi-page tutorial. To start from the beginning and see instructions for setting up, go to Creating a Guestbook.

Storing the submitted greetings

Data is written to Cloud Datastore in objects known as entities. Each entity has a key that uniquely identifies it. An entity can optionally designate another entity as its parent; the first entity is a child of the parent entity. The entities in the data store thus form a hierarchically-structured space similar to the directory structure of a file system. For detailed information, see Structuring Data for Strong Consistency.

App Engine includes a data modeling API for Python. To use the data modeling API, the sample app imports the google.appengine.ext.ndb module. Each greeting includes the author's name, the message content, and the date and time the message was posted. The app displays messages in chronological order. The following code defines the data model:

The code defines a Greeting model with three properties: author whose value is an Author object with the email address and the author's identity, content whose value is a string, and date whose value is a datetime.datetime.

Some property constructors take parameters to further configure their behavior. Passing the ndb.StringProperty constructor the indexed=False parameter says that values for this property will not be indexed. This saves writes which aren't needed because the app never uses that property in a query. Passing the ndb.DateTimeProperty constructor an auto_now_add=True parameter configures the model to automatically give new objects a datetime stamp of the time the object is created, if the application doesn't otherwise provide a value. For a complete list of property types and their options, see NDB Properties.

The application uses the data model to create new Greeting objects and put them into Cloud Datastore. The Guestbook handler creates new greetings and saves them to the data store:

This Guestbook handler creates a new Greeting object, then sets its author and content properties with the data posted by the user. The parent of Greeting is a Guestbook entity. There's no need to create the Guestbook entity before setting it to be the parent of another entity. In this example, the parent is used as a placeholder for transaction and consistency purposes. See the Transactions page for more information. Objects that share a common ancestor belong to the same entity group. The code does not set the date property, so date is automatically set to the present, using auto_now_add=True.

Finally, greeting.put() saves the new object to the data store. If we had acquired this object from a query, put() would have updated the existing object. Because we created this object with the model constructor, put() adds the new object to the data store.

Because querying in Cloud Datastore is strongly consistent only within entity groups, the code assigns all of one book's greetings to the same entity group by setting the same parent for each greeting. This means the user always sees a greeting immediately after it is written. However, the rate at which you can write to the same entity group is limited to one write to the entity group per second. When you design a real application you'll need to keep this fact in mind. Note that by using services such as Memcache, you can lower the chance that a user sees stale results when querying across entity groups after a write.

Retrieving submitted greetings

Cloud Datastore has a sophisticated query engine for data models. Because Cloud Datastore is not a traditional relational database, queries are not specified using SQL. Instead, data is queried one of two ways: either by using Datastore queries, or by using an SQL-like query language called GQL. To access the full range of Cloud Datastore's query capabilities, we recommend using queries over GQL.

The MainPage handler retrieves and displays previously submitted greetings. The greetings_query.fetch(10) call performs the query.

More about Cloud Datastore indexes

Every query in Cloud Datastore is computed from one or more indexes—tables that map ordered property values to entity keys. This is how App Engine is able to serve results quickly regardless of the size of your application's data store. Many queries can be computed from the built-in indexes, but for queries that are more complex, Cloud Datastore requires a custom index. Without a custom index, Cloud Datastore can't execute these queries efficiently.

For example, the Guestbook application filters by guestbook, and orders by date, using an ancestor query and a sort order. This requires a custom index to be specified in the application's index.yaml file. You can edit this file manually, or you can take care of it automatically by running the queries in the application locally. After the index is defined in index.yaml, deploying the application will also deploy the custom index information.

The definition for the query in index.yaml looks like this:

You can read all about Cloud Datastore indexes in the Datastore Indexes page. You can read about the proper specification for index.yaml files in Python Datastore Index Configuration.

Send feedback about...