Using Cloud Datastore with Python

This page of the Bookshelf tutorial shows how the sample app stores its persistent data in Google Cloud Datastore. The sample code for this step provides examples of how to create, read, update, and delete (CRUD) data stored in Cloud Datastore.

This page is part of a multi-page tutorial. To start from the beginning and see instructions for setting up, go to Python Bookshelf App.

Configuring settings

This section uses code in the 2-structured-data directory. Edit the files and run commands in this directory.

  1. Open config.py for editing.
  2. Set the value of PROJECT_ID to your project ID, which is visible in the GCP Console.
  3. Set the value of DATA_BACKEND to datastore.
  4. Save and close config.py.
Cloud Datastore is a fully managed service that is automatically initialized and connected to your App Engine app. No further configuration is required.

Installing dependencies

Enter these commands to create a virtual environment and install dependencies:

Linux/macOS

virtualenv -p python3 env
source env/bin/activate
pip install -r requirements.txt

Windows

virtualenv -p python3 env
env\scripts\activate
pip install -r requirements.txt

Running the app on your local machine:

  1. Start a local web server:

    python main.py
    
  2. In your web browser, enter this address:

    http://localhost:8080

Now you can browse the app's web pages and add, edit, and delete books.

Press Control+C to exit the local web server.

Deploying the app to the App Engine flexible environment

  1. Deploy the sample app:

    gcloud app deploy
    
  2. In your web browser, enter this address. Replace [YOUR_PROJECT_ID] with your project ID:

    https://[YOUR_PROJECT_ID].appspot.com
    

If you update your app, you can deploy the updated version by entering the same command you used to deploy the app the first time. The new deployment creates a new version of your app and promotes it to the default version. The older versions of your app remain, as do their associated VM instances. Be aware that all of these app versions and VM instances are billable resources.

You can reduce costs by deleting the non-default versions of your app.

To delete an app version:

  1. In the GCP Console, go to the App Engine Versions page.

    Go to the Versions page

  2. Click the checkbox next to the non-default app version you want to delete.
  3. Click the Delete button at the top of the page to delete the app version.

For complete information about cleaning up billable resources, see the Cleaning up section in the final step of this tutorial.

Application structure

Bookshelf app deployment process and structure

The application stores all persistent data in Cloud Datastore.

Understanding the code

This section walks you through the application code and explains how it works.

Handling user submissions with forms

The add/edit HTML form allows users to add and edit book submissions within the app.

Image of add/edit Form

The HTML form is created using Flask's default template engine, Jinja2. The following template defines a simple form that includes input fields for Title, Author, Date Published, and Description:

{% extends "base.html" %}

{% block content %}
<h3>{{action}} book</h3>

<form method="POST" enctype="multipart/form-data">

  <div class="form-group">
    <label for="title">Title</label>
    <input type="text" name="title" id="title" value="{{book.title}}" class="form-control"/>
  </div>

  <div class="form-group">
    <label for="author">Author</label>
    <input type="text" name="author" id="author" value="{{book.author}}" class="form-control"/>
  </div>

  <div class="form-group">
    <label for="publishedDate">Date Published</label>
    <input type="text" name="publishedDate" id="publishedDate" value="{{book.publishedDate}}" class="form-control"/>
  </div>

  <div class="form-group">
    <label for="description">Description</label>
    <textarea name="description" id="description" class="form-control">{{book.description}}</textarea>
  </div>

  <button type="submit" class="btn btn-success">Save</button>
</form>

{% endblock %}

Handling form submissions

When a user clicks Add Book, the crud.add view displays the form. When a user fills out the Add book form and then clicks Save, the same view handles the form's HTTP POST action. This initiates the process of sending the submitted data to Cloud Datastore by passing the data to the get_model().create function.

@crud.route('/add', methods=['GET', 'POST'])
def add():
    if request.method == 'POST':
        data = request.form.to_dict(flat=True)

        book = get_model().create(data)

        return redirect(url_for('.view', id=book['id']))

    return render_template("form.html", action="Add", book={})

The bookshelf/model_datastore.py file contains the code that performs CRUD functions for data stored in Cloud Datastore. For example the get_model().create statement calls the create function in bookshelf/model_datastore.py, which sends the user's submitted data to the update function. A null value is passed in the first parameter, which indicates that this is a new book submission. Here is the update function that does the actual work of saving the user's submitted data to Cloud Datastore.

def update(data, id=None):
    ds = get_client()
    if id:
        key = ds.key('Book', int(id))
    else:
        key = ds.key('Book')

    entity = datastore.Entity(
        key=key,
        exclude_from_indexes=['description'])

    entity.update(data)
    ds.put(entity)
    return from_datastore(entity)


create = update

The get_client helper function creates a client to interact with the Cloud Datastore API:

def get_client():
    return datastore.Client(current_app.config['PROJECT_ID'])

The from_datastore helper function is used to translate the datastore's entity key into an id for use by the application:

def from_datastore(entity):
    """Translates Datastore results into the format expected by the
    application.

    Datastore typically returns:
        [Entity{key: (kind, id), prop: val, ...}]

    This returns:
        {id: id, prop: val, ...}
    """
    if not entity:
        return None
    if isinstance(entity, builtin_list):
        entity = entity.pop()

    entity['id'] = entity.key.id
    return entity

After users have added books, clicking the Books link navigates to the /books page, which list all the books currently stored in Cloud Datastore. The model_datastore.list function does the work of listing all the books using data retrieved from Cloud Datastore.

def list(limit=10, cursor=None):
    ds = get_client()

    query = ds.query(kind='Book', order=['title'])
    query_iterator = query.fetch(limit=limit, start_cursor=cursor)
    page = next(query_iterator.pages)

    entities = builtin_list(map(from_datastore, page))
    next_cursor = (
        query_iterator.next_page_token.decode('utf-8')
        if query_iterator.next_page_token else None)

    return entities, next_cursor

This code queries the datastore using datastore.Query and gets all Book entities and returns them ordered by title. You can also apply filters to queries. See the Datastore documentation for more details.

We use query.fetch to get an iterator the query requests. We get results one page at a time and return a cursor to allow the user to load the next page of results.

Was this page helpful? Let us know how we did:

Send feedback about...