Getting Started: Cloud Datastore

Learn how to create an App Engine app that stores data in Google Cloud Datastore, a non-relational (NoSQL) database.

Cloud Datastore is one storage option available with App Engine that can be easily integrated into apps and store text data. Compare Cloud Datastore, Cloud SQL, and Cloud Storage and choose the one that meets your app's requirements.

This sample builds upon a series of guides and shows how to store blog post data in Cloud Datastore.

Before you begin

Configure your development environment and create your App Engine project.

Importing libraries

The sample uses the following libraries:

import com.google.appengine.api.datastore.DatastoreFailureException;
import com.google.appengine.api.datastore.DatastoreService;
import com.google.appengine.api.datastore.DatastoreServiceFactory;
import com.google.appengine.api.datastore.KeyFactory;
import com.google.appengine.api.datastore.Entity;
import com.google.appengine.api.datastore.FetchOptions;
import com.google.appengine.api.datastore.PreparedQuery;
import com.google.appengine.api.datastore.Query;
import com.google.appengine.api.datastore.Query.FilterOperator;
import com.google.appengine.api.datastore.Query.FilterPredicate;

import org.jsoup.Jsoup;
import org.jsoup.safety.Whitelist;

The libraries you use in your own work might be different, depending on which datastore features you are using. Not all tasks require all the libraries listed, and it is good practice to import only required libraries.

Setting up a Cloud Datastore connection

Before reading or writing to Cloud Datastore, you have to create a DatastoreService object and use it to communicate with Cloud Datastore. The following code creates a connection:

DatastoreService datastore;
datastore = DatastoreServiceFactory.getDatastoreService();

DatastoreService is a heavyweight object, so be sure to minimize the number of calls by using batch operations.

Creating and storing entities

Objects are stored in Cloud Datastore as entities, with object data stored in entity properties. This example stores data in an entity called post that has four properties: title, body, author, and timestamp.

The data gets submitted through an HTML form, as documented in Handling user-submitted form data.

DatastoreService datastore;

@Override
public void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException {

  // Create a map of the httpParameters that we want and run it through jSoup
  Map<String, String> blogContent =
      req.getParameterMap()
          .entrySet()
          .stream()
          .filter(a -> a.getKey().startsWith("blogContent_"))
          .collect(
              Collectors.toMap(
                  p -> p.getKey(), p -> Jsoup.clean(p.getValue()[0], Whitelist.basic())));

  Entity post = new Entity("Blogpost"); // create a new entity

  post.setProperty("title", blogContent.get("blogContent_title"));
  post.setProperty("author", blogContent.get("blogContent_author"));
  post.setProperty("body", blogContent.get("blogContent_description"));
  post.setProperty("timestamp", new Date().getTime());

  try {
    datastore.put(post); // store the entity

    // Send the user to the confirmation page with personalised confirmation text
    String confirmation = "Post with title " + blogContent.get("blogContent_title") + " created.";

    req.setAttribute("confirmation", confirmation);
    req.getRequestDispatcher("/confirm.jsp").forward(req, resp);
  } catch (DatastoreFailureException e) {
    throw new ServletException("Datastore error", e);
  }
}

@Override
public void init() throws ServletException {

  // setup datastore service
  datastore = DatastoreServiceFactory.getDatastoreService();
}

In the sample, each post entity has an identifier automatically created by Cloud Datastore. If you want to assign your own identifier instead, specify one when you create the entity:

Entity post = new Entity("Blogpost", "[IDENTIFIER]");

The ability to specify an identifier is important when you are updating an existing entity which is documented in Updating Entities.

After post entity creation, set its title, author, body, and timestamp properties with setProperty(), and store the fully populated entity in Cloud Datastore using put.

datastore.put(post);

Reading entities

To read entities from Cloud Datastore, you must use a query. The sample uses a query that displays five blog posts on the front page:

final Query q =
    new Query("Blogpost").setFilter(new FilterPredicate("title", FilterOperator.NOT_EQUAL, ""));

PreparedQuery pq = datastore.prepare(q);
List<Entity> posts = pq.asList(FetchOptions.Builder.withLimit(5)); // Retrieve up to five posts

posts.forEach(
    (result) -> {
      // Grab the key and convert it into a string in preparation for encoding
      String keyString = KeyFactory.keyToString(result.getKey());

      // Encode the entity's key with Base64
      String encodedID = new String(Base64.getUrlEncoder().encodeToString(String.valueOf(keyString).getBytes()));

      // Build up string with values from the Datastore entity
      String recordOutput =
          String.format(blogPostDisplayFormat, result.getProperty("title"), result.getProperty("timestamp"),
              result.getProperty("author"), encodedID, encodedID, result.getProperty("body"));

      out.println(recordOutput); // Print out HTML
    });

The Cloud Datastore query object supports filtering using setFilter, which is similar the SQL WHERE clause. This example uses the FetchOptions.Builder class to fetch five blog posts.

To see an example that displays an unlimited number of items in batches using Cursors, refer to the Bookshelf tutorial.

Updating entities

To update an entity, create a new entity with the same key as the original entity. This example creates a new post entity that has the same key as the original, but populates it with updated text before storing it.

// Decode the websafe ID
String decodedKey = new String(Base64.getUrlDecoder().decode(blogContent.get("blogContent_id")));

// Create a key from the decoded websafe string
Key originalPostKey = KeyFactory.stringToKey(decodedKey);

// Create a new entity with the same key as the original
Entity post = new Entity("Blogpost", originalPostKey.getId());

// Populate the new entity with the updated blog post contents
post.setProperty("title", blogContent.get("blogContent_title"));
post.setProperty("author", blogContent.get("blogContent_author"));
post.setProperty("body", blogContent.get("blogContent_body"));
post.setProperty("timestamp", new Date());

try {
  datastore.put(post); // Store the post

  // Send the user to the confirmation page with personalised confirmation text
  String confirmation = "Post with title " + blogContent.get("blogContent_title") + " updated.";

  req.setAttribute("confirmation", confirmation);
  req.getRequestDispatcher("/confirm.jsp").forward(req, resp);
} catch (DatastoreFailureException e) {
  throw new ServletException("Datastore error", e);
}

Deleting entities

To delete entities in Cloud Datastore, call the delete method passing the Key of the post to be deleted.

// Get the websafe cursor and then convert it into a usable key
// for retrieving a particular post
Map<String, String[]> blogContent = req.getParameterMap();
String[] postId = blogContent.get("id");
String decodedKey = new String(Base64.getUrlDecoder().decode(postId[0])); // Decode the websafe ID

try {
  try {
    Entity deletePost = datastore.get(KeyFactory.stringToKey(decodedKey));
    // Delete the entity based on its key
    datastore.delete(deletePost.getKey());

    // Send the user to the confirmation page with personalised confirmation text
    String confirmation = "Post " + deletePost.getProperty("title") + " has been deleted.";

    req.setAttribute("confirmation", confirmation);
    req.getRequestDispatcher("/confirm.jsp").forward(req, resp);

  } catch (EntityNotFoundException e) {
    throw new ServletException("Datastore error", e);
  }
} catch (DatastoreFailureException e) {
  throw new ServletException("Datastore error", e);
}

The encoded original key ID in this example gets passed through the URL to the sample app, where it gets decoded and used as the key value in a call to createKey from the KeyFactory class. createKey requires an entity type, a Blogpost, and an identifier that is retrieved from the servlet request variables and converted to a Long integer.

Deploying to App Engine

You can deploy your app to App Engine using Maven.

Go to the root directory of your project and type:

mvn appengine:deploy

After Maven deploys your app, open a web browser tab automatically at your new app by typing:

gcloud app browse

What's next

Cloud Datastore is useful for storing text-based data; however if you want to store images you should consider using Cloud Storage.

Using Cloud Storage

Send feedback about...

App Engine standard environment for Java