Getting started with the Google Cloud Datastore API

This page provides a short exercise in building a simple command-line TaskList application with the Google Cloud Datastore API. The TaskList application stores, lists, updates, and removes tasks.

Prerequisites

  1. Ability to write and run a command line application in the programming languages used in this topic
    In addition to a basic understand of how to develop applications, you should be able to download and install additional libraries before attempting this tutorial.
  2. A Google Cloud Platform Console project with the Cloud Datastore API enabled
    Applications that use Cloud Datastore must be associated with a Google Cloud Platform Console project with the Cloud Datastore API enabled. This project provides authentication credentials you use in your application to identify it to Google and authorize its use of the Cloud Datastore API.
    Follow these instructions to create a project, enable the Cloud Datastore API for it, and set up your local development environment with authentication credentials using the gcloud auth login command. Make a note of the project's ID, which you'll use later on.

Installation and setup

Install client libraries and configure any additional settings for your development environment.

Java

  1. Ensure you have Maven and Java (version 7 or later) installed.

  2. Download the TaskList sample application from here.

  3. At a command prompt, unzip the download:

    unzip java-docs-samples-master.zip
    
  4. Change directories to the TaskList application:

    cd java-docs-samples-master/datastore
    
  5. Run the following, where <project-id> is the ID of your Google Cloud Platform project.

    gcloud config set project <project-id>
    
  6. Compile and run the application!

    mvn clean compile
    mvn exec:java
    

Node.js

  1. Ensure you have Node.js and npm installed.

  2. Download the TaskList sample application from here.

  3. Unzip the download:

    unzip nodejs-docs-samples-master.zip
    
  4. Change directories to the TaskList application:

    cd nodejs-docs-samples-master/datastore
    
  5. Install the dependencies and link the application:

    npm install
    
  6. At a command prompt, run the following, where <project-id> is the ID of your Google Cloud Platform project.

    export GCLOUD_PROJECT=<project-id>
    
  7. Run the application!

    node tasks.js
    

Go

  1. Install the Go client library for the Cloud Datastore API.

    go get google.golang.org/cloud/datastore
    
  2. Clone the TaskList sample application.

    git clone https://github.com/GoogleCloudPlatform/golang-samples.git
    
  3. Change directories to where you cloned the sample:

    cd golang-samples/datastore/tasks
    
  4. At a command prompt, run the following, where <project-id> is the ID of your Google Cloud Platform project.

    export DATASTORE_PROJECT_ID=<project-id>
    

    (Windows users: use set instead of export.)

  5. Run the application!

    go run tasks.go
    

Python

  1. Ensure you have Python (version 2.7.9 or later), pip, and virtualenv installed.
  2. Activate a virtualenv session.

    virtualenv venv
    source venv/bin/activate
    
  3. Download the TaskList sample application from here.

  4. Unzip the download:

    unzip python-docs-samples-master.zip
    
  5. Change directories to the TaskList application:

    cd python-docs-samples-master/datastore/api
    
  6. Install dependencies:

    pip install -r requirements.txt
    
  7. Run the application! Use the ID of your Google Cloud Platform project for <project-id>.

    python tasks.py --project-id <project-id>
    

Creating an Authorized Service Object

In order to make authenticated requests to Google Cloud APIs using the Google APIs Client libraries, you must:

  • Fetch the credential to use for requests.
  • Create a service object that uses that credential.

You can then make API calls by calling methods on the Cloud Datastore service object.

For this example, you'll fetch Application Default Credentials from the environment, and pass it as an argument to create the service object.

Here's the call to create the authorized Cloud Datastore service object:

Java

// Create an authorized Datastore service using Application Default Credentials.
private final Datastore datastore = DatastoreOptions.defaultInstance().service();

// Create a Key factory to construct keys associated with this project.
private final KeyFactory keyFactory = datastore.newKeyFactory().kind("Task");

Node.js

// By default, gcloud will authenticate using the service account file specified
// by the GOOGLE_APPLICATION_CREDENTIALS environment variable and use the
// project specified by the GCLOUD_PROJECT environment variable. See
// https://googlecloudplatform.github.io/gcloud-node/#/docs/guides/authentication
var gcloud = require('gcloud');
var datastore = gcloud.datastore();

Go

ctx := context.Background()
client, err := datastore.NewClient(ctx, projID)

Python

from gcloud import datastore


def create_client(project_id):
    return datastore.Client(project_id)

Storing data

Objects in Cloud Datastore are known as entities, and each entity is of a particular kind. The TaskList application will store entities of kind Task, with the following properties:

  • description: a string specified by the user as the task description
  • created: a date that shows when the task was initially created
  • done: a boolean that indicates whether the task has been completed

When the user adds a new task, the TaskList application will create a Task entity with values for the properties listed above:

Java

/**
 * Adds a task entity to the Datastore.
 *
 * @param description The task description
 * @return The {@link Key} of the entity
 * @throws DatastoreException if the ID allocation or put fails
 */
Key addTask(String description) {
  Key key = datastore.allocateId(keyFactory.newKey());
  Entity task = Entity.builder(key)
      .set("description", StringValue.builder(description).excludeFromIndexes(true).build())
      .set("created", DateTime.now())
      .set("done", false)
      .build();
  datastore.put(task);
  return key;
}

Node.js

function addTask (description, callback) {
  var taskKey = datastore.key('Task');

  datastore.save({
    key: taskKey,
    data:  [
      {
        name: 'created',
        value: new Date().toJSON()
      },
      {
        name: 'description',
        value: description,
        excludeFromIndexes: true
      },
      {
        name: 'done',
        value: false
      }
    ]
  }, function (err) {
    if (err) {
      callback(err);
      return;
    }

    callback(null, taskKey);
  });
}

Go

// Task is the model used to store tasks in the datastore.
type Task struct {
	Desc    string    `datastore:"description"`
	Created time.Time `datastore:"created"`
	Done    bool      `datastore:"done"`
	id      int64     // The integer ID used in the datastore.
}

// AddTask adds a task with the given description to the datastore,
// returning the key of the newly created entity.
func AddTask(ctx context.Context, client *datastore.Client, desc string) (*datastore.Key, error) {
	task := &Task{
		Desc:    desc,
		Created: time.Now(),
	}
	key := datastore.NewIncompleteKey(ctx, "Task", nil)
	return client.Put(ctx, key, task)
}

Python

def add_task(client, description):
    key = client.key('Task')

    task = datastore.Entity(
        key, exclude_from_indexes=['description'])

    task.update({
        'created': datetime.datetime.utcnow(),
        'description': description,
        'done': False
    })

    client.put(task)

    return task.key

For this application, we'll also provide a method to update the done property, to indicate the task is complete:

Java

/**
 * Marks a task entity as done.
 *
 * @param id The ID of the task entity as given by {@link Key#id()}
 * @return true if the task was found, false if not
 * @throws DatastoreException if the transaction fails
 */
boolean markDone(long id) {
  Transaction transaction = datastore.newTransaction();
  try {
    Entity task = transaction.get(keyFactory.newKey(id));
    if (task != null) {
      transaction.put(Entity.builder(task).set("done", true).build());
    }
    transaction.commit();
    return task != null;
  } finally {
    if (transaction.active()) {
      transaction.rollback();
    }
  }
}

Node.js

function markDone (taskId, callback) {
  var error;

  datastore.runInTransaction(function (transaction, done) {
    var taskKey = datastore.key([
      'Task',
      taskId
    ]);

    transaction.get(taskKey, function (err, task) {
      if (err) {
        // An error occurred while getting the values.
        error = err;
        transaction.rollback(done);
        return;
      }

      task.data.done = true;

      transaction.save(task);

      // Commit the transaction.
      done();
    });
  }, function (transactionError) {
    if (transactionError || error) {
      return callback(transactionError || error);
    }
    // The transaction completed successfully.
    callback();
  });
}

Go

// MarkDone marks the task done with the given ID.
func MarkDone(ctx context.Context, client *datastore.Client, taskID int64) error {
	// Create a key using the given integer ID.
	key := datastore.NewKey(ctx, "Task", "", taskID, nil)

	// In a transaction load each task, set done to true and store.
	_, err := client.RunInTransaction(ctx, func(tx *datastore.Transaction) error {
		var task Task
		if err := tx.Get(key, &task); err != nil {
			return err
		}
		task.Done = true
		_, err := tx.Put(key, &task)
		return err
	})
	return err
}

Python

def mark_done(client, task_id):
    with client.transaction():
        key = client.key('Task', task_id)
        task = client.get(key)

        if not task:
            raise ValueError(
                'Task {} does not exist.'.format(task_id))

        task['done'] = True

        client.put(task)

Here's the method to delete a Task entity, using the Task entity's key:

Java

/**
 * Deletes a task entity.
 *
 * @param id The ID of the task entity as given by {@link Key#id()}
 * @throws DatastoreException if the delete fails
 */
void deleteTask(long id) {
  datastore.delete(keyFactory.newKey(id));
}

Node.js

function deleteTask (taskId, callback) {
  var taskKey = datastore.key([
    'Task',
    taskId
  ]);

  datastore.delete(taskKey, callback);
}

Go

// DeleteTask deletes the task with the given ID.
func DeleteTask(ctx context.Context, client *datastore.Client, taskID int64) error {
	return client.Delete(ctx, datastore.NewKey(ctx, "Task", "", taskID, nil))
}

Python

def delete_task(client, task_id):
    key = client.key('Task', task_id)
    client.delete(key)

Running a query

In addition to retrieving entities from Cloud Datastore directly by their keys, an application can perform a query to retrieve them by the values of their properties. A typical query includes the following:

  • An entity kind to which the query applies
  • Zero or more filters, for example to select kinds whose properties match a value
  • Zero or more sort orders, to sequence the results

For this application, we'll query Cloud Datastore for Task entities sorted by creation time:

Java

/**
 * Returns a list of all task entities in ascending order of creation time.
 *
 * @throws DatastoreException if the query fails
 */
Iterator<Entity> listTasks() {
  Query<Entity> query =
      Query.entityQueryBuilder().kind("Task").orderBy(OrderBy.asc("created")).build();
  return datastore.run(query);
}

Node.js

function listTasks (callback) {
  var query = datastore.createQuery('Task')
    .order('created');

  datastore.runQuery(query, callback);
}

Go

// ListTasks returns all the tasks in ascending order of creation time.
func ListTasks(ctx context.Context, client *datastore.Client) ([]*Task, error) {
	var tasks []*Task

	// Create a query to fetch all Task entities, ordered by "created".
	query := datastore.NewQuery("Task").Order("created")
	keys, err := client.GetAll(ctx, query, &tasks)
	if err != nil {
		return nil, err
	}

	// Set the id field on each Task from the corresponding key.
	for i, key := range keys {
		tasks[i].id = key.ID()
	}

	return tasks, nil
}

Python

def list_tasks(client):
    query = client.query(kind='Task')
    query.order = ['created']

    return list(query.fetch())

Next Steps

This tutorial covers only the most basic steps necessary to make calls to the Cloud Datastore API from a command-line application. Cloud Datastore supports fast and highly scalable ACID transactions, SQL-like queries, indexes and more.

Send feedback about...

Cloud Datastore