Using Cloud Datastore

Python |Java |Node.js |Go |Ruby |PHP |.NET

You can use Google Cloud Datastore to store application data from your App Engine app. Cloud Datastore is a database built for automatic scaling, high performance, and ease of application development.

This document describes how to use the Google Cloud Client Library to store and retrieve data using Cloud Datastore in an App Engine app.

Prerequisites and setup

Follow the instructions in "Hello, World!" for Go on App Engine to set up your environment and project, and to understand how Go apps are structured in App Engine. Write down and save your project ID, because you will need it to run the sample application described in this document.

Clone the repository

Download (clone) the sample:

go get -u -d github.com/GoogleCloudPlatform/golang-samples/appengine_flexible/datastore
cd $GOPATH/src/github.com/GoogleCloudPlatform/golang-samples/appengine_flexible/datastore

Edit project configuration and set dependencies

In app.yaml, set GCLOUD_DATASET_ID for your project. This value is your project ID.

appengine_flexible/datastore/app.yaml

View on GitHub

runtime: go
env: flex

automatic_scaling:
  min_num_instances: 1

#[START env_variables]
env_variables:
  GCLOUD_DATASET_ID: your-project-id
#[END env_variables]

Application code

The sample application logs, retrieves, and displays visitor IPs. You can see that a log entry is a simple two-field class that is given the type visit, and is saved to Cloud Datastore using the Datastore Client put command. Then, the ten most recent visits are retrieved in descending order, using the Datastore Client NewQuery and GetAll commands.

appengine_flexible/datastore/datastore.go

View on GitHub

// Copyright 2015 Google Inc. All rights reserved.
// Use of this source code is governed by the Apache 2.0
// license that can be found in the LICENSE file.

// Sample datastore demonstrates use of the cloud.google.com/go/datastore package from App Engine flexible.
package main

import (
	"fmt"
	"log"
	"net/http"
	"os"
	"time"

	"cloud.google.com/go/datastore"

	"golang.org/x/net/context"

	"google.golang.org/appengine"
)

var datastoreClient *datastore.Client

func main() {
	ctx := context.Background()

	// Set this in app.yaml when running in production.
	projectID := os.Getenv("GCLOUD_DATASET_ID")

	var err error
	datastoreClient, err = datastore.NewClient(ctx, projectID)
	if err != nil {
		log.Fatal(err)
	}

	http.HandleFunc("/", handle)
	appengine.Main()
}

func handle(w http.ResponseWriter, r *http.Request) {
	if r.URL.Path != "/" {
		http.NotFound(w, r)
		return
	}

	ctx := context.Background()

	// Get a list of the most recent visits.
	visits, err := queryVisits(ctx, 10)
	if err != nil {
		msg := fmt.Sprintf("Could not get recent visits: %v", err)
		http.Error(w, msg, http.StatusInternalServerError)
		return
	}

	// Record this visit.
	if err := recordVisit(ctx, time.Now(), r.RemoteAddr); err != nil {
		msg := fmt.Sprintf("Could not save visit: %v", err)
		http.Error(w, msg, http.StatusInternalServerError)
		return
	}

	fmt.Fprintln(w, "Previous visits:")
	for _, v := range visits {
		fmt.Fprintf(w, "[%s] %s\n", v.Timestamp, v.UserIP)
	}
	fmt.Fprintln(w, "\nSuccessfully stored an entry of the current request.")
}

type visit struct {
	Timestamp time.Time
	UserIP    string
}

func recordVisit(ctx context.Context, now time.Time, userIP string) error {
	v := &visit{
		Timestamp: now,
		UserIP:    userIP,
	}

	k := datastore.IncompleteKey("Visit", nil)

	_, err := datastoreClient.Put(ctx, k, v)
	return err
}

func queryVisits(ctx context.Context, limit int64) ([]*visit, error) {
	// Print out previous visits.
	q := datastore.NewQuery("Visit").
		Order("-Timestamp").
		Limit(10)

	visits := make([]*visit, 0)
	_, err := datastoreClient.GetAll(ctx, q, &visits)
	return visits, err
}

Using index.yaml files

The sample app performs simple queries. More elaborate Datastore queries require one or more indexes, which you must specify in an index.yaml file that you upload along with your app. This file may be created manually, or generated automatically while testing your app locally.

For more information

For complete information on Cloud Datastore, including optimizations and concepts, see the Cloud Datastore documentation.