Using Cloud Datastore

Python |Java |Node.js |Go |Ruby |PHP |.NET

You can use Google Cloud Datastore to store application data from your App Engine app. Cloud Datastore is a database built for automatic scaling, high performance, and ease of application development.

This document describes how to use the Google Cloud Client Library to store and retrieve data using Cloud Datastore in an App Engine app.

Prerequisites and setup

Follow the instructions in "Hello, World!" for Go on App Engine to set up your environment and project, and to understand how Go apps are structured in App Engine. Write down and save your project ID, because you will need it to run the sample application described in this document.

Clone the repository

Download (clone) the sample:

go get -u -d -v github.com/GoogleCloudPlatform/golang-samples/datastore
cd $GOPATH/src/github.com/GoogleCloudPlatform/golang-samples/appengine_flexible/datastore

Edit project configuration and set dependencies

In app.yaml, set GCLOUD_DATASET_ID for your project. This value is your project ID.

appengine_flexible/datastore/app.yaml

View on GitHub

# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

runtime: go
env: flex

automatic_scaling:
  min_num_instances: 1

env_variables:
  GCLOUD_DATASET_ID: your-project-id

Application code

The sample application logs, retrieves, and displays visitor IPs. You can see that a log entry is a simple two-field class that is given the type visit, and is saved to Cloud Datastore using the Datastore Client put command. Then, the ten most recent visits are retrieved in descending order, using the Datastore Client NewQuery and GetAll commands.

appengine_flexible/datastore/datastore.go

View on GitHub

// Sample datastore demonstrates use of the cloud.google.com/go/datastore package from App Engine flexible.
package main

import (
	"context"
	"fmt"
	"log"
	"net/http"
	"os"
	"time"

	"cloud.google.com/go/datastore"
	"google.golang.org/appengine"
)

var datastoreClient *datastore.Client

func main() {
	ctx := context.Background()

	// Set this in app.yaml when running in production.
	projectID := os.Getenv("GCLOUD_DATASET_ID")

	var err error
	datastoreClient, err = datastore.NewClient(ctx, projectID)
	if err != nil {
		log.Fatal(err)
	}

	http.HandleFunc("/", handle)
	appengine.Main()
}

func handle(w http.ResponseWriter, r *http.Request) {
	if r.URL.Path != "/" {
		http.NotFound(w, r)
		return
	}

	ctx := context.Background()

	// Get a list of the most recent visits.
	visits, err := queryVisits(ctx, 10)
	if err != nil {
		msg := fmt.Sprintf("Could not get recent visits: %v", err)
		http.Error(w, msg, http.StatusInternalServerError)
		return
	}

	// Record this visit.
	if err := recordVisit(ctx, time.Now(), r.RemoteAddr); err != nil {
		msg := fmt.Sprintf("Could not save visit: %v", err)
		http.Error(w, msg, http.StatusInternalServerError)
		return
	}

	fmt.Fprintln(w, "Previous visits:")
	for _, v := range visits {
		fmt.Fprintf(w, "[%s] %s\n", v.Timestamp, v.UserIP)
	}
	fmt.Fprintln(w, "\nSuccessfully stored an entry of the current request.")
}

type visit struct {
	Timestamp time.Time
	UserIP    string
}

func recordVisit(ctx context.Context, now time.Time, userIP string) error {
	v := &visit{
		Timestamp: now,
		UserIP:    userIP,
	}

	k := datastore.IncompleteKey("Visit", nil)

	_, err := datastoreClient.Put(ctx, k, v)
	return err
}

func queryVisits(ctx context.Context, limit int64) ([]*visit, error) {
	// Print out previous visits.
	q := datastore.NewQuery("Visit").
		Order("-Timestamp").
		Limit(10)

	visits := make([]*visit, 0)
	_, err := datastoreClient.GetAll(ctx, q, &visits)
	return visits, err
}

Using index.yaml files

The sample app performs simple queries. More elaborate Datastore queries require one or more indexes, which you must specify in an index.yaml file that you upload along with your app. This file may be created manually, or generated automatically while testing your app locally.

Local testing

If you need to develop and test your application locally, you can use the Cloud Datastore emulator.

For more information

For complete information on Cloud Datastore, including optimizations and concepts, see the Cloud Datastore documentation.