Cloud Datastore Overview

Cloud Datastore is a NoSQL document database built for automatic scaling, high performance, and ease of application development. Cloud Datastore features include:

  • Atomic transactions. Cloud Datastore can execute a set of operations where either all succeed, or none occur.
  • High availability of reads and writes. Cloud Datastore runs in Google data centers, which use redundancy to minimize impact from points of failure.
  • Massive scalability with high performance. Cloud Datastore uses a distributed architecture to automatically manage scaling. Cloud Datastore uses a mix of indexes and query constraints so your queries scale with the size of your result set, not the size of your data set.
  • Flexible storage and querying of data. Cloud Datastore maps naturally to object-oriented and scripting languages, and is exposed to applications through multiple clients. It also provides a SQL-like query language.
  • Balance of strong and eventual consistency. Cloud Datastore ensures that entity lookups by key and ancestor queries always receive strongly consistent data. All other queries are eventually consistent. The consistency models allow your application to deliver a great user experience while handling large amounts of data and users.

  • Encryption at rest. Cloud Datastore automatically encrypts all data before it is written to disk and automatically decrypts the data when read by an authorized user. For more information, see Server-Side Encryption.

  • Fully managed with no planned downtime. Google handles the administration of the Cloud Datastore service so you can focus on your application. Your application can still use Cloud Datastore when the service receives a planned upgrade.

Cloud Firestore in Datastore mode

Cloud Firestore is the newest version of Cloud Datastore and introduces several improvements over Cloud Datastore. Existing Cloud Datastore users can access these improvements by creating a new Cloud Firestore in Datastore mode database instance. In the future, all existing Cloud Datastore databases will be automatically upgraded to Cloud Firestore in Datastore mode.

Comparison with traditional databases

While the Cloud Datastore interface has many of the same features as traditional databases, as a NoSQL database it differs from them in the way it describes relationships between data objects. Here’s a high-level comparison of Cloud Datastore and relational database concepts:

Concept Cloud Datastore Cloud Firestore Relational database
Category of object Kind Collection group Table
One object Entity Document Row
Individual data for an object Property Field Column
Unique ID for an object Key Document ID Primary key

Unlike rows in a relational database table, Cloud Datastore entities of the same kind can have different properties, and different entities can have properties with the same name but different value types. These unique characteristics imply a different way of designing and managing data to take advantage of the ability to scale automatically. In particular, Cloud Datastore differs from a traditional relational database in the following important ways:

  • Cloud Datastore is designed to automatically scale to very large data sets, allowing applications to maintain high performance as they receive more traffic:
    • Cloud Datastore writes scale by automatically distributing data as necessary.
    • Cloud Datastore reads scale because the only queries supported are those whose performance scales with the size of the result set (as opposed to the data set). This means that a query whose result set contains 100 entities performs the same whether it searches over a hundred entities or a million. This property is the key reason some types of queries are not supported.
  • Because all queries are served by previously built indexes, the types of queries that can be executed are more restrictive than those allowed on a relational database with SQL. In particular, Cloud Datastore does not include support for join operations, inequality filtering on multiple properties, or filtering on data based on results of a subquery.
  • Unlike traditional relational databases which enforce a schema, Cloud Datastore is schemaless. It doesn't require entities of the same kind to have a consistent set of properties (although you can choose to enforce such a requirement in your own application code).

What it's good for

Cloud Datastore is ideal for applications that rely on highly available structured data at scale. You can use Cloud Datastore to store and query all of the following types of data:

  • Product catalogs that provide real-time inventory and product details for a retailer.
  • User profiles that deliver a customized experience based on the user’s past activities and preferences.
  • Transactions based on ACID properties, for example, transferring funds from one bank account to another.

Other storage and database options

Cloud Datastore is not ideal for every use case. For example, Cloud Datastore is not a relational database, and it is not an effective solution for analytic data.

Here are some common scenarios where you should probably consider an alternative to Cloud Datastore:

  • If you need a relational database with full SQL support for an online transaction processing (OLTP) system, consider Cloud SQL.
  • If you don’t require support for ACID transactions or if your data is not highly structured, consider Cloud Bigtable.
  • If you need interactive querying in an online analytical processing (OLAP) system, consider BigQuery.
  • If you need to store large immutable blobs, such as large images or movies, consider Cloud Storage.

For more information about other database options, see the overview of database services.

Connecting to Cloud Datastore with App Engine

App Engine's Go standard runtime connects to Cloud Datastore using the Go Datastore API. For a complete list of the contents of the datastore package, see the datastore package reference.

You cannot use the Cloud Datastore client library with Go applications in the App Engine standard environment.

Was this page helpful? Let us know how we did:

Send feedback about...

App Engine standard environment for Go