Entities, Properties, and Keys

Data objects in Cloud Datastore are known as entities. An entity has one or more named properties, each of which can have one or more values. Entities of the same kind do to not need have the same properties, and an entity's values for a given property do not all need to be of the same data type. (If necessary, an application can establish and enforce such restrictions in its own data model.)

Cloud Datastore supports a variety of data types for property values. These include, among others:

  • Integers
  • Floating-point numbers
  • Strings
  • Dates
  • Binary data

For a full list of types, see Properties and value types.

Each entity in Cloud Datastore has a key that uniquely identifies it. The key consists of the following components:

  • The namespace of the entity, which allows for multitenancy
  • The kind of the entity, which categorizes it for the purpose of Cloud Datastore queries
  • An identifier for the individual entity, which can be either
    • a key name string
    • an integer numeric ID
  • An optional ancestor path locating the entity within the Cloud Datastore hierarchy

An application can fetch an individual entity from Cloud Datastore using the entity's key, or it can retrieve one or more entities by issuing a query based on the entities' keys or property values.

The Java App Engine SDK includes a simple API, provided in the package com.google.appengine.api.datastore, that supports the features of Cloud Datastore directly. All of the examples in this document are based on this low-level API; you can choose to use it either directly in your application or as a basis on which to build your own data management layer.

Cloud Datastore itself does not enforce any restrictions on the structure of entities, such as whether a given property has a value of a particular type; this task is left to the application.

Kinds and identifiers

Each Cloud Datastore entity is of a particular kind, which categorizes the entity for the purpose of queries: for instance, a human resources application might represent each employee at a company with an entity of kind Employee. In the Java Datastore API, you specify an entity's kind when you create it, as an argument to the Entity() constructor. All kind names that begin with two underscores (__) are reserved and may not be used.

The following example creates an entity of kind Employee, populates its property values, and saves it to Datastore:

DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();

Entity employee = new Entity("Employee", "asalieri");
employee.setProperty("firstName", "Antonio");
employee.setProperty("lastName", "Salieri");
employee.setProperty("hireDate", new Date());
employee.setProperty("attendedHrTraining", true);

datastore.put(employee);

In addition to a kind, each entity has an identifier, assigned when the entity is created. Because it is part of the entity's key, the identifier is associated permanently with the entity and cannot be changed. It can be assigned in either of two ways:

  • Your application can specify its own key name string for the entity.
  • You can have Cloud Datastore automatically assign the entity an integer numeric ID.

To assign an entity a key name, provide the name as the second argument to the constructor when you create the entity:

Entity employee = new Entity("Employee", "asalieri");

To have Cloud Datastore assign a numeric ID automatically, omit this argument:

Entity employee = new Entity("Employee");

Assigning identifiers

Cloud Datastore can be configured to generate auto IDs using two different auto id policies:

  • The default policy generates a random sequence of unused IDs that are approximately uniformly distributed. Each ID can be up to 16 decimal digits long.
  • The legacy policy creates a sequence of non-consecutive smaller integer IDs.

If you want to display the entity IDs to the user, and/or depend upon their order, the best thing to do is use manual allocation.

Cloud Datastore generates a random sequence of unused IDs that are approximately uniformly distributed. Each ID can be up to 16 decimal digits long.

System-allocated ID values are guaranteed unique to the entity group. If you copy an entity from one entity group or namespace to another and wish to preserve the ID part of the key, be sure to allocate the ID first to prevent Cloud Datastore from selecting that ID for a future assignment.

Ancestor paths

Entities in Cloud Datastore form a hierarchically structured space similar to the directory structure of a file system. When you create an entity, you can optionally designate another entity as its parent; the new entity is a child of the parent entity (note that unlike in a file system, the parent entity need not actually exist). An entity without a parent is a root entity. The association between an entity and its parent is permanent, and cannot be changed once the entity is created. Cloud Datastore will never assign the same numeric ID to two entities with the same parent, or to two root entities (those without a parent).

An entity's parent, parent's parent, and so on recursively, are its ancestors; its children, children's children, and so on, are its descendants. A root entity and all of its descendants belong to the same entity group. The sequence of entities beginning with a root entity and proceeding from parent to child, leading to a given entity, constitute that entity's ancestor path. The complete key identifying the entity consists of a sequence of kind-identifier pairs specifying its ancestor path and terminating with those of the entity itself:

[Person:GreatGrandpa, Person:Grandpa, Person:Dad, Person:Me]

For a root entity, the ancestor path is empty and the key consists solely of the entity's own kind and identifier:

[Person:GreatGrandpa]

This concept is illustrated by the following diagram:

Entity group

To designate an entity's parent, provide the parent entity's key as an argument to the Entity() constructor when creating the child entity. You can get the key by calling the parent entity's getKey() method:

Entity employee = new Entity("Employee");
datastore.put(employee);

Entity address = new Entity("Address", employee.getKey());
datastore.put(address);

If the new entity also has a key name, provide the key name as the second argument to the Entity() constructor and the key of the parent entity as the third argument:

Entity address = new Entity("Address", "addr1", employee.getKey());

Transactions and entity groups

Every attempt to create, update, or delete an entity takes place in the context of a transaction. A single transaction can include any number of such operations. To maintain the consistency of the data, the transaction ensures that all of the operations it contains are applied to Cloud Datastore as a unit or, if any of the operations fails, that none of them are applied. Furthermore, all strongly-consistent reads (ancestory queries or gets) performed within the same transaction observe a consistent snapshot of the data.

As mentioned above, an entity group is a set of entities connected through ancestry to a common root element. The organization of data into entity groups can limit what transactions can be performed:

  • All the data accessed by a transaction must be contained in at most 25 entity groups.
  • If you want to use queries within a transaction, your data must be organized into entity groups in such a way that you can specify ancestor filters that will match the right data.
  • There is a write throughput limit of about one transaction per second within a single entity group. This limitation exists because Cloud Datastore performs masterless, synchronous replication of each entity group over a wide geographic area to provide high reliability and fault tolerance.

In many applications, it is acceptable to use eventual consistency (i.e. a non-ancestor query spanning multiple entity groups, which may at times return slightly stale data) when obtaining a broad view of unrelated data, and then to use strong consistency (an ancestory query, or a get of a single entity) when viewing or editing a single set of highly related data. In such applications, it is usually a good approach to use a separate entity group for each set of highly related data. For more information, see Structuring for Strong Consistency.

<