Datastore Queries

View on GitHub

var query = datastore.createQuery('Task')
  .filter('done', '=', false)
  .filter('priority', '>=', 4)
  .order('priority', {
    descending: true
  });

Go

View on GitHub

query := datastore.NewQuery("Task").
	Filter("Done =", false).
	Filter("Priority >=", 4).
	Order("-Priority")

Python

View on GitHub

query = client.query(kind='Task')
query.add_filter('done', '=', False)
query.add_filter('priority', '>=', 4)
query.order = ['-priority']

C#

datastore/src/main/java/com/google/datastore/snippets/Concepts.java

View on GitHub

Query query = new Query("Task")
{
    Filter = Filter.And(Filter.Equal("done", false),
        Filter.GreaterThanOrEqual("priority", 4)),
    Order = { { "priority", PropertyOrder.Types.Direction.Descending } }
};

GQL


SELECT * FROM Task
WHERE done = FALSE AND priority >= 4
ORDER BY priority DESC

Here's how to run a query:

Java

View on GitHub

QueryResults<Entity> tasks = datastore.run(query);

Node.js

View on GitHub

datastore.runQuery(query, function (err, tasks) {
  if (!err) {
    // Task entities found.
  }
});

Go

View on GitHub

it := client.Run(ctx, query)
for {
	var task Task
	_, err := it.Next(&task)
	if err == datastore.Done {
		break
	}
	if err != nil {
		log.Fatalf("Error fetching next task: %v", err)
	}
	fmt.Printf("Task %q, Priority %d\n", task.Description, task.Priority)
}

Python

View on GitHub

query = client.query()
results = list(query.fetch())

C#

datastore/src/main/java/com/google/datastore/snippets/Concepts.java

View on GitHub

Query query = new Query("Task");
DatastoreQueryResults tasks = _db.RunQuery(query);

GQL

Not Applicable

Query structure

A query can specify an entity kind, zero or more filters, and zero or more sort orders.

Filters

A query's filters set constraints on the properties, keys, and ancestors of the entities to be retrieved.

Property filters

A property filter specifies

A property name
A comparison operator
A property value

This example returns Task entities that are marked not done:

Java

View on GitHub

Query<Entity> query =
    Query.entityQueryBuilder().kind("Task").filter(PropertyFilter.eq("done", false)).build();

Node.js

View on GitHub

var query = datastore.createQuery('Task')
  .filter('done', '=', false);

Go

View on GitHub

query := datastore.NewQuery("Task").Filter("Done =", false)

Python

View on GitHub

query = client.query(kind='Task')
query.add_filter('done', '=', False)

C#

datastore/src/main/java/com/google/datastore/snippets/Concepts.java

View on GitHub

Query query = new Query("Task")
{
    Filter = Filter.Equal("done", false)
};

GQL


SELECT * FROM Task WHERE done = FALSE

The property value must be supplied by the application; it cannot refer to or be calculated in terms of other properties. An entity satisfies the filter if it has a property of the given name whose value compares to the value specified in the filter in the manner described by the comparison operator. If the property of the given name is array-valued, the entity satisfies the filter if any of the values compares to the value specified in the filter in the manner described by the comparison operator.

The comparison operator can be any of the following:

Operator	Meaning
`EQUAL`	Equal to
`LESS_THAN`	Less than
`LESS_THAN_OR_EQUAL`	Less than or equal to
`GREATER_THAN`	Greater than
`GREATER_THAN_OR_EQUAL`	Greater than or equal to

Composite filters

A composite filter consists of more than one property filter. This example returns Task entities that are marked not done and have a priority of 4:

Java

View on GitHub

Query<Entity> query = Query.entityQueryBuilder()
    .kind("Task")
    .filter(
        CompositeFilter.and(PropertyFilter.eq("done", false), PropertyFilter.eq("priority", 4)))
    .build();

Node.js

View on GitHub

var query = datastore.createQuery('Task')
  .filter('done', '=', false)
  .filter('priority', '=', 4);

Go

View on GitHub

query := datastore.NewQuery("Task").Filter("Done =", false).Filter("Priority =", 4)

Python

View on GitHub

query = client.query(kind='Task')
query.add_filter('done', '=', False)
query.add_filter('priority', '=', 4)

C#

datastore/src/main/java/com/google/datastore/snippets/Concepts.java

View on GitHub

Query query = new Query("Task")
{
    Filter = Filter.And(Filter.Equal("done", false),
        Filter.Equal("priority", 4)),
};

GQL


SELECT * FROM Task WHERE done = TRUE AND priority = 4

Cloud Datastore currently only natively supports combining filters with the AND operator.

Key filters

To filter on the value of an entity's key, use the special property __key__:

Java

View on GitHub

Query<Entity> query = Query.entityQueryBuilder()
    .kind("Task")
    .filter(PropertyFilter.gt("__key__", keyFactory.newKey("someTask")))
    .build();

Node.js

View on GitHub

var query = datastore.createQuery('Task')
  .filter('__key__', '>', datastore.key(['Task', 'someTask']));

Go

View on GitHub

key := datastore.NewKey(ctx, "Task", "someTask", 0, nil)
query := datastore.NewQuery("Task").Filter("__key__ >", key)

Python

View on GitHub

query = client.query(kind='Task')
first_key = client.key('Task', 'first_task')
query.key_filter(first_key, '>')

C#

datastore/src/main/java/com/google/datastore/snippets/Concepts.java

View on GitHub

Query query = new Query("Task")
{
    Filter = Filter.GreaterThan("__key__", _keyFactory.CreateKey("aTask"))
};

GQL


SELECT * FROM Task WHERE __key__ > KEY(Task, 'someTask')

When comparing for inequality, keys are ordered by the following criteria, in order:

Ancestor path
Entity kind
Identifier (key name or numeric ID)

Elements of the ancestor path are compared similarly: by kind (string), then by key name or numeric ID. Kinds and key names are strings and are ordered by byte value; numeric IDs are integers and are ordered numerically. If entities with the same parent and kind use a mix of key name strings and numeric IDs, those with numeric IDs precede those with key names.

Queries on keys use indexes just like queries on properties and require custom indexes in the same cases, with a couple of exceptions: inequality filters or an ascending sort order on the key do not require a custom index, but a descending sort order on the key does. As with all queries, the development server creates appropriate entries in the index configuration file when a query that needs a custom index is used in the development environment.

Sort orders

A query sort order specifies

A property name.
A sort direction (ascending or descending). By default the sort order is ascending.

This example sorts Task entities by creation time in ascending order:

Java

View on GitHub

Query<Entity> query =
    Query.entityQueryBuilder().kind("Task").orderBy(OrderBy.asc("created")).build();

Node.js

View on GitHub

var query = datastore.createQuery('Task')
  .order('created');

Go

View on GitHub

query := datastore.NewQuery("Task").Order("created")

Python

View on GitHub

query = client.query(kind='Task')
query.order = ['created']

C#

datastore/src/main/java/com/google/datastore/snippets/Concepts.java

View on GitHub

Query query = new Query("Task")
{
    Order = { { "created", PropertyOrder.Types.Direction.Ascending } }
};

GQL


SELECT * FROM Task ORDER BY created ASC

This example sorts Task entities by creation time in descending order:

Java

View on GitHub

Query<Entity> query =
    Query.entityQueryBuilder().kind("Task").orderBy(OrderBy.desc("created")).build();

Node.js

View on GitHub

var query = datastore.createQuery('Task')
  .order('created', {
    descending: true
  });

Go

View on GitHub

query := datastore.NewQuery("Task").Order("-created")

Python

View on GitHub

query = client.query(kind='Task')
query.order = ['-created']

C#

datastore/src/main/java/com/google/datastore/snippets/Concepts.java

View on GitHub

Query query = new Query("Task")
{
    Order = { { "created", PropertyOrder.Types.Direction.Descending } }
};

GQL


SELECT * FROM Task ORDER BY created DESC

If a query includes multiple sort orders, they are applied in the sequence specified. The following example sorts first by descending priority and then by ascending creation time:

Java

View on GitHub

Query<Entity> query = Query.entityQueryBuilder()
    .kind("Task")
    .orderBy(OrderBy.desc("priority"), OrderBy.asc("created"))
    .build();

Node.js

View on GitHub

var query = datastore.createQuery('Task')
  .order('priority', {
    descending: true
  })
  .order('created');

Go

View on GitHub

query := datastore.NewQuery("Task").Order("-priority").Order("created")

Python

View on GitHub

query = client.query(kind='Task')
query.order = [
    '-priority',
    'created'
]

C#

datastore/src/main/java/com/google/datastore/snippets/Concepts.java

View on GitHub

Query query = new Query("Task")
{
    Order = { { "priority", PropertyOrder.Types.Direction.Descending },
        { "created", PropertyOrder.Types.Direction.Ascending } }
};

GQL


SELECT * FROM Task ORDER BY priority DESC, created ASC

If no sort orders are specified, the results are returned in the order they are retrieved from Cloud Datastore.

Note: Because of the way Cloud Datastore executes queries, if a query specifies inequality filters on a property and sort orders on other properties, the property used in the inequality filters must be ordered before the other properties.

Special query types

Some specific types of query deserve special mention:

Ancestor queries

An ancestor query limits its results to the specified entity and its descendants. This example returns all Task entities that have the specified TaskList entity as an ancestor:

Java

View on GitHub

Query<Entity> query = Query.entityQueryBuilder()
    .kind("Task")
    .filter(PropertyFilter.hasAncestor(
        datastore.newKeyFactory().kind("TaskList").newKey("default")))
    .build();

Node.js

View on GitHub

var ancestorKey = datastore.key(['TaskList', 'default']);

var query = datastore.createQuery('Task')
  .hasAncestor(ancestorKey);

Go

View on GitHub

ancestor := datastore.NewKey(ctx, "TaskList", "default", 0, nil)
query := datastore.NewQuery("Task").Ancestor(ancestor)

Python

View on GitHub

ancestor = client.key('TaskList', 'default')
query = client.query(kind='Task', ancestor=ancestor)

C#

datastore/src/main/java/com/google/datastore/snippets/Concepts.java

View on GitHub

Query query = new Query("Task")
{
    Filter = Filter.HasAncestor(_db.CreateKeyFactory("TaskList")
        .CreateKey("default"))
};

GQL


SELECT * FROM Task WHERE __key__ HAS ANCESTOR KEY(TaskList, 'default')

Kindless queries

A query with no kind and no ancestor retrieves all of the entities of an application from Cloud Datastore. Such kindless queries cannot include filters or sort orders on property values. They can, however, filter on entity keys and use ancestor filters. Key filters can be used by specifying __key__ as the property name:

Java

View on GitHub

Query<Entity> query =
    Query.entityQueryBuilder().filter(PropertyFilter.gt("__key__", lastSeenKey)).build();

Node.js

View on GitHub

var query = datastore.createQuery()
  .filter('__key__', '>', lastSeenKey)
  .limit(1);

Go

View on GitHub

query := datastore.NewQuery("").Filter("__key__ >", lastSeenKey)

Python

View on GitHub

query = client.query()
query.key_filter(last_seen_key, '>')

C#

datastore/src/main/java/com/google/datastore/snippets/Concepts.java

View on GitHub

Query query = new Query()
{
    Filter = Filter.GreaterThan("__key__",
        _keyFactory.CreateKey("aTask"))
};

GQL


SELECT * WHERE __key__ > KEY(Task, 'someTask')

Projection queries

Most Cloud Datastore queries return whole entities as their results, but often an application is actually interested in only a few of the entity's properties. Projection queries allow you to query Cloud Datastore for just those specific properties of an entity that you actually need, at lower latency and cost than retrieving the entire entity.

Keys-only queries

A keys-only query (which is a type of projection query) returns just the keys of the result entities instead of the entities themselves, at lower latency and cost than retrieving entire entities.

It is often more economical to do a keys-only query first, and then fetch a subset of entities from the results, rather than executing a general query which may fetch more entities than you actually need.

Here's how to create a keys-only query:

Java

View on GitHub

Query<Key> query = Query.keyQueryBuilder().kind("Task").build();

Node.js

View on GitHub

var query = datastore.createQuery()
  .select('__key__')
  .limit(1);

Go

View on GitHub

query := datastore.NewQuery("Task").KeysOnly()

Python

View on GitHub

query = client.query()
query.keys_only()

C#

datastore/src/main/java/com/google/datastore/snippets/Concepts.java

View on GitHub

Query query = new Query("Task")
{
    Projection = { "__key__" }
};

GQL


SELECT __key__ FROM Task

A keys-only query is a small operation and counts as only a single entity read for the query itself.

Projections

Projection queries are similar to SQL queries of the form

SELECT priority, percent_complete FROM Task

You can use all of the filtering and sorting features available for standard entity queries, subject to the limitations described below. The query returns abridged results with only the specified properties (priority and percent_complete in the example) populated with values; all other properties are not populated.

Here's an example that demonstrates the construction of a projection query:

Java

View on GitHub

Query<ProjectionEntity> query = Query.projectionEntityQueryBuilder()
    .kind("Task")
    .projection("priority", "percent_complete")
    .build();

Node.js

View on GitHub

var query = datastore.createQuery('Task')
  .select(['priority', 'percent_complete']);

Go

View on GitHub

query := datastore.NewQuery("Task").Project("Priority", "PercentComplete")

Python

View on GitHub

query = client.query(kind='Task')
query.projection = ['priority', 'percent_complete']

C#

datastore/src/main/java/com/google/datastore/snippets/Concepts.java

View on GitHub

Query query = new Query("Task")
{
    Projection = { "priority", "percent_complete" }
};

GQL


SELECT priority, percent_complete FROM Task

And here's how to run the projection query:

Java

View on GitHub

List<Long> priorities = new LinkedList<>();
List<Double> percentCompletes = new LinkedList<>();
QueryResults<ProjectionEntity> tasks = datastore.run(query);
while (tasks.hasNext()) {
  ProjectionEntity task = tasks.next();
  priorities.add(task.getLong("priority"));
  percentCompletes.add(task.getDouble("percent_complete"));
}

Node.js

View on GitHub

var priorities = [];
var percentCompletes = [];

datastore.runQuery(query, function (err, tasks) {
  if (err) {
    // An error occurred while running the query.
    return;
  }

  tasks.forEach(function (task) {
    priorities.push(task.data.priority);
    percentCompletes.push(task.data.percent_complete);
  });
});

Go

View on GitHub

var priorities []int
var percents []float64
it := client.Run(ctx, query)
for {
	var task Task
	if _, err := it.Next(&task); err == datastore.Done {
		break
	} else if err != nil {
		log.Fatal(err)
	}
	priorities = append(priorities, task.Priority)
	percents = append(percents, task.PercentComplete)
}

Python

View on GitHub

priorities = []
percent_completes = []

for task in query.fetch():
    priorities.append(task['priority'])
    percent_completes.append(task['percent_complete'])

C#

datastore/src/main/java/com/google/datastore/snippets/Concepts.java

View on GitHub

Query query = new Query("Task")
{
    Projection = { "priority", "percent_complete" }
};
List<long> priorities = new List<long>();
List<double> percentCompletes = new List<double>();
foreach (var entity in _db.RunQuery(query))
{
    priorities.Add((long)entity["priority"]);
    percentCompletes.Add((double)entity["percent_complete"]);
}

GQL

Not Applicable

A projection query that does not use the distinct on clause is a small operation and counts as only a single entity read for the query itself.

Grouping

Projection queries can use the distinct on clause to ensure that only the first result for each distinct combination of values for the specified properties will be returned. This will return only the first result for entities which have the same values for the properties that are being projected.

Java

View on GitHub

Query<ProjectionEntity> query = Query.projectionEntityQueryBuilder()
    .kind("Task")
    .projection("category", "priority")
    .distinctOn("category")
    .orderBy(OrderBy.asc("category"), OrderBy.asc("priority"))
    .build();

Node.js

View on GitHub

var query = datastore.createQuery('Task')
  .groupBy('category')
  .order('category')
  .order('priority');

Go

View on GitHub

// DISTINCT ON not supported in Go API

Python

View on GitHub

query = client.query(kind='Task')
query.distinct_on = ['category']
query.order = ['category', 'priority']

C#

datastore/src/main/java/com/google/datastore/snippets/Concepts.java

View on GitHub

Query query = new Query("Task")
{
    Projection = { "category", "priority" },
    DistinctOn = { "category" },
    Order = { { "category", PropertyOrder.Types.Direction.Ascending},
        {"priority", PropertyOrder.Types.Direction.Ascending } }
};

GQL


SELECT DISTINCT ON (category) category, priority FROM Task
ORDER BY category, priority

Array values

Consider the following when your query includes properties with array values.

Properties with array values can behave in surprising ways

Because of the way they're indexed, entities with multiple values for the same property can sometimes interact with query filters and sort orders in unexpected and surprising ways.

If a query has multiple inequality filters on a given property, an entity will match the query only if at least one of its individual values for the property satisfies all of the filters. For example, if an entity of kind Task has values fun and programming for property tag, it will not match the query:

Java

View on GitHub

Query<Entity> query = Query.entityQueryBuilder()
    .kind("Task")
    .filter(CompositeFilter.and(
        PropertyFilter.gt("tag", "learn"), PropertyFilter.lt("tag", "math")))
    .build();

Node.js

View on GitHub

var query = datastore.createQuery('Task')
  .filter('tag', '>', 'learn')
  .filter('tag', '<', 'math');

Go

View on GitHub

query := datastore.NewQuery("Task").
	Filter("Tag >", "learn").
	Filter("Tag <", "math")

Python

View on GitHub

query = client.query(kind='Task')
query.add_filter('tag', '>', 'learn')
query.add_filter('tag', '<', 'math')

C#

datastore/src/main/java/com/google/datastore/snippets/Concepts.java

View on GitHub

Query query = new Query("Task")
{
    Filter = Filter.And(Filter.GreaterThan("tag", "learn"),
        Filter.LessThan("tag", "math"))
};

GQL


SELECT * FROM Task WHERE tag > 'learn' AND tag < 'math'

Each of the entity's tag values satisfies one of the filters, but neither single value satisfies both. Note that this does not apply to equality filters. For example, the same entity will satisfy the query

Java

View on GitHub

Query<Entity> query = Query.entityQueryBuilder()
    .kind("Task")
    .filter(CompositeFilter.and(
        PropertyFilter.eq("tag", "fun"), PropertyFilter.eq("tag", "programming")))
    .build();

Node.js

View on GitHub

var query = datastore.createQuery('Task')
  .filter('tag', '=', 'fun')
  .filter('tag', '=', 'programming');

Go

View on GitHub

query := datastore.NewQuery("Task").
	Filter("Tag =", "fun").
	Filter("Tag =", "programming")

Python

View on GitHub

query = client.query(kind='Task')
query.add_filter('tag', '=', 'fun')
query.add_filter('tag', '=', 'programming')

C#

datastore/src/main/java/com/google/datastore/snippets/Concepts.java

View on GitHub

Query query = new Query("Task")
{
    Filter = Filter.And(Filter.Equal("tag", "fun"),
        Filter.Equal("tag", "programming"))
};

GQL


SELECT * FROM Task WHERE tag = 'fun' AND tag = 'programming'

even though neither of the entity's individual tag values satisfies both filter conditions.

Similarly, the sort order for multiple-valued properties is unusual. Because such properties appear once in the index for each unique value, the first value seen in the index determines an entity's sort order.

If a multi-valued property is not used in any filter:

and the query results are sorted in ascending order by the property, the smallest value of the property is used for ordering.
and the query results are sorted in descending order by the property, the greatest value is used for ordering.
other values do not affect the sort order, nor does the number of values.

This has the unusual consequence that an entity with property values 1 and 9 precedes an entity with values 4, 5, 6, and 7 in both ascending and descending order.

If a multi-valued property is used in an equality filter, any sort order on that property is ignored.

If a multi-valued property is used in an inequality filter:

and the query results are sorted in ascending order by the property, the smallest value that satisfies all of the query's inequality filters is used for ordering.
and the query results are sorted in descending order by the property, the greatest value that satisfies all of the query's inequality filters is used for ordering.

Note that if a set of inequality filters on a property translate into an equality filter, such as

WHERE tags >= 'math' AND tags <= 'math'

any sort order on that property is ignored, as the filters evaluate the same as the equality filter

WHERE tags = 'math'

Projections and array-valued properties

Projecting a property with array values will not populate all values for that property. Instead, a separate entity will be returned for each unique combination of projected values matching the query. For example, suppose you have an entity of kind Task with two multiple-valued properties, tags and collaborators:

Java

View on GitHub

Entity task = Entity.builder(taskKey)
    .set("tags", "fun", "programming")
    .set("collaborators", "alice", "bob")
    .build();

Node.js

View on GitHub

var task = {
  tags: [
    'fun',
    'programming'
  ],
  collaborators: [
    'alice',
    'bob'
  ]
};

Go

View on GitHub

type Task struct {
	Tags          []string
	Collaborators []string
}
task := &Task{
	Tags:          []string{"fun", "programming"},
	Collaborators: []string{"alice", "bob"},
}

Python

View on GitHub

task = datastore.Entity(key)
task.update({
    'tags': [
        'fun',
        'programming'
    ],
    'collaborators': [
        'alice',
        'bob'
    ]
})

C#

datastore/src/main/java/com/google/datastore/snippets/Concepts.java

View on GitHub

Entity task = new Entity()
{
    Key = _db.CreateKeyFactory("Task").CreateKey("sampleTask"),
    ["collaborators"] = new ArrayValue() { Values = { "alice", "bob" } },
    ["tags"] = new ArrayValue() { Values = { "fun", "programming" } }
};

GQL

Not Applicable

Then the projection query

SELECT tags, collaborators FROM Task WHERE collaborators < 'charlie'

will return four entities with the following combinations of values:

tags = 'fun', collaborators = 'alice'
tags = 'fun', collaborators = 'bob'
tags = 'programming', collaborators = 'alice'
tags = 'programming', collaborators = 'bob'

Cursors, limits, and offsets

You can specify a limit for your query to control the maximum number of results returned in one batch. The following example retrieves at most five Task entities:

Java

View on GitHub

Query<Entity> query = Query.entityQueryBuilder().kind("Task").limit(5).build();

Node.js

View on GitHub

var query = datastore.createQuery('Task')
  .limit(5);

Go

View on GitHub

query := datastore.NewQuery("Task").Limit(5)

Python

View on GitHub

query = client.query()
tasks = list(query.fetch(limit=5))

C#

datastore/src/main/java/com/google/datastore/snippets/Concepts.java

View on GitHub

Query query = new Query("Task")
{
    Limit = 5,
};

GQL


SELECT * FROM Task LIMIT 5

Query cursors allow an application to retrieve a query's results in convenient batches without incurring the overhead of a query offset. After performing a retrieval operation, the application can obtain a cursor, which is an opaque byte string marking the index position of the last result retrieved. The application can save this string (for instance in Cloud Datastore, a cache, or embedded in a web page as a base-64 encoded HTTP GET or POST parameter), and can then use the cursor as the starting point for a subsequent retrieval operation to obtain the next batch of results from the point where the previous retrieval ended. A retrieval can also specify an end cursor, to limit the extent of the result set returned.

The following example demonstrates the use of cursors for pagination:

Java

View on GitHub

EntityQuery.Builder queryBuilder = Query.entityQueryBuilder().kind("Task").limit(pageSize);
if (pageCursor != null) {
  queryBuilder.startCursor(pageCursor);
}
QueryResults<Entity> tasks = datastore.run(queryBuilder.build());
while (tasks.hasNext()) {
  Entity task = tasks.next();
  // do something with the task
}
Cursor nextPageCursor = tasks.cursorAfter();

Node.js

View on GitHub

// By default, gcloud-node will paginate through all of the results that match
// a query, push them into an array, then return them to your callback after
// they have all been retrieved. You must execute `.autoPaginate(false)` on
// your query to disable this behavior.
var query = datastore.createQuery('Task')
  .autoPaginate(false)
  .limit(pageSize)
  .start(pageCursor);

datastore.runQuery(query, function (err, results, nextQuery) {
  if (err) {
    // An error occurred while running the query.
    return;
  }

  var nextPageCursor;

  if (nextQuery) {
    // If there are more results to retrieve, the start cursor is
    // automatically set on `nextQuery`. To get this value directly, access
    // the `startVal` property.
    nextPageCursor = nextQuery.startVal;
  } else {
    // No more results exist.
  }
  console.log('nextPageCursor', nextPageCursor);
});

Go

View on GitHub

const pageSize = 5
query := datastore.NewQuery("Tasks").Limit(pageSize)
if cursorStr != "" {
	cursor, err := datastore.DecodeCursor(cursorStr)
	if err != nil {
		log.Fatalf("Bad cursor %q: %v", cursorStr, err)
	}
	query = query.Start(cursor)
}

// Read the tasks.
var tasks []Task
var task Task
it := client.Run(ctx, query)
_, err := it.Next(&task)
for err == nil {
	tasks = append(tasks, task)
	_, err = it.Next(&task)
}
if err != datastore.Done {
	log.Fatalf("Failed fetching results: %v", err)
}

// Get the cursor for the next page of results.
nextCursor, err := it.Cursor()

Python

View on GitHub

def get_one_page_of_tasks(cursor=None):
    query = client.query(kind='Task')
    query_iter = query.fetch(start_cursor=cursor, limit=5)
    tasks, _, cursor = query_iter.next_page()

    return tasks, cursor

C#

datastore/src/main/java/com/google/datastore/snippets/Concepts.java

View on GitHub

            Query query = new Query("Task")
            {
                Limit = pageSize,
            };
            if (!string.IsNullOrEmpty(pageCursor))
                query.StartCursor = ByteString.FromBase64(pageCursor);

            ByteString finalCursor = null;
            foreach (EntityResult result in _db.RunQuery(query)
                .AsEntityResults())
            {
                var task = result.Entity;
                // Do something with the task.
                finalCursor = result.Cursor;
            }
            return finalCursor?.ToBase64();

GQL

Not Applicable

Although Cloud Datastore supports integer offsets, you should avoid using them. Instead, use cursors. Using an offset only avoids returning the skipped entities to your application, but these entities are still retrieved internally. The skipped entities do affect the latency of the query, and your application is billed for the read operations required to retrieve them. Using cursors instead of offsets lets you avoid all these costs.

Limitations of cursors

Cursors are subject to the following limitations:

A cursor can be used only by the same project that performed the original query, and only to continue the same query. It is not possible to retrieve results using a cursor without setting up the same query from which it was originally generated.
If any of the following items are changed, a cursor can still be used for subsequent retrievals.
- start cursor
- end cursor
- offset
- limit
If any of the following items are changed, a cursor cannot be used for subsequent retrievals.
- projection
- kind
- ancestor
- filter
- distinct on
- sort order
  An exception is if the original query's final sort order was on __key__. In that case, you can use the cursor in a reverse query, which is the original query with each sort order reversed. The reverse query can modify the start cursor, end cursor, offset, and limit.
Cursors don't always work as expected with a query that uses an inequality filter or a sort order on a property with multiple values. The de-duplication logic for such multiple-valued properties does not persist between retrievals, possibly causing the same result to be returned more than once.
New Cloud Datastore releases may change internal implementation details, invalidating cursors that depend on them. If an application attempts to use a cursor that is no longer valid, Cloud Datastore raises an exception.

Cursors and data updates

The cursor represents the location in the result list after the last result returned. A cursor is not a relative position in the list (it's not an offset); it's a marker to which Cloud Datastore can jump when starting an index scan for results. If the results for a query change between uses of a cursor, the query notices only changes that occur in results after the cursor. If a new result appears before the cursor's position for the query, it will not be returned when the results after the cursor are fetched. Similarly, if an entity is no longer a result for a query but had appeared before the cursor, the results that appear after the cursor do not change. If the last result returned is removed from the result set, the cursor still knows how to locate the next result.

When retrieving query results, you can use both a start cursor and an end cursor to return a continuous group of results from Cloud Datastore. When using a start and end cursor to retrieve the results, you are not guaranteed that the size of the results will be the same as when you generated the cursors. Entities may be added or deleted from Cloud Datastore between the time the cursors are generated and when they are used in a query.

Restrictions on queries

The nature of the index query mechanism imposes certain restrictions on what a query can do. Cloud Datastore queries do not support substring matches, case-insensitive matches, or so-called full-text search. The NOT, OR, and != operators are not natively supported, but some client libraries may add support on top of Cloud Datastore. Additionally:

Entities lacking a property named in the query are ignored

Entities of the same kind need not have the same properties. To be eligible as a query result, an entity must possess a value (possibly null) for every property named in the query's filters and sort orders. If not, the entity is omitted from the indexes used to execute the query and consequently will not be included in the query's results.

Filtering on unindexed properties returns no results

A query can't find property values that aren't indexed, nor can it sort on such properties. See the Unindexed properties section for a detailed discussion of unindexed properties.

Inequality filters are limited to at most one property

To avoid having to scan the entire index, the query mechanism relies on all of a query's potential results being adjacent to one another in the index. To satisfy this constraint, a single query may not use inequality comparisons (LESS_THAN, LESS_THAN_OR_EQUAL, GREATER_THAN, GREATER_THAN_OR_EQUAL) on more than one property across all of its filters. For example, the following query is valid, because both inequality filters apply to the same property:

Java

View on GitHub

Query<Entity> query = Query.entityQueryBuilder()
    .kind("Task")
    .filter(CompositeFilter.and(
        PropertyFilter.gt("created", startDate), PropertyFilter.lt("created", endDate)))
    .build();

Node.js

View on GitHub

var query = datastore.createQuery('Task')
  .filter('created', '>', new Date('1990-01-01T00:00:00z'))
  .filter('created', '<', new Date('2000-12-31T23:59:59z'));

Go

View on GitHub

query := datastore.NewQuery("Task").
	Filter("Created >", time.Date(1990, 1, 1, 0, 0, 0, 0, time.UTC)).
	Filter("Created <", time.Date(2000, 1, 1, 0, 0, 0, 0, time.UTC))

Python

View on GitHub

start_date = datetime.datetime(1990, 1, 1)
end_date = datetime.datetime(2000, 1, 1)
query = client.query(kind='Task')
query.add_filter(
    'created', '>', start_date)
query.add_filter(
    'created', '<', end_date)

C#

datastore/src/main/java/com/google/datastore/snippets/Concepts.java

View on GitHub

Query query = new Query("Task")
{
    Filter = Filter.And(Filter.GreaterThan("created", _startDate),
        Filter.LessThan("created", _endDate))
};

GQL


SELECT * FROM Task
WHERE created > DATETIME('1990-01-01T00:00:00z')
  AND created < DATETIME('2000-12-31T23:59:59z')

However, this query is not valid, because it uses inequality filters on two different properties:

Java

View on GitHub

Query<Entity> query = Query.entityQueryBuilder()
    .kind("Task")
    .filter(CompositeFilter.and(
        PropertyFilter.gt("created", startDate), PropertyFilter.gt("priority", 3)))
    .build();

Node.js

View on GitHub

var query = datastore.createQuery('Task')
  .filter('priority', '>', 3)
  .filter('created', '>', new Date('1990-01-01T00:00:00z'));

Go

View on GitHub

query := datastore.NewQuery("Task").
	Filter("Created >", time.Date(1990, 1, 1, 0, 0, 0, 0, time.UTC)).
	Filter("Priority >", 3)

Python

View on GitHub

start_date = datetime.datetime(1990, 1, 1)
query = client.query(kind='Task')
query.add_filter(
    'created', '>', start_date)
query.add_filter(
    'priority', '>', 3)

C#

datastore/src/main/java/com/google/datastore/snippets/Concepts.java

View on GitHub

Query query = new Query("Task")
{
    Filter = Filter.And(Filter.GreaterThan("created", _startDate),
        Filter.GreaterThan("priority", 3))
};

GQL


# Invalid query!
SELECT * FROM Task
WHERE created > DATETIME('1990-01-01T00:00:00z')
AND priority > 3

Note that a query can combine equality (EQUAL) filters for different properties, along with one or more inequality filters on a single property. Thus the following is a valid query:

Java

View on GitHub

Query<Entity> query = Query.entityQueryBuilder()
    .kind("Task")
    .filter(CompositeFilter.and(PropertyFilter.eq("priority", 4),
        PropertyFilter.gt("created", startDate), PropertyFilter.lt("created", endDate)))
    .build();

Node.js

View on GitHub

var query = datastore.createQuery('Task')
  .filter('priority', '=', 4)
  .filter('done', '=', false)
  .filter('created', '>', new Date('1990-01-01T00:00:00z'))
  .filter('created', '<', new Date('2000-12-31T23:59:59z'));

Go

View on GitHub

query := datastore.NewQuery("Task").
	Filter("Priority =", 4).
	Filter("Done =", false).
	Filter("Created >", time.Date(1990, 1, 1, 0, 0, 0, 0, time.UTC)).
	Filter("Created <", time.Date(2000, 1, 1, 0, 0, 0, 0, time.UTC))

Python

View on GitHub

start_date = datetime.datetime(1990, 1, 1)
end_date = datetime.datetime(2000, 12, 31, 23, 59, 59)
query = client.query(kind='Task')
query.add_filter('priority', '=', 4)
query.add_filter('done', '=', False)
query.add_filter(
    'created', '>', start_date)
query.add_filter(
    'created', '<', end_date)

C#

datastore/src/main/java/com/google/datastore/snippets/Concepts.java

View on GitHub

Query query = new Query("Task")
{
    Filter = Filter.And(Filter.Equal("priority", 4),
        Filter.GreaterThan("created", _startDate),
        Filter.LessThan("created", _endDate))
};

GQL


SELECT * FROM Task
WHERE priority = 4
  AND done = FALSE
  AND created > DATETIME('1990-01-01T00:00:00z')
  AND created < DATETIME('2000-12-31T23:59:59z')

Ordering of query results is undefined when no sort order is specified

When a query does not specify a sort order, the results are returned in the order they are retrieved. As Cloud Datastore implementation evolves (or if a project's indexes change), this order may change. Therefore, if your application requires its query results in a particular order, be sure to specify that sort order explicitly in the query.

Sort orders are ignored on properties with equality filters

Queries that include an equality filter for a given property ignore any sort order specified for that property. This is a simple optimization to save needless processing for single-valued properties, since all results have the same value for the property and so no further sorting is needed. Multiple-valued properties, however, may have additional values besides the one matched by the equality filter. Because this use case is rare and applying the sort order would be expensive and require extra indexes, the Cloud Datastore query planner simply ignores the sort order even in the multiple-valued case. This may cause query results to be returned in a different order than the sort order appears to imply.

Properties used in inequality filters must be sorted first

To retrieve all results that match an inequality filter, a query scans the index for the first row matching the filter, then scans forward until it encounters a nonmatching row. For the consecutive rows to encompass the complete result set, they must be ordered by the property used in the inequality filter before any other properties. Thus if a query specifies one or more inequality filters along with one or more sort orders, the first sort order must refer to the same property named in the inequality filters. The following is a valid query:

Java

View on GitHub

Query<Entity> query = Query.entityQueryBuilder()
    .kind("Task")
    .filter(PropertyFilter.gt("priority", 3))
    .orderBy(OrderBy.asc("priority"), OrderBy.asc("created"))
    .build();

Node.js

View on GitHub

var query = datastore.createQuery('Task')
  .filter('priority', '>', 3)
  .order('priority')
  .order('created');

Go

View on GitHub

query := datastore.NewQuery("Task").
	Filter("Priority >", 3).
	Order("Priority").
	Order("Created")

Python

View on GitHub

query = client.query(kind='Task')
query.add_filter('priority', '>', 3)
query.order = ['priority', 'created']

C#

datastore/src/main/java/com/google/datastore/snippets/Concepts.java

View on GitHub

Query query = new Query("Task")
{
    Filter = Filter.GreaterThan("priority", 3),
    Order = { { "priority", PropertyOrder.Types.Direction.Ascending},
        {"created", PropertyOrder.Types.Direction.Ascending } }
};

GQL


SELECT * FROM Task WHERE priority > 3 ORDER BY priority, created

This query is not valid, because it doesn't sort on the property used in the inequality filter:

Java

View on GitHub

Query<Entity> query = Query.entityQueryBuilder()
    .kind("Task")
    .filter(PropertyFilter.gt("priority", 3))
    .orderBy(OrderBy.asc("created"))
    .build();

Node.js

View on GitHub

var query = datastore.createQuery('Task')
  .filter('priority', '>', 3)
  .order('created');

Go

View on GitHub

query := datastore.NewQuery("Task").
	Filter("Priority >", 3).
	Order("Created")

Python

View on GitHub

query = client.query(kind='Task')
query.add_filter('priority', '>', 3)
query.order = ['created']

C#

datastore/src/main/java/com/google/datastore/snippets/Concepts.java

View on GitHub

Query query = new Query("Task")
{
    Filter = Filter.GreaterThan("priority", 3),
    Order = { { "created", PropertyOrder.Types.Direction.Ascending } }
};

GQL


# Invalid query!
SELECT * FROM Task WHERE priority > 3 ORDER BY created

Similarly, this query is not valid because the property used in the inequality filter is not the first one sorted:

Java

View on GitHub

Query<Entity> query = Query.entityQueryBuilder()
    .kind("Task")
    .filter(PropertyFilter.gt("priority", 3))
    .orderBy(OrderBy.asc("created"), OrderBy.asc("priority"))
    .build();

Node.js

View on GitHub

var query = datastore.createQuery('Task')
  .filter('priority', '>', 3)
  .order('created')
  .order('priority');

Go

View on GitHub

query := datastore.NewQuery("Task").
	Filter("Priority >", 3).
	Order("Created").
	Order("Priority")

Python

View on GitHub

query = client.query(kind='Task')
query.add_filter('priority', '>', 3)
query.order = ['created', 'priority']

C#