google.appengine.ext.ndb.query module

Summary

Higher-level Query wrapper.

The fundamental API here overloads the 6 comparisons operators to represent filters on property values, and supports AND and OR operations (implemented as functions – Python’s ‘and’ and ‘or’ operators cannot be overloaded, and the ‘&’ and ‘|’ operators have a priority that conflicts with the priority of comparison operators). For example:

class Employee(Model):
  name = StringProperty()
  age = IntegerProperty()
  rank = IntegerProperty()

  @classmethod
  def demographic(cls, min_age, max_age):
    return cls.query().filter(AND(cls.age >= min_age, cls.age <= max_age))

  @classmethod
  def ranked(cls, rank):
    return cls.query(cls.rank == rank).order(cls.age)

for emp in Employee.seniors(42, 5):
  print emp.name, emp.age, emp.rank

The ‘in’ operator cannot be overloaded, but is supported through the IN() method. For example:

Employee.query().filter(Employee.rank.IN([4, 5, 6]))

Sort orders are supported through the order() method; unary minus is overloaded on the Property class to represent a descending order:

Employee.query().order(Employee.name, -Employee.age)

Besides using AND() and OR(), filters can also be combined by repeatedly calling .filter():

q1 = Employee.query()  # A query that returns all employees
q2 = q1.filter(Employee.age >= 30)  # Only those over 30
q3 = q2.filter(Employee.age < 40)  # Only those in their 30s

A further shortcut is calling .filter() with multiple arguments; this implies AND():

q1 = Employee.query()  # A query that returns all employees
q3 = q1.filter(Employee.age >= 30,
               Employee.age < 40)  # Only those in their 30s

And finally you can also pass one or more filter expressions directly to the .query() method:

q3 = Employee.query(Employee.age >= 30,
                    Employee.age < 40)  # Only those in their 30s

Query objects are immutable, so these methods always return a new Query object; the above calls to filter() do not affect q1. (On the other hand, operations that are effectively no-ops may return the original Query object.)

Sort orders can also be combined this way, and .filter() and .order() calls may be intermixed:

q4 = q3.order(-Employee.age)
q5 = q4.order(Employee.name)
q6 = q5.filter(Employee.rank == 5)

Again, multiple .order() calls can be combined:

q5 = q3.order(-Employee.age, Employee.name)

The simplest way to retrieve Query results is a for-loop:

for emp in q3:
  print emp.name, emp.age

Some other methods to run a query and access its results:

q.iter() # Return an iterator; same as iter(q) but more flexible
q.map(callback) # Call the callback function for each query result
q.fetch(N) # Return a list of the first N results
q.get() # Return the first result
q.count(N) # Return the number of results, with a maximum of N
q.fetch_page(N, start_cursor=cursor) # Return (results, cursor, has_more)

All of the above methods take a standard set of additional query options, either in the form of keyword arguments such as keys_only=True, or as QueryOptions object passed with options=QueryOptions(…). The most important query options are:

• keys_only: bool, if set the results are keys instead of entities

• limit: int, limits the number of results returned

• offset: int, skips this many results first

• start_cursor: Cursor, start returning results after this position

• end_cursor: Cursor, stop returning results after this position

• batch_size: int, hint for the number of results returned per RPC

• prefetch_size: int, hint for the number of results in the first RPC

• produce_cursors: bool, return Cursor objects with the results

For additional (obscure) query options and more details on them, including an explanation of Cursors, see datastore_query.py.

All of the above methods except for iter() have asynchronous variants as well, which return a Future; to get the operation’s ultimate result, yield the Future (when inside a tasklet) or call the Future’s get_result() method (outside a tasklet):

q.map_async(callback)  # Callback may be a task or a plain function
q.fetch_async(N)
q.get_async()
q.count_async(N)
q.fetch_page_async(N, start_cursor=cursor)

Finally, there’s an idiom to efficiently loop over the Query results in a tasklet, properly yielding when appropriate:

it = q.iter()
while (yield it.has_next_async()):
  emp = it.next()
  print emp.name, emp.age

Contents

class google.appengine.ext.ndb.query.Query(*args, **kwds)source

Bases: object

Query object.

Usually constructed by calling Model.query().

See module docstring for examples.

Note that not all operations on Queries are supported by _MultiQuery instances; the latter are generated as necessary when any of the operators !=, IN or OR is used.

analyze()source

Return a list giving the parameters required by a query.

ancestor

Accessor for the ancestor (a Key or None).

app

Accessor for the app (a string or None).

bind(*args, **kwds)source

Bind parameter values. Returns a new Query object.

count(*args, **kwds)source

Count the number of query results, up to a limit.

This returns the same result as len(q.fetch(limit)) but more efficiently.

Note that you must pass a maximum value to limit the amount of work done by the query.

Parameters

• limit – How many results to count at most.

• **q_options – All query options keyword arguments are supported.

Returns:

count_async(*args, **kwds)source

Count the number of query results, up to a limit.

This is the asynchronous version of Query.count().

default_options

Accessor for the default_options (a QueryOptions instance or None).

fetch(*args, **kwds)source

Fetch a list of query results, up to a limit.

Parameters

• limit – How many results to retrieve at most.

• **q_options – All query options keyword arguments are supported.

Returns

A list of results.

fetch_async(*args, **kwds)source

Fetch a list of query results, up to a limit.

This is the asynchronous version of Query.fetch().

fetch_page(*args, **kwds)source

Fetch a page of results.

This is a specialized method for use by paging user interfaces.

Parameters

page_size – The requested page size. At most this many results will be returned.

In addition, any keyword argument supported by the QueryOptions class is supported. In particular, to fetch the next page, you pass the cursor returned by one call to the next call using start_cursor=<cursor>. A common idiom is to pass the cursor to the client using <cursor>.to_websafe_string() and to reconstruct that cursor on a subsequent request using Cursor.from_websafe_string(<string>).

Returns

A tuple (results, cursor, more) where results is a list of query results, cursor is a cursor pointing just after the last result returned, and more is a bool indicating whether there are (likely) more results after that.

fetch_page_async(*args, **kwds)source

Fetch a page of results.

This is the asynchronous version of Query.fetch_page().

filter(*args)source

Return a new Query with additional filter(s) applied.

filters

Accessor for the filters (a Node or None).

get(**q_options)source

Get the first query result, if any.

This is similar to calling q.fetch(1) and returning the first item of the list of results, if any, otherwise None.

Parameters

**q_options – All query options keyword arguments are supported.

Returns

A single result, or None if there are no results.

get_async(**q_options)source

Get the first query result, if any.

This is the asynchronous version of Query.get().

group_by

Accessor for the group by properties (a tuple instance or None).

is_distinct

True if results are guaranteed to contain a unique set of property values.

This happens when every property in the group_by is also in the projection.

iter(**q_options)source

Construct an iterator over the query.

Parameters

**q_options – All query options keyword arguments are supported.

Returns

A QueryIterator object.

kind

Accessor for the kind (a string or None).

map(*args, **kwds)source

Map a callback function or tasklet over the query results.

Parameters

• callback – A function or tasklet to be applied to each result; see below.

• merge_future – Optional Future subclass; see below.

• **q_options – All query options keyword arguments are supported.

Callback signature: The callback is normally called with an entity as argument. However if keys_only=True is given, it is called with a Key. Also, when pass_batch_into_callback is True, it is called with three arguments: the current batch, the index within the batch, and the entity or Key at that index. The callback can return whatever it wants. If the callback is None, a trivial callback is assumed that just returns the entity or key passed in (ignoring produce_cursors).

Optional merge future: The merge_future is an advanced argument that can be used to override how the callback results are combined into the overall map() return value. By default a list of callback return values is produced. By substituting one of a small number of specialized alternatives you can arrange otherwise. See tasklets.MultiFuture for the default implementation and a description of the protocol the merge_future object must implement the default. Alternatives from the same module include QueueFuture, SerialQueueFuture and ReducingFuture.

Returns

When the query has run to completion and all callbacks have returned, map() returns a list of the results of all callbacks. (But see ‘optional merge future’ above.)

map_async(*args, **kwds)source

Map a callback function or tasklet over the query results.

This is the asynchronous version of Query.map().

namespace

Accessor for the namespace (a string or None).

order(*args)source

Return a new Query with additional sort order(s) applied.

orders

Accessor for the filters (a datastore_query.Order or None).

projection

Accessor for the projected properties (a tuple instance or None).

run_to_queue(*args, **kwds)source

Run this query, putting entities into the given queue.

class google.appengine.ext.ndb.query.QueryOptionssource

Bases: google.appengine.ext.ndb.context.ContextOptions, google.appengine.datastore.datastore_query.QueryOptions

Support both context options and query options (esp. use_cache).

class google.appengine.ext.ndb.query.Cursor(*args, **kwds)source

Bases: google.appengine.datastore.datastore_query._BaseComponent

An immutable class that represents a relative position in a query.

The position denoted by a Cursor is relative to a result in a query even if the result has been removed from the given query. Usually to position immediately after the last result returned by a batch.

A cursor should only be used on a query with an identical signature to the one that produced it or on a query with its sort order reversed.

advance(offset, query, conn)source

Advances a Cursor by the given offset.

Parameters

• offset – The amount to advance the current query.

• query – A Query identical to the one this cursor was created from.

• conn – The datastore_rpc.Connection to use.

Returns

A new cursor that is advanced by offset using the given query.

static from_bytes(cursor)source

Gets a Cursor given its byte string serialized form.

The serialized form of a cursor may change in a non-backwards compatible way. In this case cursors must be regenerated from a new Query request.

Parameters

cursor – A serialized cursor as returned by .to_bytes.

Returns

A Cursor.

Raises

• datastore_errors.BadValueError if the cursor argument does not represent a

• serialized cursor.

static from_websafe_string(cursor)source

Gets a Cursor given its websafe serialized form.

The serialized form of a cursor may change in a non-backwards compatible way. In this case cursors must be regenerated from a new Query request.

Parameters

cursor – A serialized cursor as returned by .to_websafe_string.

Returns

A Cursor.

Raises

• datastore_errors.BadValueError if the cursor argument is not a string

• type of does not represent a serialized cursor.

reversed()source

DEPRECATED. It is no longer necessary to call reversed() on cursors.

A cursor returned by a query may also be used in a query whose sort order has been reversed. This method returns a copy of the original cursor.

to_bytes()source

Serialize cursor as a byte string.

to_websafe_string()source

Serialize cursor as a websafe string.

Returns

A base64-encoded serialized cursor.

urlsafe()source

Serialize cursor as a websafe string.

Returns

A base64-encoded serialized cursor.

class google.appengine.ext.ndb.query.QueryIterator(*args, **kwds)source

Bases: object

This iterator works both for synchronous and async callers!

For synchronous callers, just use:

for entity in Account.query():

<use entity>

Async callers use this idiom:

it = iter(Account.query()) while (yield it.has_next_async()):

entity = it.next() <use entity>

You can also use q.iter([options]) instead of iter(q); this allows passing query options such as keys_only or produce_cursors.

When keys_only is set, it.next() returns a key instead of an entity.

When produce_cursors is set, the methods it.cursor_before() and it.cursor_after() return Cursor objects corresponding to the query position just before and after the item returned by it.next(). Before it.next() is called for the first time, both raise an exception. Once the loop is exhausted, both return the cursor after the last item returned. Calling it.has_next() does not affect the cursors; you must call it.next() before the cursors move. Note that sometimes requesting a cursor requires a Cloud Datastore roundtrip (but not if you happen to request a cursor corresponding to a batch boundary). If produce_cursors is not set, both methods always raise an exception.

Note that queries requiring in-memory merging of multiple queries (i.e. queries using the IN, != or OR operators) do not support query options.

cursor_after()source

Return the cursor after the current item.

You must pass a QueryOptions object with produce_cursors=True for this to work.

If there is no cursor or no current item, raise BadArgumentError. Before next() has returned there is no cursor. Once the loop is exhausted, this returns the cursor after the last item.

cursor_before()source

Return the cursor before the current item.

You must pass a QueryOptions object with produce_cursors=True for this to work.

If there is no cursor or no current item, raise BadArgumentError. Before next() has returned there is no cursor. Once the loop is exhausted, this returns the cursor after the last item.

has_next()source

Return whether a next item is available.

See the module docstring for the usage pattern.

has_next_async(*args, **kwds)source

Return a Future whose result will say whether a next item is available.

See the module docstring for the usage pattern.

index_list()source

Return the list of indexes used for this query.

This returns a list of index representations, where an index representation is the same as what is returned by get_indexes().

Before the first result, the information is unavailable, and then None is returned. This is not the same as an empty list – the empty list means that no index was used to execute the query. (In the dev_appserver, an empty list may also mean that only built-in indexes were used; metadata queries also return an empty list here.)

Proper use is as follows:

q = <modelclass>.query(<filters>) i = q.iter() try:

i.next()

except Stopiteration:

pass

indexes = i.index_list() assert isinstance(indexes, list)

Notes: - Forcing produce_cursors=False makes this always return None. - This always returns None for a multi-query.

next()source

Iterator protocol: get next item or raise StopIteration.

probably_has_next()source

Return whether a next item is (probably) available.

This is not quite the same as has_next(), because when

produce_cursors is set, some shortcuts are possible. However, in some cases (e.g. when the query has a post_filter) we can get a false positive (returns True but next() will raise StopIteration). There are no false negatives.

class google.appengine.ext.ndb.query.RepeatedStructuredPropertyPredicate(match_keys, pb, key_prefix)source

Bases: google.appengine.datastore.datastore_query.FilterPredicate

google.appengine.ext.ndb.query.AND

alias of ConjunctionNode

google.appengine.ext.ndb.query.OR

alias of DisjunctionNode

class google.appengine.ext.ndb.query.ConjunctionNodesource

Bases: google.appengine.ext.ndb.query.Node

Tree node representing a Boolean AND operator on two or more nodes.

resolve(bindings, used)source

class google.appengine.ext.ndb.query.DisjunctionNodesource

Bases: google.appengine.ext.ndb.query.Node

Tree node representing a Boolean OR operator on two or more nodes.

resolve(bindings, used)source

class google.appengine.ext.ndb.query.FilterNodesource

Bases: google.appengine.ext.ndb.query.Node

Tree node for a single filter expression.

class google.appengine.ext.ndb.query.PostFilterNodesource

Bases: google.appengine.ext.ndb.query.Node

Tree node representing an in-memory filtering operation.

This is used to represent filters that cannot be executed by the datastore, for example a query for a structured value.

class google.appengine.ext.ndb.query.FalseNodesource

Bases: google.appengine.ext.ndb.query.Node

Tree node for an always-failing filter.

class google.appengine.ext.ndb.query.Nodesource

Bases: object

Base class for filter expression tree nodes.

Tree nodes are considered immutable, even though they can contain Parameter instances, which are not. In particular, two identical trees may be represented by the same Node object in different contexts.

resolve(bindings, used)source

Return a Node with Parameters replaced by the selected values.

Parameters

• bindings – A dict mapping integers and strings to values.

• used – A dict into which use of use of a binding is recorded.

Returns

A Node instance.

class google.appengine.ext.ndb.query.ParameterNodesource

Bases: google.appengine.ext.ndb.query.Node

Tree node for a parameterized filter.

resolve(bindings, used)source

class google.appengine.ext.ndb.query.ParameterizedThingsource

Bases: object

Base class for Parameter and ParameterizedFunction.

This exists purely for isinstance() checks.

class google.appengine.ext.ndb.query.Parameter(key)source

Bases: google.appengine.ext.ndb.query.ParameterizedThing

Represents a bound variable in a GQL query.

Parameter(1) corresponds to a slot labeled “:1” in a GQL query. Parameter(‘xyz’) corresponds to a slot labeled “:xyz”.

The value must be set (bound) separately by calling .set(value).

key

Retrieve the key.

resolve(bindings, used)source

class google.appengine.ext.ndb.query.ParameterizedFunction(func, values)source

Bases: google.appengine.ext.ndb.query.ParameterizedThing

Represents a GQL function with parameterized arguments.

For example, ParameterizedFunction(‘key’, [Parameter(1)]) stands for the GQL syntax KEY(:1).

func

is_parameterized()source

resolve(bindings, used)source

values

google.appengine.ext.ndb.query.gql(query_string, *args, **kwds)source

Parse a GQL query string.

Parameters

• query_string – Full GQL query, e.g. ‘SELECT * FROM Kind WHERE prop = 1’.

• **kwds (*args,) –

  If present, used to call bind().

Returns

An instance of query_class.