google.appengine.ext.key_range package

Summary

Key range representation and splitting.

Contents

exception google.appengine.ext.key_range.Errorsource

Bases: exceptions.Exception

Base class for exceptions in this module.

class google.appengine.ext.key_range.KeyRange(key_start=None, key_end=None, direction=None, include_start=True, include_end=True, namespace=None, _app=None)source

Bases: object

Represents a range of keys in the datastore.

A KeyRange object represents a key range

(key_start, include_start, key_end, include_end)

and a scan direction (KeyRange.DESC or KeyRange.ASC).

ASC = 'ASC'

DESC = 'DESC'

advance(key)source

Updates the start of the range immediately past the specified key.

Parameters

key – A db.Key or ndb.Key.

static bisect_string_range(start, end)source

Returns a string that is approximately in the middle of the range.

(start, end) is treated as a string range, and it is assumed start <= end in the usual lexicographic string ordering. The output key mid is guaranteed to satisfy start <= mid <= end.

The method proceeds by comparing initial characters of start and end. When the characters are equal, they are appended to the mid string. In the first place that the characters differ, the difference characters are averaged and this average is appended to the mid string. If averaging resulted in rounding down, and additional character is added to the mid string to make up for the rounding down. This extra step is necessary for correctness in the case that the average of the two characters is equal to the character in the start string.

This method makes the assumption that most keys are ascii and it attempts to perform splitting within the ascii range when that results in a valid split.

Parameters

• start – A string.

• end – A string such that start <= end.

Returns

A string mid such that start <= mid <= end.

classmethod compute_split_points(kind, count)source

Computes a set of KeyRanges that are split points for a kind.

Parameters

• kind – String with the entity kind to split.

• count – Number of non-overlapping KeyRanges to generate.

Returns

A list of KeyRange objects that are non-overlapping. At most “count” + 1 KeyRange objects will be returned. At least one will be returned.

filter_datastore_query(query, filters=None)source

Add query filter to restrict to this key range.

Parameters

• query – A datastore.Query instance.

• filters – optional list of filters to apply to the query. Each filter is a tuple: (<property_name_as_str>, <query_operation_as_str>, <value>). User filters are applied first.

Returns

The input query restricted to this key range.

filter_ndb_query(query, filters=None)source

Add query filter to restrict to this key range.

Parameters

• query – An ndb.Query instance.

• filters – optional list of filters to apply to the query. Each filter is a tuple: (<property_name_as_str>, <query_operation_as_str>, <value>). User filters are applied first.

Returns

The input query restricted to this key range.

filter_query(query, filters=None)source

Add query filter to restrict to this key range.

Parameters

• query – A db.Query or ndb.Query instance.

• filters – optional list of filters to apply to the query. Each filter is a tuple: (<property_name_as_str>, <query_operation_as_str>, <value>). User filters are applied first.

Returns

The input query restricted to this key range.

static from_json(json_str)source

Deserialize KeyRange from its json representation.

Parameters

json_str – string with json representation created by key_range_to_json.

Returns

deserialized KeyRange instance.

static guess_end_key(kind, key_start, probe_count=30, split_rate=5)source

Guess the end of a key range with a binary search of probe queries.

When the ‘key_start’ parameter has a key hierarchy, this function will only determine the key range for keys in a similar hierarchy. That means if the keys are in the form:

kind=Foo, name=bar/kind=Stuff, name=meep

only this range will be probed:

kind=Foo, name=*/kind=Stuff, name=*

That means other entities of kind ‘Stuff’ that are children of another parent entity kind will be skipped:

kind=Other, name=cookie/kind=Stuff, name=meep

Parameters

• key_start – The starting key of the search range. In most cases this should be id = 0 or name = ‘’. May be db.Key or ndb.Key.

• kind – String name of the entity kind.

• probe_count – Optional, how many probe queries to run.

• split_rate – Exponential rate to use for splitting the range on the way down from the full key space. For smaller ranges this should be higher so more of the keyspace is skipped on initial descent.

Returns

db.Key that is guaranteed to be as high or higher than the highest key existing for this Kind. Doing a query between ‘key_start’ and this returned Key (inclusive) will contain all entities of this Kind.

NOTE: Even though an ndb.Key instance is accepted as argument, the return value is always a db.Key instance.

make_ascending_datastore_query(kind, keys_only=False, filters=None)source

Construct a query for this key range without setting the scan direction.

Parameters

• kind – A string.

• keys_only – bool, default False, use keys_only on Query?

• filters – optional list of filters to apply to the query. Each filter is a tuple: (<property_name_as_str>, <query_operation_as_str>, <value>). User filters are applied first.

Returns

A datastore.Query instance.

make_ascending_ndb_query(kind_class, keys_only=False, filters=None)source

Construct an NDB query for this key range, without the scan direction.

Parameters

• kind_class – An ndb.Model subclass.

• keys_only – bool, default False, query only for keys.

Returns

An ndb.Query instance.

make_ascending_query(kind_class, keys_only=False, filters=None)source

Construct a query for this key range without setting the scan direction.

Parameters

• kind_class – A kind implementation class (a subclass of either db.Model or ndb.Model).

• keys_only – bool, default False, query only for keys.

• filters – optional list of filters to apply to the query. Each filter is a tuple: (<property_name_as_str>, <query_operation_as_str>, <value>). User filters are applied first.

Returns

A db.Query or ndb.Query instance (corresponding to kind_class).

make_directed_datastore_query(kind, keys_only=False)source

Construct a query for this key range, including the scan direction.

Parameters

• kind – A string.

• keys_only – bool, default False, use keys_only on Query?

Returns

A datastore.Query instance.

Raises

KeyRangeError – if self.direction is not in (KeyRange.ASC, KeyRange.DESC).

make_directed_ndb_query(kind_class, keys_only=False)source

Construct an NDB query for this key range, including the scan direction.

Parameters

• kind_class – An ndb.Model subclass.

• keys_only – bool, default False, use keys_only on Query?

Returns

An ndb.Query instance.

Raises

KeyRangeError – if self.direction is not in (KeyRange.ASC, KeyRange.DESC).

make_directed_query(kind_class, keys_only=False)source

Construct a query for this key range, including the scan direction.

Parameters

• kind_class – A kind implementation class (a subclass of either db.Model or ndb.Model).

• keys_only – bool, default False, use keys_only on Query?

Returns

A db.Query or ndb.Query instance (corresponding to kind_class).

Raises

KeyRangeError – if self.direction is not in (KeyRange.ASC, KeyRange.DESC).

static split_keys(key_start, key_end, batch_size)source

Return a key that is between key_start and key_end inclusive.

This method compares components of the ancestor paths of key_start and key_end. The first place in the path that differs is approximately split in half. If the kind components differ, a new non-existent kind halfway between the two is used to split the space. If the id_or_name components differ, then a new id_or_name that is halfway between the two is selected. If the lower id_or_name is numeric and the upper id_or_name is a string, then the minumum string key u’’ is used as the split id_or_name. The key that is returned is the shared portion of the ancestor path followed by the generated split component.

Parameters

• key_start – A db.Key or ndb.Key instance for the lower end of a range.

• key_end – A db.Key or ndb.Key instance for the upper end of a range.

• batch_size – The maximum size of a range that should not be split.

Returns

A db.Key instance, k, such that key_start <= k <= key_end.

NOTE: Even though ndb.Key instances are accepted as arguments, the return value is always a db.Key instance.

split_range(batch_size=0)source

Split this key range into a list of at most two ranges.

This method attempts to split the key range approximately in half. Numeric ranges are split in the middle into two equal ranges and string ranges are split lexicographically in the middle. If the key range is smaller than batch_size it is left unsplit.

Note that splitting is done without knowledge of the distribution of actual entities in the key range, so there is no guarantee (nor any particular reason to believe) that the entities of the range are evenly split.

Parameters

batch_size – The maximum size of a key range that should not be split.

Returns

A list of one or two key ranges covering the same space as this range.

to_json()source

Serialize KeyRange to json.

Returns

string with KeyRange json representation.

exception google.appengine.ext.key_range.KeyRangeErrorsource

Bases: google.appengine.ext.key_range.Error

Error while trying to generate a KeyRange.

exception google.appengine.ext.key_range.SimplejsonUnavailableErrorsource

Bases: google.appengine.ext.key_range.Error

Error using json functionality with unavailable json and simplejson.