google.appengine.ext.bulkload.transform module

Summary

Bulkloader Transform Helper functions.

A collection of helper functions for bulkloading data, typically referenced from a bulkloader.yaml file.

Contents

google.appengine.ext.bulkload.transform.blob_to_file(filename_hint_propertyname=None, directory_hint='')source

Write the blob contents to a file, and replace them with the filename.

Parameters
  • filename_hint_propertyname -- If present, the filename will begin with the contents of this value in the entity being exported.

  • directory_hint -- If present, the files will be stored in this directory.

Returns

A function which writes the input blob to a file.

google.appengine.ext.bulkload.transform.blobproperty_from_base64(value)source
google.appengine.ext.bulkload.transform.bytestring_from_base64(value)source
google.appengine.ext.bulkload.transform.child_node_from_list(child_node_name)source

Return a value suitable for generating an XML child node on export.

The return value is a list of tuples which the simplexml connector will use to build a child node.

See also list_from_child_node

Parameters

child_node_name -- The name to use for each child node.

Returns

Transform function which works as described in the args.

google.appengine.ext.bulkload.transform.create_deep_key(*path_info)source

A method to make multi-level Key objects.

Generates multi-level key from multiple fields in the input dictionary.

This is typically used for Keys for entities which have variable parent keys, e.g. ones with owned relationships. It can used for both __key__ and references.

Use create_foreign_key as a simpler way to create single level keys.

Parameters

path_info --

List of tuples, describing (kind, property, is_id=False). kind: The kind name. property: The external property in the current import dictionary, or

transform.CURRENT_PROPERTY for the value passed to the transform.

is_id: Converts value to int and treats as numeric ID if True, otherwise

the value is a string name. Default is False.

Example
create_deep_key(('rootkind', 'rootcolumn'),

('childkind', 'childcolumn', True), ('leafkind', transform.CURRENT_PROPERTY))

Returns

Transform method which parses the info from the current neutral dictionary into a Key with parents as described by path_info.

google.appengine.ext.bulkload.transform.create_foreign_key(kind, key_is_id=False)source

A method to make one-level Key objects.

These are typically used in ReferenceProperty in Python, where the reference value is a key with kind (or model) name name.

This helper method does not support keys with parents. Use create_deep_key instead to create keys with parents.

Parameters
  • kind -- The kind name of the reference as a string.

  • key_is_id -- If true, convert the key into an integer to be used as an id. If false, leave the key in the input format (typically a string).

Returns

Single argument method which parses a value into a Key of kind entity_kind.

google.appengine.ext.bulkload.transform.empty_if_none(fn)source

A wrapper for a value to return '' if it's None. Useful on export.

Can be used in config files (e.g. "transform.empty_if_none(unicode)" or as a decorator.

Parameters

fn -- Single argument transform function.

Returns

Wrapped function.

google.appengine.ext.bulkload.transform.export_date_time(format)source

A wrapper around strftime. Also returns '' if the input is None.

Parameters

format -- Format string for strftime.

Returns

Single argument method which convers a datetime into a string using format.

google.appengine.ext.bulkload.transform.import_date_time(format, _strptime=None)source

A wrapper around strptime. Also returns None if the input is empty.

Parameters

format -- Format string for strptime.

Returns

Single argument method which parses a string into a datetime using format.

google.appengine.ext.bulkload.transform.join_list(delimeter)source

Join a list into a string using the delimeter.

This is just a wrapper for string.join.

Parameters

delimeter -- The delimiter to use when joining the string.

Returns

Method which joins the list into a string with the delimeter.

google.appengine.ext.bulkload.transform.key_id_or_name_as_string(key)source
google.appengine.ext.bulkload.transform.key_id_or_name_as_string_n(index)source

Pull out the nth (0-based) key id or name from a key which has parents.

If a key is present, return its id or name as a string.

Note that this loses the distinction between integer IDs and strings which happen to look like integers. Use key_type to distinguish them.

This is a useful complement to create_deep_key.

Parameters

index -- The depth of the id or name to extract. Zero is the root key. Negative one is the leaf key.

Returns

Function extracting the name or ID of the key at depth index, as a unicode string. Returns '' if key is empty (unsaved), otherwise raises IndexError if the key is not as deep as described.

google.appengine.ext.bulkload.transform.key_kind(value)source
google.appengine.ext.bulkload.transform.key_kind_n(index)source

Pull out the nth (0-based) key kind from a key which has parents.

This is a useful complement to create_deep_key.

Parameters

index -- The depth of the id or name to extract. Zero is the root key. Negative one is the leaf key.

Returns

Function returning the kind of the key at depth index, or raising IndexError if the key is not as deep as described.

google.appengine.ext.bulkload.transform.key_type(key)source
google.appengine.ext.bulkload.transform.key_type_n(index)source

Pull out the nth (0-based) key type from a key which has parents.

This is most useful when paired with key_id_or_name_as_string_n. This is a useful complement to create_deep_key.

Parameters

index -- The depth of the id or name to extract. Zero is the root key. Negative one is the leaf key.

Returns

Method returning the type ('ID' or 'name') of the key at depth index. Returns '' if key is empty (unsaved), otherwise raises IndexError if the key is not as deep as described.

google.appengine.ext.bulkload.transform.list_from_child_node(xpath, suppress_blank=False)source

Return a list property from child nodes of the current xml node.

This applies only the simplexml helper, as it assumes __node__, the current ElementTree node corresponding to the import record.

Sample usage for structure:
<Visit>
<VisitActivities>

<Activity>A1</Activity> <Activity>A2</Activity>

</VisitActivities>

</Visit>

property: activities external_name: VisitActivities # Ignored on import, used on export. import_transform: list_from_xml_node('VisitActivities/Activity') export_transform: child_node_from_list('Activity')

Parameters
  • xpath -- XPath to run on the current node.

  • suppress_blank -- if True, ndoes with no text will be skipped.

Returns

Transform function which works as described in the args.

google.appengine.ext.bulkload.transform.list_from_multiproperty(*external_names)source

Create a list from multiple properties.

Parameters

external_names -- List of the properties to use.

Returns

Transform function which returns a list of the properties in external_names.

google.appengine.ext.bulkload.transform.none_if_empty(fn)source

A decorator which returns None if its input is empty else fn(x).

Useful on import. Can be used in config files (e.g. "transform.none_if_empty(int)" or as a decorator.

Parameters

fn -- Single argument transform function.

Returns

Wrapped function.

google.appengine.ext.bulkload.transform.property_from_list(index)source

Return the Nth item from a list, or '' if the list is shorter.

Parameters

index -- Item in the list to return.

Returns

Function returning the item from a list, or '' if the list is too short.

google.appengine.ext.bulkload.transform.regexp_bool(regexp, flags=0)source

Return a boolean if the expression matches with re.match.

Note that re.match anchors at the start but not end of the string.

Parameters
  • regexp -- String, regular expression.

  • flags -- Optional flags to pass to re.match.

Returns

Method which returns a Boolean if the expression matches.

google.appengine.ext.bulkload.transform.regexp_extract(pattern, method=match, group=1)source

Return first group in the value matching the pattern using re.match.

Parameters
  • pattern -- A regular expression to match on with at least one group.

  • method -- The method to use for matching; normally re.match or re.search.

  • group -- The group to use for extracting a value.

Returns

A single argument method which returns the group_arg group matched, or None if no match was found or the input was empty.

google.appengine.ext.bulkload.transform.regexp_to_list(pattern)source

Return function that returns a list of objects that match the regex.

Useful on import. Uses the provided regex to split a string value into a list of strings. Wrapped by none_if_input_or_result_empty, so returns none if there are no matches for the regex and none if the input is empty.

Parameters

pattern -- A regular expression pattern to match against the input string.

Returns

None if the input was none or no matches were found, otherwise a list of strings matching the input expression.

google.appengine.ext.bulkload.transform.split_string(delimeter)source

Split a string using the delimeter into a list.

This is just a wrapper for string.split.

Parameters

delimeter -- The delimiter to split the string on.

Returns

Method which splits the string into a list along the delimeter.

Send feedback about...

App Engine standard environment for Python