Notice: Over the next few months, we're reorganizing the App Engine documentation site to make it easier to find content and better align with the rest of Google Cloud products. The same content will be available, but the navigation will now match the rest of the Cloud products. If you have feedback or questions as you navigate the site, click Send Feedback.

Python 2 is no longer supported by the community. We recommend that you migrate Python 2 apps to Python 3.
Stay organized with collections Save and categorize content based on your preferences.

google.appengine.ext.bulkload.bulkloader_parser module

Summary

Bulkloader Config Parser and runner.

A library to read bulkloader yaml configs. Returns a BulkloaderEntry object which describes the bulkloader.yaml in object form, including some additional parsing of things like Python lambdas.

Contents

class google.appengine.ext.bulkload.bulkloader_parser.BulkloaderEntry(**attributes)source

Bases: google.appengine.api.validation.Validated

Root of the bulkloader configuration.

ATTRIBUTES = {'transformers': <google.appengine.api.validation.Repeated object>, 'python_preamble': <google.appengine.api.validation.Optional object>}
class google.appengine.ext.bulkload.bulkloader_parser.ConnectorOptions(**attributes)source

Bases: google.appengine.api.validation.Validated

Connector options.

ATTRIBUTES = {'xpath_to_nodes': <google.appengine.api.validation.Optional object>, 'template': <google.appengine.api.validation.Optional object>, 'encoding': <google.appengine.api.validation.Optional object>, 'skip_import_header_row': <google.appengine.api.validation.Optional object>, 'epilog': <google.appengine.api.validation.Optional object>, 'style': <google.appengine.api.validation.Optional object>, 'import_options': <google.appengine.api.validation.Optional object>, 'print_export_header_row': <google.appengine.api.validation.Optional object>, 'mode': <google.appengine.api.validation.Optional object>, 'column_list': <google.appengine.api.validation.Optional object>, 'export_options': <google.appengine.api.validation.Optional object>, 'columns': <google.appengine.api.validation.Optional object>, 'prolog': <google.appengine.api.validation.Optional object>}
CheckInitialized()source

Post-loading ‘validation’. Really used to fix up yaml hackyness.

class google.appengine.ext.bulkload.bulkloader_parser.ConnectorSubOptions(**attributes)source

Bases: google.appengine.api.validation.Validated

Connector options.

ATTRIBUTES = {'dialect': <google.appengine.api.validation.Optional object>, 'delimiter': <google.appengine.api.validation.Optional object>}
class google.appengine.ext.bulkload.bulkloader_parser.EvaluatedCallablesource

Bases: google.appengine.api.validation.Validator

Validates that a string evaluates to a Python callable.

Calls eval at validation time and stores the results as a ParsedMethod object. The ParsedMethod object can be used as a string (original value) or callable (parsed method). It also exposes supports_bulkload_state if the callable has a kwarg called ‘bulkload_state’, which is used to determine how to call the *_transform methods.

class ParsedMethod(value, key)source

Bases: object

Wrap the string, the eval’d method, and supports_bulkload_state.

ToValue(value)source

Returns the code string for this value.

Validate(value, key)source

Validates that the string compiles as a Python callable.

Parameters
  • value – String to compile as a regular expression.

  • key – The YAML field name.

Returns

Value wrapped in an object with properties ‘value’ and ‘fn’.

Raises

InvalidCodeInConfiguration when value does not compile.

class google.appengine.ext.bulkload.bulkloader_parser.ExportEntry(**attributes)source

Bases: google.appengine.api.validation.Validated

Describes the optional export transform for a single property.

ATTRIBUTES = {'external_name': <google.appengine.api.validation.Optional object>, 'export_transform': <google.appengine.api.validation.Optional object>}
class google.appengine.ext.bulkload.bulkloader_parser.PropertyEntry(**attributes)source

Bases: google.appengine.api.validation.Validated

Describes the transform for a single property.

ATTRIBUTES = {'default_value': <google.appengine.api.validation.Optional object>, 'export': <google.appengine.api.validation.Optional object>, 'import_template': <google.appengine.api.validation.Optional object>, 'external_name': <google.appengine.api.validation.Optional object>, 'import_transform': <google.appengine.api.validation.Optional object>, 'property': <google.appengine.api.validation.Type object>, 'export_transform': <google.appengine.api.validation.Optional object>}
CheckInitialized()source

Check that all required (combinations) of fields are set.

Also fills in computed properties.

Raises

InvalidConfiguration – If the config is invalid.

class google.appengine.ext.bulkload.bulkloader_parser.PythonPreambleEntry(**attributes)source

Bases: google.appengine.api.validation.Validated

Python modules to import at initialization time, typically models.

ATTRIBUTES = {'import': <google.appengine.api.validation.Type object>, 'as': <google.appengine.api.validation.Optional object>}
CheckInitialized()source

Check that all required fields are set, and update global state.

The imports specified in the preamble are imported at this time.

class google.appengine.ext.bulkload.bulkloader_parser.TransformerEntry(**attributes)source

Bases: google.appengine.api.validation.Validated

Describes the transform for an entity (or model) kind.

ATTRIBUTES = {'connector': <google.appengine.api.validation.Type object>, 'kind': <google.appengine.api.validation.Optional object>, 'sort_key_from_entity': <google.appengine.api.validation.Optional object>, 'name': <google.appengine.api.validation.Optional object>, 'connector_options': <google.appengine.api.validation.Optional object>, 'property_map': <google.appengine.api.validation.Repeated object>, 'model': <google.appengine.api.validation.Optional object>, 'post_export_function': <google.appengine.api.validation.Optional object>, 'post_import_function': <google.appengine.api.validation.Optional object>, 'use_model_on_export': <google.appengine.api.validation.Optional object>}
CheckInitialized()source

Check that all required (combinations) of fields are set.

Also fills in computed properties.

Raises

InvalidConfiguration – if the config is invalid.

google.appengine.ext.bulkload.bulkloader_parser.load_config(stream, config_globals)source

Load a configuration file and generate importer and exporter classes.

Parameters
  • stream – Stream containing config YAML.

  • config_globals – Dict to use to reference globals for code in the config.

Returns

BulkloaderEntry

Raises

InvalidConfiguration – If the config is invalid.