Notice: Over the next few months, we're reorganizing the App Engine documentation site to make it easier to find content and better align with the rest of Google Cloud products. The same content will be available, but the navigation will now match the rest of the Cloud products. If you have feedback or questions as you navigate the site, click Send Feedback.

Python 2 is no longer supported by the community. We recommend that you migrate Python 2 apps to Python 3.
Stay organized with collections Save and categorize content based on your preferences.

google.appengine.ext.bulkload.connector_interface module

Summary

Bulkloader interfaces for the format reader/writers.

Contents

class google.appengine.ext.bulkload.connector_interface.ConnectorInterfacesource

Bases: object

Abstract base class describing the external Connector interface.

The External Connector interface describes the transition between an external data source, e.g. CSV file, XML file, or some sort of database interface, and the intermediate bulkloader format, which is a dictionary or similar structure representing the external transformation of the data.

On import, the generate_import_record generator is the only method called.

On export, the initialize_export method is called once, followed by a call to write_dict for each record, followed by a call to finalize_export.

The bulkload_state is a BulkloadState object from google.appengine.ext.bulkload.bulkload_config. The interesting properties to a Connector object are the loader_opts and exporter_opts, which are strings passed in from the bulkloader command line.

finalize_export()source

Performs finalization actions after every record is written.

generate_import_record(filename, bulkload_state)source

A function which returns an iterator over dictionaries.

This is the only method used on import.

Parameters

  • filename – The –filename argument passed in on the bulkloader command line. This value is opaque to the bulkloader and thus could specify any sort of descriptor for your generator.

  • bulkload_state – Passed in BulkloadConfig.BulkloadState object.

Returns

An iterator describing an individual record. Typically a dictionary, to be used with dict_to_model. Typically implemented as a generator.

initialize_export(filename, bulkload_state)source

Initialize the output file.

Parameters

  • filename – The string given as the –filename flag argument.

  • bulkload_state – Passed in BulkloadConfig.BulkloadState object.

These values are opaque to the bulkloader and thus could specify any sort of descriptor for your exporter.

write_dict(dictionary)source

Write one record for the specified entity.

Parameters

dictionary – A post-transform dictionary.

google.appengine.ext.bulkload.connector_interface.create_from_options(options, name='unknown')source

Factory using an options dictionary.

This is frequently implemented as the constructor on the connector class, or a static or class method on the connector class.

Parameters

  • options – Parsed dictionary from yaml file, interpretation is up to the implementor of this class.

  • name – Identifier of this transform to be used in error messages.

Returns

An object which implements the ConnectorInterface interface.