google.appengine.api.files.shuffler module

Summary

Files API.

Deprecated since version 1.8.1: Use Google Cloud Storage Client library instead.

Files API Shuffler interface

Contents

google.appengine.api.files.shuffler.shuffle(job_name, input_file_list, output_file_list, callback)source

Shuffle mapreduce files using the shuffler service.

Parameters

  • job_name – unique shuffle job name as string.

  • input_file_list – list of files api file names to shuffle. Files should be in records format with serialized KeyValue protocol buffer as record.

  • output_file_list – list of files api file names to store shuffle result. Files should not be finalized. They will be of records format with serialized KeyValues protocol buffer as record.

  • callback –

    shuffle service call back specification. Can be either url - the task in default queue with default parameters will be enqueued. It can also be a dict with following keys:

    url: url to call back version: app version to call method: HTTP method to use (POST or GET) queue: queue name to enqueue a task in.

Raises

ShufflerUnavailableError if shuffler service is not available.

google.appengine.api.files.shuffler.available()source

Determine if shuffler service is available for the app.

Returns

True if shuffler service is available, False otherwise.