com.google.cloud.dataflow.sdk.runners (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

Interface Summary
Interface	Description
DataflowPipelineTranslator.TransformTranslator<TransformT extends PTransform>	A `DataflowPipelineTranslator.TransformTranslator` knows how to translate a particular subclass of `PTransform` for the Cloud Dataflow service.
DataflowPipelineTranslator.TranslationContext	The interface provided to registered callbacks for interacting with the `DataflowPipelineRunner`, including reading and writing the values of `PCollection`s and side inputs (`PCollectionView`s).
DirectPipelineRunner.EvaluationContext	The interface provided to registered callbacks for interacting with the `DirectPipelineRunner`, including reading and writing the values of `PCollection`s and `PCollectionView`s.
DirectPipelineRunner.EvaluationResults	The interface provided to registered callbacks for interacting with the `DirectPipelineRunner`, including reading and writing the values of `PCollection`s and `PCollectionView`s.
DirectPipelineRunner.TransformEvaluator<TransformT extends PTransform>	An evaluator of a PTransform.
PipelineRunnerRegistrar	`PipelineRunner` creators have the ability to automatically have their `PipelineRunner` registered with this SDK by creating a `ServiceLoader` entry and a concrete implementation of this interface.

Class Summary
Class	Description
AggregatorPipelineExtractor	Retrieves `Aggregators` at each `ParDo` and returns a `Map` of `Aggregator` to the `PTransforms` in which it is present.
AggregatorValues<T>	A collection of values associated with an `Aggregator`.
BlockingDataflowPipelineRunner	A `PipelineRunner` that's like `DataflowPipelineRunner` but that waits for the launched job to finish.
DataflowPipeline	A `DataflowPipeline` is a `Pipeline` that returns a `DataflowPipelineJob` when it is `Pipeline.run()`.
DataflowPipelineJob	A DataflowPipelineJob represents a job submitted to Dataflow using `DataflowPipelineRunner`.
DataflowPipelineRegistrar	Contains the `PipelineOptionsRegistrar` and `PipelineRunnerRegistrar` for the `DataflowPipeline`.
DataflowPipelineRegistrar.Options	Register the `DataflowPipelineOptions` and `BlockingDataflowPipelineOptions`.
DataflowPipelineRegistrar.Runner	Register the `DataflowPipelineRunner` and `BlockingDataflowPipelineRunner`.
DataflowPipelineRunner	A `PipelineRunner` that executes the operations in the pipeline by first translating them to the Dataflow representation using the `DataflowPipelineTranslator` and then submitting them to a Dataflow service for execution.
DataflowPipelineRunner.BatchBigQueryIONativeReadTranslator	Implements BigQueryIO Read translation for the Dataflow backend.
DataflowPipelineRunnerHooks	An instance of this class can be passed to the `DataflowPipelineRunner` to add user defined hooks to be invoked at various times during pipeline execution.
DataflowPipelineTranslator	`DataflowPipelineTranslator` knows how to translate `Pipeline` objects into Cloud Dataflow Service API `Job`s.
DataflowPipelineTranslator.JobSpecification	The result of a job translation.
DirectPipeline	A `DirectPipeline` is a `Pipeline` that returns `DirectPipelineRunner.EvaluationResults` when it is `Pipeline.run()`.
DirectPipelineRegistrar	Contains the `PipelineOptionsRegistrar` and `PipelineRunnerRegistrar` for the `DirectPipeline`.
DirectPipelineRegistrar.Options	Register the `DirectPipelineOptions`.
DirectPipelineRegistrar.Runner	Register the `DirectPipelineRunner`.
DirectPipelineRunner	Executes the operations in the pipeline directly, in this process, without any optimization.
DirectPipelineRunner.TestCombineDoFn<K,InputT,AccumT,OutputT>	The implementation may split the `Combine.KeyedCombineFn` into ADD, MERGE and EXTRACT phases ( see `com.google.cloud.dataflow.sdk.runners.worker.CombineValuesFn`).
DirectPipelineRunner.ValueWithMetadata<V>	An immutable (value, timestamp) pair, along with other metadata necessary for the implementation of `DirectPipelineRunner`.
PipelineRunner<ResultT extends PipelineResult>	A `PipelineRunner` can execute, translate, or otherwise process a `Pipeline`.
RecordingPipelineVisitor	Provides a simple `Pipeline.PipelineVisitor` that records the transformation tree.
TemplatingDataflowPipelineRunner	A `PipelineRunner` that's like `DataflowPipelineRunner` but only stores a template of a job.
TemplatingDataflowPipelineRunner.Runner	Register the `TemplatingDataflowPipelineRunner`.
TransformHierarchy	Captures information about a collection of transformations and their associated `PValue`s.
TransformTreeNode	Provides internal tracking of transform relationships with helper methods for initialization and ordered visitation.

Exception Summary
Exception	Description
AggregatorRetrievalException	Signals that an exception has occurred while retrieving `Aggregator`s.
DataflowJobAlreadyExistsException	An exception that is thrown if the unique job name constraint of the Dataflow service is broken because an existing job with the same job name is currently active.
DataflowJobAlreadyUpdatedException	An exception that is thrown if the existing job has already been updated within the Dataflow service and is no longer able to be updated.
DataflowJobCancelledException	Signals that a job run by a `BlockingDataflowPipelineRunner` was updated during execution.
DataflowJobException	A `RuntimeException` that contains information about a `DataflowPipelineJob`.
DataflowJobExecutionException	Signals that a job run by a `BlockingDataflowPipelineRunner` fails during execution, and provides access to the failed job.
DataflowJobUpdatedException	Signals that a job run by a `BlockingDataflowPipelineRunner` was updated during execution.
DataflowServiceException	Signals there was an error retrieving information about a job from the Cloud Dataflow Service.

Package com.google.cloud.dataflow.sdk.runners Description

Defines runners for executing Pipelines in different modes, including DirectPipelineRunner and DataflowPipelineRunner.

DirectPipelineRunner executes a Pipeline locally, without contacting the Dataflow service. DataflowPipelineRunner submits a Pipeline to the Dataflow service, which executes it on Dataflow-managed Compute Engine instances. DataflowPipelineRunner returns as soon as the Pipeline has been submitted. Use BlockingDataflowPipelineRunner to have execution updates printed to the console.

The runner is specified as part PipelineOptions.