Google Cloud Dataflow SDK for Java, version 1.9.1
com.google.cloud.dataflow.sdk.runners
Class BlockingDataflowPipelineRunner
- java.lang.Object
-
- com.google.cloud.dataflow.sdk.runners.PipelineRunner<DataflowPipelineJob>
-
- com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner
-
public class BlockingDataflowPipelineRunner extends PipelineRunner<DataflowPipelineJob>
APipelineRunner
that's likeDataflowPipelineRunner
but that waits for the launched job to finish.Prints out job status updates and console messages while it waits.
Returns the final job state, or throws an exception if the job fails or cannot be monitored.
Permissions
When reading from a Dataflow source or writing to a Dataflow sink usingBlockingDataflowPipelineRunner
, the Google cloud services account and the Google compute engine service account of the GCP project running the Dataflow Job will need access to the corresponding source/sink.Please see Google Cloud Dataflow Security and Permissions for more details.
-
-
Constructor Summary
Constructors Modifier Constructor and Description protected
BlockingDataflowPipelineRunner(DataflowPipelineRunner internalRunner, BlockingDataflowPipelineOptions options)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method and Description <OutputT extends POutput,InputT extends PInput>
OutputTapply(PTransform<InputT,OutputT> transform, InputT input)
Applies a transform to the given input, returning the output.static BlockingDataflowPipelineRunner
fromOptions(PipelineOptions options)
Constructs a runner from the provided options.DataflowPipelineJob
run(Pipeline p)
Processes the given Pipeline, returning the results.void
setHooks(DataflowPipelineRunnerHooks hooks)
Sets callbacks to invoke during execution.String
toString()
-
-
-
Constructor Detail
-
BlockingDataflowPipelineRunner
protected BlockingDataflowPipelineRunner(DataflowPipelineRunner internalRunner, BlockingDataflowPipelineOptions options)
-
-
Method Detail
-
fromOptions
public static BlockingDataflowPipelineRunner fromOptions(PipelineOptions options)
Constructs a runner from the provided options.
-
run
public DataflowPipelineJob run(Pipeline p)
Processes the given Pipeline, returning the results.- Specified by:
run
in classPipelineRunner<DataflowPipelineJob>
- Throws:
DataflowJobExecutionException
- if there is an exception during job execution.DataflowServiceException
- if there is an exception retrieving information about the job.
-
apply
public <OutputT extends POutput,InputT extends PInput> OutputT apply(PTransform<InputT,OutputT> transform, InputT input)
Description copied from class:PipelineRunner
Applies a transform to the given input, returning the output.The default implementation calls PTransform.apply(input), but can be overridden to customize behavior for a particular runner.
- Overrides:
apply
in classPipelineRunner<DataflowPipelineJob>
-
setHooks
@Experimental public void setHooks(DataflowPipelineRunnerHooks hooks)
Sets callbacks to invoke during execution. SeeDataflowPipelineRunnerHooks
.
-
-