BlockingDataflowPipelineRunner (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

Warning: Dataflow SDK 1.x for Java is unsupported as of October 16, 2018. After August 12, 2020, you can use unsupported SDKs temporarily while you upgrade your pipelines. For more information, see Using an unsupported SDK.

For information on migrating from the 1.x SDK, see Migrating from Dataflow SDK 1.x for Java.

Note: If you are using Cloud Dataflow SDK version 2.x, see the Apache Beam API reference documentation.

com.google.cloud.dataflow.sdk.runners

Class BlockingDataflowPipelineRunner

• java.lang.Object
• • com.google.cloud.dataflow.sdk.runners.PipelineRunner<DataflowPipelineJob>
  • • com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner

• ---

  
  

  public class BlockingDataflowPipelineRunner
  extends PipelineRunner<DataflowPipelineJob>

  A PipelineRunner that's like DataflowPipelineRunner but that waits for the launched job to finish.

  Prints out job status updates and console messages while it waits.

  Returns the final job state, or throws an exception if the job fails or cannot be monitored.

  Permissions

  When reading from a Dataflow source or writing to a Dataflow sink using BlockingDataflowPipelineRunner, the Google cloud services account and the Google compute engine service account of the GCP project running the Dataflow Job will need access to the corresponding source/sink.

  Please see Google Cloud Dataflow Security and Permissions for more details.

• • Constructor Summary

    Constructors 

    Modifier	Constructor and Description
    protected	BlockingDataflowPipelineRunner(DataflowPipelineRunner internalRunner, BlockingDataflowPipelineOptions options)

  • Method Summary

    All Methods Static Methods Instance Methods Concrete Methods 

    Modifier and Type	Method and Description
    <OutputT extends POutput,InputT extends PInput> OutputT	apply(PTransform<InputT,OutputT> transform, InputT input) Applies a transform to the given input, returning the output.
    static BlockingDataflowPipelineRunner	fromOptions(PipelineOptions options) Constructs a runner from the provided options.
    DataflowPipelineJob	run(Pipeline p) Processes the given Pipeline, returning the results.
    void	setHooks(DataflowPipelineRunnerHooks hooks) Sets callbacks to invoke during execution.
    String	toString()

    • Methods inherited from class java.lang.Object

      clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

• • Constructor Detail

    • BlockingDataflowPipelineRunner

      protected BlockingDataflowPipelineRunner(DataflowPipelineRunner internalRunner,
                                               BlockingDataflowPipelineOptions options)

  • Method Detail

    • fromOptions

      public static BlockingDataflowPipelineRunner fromOptions(PipelineOptions options)

      Constructs a runner from the provided options.

    • run

      public DataflowPipelineJob run(Pipeline p)

      Processes the given Pipeline, returning the results.

      Specified by:

      run in class PipelineRunner<DataflowPipelineJob>

      Throws:

      DataflowJobExecutionException - if there is an exception during job execution.

      DataflowServiceException - if there is an exception retrieving information about the job.

    • apply

      public <OutputT extends POutput,InputT extends PInput> OutputT apply(PTransform<InputT,OutputT> transform,
                                                                           InputT input)

      Description copied from class: PipelineRunner
      Applies a transform to the given input, returning the output.

      The default implementation calls PTransform.apply(input), but can be overridden to customize behavior for a particular runner.

      Overrides:

      apply in class PipelineRunner<DataflowPipelineJob>

    • setHooks

      @Experimental
      public void setHooks(DataflowPipelineRunnerHooks hooks)

      Sets callbacks to invoke during execution. See DataflowPipelineRunnerHooks.

    • toString

      public String toString()

      Overrides:

      toString in class Object