DataflowPipelineJob (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

Warning: Dataflow SDK 1.x for Java is unsupported as of October 16, 2018. After August 12, 2020, you can use unsupported SDKs temporarily while you upgrade your pipelines. For more information, see Using an unsupported SDK.

For information on migrating from the 1.x SDK, see Migrating from Dataflow SDK 1.x for Java.

Note: If you are using Cloud Dataflow SDK version 2.x, see the Apache Beam API reference documentation.

com.google.cloud.dataflow.sdk.runners

Class DataflowPipelineJob

• java.lang.Object
• • com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob

• All Implemented Interfaces:

  PipelineResult

  ---

  
  

  public class DataflowPipelineJob
  extends Object
  implements PipelineResult

  A DataflowPipelineJob represents a job submitted to Dataflow using DataflowPipelineRunner.

• • Nested Class Summary

    • Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.PipelineResult

      PipelineResult.State

  • Field Summary

    Fields 

    Modifier and Type	Field and Description
    protected static com.google.cloud.dataflow.sdk.util.FluentBackoff	STATUS_BACKOFF_FACTORY

  • Constructor Summary

    Constructors 

    Constructor and Description
    DataflowPipelineJob(String projectId, String jobId, com.google.api.services.dataflow.Dataflow dataflowClient, com.google.cloud.dataflow.sdk.runners.dataflow.DataflowAggregatorTransforms aggregatorTransforms) Constructs the job.

  • Method Summary

    All Methods Instance Methods Concrete Methods 

    Modifier and Type	Method and Description
    void	cancel() Cancels the job.
    <OutputT> AggregatorValues<OutputT>	getAggregatorValues(Aggregator<?,OutputT> aggregator) Retrieves the current value of the provided Aggregator.
    com.google.api.services.dataflow.Dataflow	getDataflowClient() Get the Cloud Dataflow API Client used by this job.
    String	getJobId() Get the id of this job.
    String	getProjectId() Get the project this job exists in.
    DataflowPipelineJob	getReplacedByJob() Returns a new DataflowPipelineJob for the job that replaced this one, if applicable.
    PipelineResult.State	getState() Retrieves the current state of the pipeline execution.
    PipelineResult.State	waitToFinish(long timeToWait, TimeUnit timeUnit, com.google.cloud.dataflow.sdk.util.MonitoringUtil.JobMessagesHandler messageHandler) Waits for the job to finish and return the final status.

    • Methods inherited from class java.lang.Object

      clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

• • Field Detail

    • STATUS_BACKOFF_FACTORY

      protected static final com.google.cloud.dataflow.sdk.util.FluentBackoff STATUS_BACKOFF_FACTORY

  • Constructor Detail

    • DataflowPipelineJob

      public DataflowPipelineJob(String projectId,
                                 String jobId,
                                 com.google.api.services.dataflow.Dataflow dataflowClient,
                                 com.google.cloud.dataflow.sdk.runners.dataflow.DataflowAggregatorTransforms aggregatorTransforms)

      Constructs the job.

      Parameters:

      projectId - the project id

      jobId - the job id

      dataflowClient - the client for the Dataflow Service

  • Method Detail

    • getJobId

      public String getJobId()

      Get the id of this job.

    • getProjectId

      public String getProjectId()

      Get the project this job exists in.

    • getReplacedByJob

      public DataflowPipelineJob getReplacedByJob()

      Returns a new DataflowPipelineJob for the job that replaced this one, if applicable.

      Throws:

      IllegalStateException - if called before the job has terminated or if the job terminated but was not updated

    • getDataflowClient

      public com.google.api.services.dataflow.Dataflow getDataflowClient()

      Get the Cloud Dataflow API Client used by this job.

    • waitToFinish

      @Nullable
      public PipelineResult.State waitToFinish(long timeToWait,
                                                         TimeUnit timeUnit,
                                                         com.google.cloud.dataflow.sdk.util.MonitoringUtil.JobMessagesHandler messageHandler)
                                                  throws IOException,
                                                         InterruptedException

      Waits for the job to finish and return the final status.

      Parameters:

      timeToWait - The time to wait in units timeUnit for the job to finish. Provide a value less than 1 ms for an infinite wait.

      timeUnit - The unit of time for timeToWait.

      messageHandler - If non null this handler will be invoked for each batch of messages received.

      Returns:

      The final state of the job or null on timeout or if the thread is interrupted.

      Throws:

      IOException - If there is a persistent problem getting job information.

      InterruptedException

    • cancel

      public void cancel()
                  throws IOException

      Cancels the job.

      Throws:

      IOException - if there is a problem executing the cancel request.

    • getState

      public PipelineResult.State getState()

      Description copied from interface: PipelineResult
      Retrieves the current state of the pipeline execution.

      Specified by:

      getState in interface PipelineResult

      Returns:

      the PipelineResult.State representing the state of this pipeline.

    • getAggregatorValues

      public <OutputT> AggregatorValues<OutputT> getAggregatorValues(Aggregator<?,OutputT> aggregator)
                                                              throws AggregatorRetrievalException

      Description copied from interface: PipelineResult
      Retrieves the current value of the provided Aggregator.

      Specified by:

      getAggregatorValues in interface PipelineResult

      Parameters:

      aggregator - the Aggregator to retrieve values for.

      Returns:

      the current values of the Aggregator, which may be empty if there are no values yet.

      Throws:

      AggregatorRetrievalException - if the Aggregator values could not be retrieved.