DataflowPipelineTranslator (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

Warning: Dataflow SDK 1.x for Java is unsupported as of October 16, 2018. After August 12, 2020, Dataflow will not run jobs using Dataflow 1.x and below. See Migrating from Dataflow SDK 1.x for Java for migration guidance.

Note: If you are using Cloud Dataflow SDK version 2.x, see the Apache Beam API reference documentation.

com.google.cloud.dataflow.sdk.runners

Class DataflowPipelineTranslator

• java.lang.Object
• • com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator

• ---

  
  

  public class DataflowPipelineTranslator
  extends Object

  DataflowPipelineTranslator knows how to translate Pipeline objects into Cloud Dataflow Service API Jobs.

• • Nested Class Summary

    Nested Classes 

    Modifier and Type	Class and Description
    static class	DataflowPipelineTranslator.JobSpecification The result of a job translation.
    static interface	DataflowPipelineTranslator.TransformTranslator<TransformT extends PTransform> A DataflowPipelineTranslator.TransformTranslator knows how to translate a particular subclass of PTransform for the Cloud Dataflow service.
    static interface	DataflowPipelineTranslator.TranslationContext The interface provided to registered callbacks for interacting with the DataflowPipelineRunner, including reading and writing the values of PCollections and side inputs (PCollectionViews).

  • Method Summary

    All Methods Static Methods Instance Methods Concrete Methods 

    Modifier and Type	Method and Description
    static DataflowPipelineTranslator	fromOptions(DataflowPipelineOptions options) Constructs a translator from the provided options.
    <TransformT extends PTransform> DataflowPipelineTranslator.TransformTranslator<TransformT>	getTransformTranslator(Class<TransformT> transformClass) Returns the DataflowPipelineTranslator.TransformTranslator to use for instances of the specified PTransform class, or null if none registered.
    static String	jobToString(com.google.api.services.dataflow.model.Job job) Renders a Job as a string.
    static <TransformT extends PTransform> void	registerTransformTranslator(Class<TransformT> transformClass, DataflowPipelineTranslator.TransformTranslator<? extends TransformT> transformTranslator) Records that instances of the specified PTransform class should be translated by default by the corresponding DataflowPipelineTranslator.TransformTranslator.
    String	toString()
    DataflowPipelineTranslator.JobSpecification	translate(Pipeline pipeline, DataflowPipelineRunner runner, List<com.google.api.services.dataflow.model.DataflowPackage> packages) Translates a Pipeline into a JobSpecification.

    • Methods inherited from class java.lang.Object

      clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

• • Method Detail

    • fromOptions

      public static DataflowPipelineTranslator fromOptions(DataflowPipelineOptions options)

      Constructs a translator from the provided options.

      Parameters:

      options - Properties that configure the translator.

      Returns:

      The newly created translator.

    • translate

      public DataflowPipelineTranslator.JobSpecification translate(Pipeline pipeline,
                                                                   DataflowPipelineRunner runner,
                                                                   List<com.google.api.services.dataflow.model.DataflowPackage> packages)

      Translates a Pipeline into a JobSpecification.

    • jobToString

      public static String jobToString(com.google.api.services.dataflow.model.Job job)

      Renders a Job as a string.

    • registerTransformTranslator

      public static <TransformT extends PTransform> void registerTransformTranslator(Class<TransformT> transformClass,
                                                                                     DataflowPipelineTranslator.TransformTranslator<? extends TransformT> transformTranslator)

      Records that instances of the specified PTransform class should be translated by default by the corresponding DataflowPipelineTranslator.TransformTranslator.

    • getTransformTranslator

      public <TransformT extends PTransform> DataflowPipelineTranslator.TransformTranslator<TransformT> getTransformTranslator(Class<TransformT> transformClass)

      Returns the DataflowPipelineTranslator.TransformTranslator to use for instances of the specified PTransform class, or null if none registered.

    • toString

      public String toString()

      Overrides:

      toString in class Object