DataflowPipelineTranslator.TranslationContext (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

Warning: Dataflow SDK 1.x for Java is unsupported as of October 16, 2018. After August 12, 2020, Dataflow will not run jobs using Dataflow 1.x and below. See Migrating from Dataflow SDK 1.x for Java for migration guidance.

Note: If you are using Cloud Dataflow SDK version 2.x, see the Apache Beam API reference documentation.

com.google.cloud.dataflow.sdk.runners

Interface DataflowPipelineTranslator.TranslationContext

• Enclosing class:

  DataflowPipelineTranslator

  ---

  
  

  public static interface DataflowPipelineTranslator.TranslationContext

  The interface provided to registered callbacks for interacting with the DataflowPipelineRunner, including reading and writing the values of PCollections and side inputs (PCollectionViews).

• • Method Summary

    All Methods Instance Methods Abstract Methods 

    Modifier and Type	Method and Description
    void	addCollectionToSingletonOutput(String name, PValue inputValue, PValue outputValue) Adds an output with the given name to the previously added CollectionToSingleton Dataflow step, consuming the specified input PValue and producing the specified output PValue.
    void	addEncodingInput(Coder<?> value) Sets the encoding for the current Dataflow step.
    void	addInput(String name, Boolean value) Adds an input with the given name and value to the current Dataflow step.
    void	addInput(String name, List<? extends Map<String,Object>> elements) Adds an input that is a list of objects.
    void	addInput(String name, Long value) Adds an input with the given name and value to the current Dataflow step.
    void	addInput(String name, Map<String,Object> elements) Adds an input that is a dictionary of strings to objects.
    void	addInput(String name, PInput value) Adds an input with the given name to the previously added Dataflow step, coming from the specified input PValue.
    void	addInput(String name, String value) Adds an input with the given name and value to the current Dataflow step.
    void	addOutput(String name, PValue value) Adds an output with the given name to the previously added Dataflow step, producing the specified output PValue, including its Coder if a TypedPValue.
    void	addStep(PTransform<?,?> transform, String type) Adds a step to the Dataflow workflow for the given transform, with the given Dataflow step type.
    void	addStep(PTransform<?,? extends PValue> transform, com.google.api.services.dataflow.model.Step step) Adds a pre-defined step to the Dataflow workflow.
    void	addValueOnlyOutput(String name, PValue value) Adds an output with the given name to the previously added Dataflow step, producing the specified output PValue, including its Coder if a TypedPValue.
    com.google.cloud.dataflow.sdk.util.OutputReference	asOutputReference(PValue value) Encode a PValue reference as an output reference.
    String	getFullName(PTransform<?,?> transform) Returns the full name of the currently being translated transform.
    <InputT extends PInput> InputT	getInput(PTransform<InputT,?> transform) Returns the input of the currently being translated transform.
    <OutputT extends POutput> OutputT	getOutput(PTransform<?,OutputT> transform) Returns the output of the currently being translated transform.
    DataflowPipelineOptions	getPipelineOptions() Returns the configured pipeline options.

• • Method Detail

    • getPipelineOptions

      DataflowPipelineOptions getPipelineOptions()

      Returns the configured pipeline options.

    • getInput

      <InputT extends PInput> InputT getInput(PTransform<InputT,?> transform)

      Returns the input of the currently being translated transform.

    • getOutput

      <OutputT extends POutput> OutputT getOutput(PTransform<?,OutputT> transform)

      Returns the output of the currently being translated transform.

    • getFullName

      String getFullName(PTransform<?,?> transform)

      Returns the full name of the currently being translated transform.

    • addStep

      void addStep(PTransform<?,?> transform,
                   String type)

      Adds a step to the Dataflow workflow for the given transform, with the given Dataflow step type. This step becomes "current" for the purpose of addInput(java.lang.String, java.lang.Boolean) and addOutput(java.lang.String, com.google.cloud.dataflow.sdk.values.PValue).

    • addStep

      void addStep(PTransform<?,? extends PValue> transform,
                   com.google.api.services.dataflow.model.Step step)

      Adds a pre-defined step to the Dataflow workflow. The given PTransform should be consistent with the Step, in terms of input, output and coder types.

      This is a low-level operation, when using this method it is up to the caller to ensure that names do not collide.

    • addEncodingInput

      void addEncodingInput(Coder<?> value)

      Sets the encoding for the current Dataflow step.

    • addInput

      void addInput(String name,
                    Boolean value)

      Adds an input with the given name and value to the current Dataflow step.

    • addInput

      void addInput(String name,
                    String value)

      Adds an input with the given name and value to the current Dataflow step.

    • addInput

      void addInput(String name,
                    Long value)

      Adds an input with the given name and value to the current Dataflow step.

    • addInput

      void addInput(String name,
                    PInput value)

      Adds an input with the given name to the previously added Dataflow step, coming from the specified input PValue.

    • addInput

      void addInput(String name,
                    Map<String,Object> elements)

      Adds an input that is a dictionary of strings to objects.

    • addInput

      void addInput(String name,
                    List<? extends Map<String,Object>> elements)

      Adds an input that is a list of objects.

    • addOutput

      void addOutput(String name,
                     PValue value)

      Adds an output with the given name to the previously added Dataflow step, producing the specified output PValue, including its Coder if a TypedPValue. If the PValue is a PCollection, wraps its coder inside a WindowedValueCoder.

    • addValueOnlyOutput

      void addValueOnlyOutput(String name,
                              PValue value)

      Adds an output with the given name to the previously added Dataflow step, producing the specified output PValue, including its Coder if a TypedPValue. If the PValue is a PCollection, wraps its coder inside a ValueOnlyCoder.

    • addCollectionToSingletonOutput

      void addCollectionToSingletonOutput(String name,
                                          PValue inputValue,
                                          PValue outputValue)

      Adds an output with the given name to the previously added CollectionToSingleton Dataflow step, consuming the specified input PValue and producing the specified output PValue. This step requires special treatment for its output encoding.

    • asOutputReference

      com.google.cloud.dataflow.sdk.util.OutputReference asOutputReference(PValue value)

      Encode a PValue reference as an output reference.