DataflowPipelineTranslator.TranslationContext (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

com.google.cloud.dataflow.sdk.runners

Interface DataflowPipelineTranslator.TranslationContext

    • Method Summary

      All Methods Instance Methods Abstract Methods 
      Modifier and Type Method and Description
      void addCollectionToSingletonOutput(String name, PValue inputValue, PValue outputValue)
      Adds an output with the given name to the previously added CollectionToSingleton Dataflow step, consuming the specified input PValue and producing the specified output PValue.
      void addEncodingInput(Coder<?> value)
      Sets the encoding for the current Dataflow step.
      void addInput(String name, Boolean value)
      Adds an input with the given name and value to the current Dataflow step.
      void addInput(String name, List<? extends Map<String,Object>> elements)
      Adds an input that is a list of objects.
      void addInput(String name, Long value)
      Adds an input with the given name and value to the current Dataflow step.
      void addInput(String name, Map<String,Object> elements)
      Adds an input that is a dictionary of strings to objects.
      void addInput(String name, PInput value)
      Adds an input with the given name to the previously added Dataflow step, coming from the specified input PValue.
      void addInput(String name, String value)
      Adds an input with the given name and value to the current Dataflow step.
      void addOutput(String name, PValue value)
      Adds an output with the given name to the previously added Dataflow step, producing the specified output PValue, including its Coder if a TypedPValue.
      void addStep(PTransform<?,?> transform, String type)
      Adds a step to the Dataflow workflow for the given transform, with the given Dataflow step type.
      void addStep(PTransform<?,? extends PValue> transform, com.google.api.services.dataflow.model.Step step)
      Adds a pre-defined step to the Dataflow workflow.
      void addValueOnlyOutput(String name, PValue value)
      Adds an output with the given name to the previously added Dataflow step, producing the specified output PValue, including its Coder if a TypedPValue.
      com.google.cloud.dataflow.sdk.util.OutputReference asOutputReference(PValue value)
      Encode a PValue reference as an output reference.
      String getFullName(PTransform<?,?> transform)
      Returns the full name of the currently being translated transform.
      <InputT extends PInput>
      InputT
      getInput(PTransform<InputT,?> transform)
      Returns the input of the currently being translated transform.
      <OutputT extends POutput>
      OutputT
      getOutput(PTransform<?,OutputT> transform)
      Returns the output of the currently being translated transform.
      DataflowPipelineOptions getPipelineOptions()
      Returns the configured pipeline options.
    • Method Detail

      • getInput

        <InputT extends PInput> InputT getInput(PTransform<InputT,?> transform)
        Returns the input of the currently being translated transform.
      • getOutput

        <OutputT extends POutput> OutputT getOutput(PTransform<?,OutputT> transform)
        Returns the output of the currently being translated transform.
      • getFullName

        String getFullName(PTransform<?,?> transform)
        Returns the full name of the currently being translated transform.
      • addStep

        void addStep(PTransform<?,? extends PValue> transform,
                     com.google.api.services.dataflow.model.Step step)
        Adds a pre-defined step to the Dataflow workflow. The given PTransform should be consistent with the Step, in terms of input, output and coder types.

        This is a low-level operation, when using this method it is up to the caller to ensure that names do not collide.

      • addEncodingInput

        void addEncodingInput(Coder<?> value)
        Sets the encoding for the current Dataflow step.
      • addInput

        void addInput(String name,
                      Boolean value)
        Adds an input with the given name and value to the current Dataflow step.
      • addInput

        void addInput(String name,
                      String value)
        Adds an input with the given name and value to the current Dataflow step.
      • addInput

        void addInput(String name,
                      Long value)
        Adds an input with the given name and value to the current Dataflow step.
      • addInput

        void addInput(String name,
                      PInput value)
        Adds an input with the given name to the previously added Dataflow step, coming from the specified input PValue.
      • addInput

        void addInput(String name,
                      Map<String,Object> elements)
        Adds an input that is a dictionary of strings to objects.
      • addInput

        void addInput(String name,
                      List<? extends Map<String,Object>> elements)
        Adds an input that is a list of objects.
      • addOutput

        void addOutput(String name,
                       PValue value)
        Adds an output with the given name to the previously added Dataflow step, producing the specified output PValue, including its Coder if a TypedPValue. If the PValue is a PCollection, wraps its coder inside a WindowedValueCoder.
      • addValueOnlyOutput

        void addValueOnlyOutput(String name,
                                PValue value)
        Adds an output with the given name to the previously added Dataflow step, producing the specified output PValue, including its Coder if a TypedPValue. If the PValue is a PCollection, wraps its coder inside a ValueOnlyCoder.
      • addCollectionToSingletonOutput

        void addCollectionToSingletonOutput(String name,
                                            PValue inputValue,
                                            PValue outputValue)
        Adds an output with the given name to the previously added CollectionToSingleton Dataflow step, consuming the specified input PValue and producing the specified output PValue. This step requires special treatment for its output encoding.
      • asOutputReference

        com.google.cloud.dataflow.sdk.util.OutputReference asOutputReference(PValue value)
        Encode a PValue reference as an output reference.


Was this page helpful? Let us know how we did:

Send feedback about...

Cloud Dataflow
Need help? Visit our support page.