DoFn.Context (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

com.google.cloud.dataflow.sdk.transforms

Class DoFn.Context

  • Direct Known Subclasses:
    DoFn.ProcessContext
    Enclosing class:
    DoFn<InputT,OutputT>


    public abstract class DoFn.Context
    extends Object
    Information accessible to all methods in this DoFn. Used primarily to output elements.
    • Constructor Detail

      • Context

        public Context()
    • Method Detail

      • getPipelineOptions

        public abstract PipelineOptions getPipelineOptions()
        Returns the PipelineOptions specified with the PipelineRunner invoking this DoFn. The PipelineOptions will be the default running via DoFnTester.
      • output

        public abstract void output(OutputT output)
        Adds the given element to the main output PCollection.

        Once passed to output the element should be considered immutable and not be modified in any way. It may be cached or retained by the Dataflow runtime or later steps in the pipeline, or used in other unspecified ways.

        If invoked from processElement, the output element will have the same timestamp and be in the same windows as the input element passed to processElement.

        If invoked from startBundle or finishBundle, this will attempt to use the WindowFn of the input PCollection to determine what windows the element should be in, throwing an exception if the WindowFn attempts to access any information about the input element. The output element will have a timestamp of negative infinity.

      • outputWithTimestamp

        public abstract void outputWithTimestamp(OutputT output,
                                                 Instant timestamp)
        Adds the given element to the main output PCollection, with the given timestamp.

        Once passed to outputWithTimestamp the element should not be modified in any way.

        If invoked from processElement, the timestamp must not be older than the input element's timestamp minus getAllowedTimestampSkew. The output element will be in the same windows as the input element.

        If invoked from startBundle or finishBundle, this will attempt to use the WindowFn of the input PCollection to determine what windows the element should be in, throwing an exception if the WindowFn attempts to access any information about the input element except for the timestamp.

      • sideOutput

        public abstract <T> void sideOutput(TupleTag<T> tag,
                                            T output)
        Adds the given element to the side output PCollection with the given tag.

        Once passed to sideOutput the element should not be modified in any way.

        The caller of ParDo uses withOutputTags to specify the tags of side outputs that it consumes. Non-consumed side outputs, e.g., outputs for monitoring purposes only, don't necessarily need to be specified.

        The output element will have the same timestamp and be in the same windows as the input element passed to processElement.

        If invoked from startBundle or finishBundle, this will attempt to use the WindowFn of the input PCollection to determine what windows the element should be in, throwing an exception if the WindowFn attempts to access any information about the input element. The output element will have a timestamp of negative infinity.

        See Also:
        ParDo.withOutputTags(com.google.cloud.dataflow.sdk.values.TupleTag<OutputT>, com.google.cloud.dataflow.sdk.values.TupleTagList)


Was this page helpful? Let us know how we did:

Send feedback about...

Cloud Dataflow
Need help? Visit our support page.