OutputTimeFn (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1


Class OutputTimeFn<W extends BoundedWindow>

  • java.lang.Object
    • com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn<W>
  • Type Parameters:
    W - the type of window. Contravariant: methods accepting any subtype of OutputTimeFn<W> should use the parameter type OutputTimeFn<? super W>.
    All Implemented Interfaces:
    Direct Known Subclasses:
    OutputTimeFn.Defaults, OutputTimeFn.DependsOnlyOnWindow

    public abstract class OutputTimeFn<W extends BoundedWindow>
    extends Object
    implements Serializable
    (Experimental) A function from timestamps of input values to the timestamp for a computed value.

    The function is represented via three components:

    1. assignOutputTime(org.joda.time.Instant, W) calculates an output timestamp for any input value in a particular window.
    2. The output timestamps for all non-late input values within a window are combined according to combine(), a commutative and associative operation on the output timestamps.
    3. The output timestamp when windows merge is provided by merge().

    This abstract class cannot be subclassed directly, by design: it may grow in consumer-compatible ways that require mutually-exclusive default implementations. To create a concrete subclass, extend OutputTimeFn.Defaults or OutputTimeFn.DependsOnlyOnWindow. Note that as long as this class remains experimental, we may also choose to change it in arbitrary backwards-incompatible ways.

    See Also:
    Serialized Form
    • Method Detail

      • assignOutputTime

        public abstract Instant assignOutputTime(Instant inputTimestamp,
                                                 W window)
        Returns the output timestamp to use for data depending on the given inputTimestamp in the specified window.

        The result of this method must be between inputTimestamp and window.maxTimestamp() (inclusive on both sides).

        This function must be monotonic across input timestamps. Specifically, if A < B, then assignOutputTime(A, window) <= assignOutputTime(B, window).

        For a WindowFn that doesn't produce overlapping windows, this can (and typically should) just return inputTimestamp. In the presence of overlapping windows, it is suggested that the result in later overlapping windows is past the end of earlier windows so that the later windows don't prevent the watermark from progressing past the end of the earlier window.

        See the overview of OutputTimeFn for the consistency properties required between assignOutputTime(org.joda.time.Instant, W), combine(org.joda.time.Instant, org.joda.time.Instant), and merge(W, java.lang.Iterable<? extends org.joda.time.Instant>).

      • combine

        public abstract Instant combine(Instant outputTime,
                                        Instant otherOutputTime)
        Combines the given output times, which must be from the same window, into an output time for a computed value.
        • combine must be commutative: combine(a, b).equals(combine(b, a)).
        • combine must be associative: combine(a, combine(b, c)).equals(combine(combine(a, b), c)).
      • dependsOnlyOnEarliestInputTimestamp

        public abstract boolean dependsOnlyOnEarliestInputTimestamp()
        Returns true if the result of combination of many output timestamps actually depends only on the earliest.

        This may allow optimizations when it is very efficient to retrieve the earliest timestamp to be combined.