OutputTimeFn.DependsOnlyOnWindow (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

Warning: Dataflow SDK 1.x for Java is unsupported as of October 16, 2018. After August 12, 2020, Dataflow will not run jobs using Dataflow 1.x and below. See Migrating from Dataflow SDK 1.x for Java for migration guidance.

Note: If you are using Cloud Dataflow SDK version 2.x, see the Apache Beam API reference documentation.

com.google.cloud.dataflow.sdk.transforms.windowing

Class OutputTimeFn.DependsOnlyOnWindow<W extends BoundedWindow>

• java.lang.Object
• • com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn<W>
  • • com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.DependsOnlyOnWindow<W>

• All Implemented Interfaces:

  Serializable

  Enclosing class:

  OutputTimeFn<W extends BoundedWindow>

  ---

  
  

  public abstract static class OutputTimeFn.DependsOnlyOnWindow<W extends BoundedWindow>
  extends OutputTimeFn<W>

  (Experimental) Default method implementations for OutputTimeFn when the output time depends only on the window.

  To complete an implementation, override assignOutputTime(BoundedWindow).

  See Also:

  Serialized Form

• • Nested Class Summary

    • Nested classes/interfaces inherited from class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn

      OutputTimeFn.Defaults<W extends BoundedWindow>, OutputTimeFn.DependsOnlyOnWindow<W extends BoundedWindow>

  • Constructor Summary

    Constructors 

    Modifier	Constructor and Description
    protected	DependsOnlyOnWindow()

  • Method Summary

    All Methods Instance Methods Abstract Methods Concrete Methods 

    Modifier and Type	Method and Description
    Instant	assignOutputTime(Instant timestamp, W window) Returns the output timestamp to use for data depending on the given inputTimestamp in the specified window.
    protected abstract Instant	assignOutputTime(W window) Returns the output timestamp to use for data in the specified window.
    Instant	combine(Instant outputTimestamp, Instant otherOutputTimestamp) Combines the given output times, which must be from the same window, into an output time for a computed value.
    boolean	dependsOnlyOnEarliestInputTimestamp() Returns true if the result of combination of many output timestamps actually depends only on the earliest.
    boolean	dependsOnlyOnWindow() Returns true if the result does not depend on what outputs were combined but only the window they are in.
    boolean	equals(Object other)
    int	hashCode()
    Instant	merge(W resultWindow, Iterable<? extends Instant> mergingTimestamps) Merges the given output times, presumed to be combined output times for windows that are merging, into an output time for the resultWindow.

    • Methods inherited from class java.lang.Object

      clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

• • Constructor Detail

    • DependsOnlyOnWindow

      protected DependsOnlyOnWindow()

  • Method Detail

    • assignOutputTime

      protected abstract Instant assignOutputTime(W window)

      Returns the output timestamp to use for data in the specified window.

      Note that the result of this method must be between the maximum possible input timestamp in window and window.maxTimestamp() (inclusive on both sides).

      For example, using Sessions.withGapDuration(gapDuration), we know that all input timestamps must lie at least gapDuration from the end of the session, so window.maxTimestamp() - gapDuration is an acceptable assigned timestamp.

      See Also:

      assignOutputTime(Instant, BoundedWindow)

    • assignOutputTime

      public final Instant assignOutputTime(Instant timestamp,
                                            W window)

      Returns the output timestamp to use for data depending on the given inputTimestamp in the specified window.

      The result of this method must be between inputTimestamp and window.maxTimestamp() (inclusive on both sides).

      This function must be monotonic across input timestamps. Specifically, if A < B, then assignOutputTime(A, window) <= assignOutputTime(B, window).

      For a WindowFn that doesn't produce overlapping windows, this can (and typically should) just return inputTimestamp. In the presence of overlapping windows, it is suggested that the result in later overlapping windows is past the end of earlier windows so that the later windows don't prevent the watermark from progressing past the end of the earlier window.

      See the overview of OutputTimeFn for the consistency properties required between OutputTimeFn.assignOutputTime(org.joda.time.Instant, W), OutputTimeFn.combine(org.joda.time.Instant, org.joda.time.Instant), and OutputTimeFn.merge(W, java.lang.Iterable<? extends org.joda.time.Instant>).

      Specified by:

      assignOutputTime in class OutputTimeFn<W extends BoundedWindow>

      Returns:

      the result of assignOutputTime(window).

    • combine

      public final Instant combine(Instant outputTimestamp,
                                   Instant otherOutputTimestamp)

      Combines the given output times, which must be from the same window, into an output time for a computed value.

      • combine must be commutative: combine(a, b).equals(combine(b, a)).
      • combine must be associative: combine(a, combine(b, c)).equals(combine(combine(a, b), c)).

      Specified by:

      combine in class OutputTimeFn<W extends BoundedWindow>

      Returns:

      the same timestamp as both argument timestamps, which are necessarily equal.

    • merge

      public final Instant merge(W resultWindow,
                                 Iterable<? extends Instant> mergingTimestamps)

      Merges the given output times, presumed to be combined output times for windows that are merging, into an output time for the resultWindow.

      When windows w1 and w2 merge to become a new window w1plus2, then OutputTimeFn.merge(W, java.lang.Iterable<? extends org.joda.time.Instant>) must be implemented such that the output time is the same as if all timestamps were assigned in w1plus2. Formally:

      fn.merge(w, fn.assignOutputTime(t1, w1), fn.assignOutputTime(t2, w2))

      must be equal to

      fn.combine(fn.assignOutputTime(t1, w1plus2), fn.assignOutputTime(t2, w1plus2))

      If the assigned time depends only on the window, the correct implementation of merge() necessarily returns the result of assignOutputTime(t1, w1plus2) (which equals assignOutputTime(t2, w1plus2). Defaults for this case are provided by OutputTimeFn.DependsOnlyOnWindow.

      For many other OutputTimeFn implementations, such as taking the earliest or latest timestamp, this will be the same as combine(). Defaults for this case are provided by OutputTimeFn.Defaults.

      Specified by:

      merge in class OutputTimeFn<W extends BoundedWindow>

      Returns:

      the result of assignOutputTime(resultWindow).

    • dependsOnlyOnWindow

      public final boolean dependsOnlyOnWindow()

      Returns true if the result does not depend on what outputs were combined but only the window they are in. The canonical example is if all timestamps are sure to be the end of the window.

      This may allow optimizations, since it is typically very efficient to retrieve the window and combining output timestamps is not necessary.

      If the assigned output time for an implementation depends only on the window, consider extending OutputTimeFn.DependsOnlyOnWindow, which returns true here and also provides a framework for easily implementing a correct OutputTimeFn.merge(W, java.lang.Iterable<? extends org.joda.time.Instant>), OutputTimeFn.combine(org.joda.time.Instant, org.joda.time.Instant) and OutputTimeFn.assignOutputTime(org.joda.time.Instant, W).

      Specified by:

      dependsOnlyOnWindow in class OutputTimeFn<W extends BoundedWindow>

      Returns:

      true.

    • dependsOnlyOnEarliestInputTimestamp

      public final boolean dependsOnlyOnEarliestInputTimestamp()

      Returns true if the result of combination of many output timestamps actually depends only on the earliest.

      This may allow optimizations when it is very efficient to retrieve the earliest timestamp to be combined.

      Specified by:

      dependsOnlyOnEarliestInputTimestamp in class OutputTimeFn<W extends BoundedWindow>

      Returns:

      true. Since the output time depends only on the window, it can certainly be ascertained given a single input timestamp.

    • equals

      public boolean equals(Object other)

      Overrides:

      equals in class Object

      Returns:

      true if the two OutputTimeFn instances have the same class, by default.

    • hashCode

      public int hashCode()

      Overrides:

      hashCode in class Object