Trigger (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

com.google.cloud.dataflow.sdk.transforms.windowing

Class Trigger<W extends BoundedWindow>

  • java.lang.Object
    • com.google.cloud.dataflow.sdk.transforms.windowing.Trigger<W>
  • Type Parameters:
    W - BoundedWindow subclass used to represent the windows used by this Trigger
    All Implemented Interfaces:
    TriggerBuilder<W>, Serializable
    Direct Known Subclasses:
    AfterEach, DefaultTrigger, Repeatedly, Trigger.OnceTrigger

    @Experimental(value=TRIGGER)
public abstract class Trigger<W extends BoundedWindow>
extends Object
implements Serializable, TriggerBuilder<W>
    Triggers control when the elements for a specific key and window are output. As elements arrive, they are put into one or more windows by a Window transform and its associated WindowFn, and then passed to the associated Trigger to determine if the Windows contents should be output.

    See GroupByKey and Window for more information about how grouping with windows works.

    The elements that are assigned to a window since the last time it was fired (or since the window was created) are placed into the current window pane. Triggers are evaluated against the elements as they are added. When the root trigger fires, the elements in the current pane will be output. When the root trigger finishes (indicating it will never fire again), the window is closed and any new elements assigned to that window are discarded.

    Several predefined triggers are provided:

    • AfterWatermark for firing when the watermark passes a timestamp determined from either the end of the window or the arrival of the first element in a pane.
    • AfterProcessingTime for firing after some amount of processing time has elapsed (typically since the first element in a pane).
    • AfterPane for firing off a property of the elements in the current pane, such as the number of elements that have been assigned to the current pane.

    In addition, triggers can be combined in a variety of ways:

    Each trigger tree is instantiated per-key and per-window. Every trigger in the tree is in one of the following states:

    • Never Existed - before the trigger has started executing, there is no state associated with it anywhere in the system. A trigger moves to the executing state as soon as it processes in the current pane.
    • Executing - while the trigger is receiving items and may fire. While it is in this state, it may persist book-keeping information to persisted state, set timers, etc.
    • Finished - after a trigger finishes, all of its book-keeping data is cleaned up, and the system remembers only that it is finished. Entering this state causes us to discard any elements in the buffer for that window, as well.

    Once finished, a trigger cannot return itself back to an earlier state, however a composite trigger could reset its sub-triggers.

    Triggers should not build up any state internally since they may be recreated between invocations of the callbacks. All important values should be persisted using state before the callback returns.

    See Also:
    Serialized Form

    • Constructor Detail

      • Trigger

        protected Trigger(@Nullable
                  List<Trigger<W>> subTriggers)

    • Method Detail

      • onMerge

        public abstract void onMerge(Trigger.OnMergeContext c)
                      throws Exception
        Called immediately after windows have been merged.

        Leaf triggers should update their state by inspecting their status and any state in the merging windows. Composite triggers should update their state by calling ExecutableTrigger.invokeOnMerge(com.google.cloud.dataflow.sdk.transforms.windowing.Trigger<W>.OnMergeContext) on their sub-triggers, and applying appropriate logic.

        A trigger such as AfterWatermark.pastEndOfWindow() may no longer be finished; it is the responsibility of the trigger itself to record this fact. It is forbidden for a trigger to become finished due to onMerge(com.google.cloud.dataflow.sdk.transforms.windowing.Trigger<W>.OnMergeContext), as it has not yet fired the pending elements that led to it being ready to fire.

        The implementation does not need to clear out any state associated with the old windows.

        Throws:
        Exception

      • shouldFire

        public abstract boolean shouldFire(Trigger.TriggerContext context)
                            throws Exception
        Returns true if the current state of the trigger indicates that its condition is satisfied and it is ready to fire.
        Throws:
        Exception

      • onFire

        public abstract void onFire(Trigger.TriggerContext context)
                     throws Exception
        Adjusts the state of the trigger to be ready for the next pane. For example, a Repeatedly trigger will reset its inner trigger, since it has fired.

        If the trigger is finished, it is the responsibility of the trigger itself to record that fact via the context.

        Throws:
        Exception

      • clear

        public void clear(Trigger.TriggerContext c)
           throws Exception
        Clear any state associated with this trigger in the given window.

        This is called after a trigger has indicated it will never fire again. The trigger system keeps enough information to know that the trigger is finished, so this trigger should clear all of its state.

        Throws:
        Exception

      • getContinuationTrigger

        public Trigger<W> getContinuationTrigger()
        Return a trigger to use after a GroupByKey to preserve the intention of this trigger. Specifically, triggers that are time based and intended to provide speculative results should continue providing speculative results. Triggers that fire once (or multiple times) should continue firing once (or multiple times).

      • getContinuationTrigger

        protected abstract Trigger<W> getContinuationTrigger(List<Trigger<W>> continuationTriggers)
        Return the getContinuationTrigger() of this Trigger. For convenience, this is provided the continuation trigger of each of the sub-triggers.

      • getWatermarkThatGuaranteesFiring

        public abstract Instant getWatermarkThatGuaranteesFiring(W window)
        Returns a bound in watermark time by which this trigger would have fired at least once for a given window had there been input data. This is a static property of a trigger that does not depend on its state.

        For triggers that do not fire based on the watermark advancing, returns BoundedWindow.TIMESTAMP_MAX_VALUE.

        This estimate is used to determine that there are no elements in a side-input window, which causes the default value to be used instead.

      • isCompatible

        public boolean isCompatible(Trigger<?> other)
        Returns whether this performs the same triggering as the given Trigger.

      • hashCode

        public int hashCode()
        Overrides:
        hashCode in class Object

      • orFinally

        public Trigger<W> orFinally(Trigger.OnceTrigger<W> until)
        Specify an ending condition for this trigger. If the until fires then the combination fires.

        The expression t1.orFinally(t2) fires every time t1 fires, and finishes as soon as either t1 finishes or t2 fires, in which case it fires one last time for t2. Both t1 and t2 are executed in parallel. This means that t1 may have fired since t2 started, so not all of the elements that t2 has seen are necessarily in the current pane.

        For example the final firing of the following trigger may only have 1 element: 

         
 Repeatedly.forever(AfterPane.elementCountAtLeast(2))
     .orFinally(AfterPane.elementCountAtLeast(5))

        Note that if t1 is Trigger.OnceTrigger, then t1.orFinally(t2) is the same as AfterFirst.of(t1, t2).