Google Cloud Dataflow SDK for Java, version 1.9.1
Class Trigger<W extends BoundedWindow>
- java.lang.Object
-
- com.google.cloud.dataflow.sdk.transforms.windowing.Trigger<W>
-
- Type Parameters:
W
-BoundedWindow
subclass used to represent the windows used by thisTrigger
- All Implemented Interfaces:
- TriggerBuilder<W>, Serializable
- Direct Known Subclasses:
- AfterEach, DefaultTrigger, Repeatedly, Trigger.OnceTrigger
@Experimental(value=TRIGGER) public abstract class Trigger<W extends BoundedWindow> extends Object implements Serializable, TriggerBuilder<W>
Triggers control when the elements for a specific key and window are output. As elements arrive, they are put into one or more windows by aWindow
transform and its associatedWindowFn
, and then passed to the associatedTrigger
to determine if theWindow
s contents should be output.See
GroupByKey
andWindow
for more information about how grouping with windows works.The elements that are assigned to a window since the last time it was fired (or since the window was created) are placed into the current window pane. Triggers are evaluated against the elements as they are added. When the root trigger fires, the elements in the current pane will be output. When the root trigger finishes (indicating it will never fire again), the window is closed and any new elements assigned to that window are discarded.
Several predefined triggers are provided:
-
AfterWatermark
for firing when the watermark passes a timestamp determined from either the end of the window or the arrival of the first element in a pane. -
AfterProcessingTime
for firing after some amount of processing time has elapsed (typically since the first element in a pane). -
AfterPane
for firing off a property of the elements in the current pane, such as the number of elements that have been assigned to the current pane.
In addition, triggers can be combined in a variety of ways:
-
Repeatedly.forever(com.google.cloud.dataflow.sdk.transforms.windowing.Trigger<W>)
to create a trigger that executes forever. Any time its argument finishes it gets reset and starts over. Can be combined withorFinally(com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnceTrigger<W>)
to specify a condition that causes the repetition to stop. -
AfterEach.inOrder(com.google.cloud.dataflow.sdk.transforms.windowing.Trigger<W>...)
to execute each trigger in sequence, firing each (and every) time that a trigger fires, and advancing to the next trigger in the sequence when it finishes. -
AfterFirst.of(com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnceTrigger<W>...)
to create a trigger that fires after at least one of its arguments fires. AnAfterFirst
trigger finishes after it fires once. -
AfterAll.of(com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnceTrigger<W>...)
to create a trigger that fires after all least one of its arguments have fired at least once. AnAfterAll
trigger finishes after it fires once.
Each trigger tree is instantiated per-key and per-window. Every trigger in the tree is in one of the following states:
- Never Existed - before the trigger has started executing, there is no state associated with it anywhere in the system. A trigger moves to the executing state as soon as it processes in the current pane.
- Executing - while the trigger is receiving items and may fire. While it is in this state, it may persist book-keeping information to persisted state, set timers, etc.
- Finished - after a trigger finishes, all of its book-keeping data is cleaned up, and the system remembers only that it is finished. Entering this state causes us to discard any elements in the buffer for that window, as well.
Once finished, a trigger cannot return itself back to an earlier state, however a composite trigger could reset its sub-triggers.
Triggers should not build up any state internally since they may be recreated between invocations of the callbacks. All important values should be persisted using state before the callback returns.
- See Also:
- Serialized Form
-
-
Nested Class Summary
Nested Classes Modifier and Type Class and Description static interface
Trigger.MergingTriggerInfo<W extends BoundedWindow>
Interact with properties of the trigger being executed, with extensions to deal with the merging windows.static class
Trigger.OnceTrigger<W extends BoundedWindow>
Triggers that are guaranteed to fire at most once should extend from this, rather than the generalTrigger
class to indicate that behavior.class
Trigger.OnElementContext
ExtendedTrigger.TriggerContext
containing information accessible to theonElement(com.google.cloud.dataflow.sdk.transforms.windowing.Trigger<W>.OnElementContext)
operational hook.class
Trigger.OnMergeContext
ExtendedTrigger.TriggerContext
containing information accessible to theonMerge(com.google.cloud.dataflow.sdk.transforms.windowing.Trigger<W>.OnMergeContext)
operational hook.class
Trigger.TriggerContext
Information accessible to all operational hooks in thisTrigger
.static interface
Trigger.TriggerInfo<W extends BoundedWindow>
Interface for accessing information about the trigger being executed and other triggers in the same tree.
-
Field Summary
Fields Modifier and Type Field and Description protected List<Trigger<W>>
subTriggers
-
Constructor Summary
Constructors Modifier Constructor and Description protected
Trigger(List<Trigger<W>> subTriggers)
-
Method Summary
All Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method and Description Trigger<W>
buildTrigger()
Return theTrigger
built by this builder.void
clear(Trigger.TriggerContext c)
Clear any state associated with this trigger in the given window.boolean
equals(Object obj)
Trigger<W>
getContinuationTrigger()
Return a trigger to use after aGroupByKey
to preserve the intention of this trigger.protected abstract Trigger<W>
getContinuationTrigger(List<Trigger<W>> continuationTriggers)
Return thegetContinuationTrigger()
of thisTrigger
.abstract Instant
getWatermarkThatGuaranteesFiring(W window)
Returns a bound in watermark time by which this trigger would have fired at least once for a given window had there been input data.int
hashCode()
boolean
isCompatible(Trigger<?> other)
Returns whether this performs the same triggering as the givenTrigger
.abstract void
onElement(Trigger.OnElementContext c)
Called immediately after an element is first incorporated into a window.abstract void
onFire(Trigger.TriggerContext context)
Adjusts the state of the trigger to be ready for the next pane.abstract void
onMerge(Trigger.OnMergeContext c)
Called immediately after windows have been merged.Trigger<W>
orFinally(Trigger.OnceTrigger<W> until)
Specify an ending condition for this trigger.void
prefetchOnElement(com.google.cloud.dataflow.sdk.util.state.StateAccessor<?> state)
Called to allow the trigger to prefetch any state it will likely need to read from during anonElement(com.google.cloud.dataflow.sdk.transforms.windowing.Trigger<W>.OnElementContext)
call.void
prefetchOnFire(com.google.cloud.dataflow.sdk.util.state.StateAccessor<?> state)
Called to allow the trigger to prefetch any state it will likely need to read from during anonFire(com.google.cloud.dataflow.sdk.transforms.windowing.Trigger<W>.TriggerContext)
call.void
prefetchOnMerge(com.google.cloud.dataflow.sdk.util.state.MergingStateAccessor<?,W> state)
Called to allow the trigger to prefetch any state it will likely need to read from during anonMerge(com.google.cloud.dataflow.sdk.transforms.windowing.Trigger<W>.OnMergeContext)
call.void
prefetchShouldFire(com.google.cloud.dataflow.sdk.util.state.StateAccessor<?> state)
Called to allow the trigger to prefetch any state it will likely need to read from during anshouldFire(com.google.cloud.dataflow.sdk.transforms.windowing.Trigger<W>.TriggerContext)
call.abstract boolean
shouldFire(Trigger.TriggerContext context)
Returnstrue
if the current state of the trigger indicates that its condition is satisfied and it is ready to fire.Iterable<Trigger<W>>
subTriggers()
String
toString()
-
-
-
Field Detail
-
subTriggers
@Nullable protected final List<Trigger<W extends BoundedWindow>> subTriggers
-
-
Method Detail
-
onElement
public abstract void onElement(Trigger.OnElementContext c) throws Exception
Called immediately after an element is first incorporated into a window.- Throws:
Exception
-
onMerge
public abstract void onMerge(Trigger.OnMergeContext c) throws Exception
Called immediately after windows have been merged.Leaf triggers should update their state by inspecting their status and any state in the merging windows. Composite triggers should update their state by calling
ExecutableTrigger.invokeOnMerge(com.google.cloud.dataflow.sdk.transforms.windowing.Trigger<W>.OnMergeContext)
on their sub-triggers, and applying appropriate logic.A trigger such as
AfterWatermark.pastEndOfWindow()
may no longer be finished; it is the responsibility of the trigger itself to record this fact. It is forbidden for a trigger to become finished due toonMerge(com.google.cloud.dataflow.sdk.transforms.windowing.Trigger<W>.OnMergeContext)
, as it has not yet fired the pending elements that led to it being ready to fire.The implementation does not need to clear out any state associated with the old windows.
- Throws:
Exception
-
shouldFire
public abstract boolean shouldFire(Trigger.TriggerContext context) throws Exception
Returnstrue
if the current state of the trigger indicates that its condition is satisfied and it is ready to fire.- Throws:
Exception
-
onFire
public abstract void onFire(Trigger.TriggerContext context) throws Exception
Adjusts the state of the trigger to be ready for the next pane. For example, aRepeatedly
trigger will reset its inner trigger, since it has fired.If the trigger is finished, it is the responsibility of the trigger itself to record that fact via the
context
.- Throws:
Exception
-
prefetchOnElement
public void prefetchOnElement(com.google.cloud.dataflow.sdk.util.state.StateAccessor<?> state)
Called to allow the trigger to prefetch any state it will likely need to read from during anonElement(com.google.cloud.dataflow.sdk.transforms.windowing.Trigger<W>.OnElementContext)
call.
-
prefetchOnMerge
public void prefetchOnMerge(com.google.cloud.dataflow.sdk.util.state.MergingStateAccessor<?,W> state)
Called to allow the trigger to prefetch any state it will likely need to read from during anonMerge(com.google.cloud.dataflow.sdk.transforms.windowing.Trigger<W>.OnMergeContext)
call.
-
prefetchShouldFire
public void prefetchShouldFire(com.google.cloud.dataflow.sdk.util.state.StateAccessor<?> state)
Called to allow the trigger to prefetch any state it will likely need to read from during anshouldFire(com.google.cloud.dataflow.sdk.transforms.windowing.Trigger<W>.TriggerContext)
call.
-
prefetchOnFire
public void prefetchOnFire(com.google.cloud.dataflow.sdk.util.state.StateAccessor<?> state)
Called to allow the trigger to prefetch any state it will likely need to read from during anonFire(com.google.cloud.dataflow.sdk.transforms.windowing.Trigger<W>.TriggerContext)
call.
-
clear
public void clear(Trigger.TriggerContext c) throws Exception
Clear any state associated with this trigger in the given window.This is called after a trigger has indicated it will never fire again. The trigger system keeps enough information to know that the trigger is finished, so this trigger should clear all of its state.
- Throws:
Exception
-
getContinuationTrigger
public Trigger<W> getContinuationTrigger()
Return a trigger to use after aGroupByKey
to preserve the intention of this trigger. Specifically, triggers that are time based and intended to provide speculative results should continue providing speculative results. Triggers that fire once (or multiple times) should continue firing once (or multiple times).
-
getContinuationTrigger
protected abstract Trigger<W> getContinuationTrigger(List<Trigger<W>> continuationTriggers)
Return thegetContinuationTrigger()
of thisTrigger
. For convenience, this is provided the continuation trigger of each of the sub-triggers.
-
getWatermarkThatGuaranteesFiring
public abstract Instant getWatermarkThatGuaranteesFiring(W window)
Returns a bound in watermark time by which this trigger would have fired at least once for a given window had there been input data. This is a static property of a trigger that does not depend on its state.For triggers that do not fire based on the watermark advancing, returns
BoundedWindow.TIMESTAMP_MAX_VALUE
.This estimate is used to determine that there are no elements in a side-input window, which causes the default value to be used instead.
-
isCompatible
public boolean isCompatible(Trigger<?> other)
Returns whether this performs the same triggering as the givenTrigger
.
-
orFinally
public Trigger<W> orFinally(Trigger.OnceTrigger<W> until)
Specify an ending condition for this trigger. If theuntil
fires then the combination fires.The expression
t1.orFinally(t2)
fires every timet1
fires, and finishes as soon as eithert1
finishes ort2
fires, in which case it fires one last time fort2
. Botht1
andt2
are executed in parallel. This means thatt1
may have fired sincet2
started, so not all of the elements thatt2
has seen are necessarily in the current pane.For example the final firing of the following trigger may only have 1 element:
Repeatedly.forever(AfterPane.elementCountAtLeast(2)) .orFinally(AfterPane.elementCountAtLeast(5))
Note that if
t1
isTrigger.OnceTrigger
, thent1.orFinally(t2)
is the same asAfterFirst.of(t1, t2)
.
-
buildTrigger
public Trigger<W> buildTrigger()
Description copied from interface:TriggerBuilder
Return theTrigger
built by this builder.- Specified by:
buildTrigger
in interfaceTriggerBuilder<W extends BoundedWindow>
-
-