Google Cloud Dataflow SDK for Java, version 1.9.1
-
Interface Summary Interface Description AfterWatermark.AfterWatermarkEarly<W extends BoundedWindow> Interface for building an AfterWatermarkTrigger with early firings already filled in.AfterWatermark.AfterWatermarkLate<W extends BoundedWindow> Interface for building an AfterWatermarkTrigger with late firings already filled in.Trigger.MergingTriggerInfo<W extends BoundedWindow> Interact with properties of the trigger being executed, with extensions to deal with the merging windows.Trigger.TriggerInfo<W extends BoundedWindow> Interface for accessing information about the trigger being executed and other triggers in the same tree.TriggerBuilder<W extends BoundedWindow> Anything that can be used to create an instance of aTrigger
implements this interface. -
Class Summary Class Description AfterAll<W extends BoundedWindow> Create aTrigger
that fires and finishes once after all of its sub-triggers have fired.AfterDelayFromFirstElement<W extends BoundedWindow> A base class for triggers that happen after a processing time delay from the arrival of the first element in a pane.AfterEach<W extends BoundedWindow> A compositeTrigger
that executes its sub-triggers in order.AfterFirst<W extends BoundedWindow> Create a compositeTrigger
that fires once after at least one of its sub-triggers have fired.AfterPane<W extends BoundedWindow> Trigger
s that fire based on properties of the elements in the current pane.AfterProcessingTime<W extends BoundedWindow> AfterProcessingTime
triggers fire based on the current processing time.AfterWatermark<W extends BoundedWindow> AfterWatermark
triggers fire based on progress of the system watermark.AfterWatermark.FromEndOfWindow<W extends BoundedWindow> A watermark trigger targeted relative to the end of the window.BoundedWindow ABoundedWindow
represents a finite grouping of elements, with an upper bound (larger timestamps represent more recent data) on the timestamps of elements that can be placed in the window.CalendarWindows A collection ofWindowFn
s that windows values into calendar-based windows such as spans of days, months, or years.CalendarWindows.DaysWindows AWindowFn
that windows elements into periods measured by days.CalendarWindows.MonthsWindows AWindowFn
that windows elements into periods measured by months.CalendarWindows.YearsWindows AWindowFn
that windows elements into periods measured by years.DefaultTrigger<W extends BoundedWindow> A trigger that is equivalent toRepeatedly.forever(AfterWatermark.pastEndOfWindow())
.FixedWindows AWindowFn
that windows values into fixed-size timestamp-based windows.GlobalWindow The default window into which all data is placed (viaGlobalWindows
).GlobalWindow.Coder GlobalWindow.Coder
for encoding and decodingGlobalWindow
s.GlobalWindows DefaultWindowFn
that assigns all data to the same window.IntervalWindow An implementation ofBoundedWindow
that represents an interval fromIntervalWindow.start
(inclusive) toIntervalWindow.end
(exclusive).InvalidWindows<W extends BoundedWindow> AWindowFn
that represents an invalid pipeline state.MergeOverlappingIntervalWindows A utility function for merging overlappingIntervalWindow
s.Never A trigger which never fires.NonMergingWindowFn<T,W extends BoundedWindow> Abstract base class forWindowFn
s that do not merge windows.OutputTimeFn<W extends BoundedWindow> (Experimental) A function from timestamps of input values to the timestamp for a computed value.OutputTimeFn.Defaults<W extends BoundedWindow> (Experimental) Default method implementations forOutputTimeFn
where the output time depends on the input element timestamps and possibly the window.OutputTimeFn.DependsOnlyOnWindow<W extends BoundedWindow> (Experimental) Default method implementations forOutputTimeFn
when the output time depends only on the window.OutputTimeFns (Experimental) Static utility methods and provided implementations forOutputTimeFn
.PaneInfo Provides information about the pane an element belongs to.PaneInfo.PaneInfoCoder A Coder for encoding PaneInfo instances.PartitioningWindowFn<T,W extends BoundedWindow> AWindowFn
that places each value into exactly one window based on its timestamp and never merges windows.Repeatedly<W extends BoundedWindow> Repeat a trigger, either until some condition is met or forever.Sessions AWindowFn
windowing values into sessions separated bySessions.gapDuration
-long periods with no elements.SlidingWindows AWindowFn
that windows values into possibly overlapping fixed-size timestamp-based windows.Trigger<W extends BoundedWindow> Trigger
s control when the elements for a specific key and window are output.Trigger.OnceTrigger<W extends BoundedWindow> Window Window
logically divides up or groups the elements of aPCollection
into finite windows according to aWindowFn
.Window.Bound<T> APTransform
that windows the elements of aPCollection<T>
, into finite windows according to a user-specifiedWindowFn
.Window.Remerge<T> PTransform
that does not change assigned windows, but will cause windows to be merged again as part of the nextGroupByKey
.Window.Unbound An incompleteWindow
transform, with unbound input/output type.WindowFn<T,W extends BoundedWindow> The argument to theWindow
transform used to assign elements into windows and to determine how windows are merged. -
Enum Summary Enum Description PaneInfo.Timing Enumerates the possibilities for the timing of this pane firing related to the input and output watermarks for its computation.Window.ClosingBehavior Specifies the conditions under which a final pane will be created when a window is permanently closed.
Package com.google.cloud.dataflow.sdk.transforms.windowing Description
Window
transform
for dividing the elements in a PCollection into windows, and the
Trigger
for controlling when those
elements are output.
Window
logically divides up or groups the elements of a
PCollection
into finite windows according to a
WindowFn
.
The output of Window
contains the same elements as input, but they
have been logically assigned to windows. The next
GroupByKey
s, including one
within composite transforms, will group by the combination of keys and
windows.
Windowing a PCollection
allows chunks of it to be processed
individually, before the entire PCollection
is available. This is
especially important for PCollection
s with unbounded size, since the full
PCollection
is never available at once.
For PCollection
s with a bounded size, by default, all data is implicitly in a
single window, and this replicates conventional batch mode. However, windowing can still be a
convenient way to express time-sliced algorithms over bounded PCollection
s.
As elements are assigned to a window, they are are placed into a pane. When the trigger fires all of the elements in the current pane are output.
The DefaultTrigger
will output a
window when the system watermark passes the end of the window. See
AfterWatermark
for details on the
watermark.