com.google.cloud.dataflow.sdk.transforms.windowing (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

Warning: Cloud Dataflow SDK 1.x for Java is unsupported as of October 16, 2018. See Migrating from Cloud Dataflow SDK 1.x for Java for migration guidance.

Note: If you are using Cloud Dataflow SDK version 2.x, see the Apache Beam API reference documentation.

Package com.google.cloud.dataflow.sdk.transforms.windowing

Defines the Window transform for dividing the elements in a PCollection into windows, and the Trigger for controlling when those elements are output.

See: Description

• Interface Summary 

  Interface	Description
  AfterWatermark.AfterWatermarkEarly<W extends BoundedWindow>	Interface for building an AfterWatermarkTrigger with early firings already filled in.
  AfterWatermark.AfterWatermarkLate<W extends BoundedWindow>	Interface for building an AfterWatermarkTrigger with late firings already filled in.
  Trigger.MergingTriggerInfo<W extends BoundedWindow>	Interact with properties of the trigger being executed, with extensions to deal with the merging windows.
  Trigger.TriggerInfo<W extends BoundedWindow>	Interface for accessing information about the trigger being executed and other triggers in the same tree.
  TriggerBuilder<W extends BoundedWindow>	Anything that can be used to create an instance of a Trigger implements this interface.

• Class Summary 

  Class	Description
  AfterAll<W extends BoundedWindow>	Create a Trigger that fires and finishes once after all of its sub-triggers have fired.
  AfterDelayFromFirstElement<W extends BoundedWindow>	A base class for triggers that happen after a processing time delay from the arrival of the first element in a pane.
  AfterEach<W extends BoundedWindow>	A composite Trigger that executes its sub-triggers in order.
  AfterFirst<W extends BoundedWindow>	Create a composite Trigger that fires once after at least one of its sub-triggers have fired.
  AfterPane<W extends BoundedWindow>	Triggers that fire based on properties of the elements in the current pane.
  AfterProcessingTime<W extends BoundedWindow>	AfterProcessingTime triggers fire based on the current processing time.
  AfterWatermark<W extends BoundedWindow>	AfterWatermark triggers fire based on progress of the system watermark.
  AfterWatermark.FromEndOfWindow<W extends BoundedWindow>	A watermark trigger targeted relative to the end of the window.
  BoundedWindow	A BoundedWindow represents a finite grouping of elements, with an upper bound (larger timestamps represent more recent data) on the timestamps of elements that can be placed in the window.
  CalendarWindows	A collection of WindowFns that windows values into calendar-based windows such as spans of days, months, or years.
  CalendarWindows.DaysWindows	A WindowFn that windows elements into periods measured by days.
  CalendarWindows.MonthsWindows	A WindowFn that windows elements into periods measured by months.
  CalendarWindows.YearsWindows	A WindowFn that windows elements into periods measured by years.
  DefaultTrigger<W extends BoundedWindow>	A trigger that is equivalent to Repeatedly.forever(AfterWatermark.pastEndOfWindow()).
  FixedWindows	A WindowFn that windows values into fixed-size timestamp-based windows.
  GlobalWindow	The default window into which all data is placed (via GlobalWindows).
  GlobalWindow.Coder	GlobalWindow.Coder for encoding and decoding GlobalWindows.
  GlobalWindows	Default WindowFn that assigns all data to the same window.
  IntervalWindow	An implementation of BoundedWindow that represents an interval from IntervalWindow.start (inclusive) to IntervalWindow.end (exclusive).
  InvalidWindows<W extends BoundedWindow>	A WindowFn that represents an invalid pipeline state.
  MergeOverlappingIntervalWindows	A utility function for merging overlapping IntervalWindows.
  Never	A trigger which never fires.
  NonMergingWindowFn<T,W extends BoundedWindow>	Abstract base class for WindowFns that do not merge windows.
  OutputTimeFn<W extends BoundedWindow>	(Experimental) A function from timestamps of input values to the timestamp for a computed value.
  OutputTimeFn.Defaults<W extends BoundedWindow>	(Experimental) Default method implementations for OutputTimeFn where the output time depends on the input element timestamps and possibly the window.
  OutputTimeFn.DependsOnlyOnWindow<W extends BoundedWindow>	(Experimental) Default method implementations for OutputTimeFn when the output time depends only on the window.
  OutputTimeFns	(Experimental) Static utility methods and provided implementations for OutputTimeFn.
  PaneInfo	Provides information about the pane an element belongs to.
  PaneInfo.PaneInfoCoder	A Coder for encoding PaneInfo instances.
  PartitioningWindowFn<T,W extends BoundedWindow>	A WindowFn that places each value into exactly one window based on its timestamp and never merges windows.
  Repeatedly<W extends BoundedWindow>	Repeat a trigger, either until some condition is met or forever.
  Sessions	A WindowFn windowing values into sessions separated by Sessions.gapDuration-long periods with no elements.
  SlidingWindows	A WindowFn that windows values into possibly overlapping fixed-size timestamp-based windows.
  Trigger<W extends BoundedWindow>	Triggers control when the elements for a specific key and window are output.
  Trigger.OnceTrigger<W extends BoundedWindow>	Triggers that are guaranteed to fire at most once should extend from this, rather than the general Trigger class to indicate that behavior.
  Window	Window logically divides up or groups the elements of a PCollection into finite windows according to a WindowFn.
  Window.Bound<T>	A PTransform that windows the elements of a PCollection<T>, into finite windows according to a user-specified WindowFn.
  Window.Remerge<T>	PTransform that does not change assigned windows, but will cause windows to be merged again as part of the next GroupByKey.
  Window.Unbound	An incomplete Window transform, with unbound input/output type.
  WindowFn<T,W extends BoundedWindow>	The argument to the Window transform used to assign elements into windows and to determine how windows are merged.

• Enum Summary 

  Enum	Description
  PaneInfo.Timing	Enumerates the possibilities for the timing of this pane firing related to the input and output watermarks for its computation.
  Window.ClosingBehavior	Specifies the conditions under which a final pane will be created when a window is permanently closed.

Package com.google.cloud.dataflow.sdk.transforms.windowing Description

Defines the Window transform for dividing the elements in a PCollection into windows, and the Trigger for controlling when those elements are output.

Window logically divides up or groups the elements of a PCollection into finite windows according to a WindowFn. The output of Window contains the same elements as input, but they have been logically assigned to windows. The next GroupByKeys, including one within composite transforms, will group by the combination of keys and windows.

Windowing a PCollection allows chunks of it to be processed individually, before the entire PCollection is available. This is especially important for PCollections with unbounded size, since the full PCollection is never available at once.

For PCollections with a bounded size, by default, all data is implicitly in a single window, and this replicates conventional batch mode. However, windowing can still be a convenient way to express time-sliced algorithms over bounded PCollections.

As elements are assigned to a window, they are are placed into a pane. When the trigger fires all of the elements in the current pane are output.

The DefaultTrigger will output a window when the system watermark passes the end of the window. See AfterWatermark for details on the watermark.