com.google.cloud.dataflow.sdk.transforms.windowing (Google Cloud Dataflow SDK 1.9.0 API)

Package com.google.cloud.dataflow.sdk.transforms.windowing

Defines the Window transform for dividing the elements in a PCollection into windows, and the Trigger for controlling when those elements are output.

See: Description

Package com.google.cloud.dataflow.sdk.transforms.windowing Description

Defines the Window transform for dividing the elements in a PCollection into windows, and the Trigger for controlling when those elements are output.

Window logically divides up or groups the elements of a PCollection into finite windows according to a WindowFn. The output of Window contains the same elements as input, but they have been logically assigned to windows. The next GroupByKeys, including one within composite transforms, will group by the combination of keys and windows.

Windowing a PCollection allows chunks of it to be processed individually, before the entire PCollection is available. This is especially important for PCollections with unbounded size, since the full PCollection is never available at once.

For PCollections with a bounded size, by default, all data is implicitly in a single window, and this replicates conventional batch mode. However, windowing can still be a convenient way to express time-sliced algorithms over bounded PCollections.

As elements are assigned to a window, they are are placed into a pane. When the trigger fires all of the elements in the current pane are output.

The DefaultTrigger will output a window when the system watermark passes the end of the window. See AfterWatermark for details on the watermark.


Send feedback about...

Cloud Dataflow