WindowFnTestUtils (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

Warning: Dataflow SDK 1.x for Java is unsupported as of October 16, 2018. After August 12, 2020, you can use unsupported SDKs temporarily while you upgrade your pipelines. For more information, see Using an unsupported SDK.

For information on migrating from the 1.x SDK, see Migrating from Dataflow SDK 1.x for Java.

Note: If you are using Cloud Dataflow SDK version 2.x, see the Apache Beam API reference documentation.

com.google.cloud.dataflow.sdk.testing

Class WindowFnTestUtils

• java.lang.Object
• • com.google.cloud.dataflow.sdk.testing.WindowFnTestUtils

• ---

  
  

  public class WindowFnTestUtils
  extends Object

  A utility class for testing WindowFns.

• • Constructor Summary

    Constructors 

    Constructor and Description
    WindowFnTestUtils()

  • Method Summary

    All Methods Static Methods Concrete Methods 

    Modifier and Type	Method and Description
    static <T,W extends BoundedWindow> Collection<W>	assignedWindows(WindowFn<T,W> windowFn, long timestamp)
    static <T,W extends BoundedWindow> Map<W,Set<String>>	runWindowFn(WindowFn<T,W> windowFn, List<Long> timestamps) Runs the WindowFn over the provided input, returning a map of windows to the timestamps in those windows.
    static Set<String>	set(long... timestamps) Creates a Set of elements to be used as expected output in runWindowFn(com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn<T, W>, java.util.List<java.lang.Long>).
    static <T,W extends BoundedWindow> void	validateGetOutputTimestamp(WindowFn<T,W> windowFn, long timestamp) Assigns the given timestamp to windows using the specified windowFn, and verifies that result of windowFn.getOutputTime for later windows (as defined by maxTimestamp won't prevent the watermark from passing the end of earlier windows.
    static <T,W extends IntervalWindow> void	validateGetOutputTimestamps(WindowFn<T,W> windowFn, OutputTimeFn<? super W> outputTimeFn, List<List<Long>> timestampsPerWindow) Verifies that later-ending merged windows from any of the timestamps hold up output of earlier-ending windows, using the provided WindowFn and OutputTimeFn.
    static <T,W extends BoundedWindow> void	validateNonInterferingOutputTimes(WindowFn<T,W> windowFn, long timestamp) Assigns the given timestamp to windows using the specified windowFn, and verifies that result of windowFn.getOutputTimestamp for each window is within the proper bound.

    • Methods inherited from class java.lang.Object

      clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

• • Constructor Detail

    • WindowFnTestUtils

      public WindowFnTestUtils()

  • Method Detail

    • set

      public static Set<String> set(long... timestamps)

      Creates a Set of elements to be used as expected output in runWindowFn(com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn<T, W>, java.util.List<java.lang.Long>).

    • runWindowFn

      public static <T,W extends BoundedWindow> Map<W,Set<String>> runWindowFn(WindowFn<T,W> windowFn,
                                                                               List<Long> timestamps)
                                                                        throws Exception

      Runs the WindowFn over the provided input, returning a map of windows to the timestamps in those windows.

      Throws:

      Exception

    • assignedWindows

      public static <T,W extends BoundedWindow> Collection<W> assignedWindows(WindowFn<T,W> windowFn,
                                                                              long timestamp)
                                                                       throws Exception

      Throws:

      Exception

    • validateNonInterferingOutputTimes

      public static <T,W extends BoundedWindow> void validateNonInterferingOutputTimes(WindowFn<T,W> windowFn,
                                                                                       long timestamp)
                                                                                throws Exception

      Assigns the given timestamp to windows using the specified windowFn, and verifies that result of windowFn.getOutputTimestamp for each window is within the proper bound.

      Throws:

      Exception

    • validateGetOutputTimestamp

      public static <T,W extends BoundedWindow> void validateGetOutputTimestamp(WindowFn<T,W> windowFn,
                                                                                long timestamp)
                                                                         throws Exception

      Assigns the given timestamp to windows using the specified windowFn, and verifies that result of windowFn.getOutputTime for later windows (as defined by maxTimestamp won't prevent the watermark from passing the end of earlier windows.

      This verifies that overlapping windows don't interfere at all. Depending on the windowFn this may be stricter than desired.

      Throws:

      Exception

    • validateGetOutputTimestamps

      public static <T,W extends IntervalWindow> void validateGetOutputTimestamps(WindowFn<T,W> windowFn,
                                                                                  OutputTimeFn<? super W> outputTimeFn,
                                                                                  List<List<Long>> timestampsPerWindow)
                                                                           throws Exception

      Verifies that later-ending merged windows from any of the timestamps hold up output of earlier-ending windows, using the provided WindowFn and OutputTimeFn.

      Given a list of lists of timestamps, where each list is expected to merge into a single window with end times in ascending order, assigns and merges windows for each list (as though each were a separate key/user session). Then maps each timestamp in the list according to outputTimeFn.assignOutputTime() and outputTimeFn.combine().

      Verifies that a overlapping windows do not hold each other up via the watermark.

      Throws:

      Exception