Flatten (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1


Class Flatten

  • public class Flatten
    extends Object
    Flatten<T> takes multiple PCollection<T>s bundled into a PCollectionList<T> and returns a single PCollection<T> containing all the elements in all the input PCollections. The name "Flatten" suggests taking a list of lists and flattening them into a single list.

    Example of use:

     PCollection<String> pc1 = ...;
     PCollection<String> pc2 = ...;
     PCollection<String> pc3 = ...;
     PCollectionList<String> pcs = PCollectionList.of(pc1).and(pc2).and(pc3);
     PCollection<String> merged = pcs.apply(Flatten.<String>pCollections());

    By default, the Coder of the output PCollection is the same as the Coder of the first PCollection in the input PCollectionList (if the PCollectionList is non-empty).

    • Constructor Detail

      • Flatten

        public Flatten()
    • Method Detail

      • pCollections

        public static <T> Flatten.FlattenPCollectionList<T> pCollections()
        Returns a PTransform that flattens a PCollectionList into a PCollection containing all the elements of all the PCollections in its input.

        All inputs must have equal WindowFns. The output elements of Flatten<T> are in the same windows and have the same timestamps as their corresponding input elements. The output PCollection will have the same WindowFn as all of the inputs.

        Type Parameters:
        T - the type of the elements in the input and output PCollections.
      • iterables

        public static <T> Flatten.FlattenIterables<T> iterables()
        Returns a PTransform that takes a PCollection<Iterable<T>> and returns a PCollection<T> containing all the elements from all the Iterables.

        Example of use:

         PCollection<Iterable<Integer>> pcOfIterables = ...;
         PCollection<Integer> pc = pcOfIterables.apply(Flatten.<Integer>iterables());

        By default, the output PCollection encodes its elements using the same Coder that the input uses for the elements in its Iterable.

        Type Parameters:
        T - the type of the elements of the input Iterable and the output PCollection

Monitor your resources on the go

Get the Google Cloud Console app to help you manage your projects.

Send feedback about...

Cloud Dataflow