Flatten (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

Note: If you are using Cloud Dataflow SDK version 2.x, see the Apache Beam API reference documentation.

com.google.cloud.dataflow.sdk.transforms

Class Flatten

• java.lang.Object
• • com.google.cloud.dataflow.sdk.transforms.Flatten

• ---

  
  

  public class Flatten
  extends Object

  Flatten<T> takes multiple PCollection<T>s bundled into a PCollectionList<T> and returns a single PCollection<T> containing all the elements in all the input PCollections. The name "Flatten" suggests taking a list of lists and flattening them into a single list.

  Example of use:

   
   PCollection<String> pc1 = ...;
   PCollection<String> pc2 = ...;
   PCollection<String> pc3 = ...;
   PCollectionList<String> pcs = PCollectionList.of(pc1).and(pc2).and(pc3);
   PCollection<String> merged = pcs.apply(Flatten.<String>pCollections());
    

  By default, the Coder of the output PCollection is the same as the Coder of the first PCollection in the input PCollectionList (if the PCollectionList is non-empty).

• • Nested Class Summary

    Nested Classes 

    Modifier and Type	Class and Description
    static class	Flatten.FlattenIterables<T> FlattenIterables<T> takes a PCollection<Iterable<T>> and returns a PCollection<T> that contains all the elements from each iterable.
    static class	Flatten.FlattenPCollectionList<T> A PTransform that flattens a PCollectionList into a PCollection containing all the elements of all the PCollections in its input.

  • Constructor Summary

    Constructors 

    Constructor and Description
    Flatten()

  • Method Summary

    All Methods Static Methods Concrete Methods 

    Modifier and Type	Method and Description
    static <T> Flatten.FlattenIterables<T>	iterables() Returns a PTransform that takes a PCollection<Iterable<T>> and returns a PCollection<T> containing all the elements from all the Iterables.
    static <T> Flatten.FlattenPCollectionList<T>	pCollections() Returns a PTransform that flattens a PCollectionList into a PCollection containing all the elements of all the PCollections in its input.

    • Methods inherited from class java.lang.Object

      clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

• • Constructor Detail

    • Flatten

      public Flatten()

  • Method Detail

    • pCollections

      public static <T> Flatten.FlattenPCollectionList<T> pCollections()

      Returns a PTransform that flattens a PCollectionList into a PCollection containing all the elements of all the PCollections in its input.

      All inputs must have equal WindowFns. The output elements of Flatten<T> are in the same windows and have the same timestamps as their corresponding input elements. The output PCollection will have the same WindowFn as all of the inputs.

      Type Parameters:

      T - the type of the elements in the input and output PCollections.

    • iterables

      public static <T> Flatten.FlattenIterables<T> iterables()

      Returns a PTransform that takes a PCollection<Iterable<T>> and returns a PCollection<T> containing all the elements from all the Iterables.

      Example of use:

       
       PCollection<Iterable<Integer>> pcOfIterables = ...;
       PCollection<Integer> pc = pcOfIterables.apply(Flatten.<Integer>iterables());
        

      By default, the output PCollection encodes its elements using the same Coder that the input uses for the elements in its Iterable.

      Type Parameters:

      T - the type of the elements of the input Iterable and the output PCollection