Flatten (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

Warning: Dataflow SDK 1.x for Java is unsupported as of October 16, 2018. After August 12, 2020, Dataflow will not run jobs using Dataflow 1.x and below. See Migrating from Dataflow SDK 1.x for Java for migration guidance.

Note: If you are using Cloud Dataflow SDK version 2.x, see the Apache Beam API reference documentation.

com.google.cloud.dataflow.sdk.transforms

Class Flatten

• java.lang.Object
• • com.google.cloud.dataflow.sdk.transforms.Flatten

• ---

  
  

  public class Flatten
  extends Object

  Flatten<T> takes multiple PCollection<T>s bundled into a PCollectionList<T> and returns a single PCollection<T> containing all the elements in all the input PCollections. The name "Flatten" suggests taking a list of lists and flattening them into a single list.

  Example of use:

   
   PCollection<String> pc1 = ...;
   PCollection<String> pc2 = ...;
   PCollection<String> pc3 = ...;
   PCollectionList<String> pcs = PCollectionList.of(pc1).and(pc2).and(pc3);
   PCollection<String> merged = pcs.apply(Flatten.<String>pCollections());
    

  By default, the Coder of the output PCollection is the same as the Coder of the first PCollection in the input PCollectionList (if the PCollectionList is non-empty).

• • Nested Class Summary

    Nested Classes 

    Modifier and Type	Class and Description
    static class	Flatten.FlattenIterables<T> FlattenIterables<T> takes a PCollection<Iterable<T>> and returns a PCollection<T> that contains all the elements from each iterable.
    static class	Flatten.FlattenPCollectionList<T> A PTransform that flattens a PCollectionList into a PCollection containing all the elements of all the PCollections in its input.

  • Constructor Summary

    Constructors 

    Constructor and Description
    Flatten()

  • Method Summary

    All Methods Static Methods Concrete Methods 

    Modifier and Type	Method and Description
    static <T> Flatten.FlattenIterables<T>	iterables() Returns a PTransform that takes a PCollection<Iterable<T>> and returns a PCollection<T> containing all the elements from all the Iterables.
    static <T> Flatten.FlattenPCollectionList<T>	pCollections() Returns a PTransform that flattens a PCollectionList into a PCollection containing all the elements of all the PCollections in its input.

    • Methods inherited from class java.lang.Object

      clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

• • Constructor Detail

    • Flatten

      public Flatten()

  • Method Detail

    • pCollections

      public static <T> Flatten.FlattenPCollectionList<T> pCollections()

      Returns a PTransform that flattens a PCollectionList into a PCollection containing all the elements of all the PCollections in its input.

      All inputs must have equal WindowFns. The output elements of Flatten<T> are in the same windows and have the same timestamps as their corresponding input elements. The output PCollection will have the same WindowFn as all of the inputs.

      Type Parameters:

      T - the type of the elements in the input and output PCollections.

    • iterables

      public static <T> Flatten.FlattenIterables<T> iterables()

      Returns a PTransform that takes a PCollection<Iterable<T>> and returns a PCollection<T> containing all the elements from all the Iterables.

      Example of use:

       
       PCollection<Iterable<Integer>> pcOfIterables = ...;
       PCollection<Integer> pc = pcOfIterables.apply(Flatten.<Integer>iterables());
        

      By default, the output PCollection encodes its elements using the same Coder that the input uses for the elements in its Iterable.

      Type Parameters:

      T - the type of the elements of the input Iterable and the output PCollection