RemoveDuplicates.WithRepresentativeValues (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

Warning: Dataflow SDK 1.x for Java is unsupported as of October 16, 2018. After August 12, 2020, you can use unsupported SDKs temporarily while you upgrade your pipelines. For more information, see Using an unsupported SDK.

For information on migrating from the 1.x SDK, see Migrating from Dataflow SDK 1.x for Java.

Note: If you are using Cloud Dataflow SDK version 2.x, see the Apache Beam API reference documentation for the Distinct transform.

com.google.cloud.dataflow.sdk.transforms

Class RemoveDuplicates.WithRepresentativeValues<T,IdT>

• java.lang.Object
• • com.google.cloud.dataflow.sdk.transforms.PTransform<PCollection<T>,PCollection<T>>
  • • com.google.cloud.dataflow.sdk.transforms.RemoveDuplicates.WithRepresentativeValues<T,IdT>

• Type Parameters:

  T - the type of input and output element

  IdT - the type of representative values used to dedup

  All Implemented Interfaces:

  HasDisplayData, Serializable

  Enclosing class:

  RemoveDuplicates<T>

  ---

  
  

  public static class RemoveDuplicates.WithRepresentativeValues<T,IdT>
  extends PTransform<PCollection<T>,PCollection<T>>

  A RemoveDuplicates PTransform that uses a SerializableFunction to obtain a representative value for each input element.

  Construct via RemoveDuplicates.withRepresentativeValueFn(SerializableFunction).

  See Also:

  Serialized Form

• • Field Summary

    • Fields inherited from class com.google.cloud.dataflow.sdk.transforms.PTransform

      name

  • Method Summary

    All Methods Instance Methods Concrete Methods 

    Modifier and Type	Method and Description
    PCollection<T>	apply(PCollection<T> in) Applies this PTransform on the given InputT, and returns its Output.
    RemoveDuplicates.WithRepresentativeValues<T,IdT>	withRepresentativeType(TypeDescriptor<IdT> type) Return a WithRepresentativeValues PTransform that is like this one, but with the specified output type descriptor.

    • Methods inherited from class com.google.cloud.dataflow.sdk.transforms.PTransform

      getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, populateDisplayData, toString, validate

    • Methods inherited from class java.lang.Object

      clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

• • Method Detail

    • apply

      public PCollection<T> apply(PCollection<T> in)

      Description copied from class: PTransform
      Applies this PTransform on the given InputT, and returns its Output.

      Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

      The default implementation throws an exception. A derived class must either implement apply, or else each runner must supply a custom implementation via PipelineRunner.apply(com.google.cloud.dataflow.sdk.transforms.PTransform<InputT, OutputT>, InputT).

      Overrides:

      apply in class PTransform<PCollection<T>,PCollection<T>>

    • withRepresentativeType

      public RemoveDuplicates.WithRepresentativeValues<T,IdT> withRepresentativeType(TypeDescriptor<IdT> type)

      Return a WithRepresentativeValues PTransform that is like this one, but with the specified output type descriptor.

      Required for use of RemoveDuplicates.withRepresentativeValueFn(SerializableFunction) in Java 8 with a lambda as the fn.

      Parameters:

      type - a TypeDescriptor describing the representative type of this WithRepresentativeValues

      Returns:

      A WithRepresentativeValues PTransform that is like this one, but with the specified output type descriptor.