IterableLikeCoder (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

Warning: Dataflow SDK 1.x for Java is unsupported as of October 16, 2018. After August 12, 2020, you can use unsupported SDKs temporarily while you upgrade your pipelines. For more information, see Using an unsupported SDK.

For information on migrating from the 1.x SDK, see Migrating from Dataflow SDK 1.x for Java.

Note: If you are using Cloud Dataflow SDK version 2.x, see the Apache Beam API reference documentation.

com.google.cloud.dataflow.sdk.coders

Class IterableLikeCoder<T,IterableT extends Iterable<T>>

• java.lang.Object
• • com.google.cloud.dataflow.sdk.coders.StandardCoder<IterableT>
  • • com.google.cloud.dataflow.sdk.coders.IterableLikeCoder<T,IterableT>

• Type Parameters:

  T - the type of the elements of the Iterables being transcoded

  IterableT - the type of the Iterables being transcoded

  All Implemented Interfaces:

  Coder<IterableT>, Serializable

  Direct Known Subclasses:

  CollectionCoder, IterableCoder, ListCoder, SetCoder

  ---

  
  

  public abstract class IterableLikeCoder<T,IterableT extends Iterable<T>>
  extends StandardCoder<IterableT>

  An abstract base class with functionality for assembling a Coder for a class that implements Iterable.

  To complete a subclass, implement the decodeToIterable(java.util.List<T>) method. This superclass will decode the elements in the input stream into a List and then pass them to that method to be converted into the appropriate iterable type. Note that this means the input iterables must fit into memory.

  The format of this coder is as follows:

  • If the input Iterable has a known and finite size, then the size is written to the output stream in big endian format, followed by all of the encoded elements.
  • If the input Iterable is not known to have a finite size, then each element of the input is preceded by true encoded as a byte (indicating "more data") followed by the encoded element, and terminated by false encoded as a byte.

  See Also:

  Serialized Form

• • Nested Class Summary

    • Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.coders.Coder

      Coder.Context, Coder.NonDeterministicException

  • Constructor Summary

    Constructors 

    Modifier	Constructor and Description
    protected	IterableLikeCoder(Coder<T> elementCoder, String iterableName)

  • Method Summary

    All Methods Static Methods Instance Methods Abstract Methods Concrete Methods 

    Modifier and Type	Method and Description
    IterableT	decode(InputStream inStream, Coder.Context context) Decodes a value of type T from the given input stream in the given context.
    protected abstract IterableT	decodeToIterable(List<T> decodedElements) Builds an instance of IterableT, this coder's associated Iterable-like subtype, from a list of decoded elements.
    void	encode(IterableT iterable, OutputStream outStream, Coder.Context context) Encodes the given value of type T onto the given output stream in the given context.
    List<? extends Coder<?>>	getCoderArguments() If this is a Coder for a parameterized type, returns the list of Coders being used for each of the parameters, or returns null if this cannot be done or this is not a parameterized type.
    Coder<T>	getElemCoder()
    protected static <T,IterableT extends Iterable<T>> List<Object>	getInstanceComponentsHelper(IterableT exampleValue) Returns the first element in the iterable-like exampleValue if it is non-empty, otherwise returns null.
    boolean	isRegisterByteSizeObserverCheap(IterableT iterable, Coder.Context context) Returns whether Coder.registerByteSizeObserver(T, com.google.cloud.dataflow.sdk.util.common.ElementByteSizeObserver, com.google.cloud.dataflow.sdk.coders.Coder.Context) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).
    void	registerByteSizeObserver(IterableT iterable, com.google.cloud.dataflow.sdk.util.common.ElementByteSizeObserver observer, Coder.Context context) Notifies the ElementByteSizeObserver about the byte size of the encoded value using this Coder.
    void	verifyDeterministic() Throw Coder.NonDeterministicException if the coding is not deterministic.

    • Methods inherited from class com.google.cloud.dataflow.sdk.coders.StandardCoder

      asCloudObject, consistentWithEquals, equals, getAllowedEncodings, getComponents, getEncodedElementByteSize, getEncodingId, hashCode, structuralValue, toString, verifyDeterministic, verifyDeterministic

    • Methods inherited from class java.lang.Object

      clone, finalize, getClass, notify, notifyAll, wait, wait, wait

• • Constructor Detail

    • IterableLikeCoder

      protected IterableLikeCoder(Coder<T> elementCoder,
                                  String iterableName)

  • Method Detail

    • getElemCoder

      public Coder<T> getElemCoder()

    • decodeToIterable

      protected abstract IterableT decodeToIterable(List<T> decodedElements)

      Builds an instance of IterableT, this coder's associated Iterable-like subtype, from a list of decoded elements.

    • getInstanceComponentsHelper

      protected static <T,IterableT extends Iterable<T>> List<Object> getInstanceComponentsHelper(IterableT exampleValue)

      Returns the first element in the iterable-like exampleValue if it is non-empty, otherwise returns null.

    • encode

      public void encode(IterableT iterable,
                         OutputStream outStream,
                         Coder.Context context)
                  throws IOException,
                         CoderException

      Description copied from interface: Coder
      Encodes the given value of type T onto the given output stream in the given context.

      Throws:

      IOException - if writing to the OutputStream fails for some reason

      CoderException - if the value could not be encoded for some reason

    • decode

      public IterableT decode(InputStream inStream,
                              Coder.Context context)
                       throws IOException,
                              CoderException

      Description copied from interface: Coder
      Decodes a value of type T from the given input stream in the given context. Returns the decoded value.

      Throws:

      IOException - if reading from the InputStream fails for some reason

      CoderException - if the value could not be decoded for some reason

    • getCoderArguments

      public List<? extends Coder<?>> getCoderArguments()

      Description copied from interface: Coder
      If this is a Coder for a parameterized type, returns the list of Coders being used for each of the parameters, or returns null if this cannot be done or this is not a parameterized type.

    • verifyDeterministic

      public void verifyDeterministic()
                               throws Coder.NonDeterministicException

      Throw Coder.NonDeterministicException if the coding is not deterministic.

      In order for a Coder to be considered deterministic, the following must be true:

      • two values that compare as equal (via Object.equals() or Comparable.compareTo(), if supported) have the same encoding.
      • the Coder always produces a canonical encoding, which is the same for an instance of an object even if produced on different computers at different times.

      Throws:

      NonDeterministicException - always. Encoding is not deterministic for the general Iterable case, as it depends upon the type of iterable. This may allow two objects to compare as equal while the encoding differs.

      Coder.NonDeterministicException - if this coder is not deterministic.

    • isRegisterByteSizeObserverCheap

      public boolean isRegisterByteSizeObserverCheap(IterableT iterable,
                                                     Coder.Context context)

      Returns whether Coder.registerByteSizeObserver(T, com.google.cloud.dataflow.sdk.util.common.ElementByteSizeObserver, com.google.cloud.dataflow.sdk.coders.Coder.Context) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).

      Not intended to be called by user code, but instead by PipelineRunner implementations.

      Specified by:

      isRegisterByteSizeObserverCheap in interface Coder<IterableT extends Iterable<T>>

      Overrides:

      isRegisterByteSizeObserverCheap in class StandardCoder<IterableT extends Iterable<T>>

      Returns:

      true if the iterable is of a known class that supports lazy counting of byte size, since that requires minimal extra computation.

    • registerByteSizeObserver

      public void registerByteSizeObserver(IterableT iterable,
                                           com.google.cloud.dataflow.sdk.util.common.ElementByteSizeObserver observer,
                                           Coder.Context context)
                                    throws Exception

      Description copied from class: StandardCoder
      Notifies the ElementByteSizeObserver about the byte size of the encoded value using this Coder.

      Not intended to be called by user code, but instead by PipelineRunner implementations.

      For StandardCoder subclasses, this notifies observer about the byte size of the encoded value using this coder as returned by StandardCoder.getEncodedElementByteSize(T, com.google.cloud.dataflow.sdk.coders.Coder.Context).

      Specified by:

      registerByteSizeObserver in interface Coder<IterableT extends Iterable<T>>

      Overrides:

      registerByteSizeObserver in class StandardCoder<IterableT extends Iterable<T>>

      Throws:

      Exception