StandardCoder (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

Warning: Dataflow SDK 1.x for Java is unsupported as of October 16, 2018. See Migrating from Dataflow SDK 1.x for Java for migration guidance.

Note: If you are using Cloud Dataflow SDK version 2.x, see the Apache Beam API reference documentation.

com.google.cloud.dataflow.sdk.coders

Class StandardCoder<T>

• java.lang.Object
• • com.google.cloud.dataflow.sdk.coders.StandardCoder<T>

• All Implemented Interfaces:

  Coder<T>, Serializable

  Direct Known Subclasses:

  AvroCoder, CoGbkResult.CoGbkResultCoder, DeterministicStandardCoder, IterableLikeCoder, KvCoderBase, MapCoderBase, NullableCoder, TimestampedValue.TimestampedValueCoder, UnionCoder

  ---

  
  

  public abstract class StandardCoder<T>
  extends Object
  implements Coder<T>

  An abstract base class to implement a Coder that defines equality, hashing, and printing via the class name and recursively using getComponents().

  To extend StandardCoder, override the following methods as appropriate:

  • getComponents(): the default implementation returns Coder.getCoderArguments().
  • getEncodedElementByteSize(T, com.google.cloud.dataflow.sdk.coders.Coder.Context) and isRegisterByteSizeObserverCheap(T, com.google.cloud.dataflow.sdk.coders.Coder.Context): the default implementation encodes values to bytes and counts the bytes, which is considered expensive.
  • getEncodingId() and getAllowedEncodings(): by default, the encoding id is the empty string, so only the canonical name of the subclass will be used for compatibility checks, and no other encoding ids are allowed.

  See Also:

  Serialized Form

• • Nested Class Summary

    • Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.coders.Coder

      Coder.Context, Coder.NonDeterministicException

  • Constructor Summary

    Constructors 

    Modifier	Constructor and Description
    protected	StandardCoder()

  • Method Summary

    All Methods Instance Methods Concrete Methods 

    Modifier and Type	Method and Description
    com.google.cloud.dataflow.sdk.util.CloudObject	asCloudObject() Returns the CloudObject that represents this Coder.
    boolean	consistentWithEquals() Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).
    boolean	equals(Object o)
    Collection<String>	getAllowedEncodings() A collection of encodings supported by Coder.decode(java.io.InputStream, com.google.cloud.dataflow.sdk.coders.Coder.Context) in addition to the encoding from Coder.getEncodingId() (which is assumed supported).
    List<? extends Coder<?>>	getComponents() Returns the list of Coders that are components of this Coder.
    protected long	getEncodedElementByteSize(T value, Coder.Context context) Returns the size in bytes of the encoded value using this coder.
    String	getEncodingId() An identifier for the binary format written by Coder.encode(T, java.io.OutputStream, com.google.cloud.dataflow.sdk.coders.Coder.Context).
    int	hashCode()
    boolean	isRegisterByteSizeObserverCheap(T value, Coder.Context context) Returns whether Coder.registerByteSizeObserver(T, com.google.cloud.dataflow.sdk.util.common.ElementByteSizeObserver, com.google.cloud.dataflow.sdk.coders.Coder.Context) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).
    void	registerByteSizeObserver(T value, com.google.cloud.dataflow.sdk.util.common.ElementByteSizeObserver observer, Coder.Context context) Notifies the ElementByteSizeObserver about the byte size of the encoded value using this Coder.
    Object	structuralValue(T value) Returns an object with an Object.equals() method that represents structural equality on the argument.
    String	toString()
    protected void	verifyDeterministic(String message, Coder<?>... coders)
    protected void	verifyDeterministic(String message, Iterable<Coder<?>> coders)

    • Methods inherited from class java.lang.Object

      clone, finalize, getClass, notify, notifyAll, wait, wait, wait

    • Methods inherited from interface com.google.cloud.dataflow.sdk.coders.Coder

      decode, encode, getCoderArguments, verifyDeterministic

• • Constructor Detail

    • StandardCoder

      protected StandardCoder()

  • Method Detail

    • getEncodingId

      public String getEncodingId()

      Description copied from interface: Coder
      An identifier for the binary format written by Coder.encode(T, java.io.OutputStream, com.google.cloud.dataflow.sdk.coders.Coder.Context).

      This value, along with the fully qualified class name, forms an identifier for the binary format of this coder. Whenever this value changes, the new encoding is considered incompatible with the prior format: It is presumed that the prior version of the coder will be unable to correctly read the new format and the new version of the coder will be unable to correctly read the old format.

      If the format is changed in a backwards-compatible way (the Coder can still accept data from the prior format), such as by adding optional fields to a Protocol Buffer or Avro definition, and you want Dataflow to understand that the new coder is compatible with the prior coder, this value must remain unchanged. It is then the responsibility of Coder.decode(java.io.InputStream, com.google.cloud.dataflow.sdk.coders.Coder.Context) to correctly read data from the prior format.

      Specified by:

      getEncodingId in interface Coder<T>

    • getAllowedEncodings

      public Collection<String> getAllowedEncodings()

      Description copied from interface: Coder
      A collection of encodings supported by Coder.decode(java.io.InputStream, com.google.cloud.dataflow.sdk.coders.Coder.Context) in addition to the encoding from Coder.getEncodingId() (which is assumed supported).

      This information is not currently used for any purpose. It is descriptive only, and this method is subject to change.

      Specified by:

      getAllowedEncodings in interface Coder<T>

      See Also:

      Coder.getEncodingId()

    • getComponents

      public List<? extends Coder<?>> getComponents()

      Returns the list of Coders that are components of this Coder.

    • equals

      public boolean equals(Object o)

      Overrides:

      equals in class Object

      Returns:

      true if the two StandardCoder instances have the same class and equal components.

    • hashCode

      public int hashCode()

      Overrides:

      hashCode in class Object

    • toString

      public String toString()

      Overrides:

      toString in class Object

    • asCloudObject

      public com.google.cloud.dataflow.sdk.util.CloudObject asCloudObject()

      Description copied from interface: Coder
      Returns the CloudObject that represents this Coder.

      Specified by:

      asCloudObject in interface Coder<T>

    • isRegisterByteSizeObserverCheap

      public boolean isRegisterByteSizeObserverCheap(T value,
                                                     Coder.Context context)

      Returns whether Coder.registerByteSizeObserver(T, com.google.cloud.dataflow.sdk.util.common.ElementByteSizeObserver, com.google.cloud.dataflow.sdk.coders.Coder.Context) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).

      Not intended to be called by user code, but instead by PipelineRunner implementations.

      Specified by:

      isRegisterByteSizeObserverCheap in interface Coder<T>

      Returns:

      false unless it is overridden. registerByteSizeObserver(T, com.google.cloud.dataflow.sdk.util.common.ElementByteSizeObserver, com.google.cloud.dataflow.sdk.coders.Coder.Context) invokes getEncodedElementByteSize(T, com.google.cloud.dataflow.sdk.coders.Coder.Context) which requires re-encoding an element unless it is overridden. This is considered expensive.

    • getEncodedElementByteSize

      protected long getEncodedElementByteSize(T value,
                                               Coder.Context context)
                                        throws Exception

      Returns the size in bytes of the encoded value using this coder.

      Throws:

      Exception

    • registerByteSizeObserver

      public void registerByteSizeObserver(T value,
                                           com.google.cloud.dataflow.sdk.util.common.ElementByteSizeObserver observer,
                                           Coder.Context context)
                                    throws Exception

      Notifies the ElementByteSizeObserver about the byte size of the encoded value using this Coder.

      Not intended to be called by user code, but instead by PipelineRunner implementations.

      For StandardCoder subclasses, this notifies observer about the byte size of the encoded value using this coder as returned by getEncodedElementByteSize(T, com.google.cloud.dataflow.sdk.coders.Coder.Context).

      Specified by:

      registerByteSizeObserver in interface Coder<T>

      Throws:

      Exception

    • verifyDeterministic

      protected void verifyDeterministic(String message,
                                         Iterable<Coder<?>> coders)
                                  throws Coder.NonDeterministicException

      Throws:

      Coder.NonDeterministicException

    • verifyDeterministic

      protected void verifyDeterministic(String message,
                                         Coder<?>... coders)
                                  throws Coder.NonDeterministicException

      Throws:

      Coder.NonDeterministicException

    • consistentWithEquals

      public boolean consistentWithEquals()

      Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

      Whenever the encoded bytes of two values are equal, then the original values are equal according to Objects.equals(). Note that this is well-defined for null.

      This condition is most notably false for arrays. More generally, this condition is false whenever equals() compares object identity, rather than performing a semantic/structural comparison.

      Specified by:

      consistentWithEquals in interface Coder<T>

      Returns:

      false for StandardCoder unless overridden.

    • structuralValue

      public Object structuralValue(T value)
                             throws Exception

      Description copied from interface: Coder
      Returns an object with an Object.equals() method that represents structural equality on the argument.

      For any two values x and y of type T, if their encoded bytes are the same, then it must be the case that structuralValue(x).equals(@code structuralValue(y).

      Most notably:

      • The structural value for an array coder should perform a structural comparison of the contents of the arrays, rather than the default behavior of comparing according to object identity.
      • The structural value for a coder accepting null should be a proper object with an equals() method, even if the input value is null.

      See also Coder.consistentWithEquals().

      Specified by:

      structuralValue in interface Coder<T>

      Throws:

      Exception