Sink.Writer (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

com.google.cloud.dataflow.sdk.io

Class Sink.Writer<T,WriteT>

  • Type Parameters:
    T - The type of object to write
    WriteT - The writer results type (e.g., the bundle's output filename, as String)
    Direct Known Subclasses:
    FileBasedSink.FileBasedWriter
    Enclosing class:
    Sink<T>

    public abstract static class Sink.Writer<T,WriteT>
extends Object
    A Writer writes a bundle of elements from a PCollection to a sink. open(java.lang.String) is called before writing begins and close() is called after all elements in the bundle have been written. write(T) writes an element to the sink.

    Note that any access to static members or methods of a Writer must be thread-safe, as multiple instances of a Writer may be instantiated in different threads on the same worker.

    See Sink for more detailed documentation about the process of writing to a Sink.

    • Constructor Detail

      • Writer

        public Writer()

    • Method Detail

      • open

        public abstract void open(String uId)
                   throws Exception
        Performs bundle initialization. For example, creates a temporary file for writing or initializes any state that will be used across calls to write(T).

        The unique id that is given to open should be used to ensure that the writer's output does not interfere with the output of other Writers, as a bundle may be executed many times for fault tolerance. See Sink for more information about bundle ids.

        Throws:
        Exception

      • write

        public abstract void write(T value)
                    throws Exception
        Called for each value in the bundle.
        Throws:
        Exception

      • getWriteOperation

        public abstract Sink.WriteOperation<T,WriteT> getWriteOperation()
        Returns the write operation this writer belongs to.