Google Cloud Dataflow SDK for Java, version 1.9.1
Class Sink.Writer<T,WriteT>
- java.lang.Object
-
- com.google.cloud.dataflow.sdk.io.Sink.Writer<T,WriteT>
-
- Type Parameters:
T
- The type of object to writeWriteT
- The writer results type (e.g., the bundle's output filename, as String)
- Direct Known Subclasses:
- FileBasedSink.FileBasedWriter
public abstract static class Sink.Writer<T,WriteT> extends Object
A Writer writes a bundle of elements from a PCollection to a sink.open(java.lang.String)
is called before writing begins andclose()
is called after all elements in the bundle have been written.write(T)
writes an element to the sink.Note that any access to static members or methods of a Writer must be thread-safe, as multiple instances of a Writer may be instantiated in different threads on the same worker.
See
Sink
for more detailed documentation about the process of writing to a Sink.
-
-
Constructor Summary
Constructors Constructor and Description Writer()
-
Method Summary
All Methods Instance Methods Abstract Methods Modifier and Type Method and Description abstract WriteT
close()
Finishes writing the bundle.abstract Sink.WriteOperation<T,WriteT>
getWriteOperation()
Returns the write operation this writer belongs to.abstract void
open(String uId)
Performs bundle initialization.abstract void
write(T value)
Called for each value in the bundle.
-
-
-
Method Detail
-
open
public abstract void open(String uId) throws Exception
Performs bundle initialization. For example, creates a temporary file for writing or initializes any state that will be used across calls towrite(T)
.The unique id that is given to open should be used to ensure that the writer's output does not interfere with the output of other Writers, as a bundle may be executed many times for fault tolerance. See
Sink
for more information about bundle ids.- Throws:
Exception
-
write
public abstract void write(T value) throws Exception
Called for each value in the bundle.- Throws:
Exception
-
close
public abstract WriteT close() throws Exception
Finishes writing the bundle. Closes any resources used for writing the bundle.Returns a writer result that will be used in the
Sink.WriteOperation
's finalization. The result should contain some way to identify the output of this bundle (using the bundle id).Sink.WriteOperation.finalize(java.lang.Iterable<WriteT>, com.google.cloud.dataflow.sdk.options.PipelineOptions)
will use the writer result to identify successful writes. SeeSink
for more information about bundle ids.- Returns:
- the writer result
- Throws:
Exception
-
getWriteOperation
public abstract Sink.WriteOperation<T,WriteT> getWriteOperation()
Returns the write operation this writer belongs to.
-
-