Google Cloud Dataflow SDK for Java, version 1.9.1
Class FileBasedSink.FileBasedWriter<T>
- java.lang.Object
-
- com.google.cloud.dataflow.sdk.io.Sink.Writer<T,FileBasedSink.FileResult>
-
- com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriter<T>
-
- Type Parameters:
T
- the type of values to write.
- Direct Known Subclasses:
- XmlSink.XmlWriter
- Enclosing class:
- FileBasedSink<T>
public abstract static class FileBasedSink.FileBasedWriter<T> extends Sink.Writer<T,FileBasedSink.FileResult>
AbstractSink.Writer
that writes a bundle to aFileBasedSink
. Subclass implementations provide a method that can write a single value to aWritableByteChannel
(Sink.Writer.write(T)
).Subclass implementations may also override methods that write headers and footers before and after the values in a bundle, respectively, as well as provide a MIME type for the output channel.
Multiple FileBasedWriter instances may be created on the same worker, and therefore any access to static members or methods should be thread safe.
-
-
Field Summary
Fields Modifier and Type Field and Description protected String
mimeType
The MIME type used in the creation of the output channel (if the file system supports it).
-
Constructor Summary
Constructors Constructor and Description FileBasedWriter(FileBasedSink.FileBasedWriteOperation<T> writeOperation)
Construct a new FileBasedWriter with a base filename.
-
Method Summary
All Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method and Description FileBasedSink.FileResult
close()
Closes the channel and return the bundle result.FileBasedSink.FileBasedWriteOperation<T>
getWriteOperation()
Return the FileBasedWriteOperation that this Writer belongs to.void
open(String uId)
Opens the channel.protected abstract void
prepareWrite(WritableByteChannel channel)
Called with the channel that a subclass will write its header, footer, and values to.protected void
writeFooter()
Writes footer at the end of output files.protected void
writeHeader()
Writes header at the beginning of output files.-
Methods inherited from class com.google.cloud.dataflow.sdk.io.Sink.Writer
write
-
-
-
-
Field Detail
-
mimeType
protected String mimeType
The MIME type used in the creation of the output channel (if the file system supports it).GCS, for example, supports writing files with Content-Type metadata.
May be overridden. Default is
MimeTypes.TEXT
. SeeMimeTypes
for other options.
-
-
Constructor Detail
-
FileBasedWriter
public FileBasedWriter(FileBasedSink.FileBasedWriteOperation<T> writeOperation)
Construct a new FileBasedWriter with a base filename.
-
-
Method Detail
-
prepareWrite
protected abstract void prepareWrite(WritableByteChannel channel) throws Exception
Called with the channel that a subclass will write its header, footer, and values to. Subclasses should either keep a reference to the channel provided or create and keep a reference to an appropriate object that they will use to write to it.Called before any subsequent calls to writeHeader, writeFooter, and write.
- Throws:
Exception
-
writeHeader
protected void writeHeader() throws Exception
Writes header at the beginning of output files. Nothing by default; subclasses may override.- Throws:
Exception
-
writeFooter
protected void writeFooter() throws Exception
Writes footer at the end of output files. Nothing by default; subclasses may override.- Throws:
Exception
-
open
public final void open(String uId) throws Exception
Opens the channel.- Specified by:
open
in classSink.Writer<T,FileBasedSink.FileResult>
- Throws:
Exception
-
close
public final FileBasedSink.FileResult close() throws Exception
Closes the channel and return the bundle result.- Specified by:
close
in classSink.Writer<T,FileBasedSink.FileResult>
- Returns:
- the writer result
- Throws:
Exception
-
getWriteOperation
public FileBasedSink.FileBasedWriteOperation<T> getWriteOperation()
Return the FileBasedWriteOperation that this Writer belongs to.- Specified by:
getWriteOperation
in classSink.Writer<T,FileBasedSink.FileResult>
-
-