Google Cloud Dataflow SDK for Java, version 1.9.1
Class TextIO.Write.Bound<T>
- java.lang.Object
-
- com.google.cloud.dataflow.sdk.transforms.PTransform<PCollection<T>,PDone>
-
- com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound<T>
-
- Type Parameters:
T
- the type of the elements of the input PCollection
- All Implemented Interfaces:
- HasDisplayData, Serializable
- Enclosing class:
- TextIO.Write
public static class TextIO.Write.Bound<T> extends PTransform<PCollection<T>,PDone>
A PTransform that writes a bounded PCollection to a text file (or multiple text files matching a sharding pattern), with each PCollection element being encoded into its own line.- See Also:
- Serialized Form
-
-
Field Summary
-
Fields inherited from class com.google.cloud.dataflow.sdk.transforms.PTransform
name
-
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method and Description PDone
apply(PCollection<T> input)
Applies thisPTransform
on the givenInputT
, and returns itsOutput
.Coder<T>
getCoder()
protected Coder<Void>
getDefaultOutputCoder()
Returns the defaultCoder
to use for the output of this single-outputPTransform
.String
getFilenamePrefix()
String
getFilenameSuffix()
String
getFooter()
String
getHeader()
int
getNumShards()
String
getShardNameTemplate()
Returns the current shard name template string.String
getShardTemplate()
TextIO.Write.Bound<T>
named(String name)
Returns a transform for writing to text files that's like this one but with the given step name.boolean
needsValidation()
void
populateDisplayData(DisplayData.Builder builder)
Register display data for the given transform or component.TextIO.Write.Bound<T>
to(String filenamePrefix)
Returns a transform for writing to text files that's like this one but that writes to the file(s) with the given filename prefix.TextIO.Write.Bound<T>
to(ValueProvider<String> filenamePrefix)
Liketo(String)
, but with aValueProvider
.<X> TextIO.Write.Bound<X>
withCoder(Coder<X> coder)
Returns a transform for writing to text files that's like this one but that uses the givenCoder
to encode each of the elements of the inputPCollection
into an output text line.TextIO.Write.Bound<T>
withFooter(String footer)
Returns a transform for writing to text files that adds a footer string to the files it writes.TextIO.Write.Bound<T>
withHeader(String header)
Returns a transform for writing to text files that adds a header string to the files it writes.TextIO.Write.Bound<T>
withNumShards(int numShards)
Returns a transform for writing to text files that's like this one but that uses the provided shard count.TextIO.Write.Bound<T>
withoutSharding()
Returns a transform for writing to text files that's like this one but that forces a single file as output.TextIO.Write.Bound<T>
withoutValidation()
Returns a transform for writing to text files that's like this one but that has Cloud Storage output path validation on pipeline creation disabled.TextIO.Write.Bound<T>
withShardNameTemplate(String shardTemplate)
Returns a transform for writing to text files that's like this one but that uses the given shard name template.TextIO.Write.Bound<T>
withSuffix(String nameExtension)
Returns a transform for writing to text files that that's like this one but that writes to the file(s) with the given filename suffix.-
Methods inherited from class com.google.cloud.dataflow.sdk.transforms.PTransform
getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, toString, validate
-
-
-
-
Method Detail
-
named
public TextIO.Write.Bound<T> named(String name)
Returns a transform for writing to text files that's like this one but with the given step name.Does not modify this object.
-
to
public TextIO.Write.Bound<T> to(String filenamePrefix)
Returns a transform for writing to text files that's like this one but that writes to the file(s) with the given filename prefix.See
Write.to(String)
for more information.Does not modify this object.
-
to
public TextIO.Write.Bound<T> to(ValueProvider<String> filenamePrefix)
Liketo(String)
, but with aValueProvider
.
-
withSuffix
public TextIO.Write.Bound<T> withSuffix(String nameExtension)
Returns a transform for writing to text files that that's like this one but that writes to the file(s) with the given filename suffix.Does not modify this object.
- See Also:
ShardNameTemplate
-
withNumShards
public TextIO.Write.Bound<T> withNumShards(int numShards)
Returns a transform for writing to text files that's like this one but that uses the provided shard count.Constraining the number of shards is likely to reduce the performance of a pipeline. Setting this value is not recommended unless you require a specific number of output files.
Does not modify this object.
- Parameters:
numShards
- the number of shards to use, or 0 to let the system decide.- See Also:
ShardNameTemplate
-
withShardNameTemplate
public TextIO.Write.Bound<T> withShardNameTemplate(String shardTemplate)
Returns a transform for writing to text files that's like this one but that uses the given shard name template.Does not modify this object.
- See Also:
ShardNameTemplate
-
withoutSharding
public TextIO.Write.Bound<T> withoutSharding()
Returns a transform for writing to text files that's like this one but that forces a single file as output.Constraining the number of shards is likely to reduce the performance of a pipeline. Using this setting is not recommended unless you truly require a single output file.
This is a shortcut for
.withNumShards(1).withShardNameTemplate("")
Does not modify this object.
-
withCoder
public <X> TextIO.Write.Bound<X> withCoder(Coder<X> coder)
Returns a transform for writing to text files that's like this one but that uses the givenCoder
to encode each of the elements of the inputPCollection
into an output text line. Does not modify this object.- Type Parameters:
X
- the type of the elements of the inputPCollection
-
withHeader
public TextIO.Write.Bound<T> withHeader(@Nullable String header)
Returns a transform for writing to text files that adds a header string to the files it writes. Note that a newline character will be added after the header.A
null
value will clear any previously configured header.Does not modify this object.
- Parameters:
header
- the string to be added as file header
-
withFooter
public TextIO.Write.Bound<T> withFooter(@Nullable String footer)
Returns a transform for writing to text files that adds a footer string to the files it writes. Note that a newline character will be added after the header.A
null
value will clear any previously configured footer.Does not modify this object.
- Parameters:
footer
- the string to be added as file footer
-
withoutValidation
public TextIO.Write.Bound<T> withoutValidation()
Returns a transform for writing to text files that's like this one but that has Cloud Storage output path validation on pipeline creation disabled.This can be useful in the case where the Cloud Storage output location does not exist at the pipeline creation time, but is expected to be available at execution time.
Does not modify this object.
-
apply
public PDone apply(PCollection<T> input)
Description copied from class:PTransform
Applies thisPTransform
on the givenInputT
, and returns itsOutput
.Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).
The default implementation throws an exception. A derived class must either implement apply, or else each runner must supply a custom implementation via
PipelineRunner.apply(com.google.cloud.dataflow.sdk.transforms.PTransform<InputT, OutputT>, InputT)
.- Overrides:
apply
in classPTransform<PCollection<T>,PDone>
-
populateDisplayData
public void populateDisplayData(DisplayData.Builder builder)
Description copied from class:PTransform
Register display data for the given transform or component.populateDisplayData(DisplayData.Builder)
is invoked by Pipeline runners to collect display data viaDisplayData.from(HasDisplayData)
. Implementations may callsuper.populateDisplayData(builder)
in order to register display data in the current namespace, but should otherwise usesubcomponent.populateDisplayData(builder)
to use the namespace of the subcomponent.By default, does not register any display data. Implementors may override this method to provide their own display data.
- Specified by:
populateDisplayData
in interfaceHasDisplayData
- Overrides:
populateDisplayData
in classPTransform<PCollection<T>,PDone>
- Parameters:
builder
- The builder to populate with display data.- See Also:
HasDisplayData
-
getShardNameTemplate
public String getShardNameTemplate()
Returns the current shard name template string.
-
getDefaultOutputCoder
protected Coder<Void> getDefaultOutputCoder()
Description copied from class:PTransform
Returns the defaultCoder
to use for the output of this single-outputPTransform
.By default, always throws
- Overrides:
getDefaultOutputCoder
in classPTransform<PCollection<T>,PDone>
-
getFilenamePrefix
public String getFilenamePrefix()
-
getShardTemplate
public String getShardTemplate()
-
getNumShards
public int getNumShards()
-
getFilenameSuffix
public String getFilenameSuffix()
-
getHeader
@Nullable public String getHeader()
-
getFooter
@Nullable public String getFooter()
-
needsValidation
public boolean needsValidation()
-
-