TextIO.Write.Bound (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

com.google.cloud.dataflow.sdk.io

Class TextIO.Write.Bound<T>

  • Type Parameters:
    T - the type of the elements of the input PCollection
    All Implemented Interfaces:
    HasDisplayData, Serializable
    Enclosing class:
    TextIO.Write


    public static class TextIO.Write.Bound<T>
    extends PTransform<PCollection<T>,PDone>
    A PTransform that writes a bounded PCollection to a text file (or multiple text files matching a sharding pattern), with each PCollection element being encoded into its own line.
    See Also:
    Serialized Form
    • Method Detail

      • named

        public TextIO.Write.Bound<T> named(String name)
        Returns a transform for writing to text files that's like this one but with the given step name.

        Does not modify this object.

      • to

        public TextIO.Write.Bound<T> to(String filenamePrefix)
        Returns a transform for writing to text files that's like this one but that writes to the file(s) with the given filename prefix.

        See Write.to(String) for more information.

        Does not modify this object.

      • withSuffix

        public TextIO.Write.Bound<T> withSuffix(String nameExtension)
        Returns a transform for writing to text files that that's like this one but that writes to the file(s) with the given filename suffix.

        Does not modify this object.

        See Also:
        ShardNameTemplate
      • withNumShards

        public TextIO.Write.Bound<T> withNumShards(int numShards)
        Returns a transform for writing to text files that's like this one but that uses the provided shard count.

        Constraining the number of shards is likely to reduce the performance of a pipeline. Setting this value is not recommended unless you require a specific number of output files.

        Does not modify this object.

        Parameters:
        numShards - the number of shards to use, or 0 to let the system decide.
        See Also:
        ShardNameTemplate
      • withShardNameTemplate

        public TextIO.Write.Bound<T> withShardNameTemplate(String shardTemplate)
        Returns a transform for writing to text files that's like this one but that uses the given shard name template.

        Does not modify this object.

        See Also:
        ShardNameTemplate
      • withoutSharding

        public TextIO.Write.Bound<T> withoutSharding()
        Returns a transform for writing to text files that's like this one but that forces a single file as output.

        Constraining the number of shards is likely to reduce the performance of a pipeline. Using this setting is not recommended unless you truly require a single output file.

        This is a shortcut for .withNumShards(1).withShardNameTemplate("")

        Does not modify this object.

      • withCoder

        public <X> TextIO.Write.Bound<X> withCoder(Coder<X> coder)
        Returns a transform for writing to text files that's like this one but that uses the given Coder to encode each of the elements of the input PCollection into an output text line. Does not modify this object.
        Type Parameters:
        X - the type of the elements of the input PCollection
      • withHeader

        public TextIO.Write.Bound<T> withHeader(@Nullable
                                                String header)
        Returns a transform for writing to text files that adds a header string to the files it writes. Note that a newline character will be added after the header.

        A null value will clear any previously configured header.

        Does not modify this object.

        Parameters:
        header - the string to be added as file header
      • withFooter

        public TextIO.Write.Bound<T> withFooter(@Nullable
                                                String footer)
        Returns a transform for writing to text files that adds a footer string to the files it writes. Note that a newline character will be added after the header.

        A null value will clear any previously configured footer.

        Does not modify this object.

        Parameters:
        footer - the string to be added as file footer
      • withoutValidation

        public TextIO.Write.Bound<T> withoutValidation()
        Returns a transform for writing to text files that's like this one but that has GCS output path validation on pipeline creation disabled.

        This can be useful in the case where the GCS output location does not exist at the pipeline creation time, but is expected to be available at execution time.

        Does not modify this object.

      • populateDisplayData

        public void populateDisplayData(DisplayData.Builder builder)
        Description copied from class: PTransform
        Register display data for the given transform or component.

        populateDisplayData(DisplayData.Builder) is invoked by Pipeline runners to collect display data via DisplayData.from(HasDisplayData). Implementations may call super.populateDisplayData(builder) in order to register display data in the current namespace, but should otherwise use subcomponent.populateDisplayData(builder) to use the namespace of the subcomponent.

        By default, does not register any display data. Implementors may override this method to provide their own display data.

        Specified by:
        populateDisplayData in interface HasDisplayData
        Overrides:
        populateDisplayData in class PTransform<PCollection<T>,PDone>
        Parameters:
        builder - The builder to populate with display data.
        See Also:
        HasDisplayData
      • getShardNameTemplate

        public String getShardNameTemplate()
        Returns the current shard name template string.
      • getFilenamePrefix

        public String getFilenamePrefix()
      • getShardTemplate

        public String getShardTemplate()
      • getNumShards

        public int getNumShards()
      • getFilenameSuffix

        public String getFilenameSuffix()
      • getCoder

        public Coder<T> getCoder()
      • needsValidation

        public boolean needsValidation()


Send feedback about...

Cloud Dataflow