TextIO.Read (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

Warning: Cloud Dataflow SDK 1.x for Java is unsupported as of October 16, 2018. See Migrating from Cloud Dataflow SDK 1.x for Java for migration guidance.

Note: If you are using Cloud Dataflow SDK version 2.x, see the Apache Beam API reference documentation.

com.google.cloud.dataflow.sdk.io

Class TextIO.Read

• java.lang.Object
• • com.google.cloud.dataflow.sdk.io.TextIO.Read

• Enclosing class:

  TextIO

  ---

  
  

  public static class TextIO.Read
  extends Object

  A PTransform that reads from a text file (or multiple text files matching a pattern) and returns a PCollection containing the decoding of each of the lines of the text file(s). The default decoding just returns each line as a String, but you may call withCoder(Coder) to change the return type.

• • Nested Class Summary

    Nested Classes 

    Modifier and Type	Class and Description
    static class	TextIO.Read.Bound<T> A PTransform that reads from one or more text files and returns a bounded PCollection containing one element for each line of the input files.

  • Method Summary

    All Methods Static Methods Concrete Methods 

    Modifier and Type	Method and Description
    static TextIO.Read.Bound<String>	from(String filepattern) Returns a transform for reading text files that reads from the file(s) with the given filename or filename pattern.
    static TextIO.Read.Bound<String>	from(ValueProvider<String> filepattern) Same as from(filepattern), but accepting a ValueProvider.
    static TextIO.Read.Bound<String>	named(String name) Returns a transform for reading text files that uses the given step name.
    static <T> TextIO.Read.Bound<T>	withCoder(Coder<T> coder) Returns a transform for reading text files that uses the given Coder<T> to decode each of the lines of the file into a value of type T.
    static TextIO.Read.Bound<String>	withCompressionType(TextIO.CompressionType compressionType) Returns a transform for reading text files that decompresses all input files using the specified compression type.
    static TextIO.Read.Bound<String>	withoutValidation() Returns a transform for reading text files that has GCS path validation on pipeline creation disabled.

    • Methods inherited from class java.lang.Object

      clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

• • Method Detail

    • named

      public static TextIO.Read.Bound<String> named(String name)

      Returns a transform for reading text files that uses the given step name.

    • from

      public static TextIO.Read.Bound<String> from(String filepattern)

      Returns a transform for reading text files that reads from the file(s) with the given filename or filename pattern. This can be a local path (if running locally), or a Google Cloud Storage filename or filename pattern of the form "gs://<bucket>/<filepath>" (if running locally or via the Google Cloud Dataflow service). Standard Java Filesystem glob patterns ("*", "?", "[..]") are supported.

    • from

      public static TextIO.Read.Bound<String> from(ValueProvider<String> filepattern)

      Same as from(filepattern), but accepting a ValueProvider.

    • withCoder

      public static <T> TextIO.Read.Bound<T> withCoder(Coder<T> coder)

      Returns a transform for reading text files that uses the given Coder<T> to decode each of the lines of the file into a value of type T.

      By default, uses StringUtf8Coder, which just returns the text lines as Java strings.

      Type Parameters:

      T - the type of the decoded elements, and the elements of the resulting PCollection

    • withoutValidation

      public static TextIO.Read.Bound<String> withoutValidation()

      Returns a transform for reading text files that has GCS path validation on pipeline creation disabled.

      This can be useful in the case where the GCS input does not exist at the pipeline creation time, but is expected to be available at execution time.

    • withCompressionType

      public static TextIO.Read.Bound<String> withCompressionType(TextIO.CompressionType compressionType)

      Returns a transform for reading text files that decompresses all input files using the specified compression type.

      If no compression type is specified, the default is TextIO.CompressionType.AUTO. In this mode, the compression type of the file is determined by its extension (e.g., *.gz is gzipped, *.bz2 is bzipped, and all other extensions are uncompressed).