AvroIO.Read (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1


Class AvroIO.Read

  • Enclosing class:

    public static class AvroIO.Read
    extends Object
    A root PTransform that reads from an Avro file (or multiple Avro files matching a pattern) and returns a PCollection containing the decoding of each record.
    • Method Detail

      • from

        public static AvroIO.Read.Bound<GenericRecord> from(String filepattern)
        Returns a PTransform that reads from the file(s) with the given name or pattern. This can be a local filename or filename pattern (if running locally), or a Google Cloud Storage filename or filename pattern of the form "gs://<bucket>/<filepath>" (if running locally or via the Google Cloud Dataflow service). Standard Java Filesystem glob patterns ("*", "?", "[..]") are supported.
      • withSchema

        public static <T> AvroIO.Read.Bound<T> withSchema(Class<T> type)
        Returns a PTransform that reads Avro file(s) containing records whose type is the specified Avro-generated class.
        Type Parameters:
        T - the type of the decoded elements, and the elements of the resulting PCollection
      • withoutValidation

        public static AvroIO.Read.Bound<GenericRecord> withoutValidation()
        Returns a PTransform that reads Avro file(s) that has GCS path validation on pipeline creation disabled.

        This can be useful in the case where the GCS input location does not exist at the pipeline creation time, but is expected to be available at execution time.