Google Cloud Dataflow SDK for Java, version 1.9.1
com.google.cloud.dataflow.sdk.io
Class AvroIO.Read
- java.lang.Object
-
- com.google.cloud.dataflow.sdk.io.AvroIO.Read
-
- Enclosing class:
- AvroIO
public static class AvroIO.Read extends Object
A rootPTransform
that reads from an Avro file (or multiple Avro files matching a pattern) and returns aPCollection
containing the decoding of each record.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class and Description static class
AvroIO.Read.Bound<T>
APTransform
that reads from an Avro file (or multiple Avro files matching a pattern) and returns a boundedPCollection
containing the decoding of each record.
-
Method Summary
All Methods Static Methods Concrete Methods Modifier and Type Method and Description static AvroIO.Read.Bound<GenericRecord>
from(String filepattern)
Returns aPTransform
that reads from the file(s) with the given name or pattern.static AvroIO.Read.Bound<GenericRecord>
named(String name)
Returns aPTransform
with the given step name.static AvroIO.Read.Bound<GenericRecord>
withoutValidation()
Returns aPTransform
that reads Avro file(s) that has GCS path validation on pipeline creation disabled.static <T> AvroIO.Read.Bound<T>
withSchema(Class<T> type)
Returns aPTransform
that reads Avro file(s) containing records whose type is the specified Avro-generated class.static AvroIO.Read.Bound<GenericRecord>
withSchema(Schema schema)
Returns aPTransform
that reads Avro file(s) containing records of the specified schema.static AvroIO.Read.Bound<GenericRecord>
withSchema(String schema)
Returns aPTransform
that reads Avro file(s) containing records of the specified schema in a JSON-encoded string form.
-
-
-
Method Detail
-
named
public static AvroIO.Read.Bound<GenericRecord> named(String name)
Returns aPTransform
with the given step name.
-
from
public static AvroIO.Read.Bound<GenericRecord> from(String filepattern)
Returns aPTransform
that reads from the file(s) with the given name or pattern. This can be a local filename or filename pattern (if running locally), or a Google Cloud Storage filename or filename pattern of the form"gs://<bucket>/<filepath>"
(if running locally or via the Google Cloud Dataflow service). Standard Java Filesystem glob patterns ("*", "?", "[..]") are supported.
-
withSchema
public static <T> AvroIO.Read.Bound<T> withSchema(Class<T> type)
Returns aPTransform
that reads Avro file(s) containing records whose type is the specified Avro-generated class.- Type Parameters:
T
- the type of the decoded elements, and the elements of the resultingPCollection
-
withSchema
public static AvroIO.Read.Bound<GenericRecord> withSchema(Schema schema)
Returns aPTransform
that reads Avro file(s) containing records of the specified schema.
-
withSchema
public static AvroIO.Read.Bound<GenericRecord> withSchema(String schema)
Returns aPTransform
that reads Avro file(s) containing records of the specified schema in a JSON-encoded string form.
-
withoutValidation
public static AvroIO.Read.Bound<GenericRecord> withoutValidation()
Returns aPTransform
that reads Avro file(s) that has GCS path validation on pipeline creation disabled.This can be useful in the case where the GCS input location does not exist at the pipeline creation time, but is expected to be available at execution time.
-
-