Google Cloud Dataflow SDK for Java, version 1.9.1
Class AvroIO.Read.Bound<T>
- java.lang.Object
-
- com.google.cloud.dataflow.sdk.transforms.PTransform<PInput,PCollection<T>>
-
- com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound<T>
-
- Type Parameters:
T
- the type of each of the elements of the resulting PCollection
- All Implemented Interfaces:
- HasDisplayData, Serializable
- Enclosing class:
- AvroIO.Read
public static class AvroIO.Read.Bound<T> extends PTransform<PInput,PCollection<T>>
APTransform
that reads from an Avro file (or multiple Avro files matching a pattern) and returns a boundedPCollection
containing the decoding of each record.- See Also:
- Serialized Form
-
-
Field Summary
-
Fields inherited from class com.google.cloud.dataflow.sdk.transforms.PTransform
name
-
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method and Description PCollection<T>
apply(PInput input)
Applies thisPTransform
on the givenInputT
, and returns itsOutput
.AvroIO.Read.Bound<T>
from(String filepattern)
Returns a newPTransform
that's like this one but that reads from the file(s) with the given name or pattern.protected Coder<T>
getDefaultOutputCoder()
Returns the defaultCoder
to use for the output of this single-outputPTransform
.String
getFilepattern()
Schema
getSchema()
AvroIO.Read.Bound<T>
named(String name)
Returns a newPTransform
that's like this one but with the given step name.boolean
needsValidation()
void
populateDisplayData(DisplayData.Builder builder)
Register display data for the given transform or component.AvroIO.Read.Bound<T>
withoutValidation()
Returns a newPTransform
that's like this one but that has GCS input path validation on pipeline creation disabled.<X> AvroIO.Read.Bound<X>
withSchema(Class<X> type)
Returns a newPTransform
that's like this one but that reads Avro file(s) containing records whose type is the specified Avro-generated class.AvroIO.Read.Bound<GenericRecord>
withSchema(Schema schema)
Returns a newPTransform
that's like this one but that reads Avro file(s) containing records of the specified schema.AvroIO.Read.Bound<GenericRecord>
withSchema(String schema)
Returns a newPTransform
that's like this one but that reads Avro file(s) containing records of the specified schema in a JSON-encoded string form.-
Methods inherited from class com.google.cloud.dataflow.sdk.transforms.PTransform
getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, toString, validate
-
-
-
-
Method Detail
-
named
public AvroIO.Read.Bound<T> named(String name)
Returns a newPTransform
that's like this one but with the given step name.Does not modify this object.
-
from
public AvroIO.Read.Bound<T> from(String filepattern)
Returns a newPTransform
that's like this one but that reads from the file(s) with the given name or pattern. (SeeAvroIO.Read.from(java.lang.String)
for a description of filepatterns.)Does not modify this object.
-
withSchema
public <X> AvroIO.Read.Bound<X> withSchema(Class<X> type)
Returns a newPTransform
that's like this one but that reads Avro file(s) containing records whose type is the specified Avro-generated class.Does not modify this object.
- Type Parameters:
X
- the type of the decoded elements and the elements of the resulting PCollection
-
withSchema
public AvroIO.Read.Bound<GenericRecord> withSchema(Schema schema)
Returns a newPTransform
that's like this one but that reads Avro file(s) containing records of the specified schema.Does not modify this object.
-
withSchema
public AvroIO.Read.Bound<GenericRecord> withSchema(String schema)
Returns a newPTransform
that's like this one but that reads Avro file(s) containing records of the specified schema in a JSON-encoded string form.Does not modify this object.
-
withoutValidation
public AvroIO.Read.Bound<T> withoutValidation()
Returns a newPTransform
that's like this one but that has GCS input path validation on pipeline creation disabled.Does not modify this object.
This can be useful in the case where the GCS input location does not exist at the pipeline creation time, but is expected to be available at execution time.
-
apply
public PCollection<T> apply(PInput input)
Description copied from class:PTransform
Applies thisPTransform
on the givenInputT
, and returns itsOutput
.Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).
The default implementation throws an exception. A derived class must either implement apply, or else each runner must supply a custom implementation via
PipelineRunner.apply(com.google.cloud.dataflow.sdk.transforms.PTransform<InputT, OutputT>, InputT)
.- Overrides:
apply
in classPTransform<PInput,PCollection<T>>
-
populateDisplayData
public void populateDisplayData(DisplayData.Builder builder)
Description copied from class:PTransform
Register display data for the given transform or component.populateDisplayData(DisplayData.Builder)
is invoked by Pipeline runners to collect display data viaDisplayData.from(HasDisplayData)
. Implementations may callsuper.populateDisplayData(builder)
in order to register display data in the current namespace, but should otherwise usesubcomponent.populateDisplayData(builder)
to use the namespace of the subcomponent.By default, does not register any display data. Implementors may override this method to provide their own display data.
- Specified by:
populateDisplayData
in interfaceHasDisplayData
- Overrides:
populateDisplayData
in classPTransform<PInput,PCollection<T>>
- Parameters:
builder
- The builder to populate with display data.- See Also:
HasDisplayData
-
getDefaultOutputCoder
protected Coder<T> getDefaultOutputCoder()
Description copied from class:PTransform
Returns the defaultCoder
to use for the output of this single-outputPTransform
.By default, always throws
- Overrides:
getDefaultOutputCoder
in classPTransform<PInput,PCollection<T>>
-
getFilepattern
public String getFilepattern()
-
getSchema
public Schema getSchema()
-
needsValidation
public boolean needsValidation()
-
-