com.google.cloud.dataflow.sdk.io (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

Interface Summary
Interface	Description
CompressedSource.DecompressingChannelFactory	Factory interface for creating channels that decompress the content of an underlying channel.
UnboundedSource.CheckpointMark	A marker representing the progress and state of an `UnboundedSource.UnboundedReader`.

Class Summary
Class	Description
AvroIO	`PTransform`s for reading and writing Avro files.
AvroIO.Read	A root `PTransform` that reads from an Avro file (or multiple Avro files matching a pattern) and returns a `PCollection` containing the decoding of each record.
AvroIO.Read.Bound<T>	A `PTransform` that reads from an Avro file (or multiple Avro files matching a pattern) and returns a bounded `PCollection` containing the decoding of each record.
AvroIO.Write	A root `PTransform` that writes a `PCollection` to an Avro file (or multiple Avro files matching a sharding pattern).
AvroIO.Write.Bound<T>	A `PTransform` that writes a bounded `PCollection` to an Avro file (or multiple Avro files matching a sharding pattern).
AvroSource<T>	A `FileBasedSource` for reading Avro files.
AvroSource.AvroReader<T>	A `BlockBasedSource.BlockBasedReader` for reading blocks from Avro files.
BigQueryIO	`PTransform`s for reading and writing BigQuery tables.
BigQueryIO.Read	A `PTransform` that reads from a BigQuery table and returns a `PCollection` of `TableRows` containing each of the rows of the table.
BigQueryIO.Read.Bound	A `PTransform` that reads from a BigQuery table and returns a bounded `PCollection` of `TableRows`.
BigQueryIO.Write	A `PTransform` that writes a `PCollection` containing `TableRows` to a BigQuery table.
BigQueryIO.Write.Bound	A `PTransform` that can write either a bounded or unbounded `PCollection` of `TableRows` to a BigQuery table.
BlockBasedSource<T>	A `BlockBasedSource` is a `FileBasedSource` where a file consists of blocks of records.
BlockBasedSource.Block<T>	A `Block` represents a block of records that can be read.
BlockBasedSource.BlockBasedReader<T>	A `Reader` that reads records from a `BlockBasedSource`.
BoundedSource<T>	A `Source` that reads a finite amount of input and, because of that, supports some additional operations.
BoundedSource.BoundedReader<T>	A `Reader` that reads a bounded amount of input and supports some additional operations, such as progress estimation and dynamic work rebalancing.
CompressedSource<T>	A Source that reads from compressed files.
CompressedSource.CompressedReader<T>	Reader for a `CompressedSource`.
CountingInput	A `PTransform` that produces longs.
CountingInput.BoundedCountingInput	A `PTransform` that will produce a specified number of `Longs` starting from 0.
CountingInput.UnboundedCountingInput	A `PTransform` that will produce numbers starting from `0` up to `Long.MAX_VALUE`.
CountingSource	A source that produces longs.
CountingSource.CounterMark	The checkpoint for an unbounded `CountingSource` is simply the last value produced.
DatastoreIO	Deprecated replaced by `DatastoreIO`
DatastoreIO.DatastoreReader	A `Source.Reader` over the records from a query of the datastore.
DatastoreIO.Sink	A `Sink` that writes a `PCollection` containing `Entities` to a Datastore kind.
DatastoreIO.Source	A `Source` that reads the result rows of a Datastore query as `Entity` objects.
FileBasedSink<T>	Abstract `Sink` for file-based output.
FileBasedSink.FileBasedWriteOperation<T>	Abstract `Sink.WriteOperation` that manages the process of writing to a `FileBasedSink`.
FileBasedSink.FileBasedWriter<T>	Abstract `Sink.Writer` that writes a bundle to a `FileBasedSink`.
FileBasedSink.FileResult	Result of a single bundle write.
FileBasedSource<T>	A common base class for all file-based `Source`s.
FileBasedSource.FileBasedReader<T>	A `reader` that implements code common to readers of `FileBasedSource`s.
OffsetBasedSource<T>	A `BoundedSource` that uses offsets to define starting and ending positions.
OffsetBasedSource.OffsetBasedReader<T>	A `Source.Reader` that implements code common to readers of all `OffsetBasedSource`s.
PubsubIO	Read and Write `PTransform`s for Cloud Pub/Sub streams.
PubsubIO.PubsubSubscription	Class representing a Cloud Pub/Sub Subscription.
PubsubIO.PubsubTopic	Class representing a Cloud Pub/Sub Topic.
PubsubIO.Read	A `PTransform` that continuously reads from a Cloud Pub/Sub stream and returns a `PCollection` of `Strings` containing the items from the stream.
PubsubIO.Read.Bound<T>	A `PTransform` that reads from a Cloud Pub/Sub source and returns a unbounded `PCollection` containing the items from the stream.
PubsubIO.Write	A `PTransform` that continuously writes a `PCollection` of `Strings` to a Cloud Pub/Sub stream.
PubsubIO.Write.Bound<T>	A `PTransform` that writes an unbounded `PCollection` of `Strings` to a Cloud Pub/Sub stream.
PubsubUnboundedSink<T>	A PTransform which streams messages to Pubsub.
PubsubUnboundedSource<T>	A PTransform which streams messages from Pubsub.
Read	A `PTransform` for reading from a `Source`.
Read.Bounded<T>	`PTransform` that reads from a `BoundedSource`.
Read.Builder	Helper class for building `Read` transforms.
Read.Unbounded<T>	`PTransform` that reads from a `UnboundedSource`.
ShardNameTemplate	Standard shard naming templates.
Sink<T>	A `Sink` represents a resource that can be written to using the `Write` transform.
Sink.WriteOperation<T,WriteT>	A `Sink.WriteOperation` defines the process of a parallel write of objects to a Sink.
Sink.Writer<T,WriteT>	A Writer writes a bundle of elements from a PCollection to a sink.
Source<T>	Base class for defining input formats and creating a `Source` for reading the input.
Source.Reader<T>	The interface that readers of custom input sources must implement.
TextIO	`PTransform`s for reading and writing text files.
TextIO.Read	A `PTransform` that reads from a text file (or multiple text files matching a pattern) and returns a `PCollection` containing the decoding of each of the lines of the text file(s).
TextIO.Read.Bound<T>	A `PTransform` that reads from one or more text files and returns a bounded `PCollection` containing one element for each line of the input files.
TextIO.Write	A `PTransform` that writes a `PCollection` to text file (or multiple text files matching a sharding pattern), with each element of the input collection encoded into its own line.
TextIO.Write.Bound<T>	A PTransform that writes a bounded PCollection to a text file (or multiple text files matching a sharding pattern), with each PCollection element being encoded into its own line.
UnboundedSource<OutputT,CheckpointMarkT extends UnboundedSource.CheckpointMark>	A `Source` that reads an unbounded amount of input and, because of that, supports some additional operations such as checkpointing, watermarks, and record ids.
UnboundedSource.UnboundedReader<OutputT>	A `Reader` that reads an unbounded amount of input.
Write	A `PTransform` that writes to a `Sink`.
Write.Bound<T>	A `PTransform` that writes to a `Sink`.
XmlSink	A `Sink` that outputs records as XML-formatted elements.
XmlSink.Bound<T>	A `FileBasedSink` that writes objects as XML elements.
XmlSink.XmlWriteOperation<T>	`Sink.WriteOperation` for XML `Sink`s.
XmlSink.XmlWriter<T>	A `Sink.Writer` that can write objects as XML elements.
XmlSource<T>	A source that can be used to read XML files.

Enum Summary
Enum	Description
BigQueryIO.Write.CreateDisposition	An enumeration type for the BigQuery create disposition strings.
BigQueryIO.Write.WriteDisposition	An enumeration type for the BigQuery write disposition strings.
CompressedSource.CompressionMode	Default compression types supported by the `CompressedSource`.
FileBasedSink.FileBasedWriteOperation.TemporaryFileRetention	Options for handling of temporary output files.
FileBasedSource.Mode	A given `FileBasedSource` represents a file resource of one of these types.
TextIO.CompressionType	Possible text file compression types.

Package com.google.cloud.dataflow.sdk.io Description

Defines transforms for reading and writing common storage formats, including AvroIO, BigQueryIO, and TextIO.

The classes in this package provide Read transforms that create PCollections from existing storage:


 PCollection<TableRow> inputData = pipeline.apply(
     BigQueryIO.Read.named("Read")
                    .from("clouddataflow-readonly:samples.weather_stations");

and Write transforms that persist PCollections to external storage:

 
 PCollection<Integer> numbers = ...;
 numbers.apply(TextIO.Write.named("WriteNumbers")
                           .to("gs://my_bucket/path/to/numbers"));