Class ReadRowsStream (0.7.0)

Version 0.7.0

• 2.27.0 (latest)
• 2.26.0
• 2.25.0
• 2.24.0
• 2.23.0
• 2.22.0
• 2.21.0
• 2.20.0
• 2.19.1
• 2.18.1
• 2.17.0
• 2.16.2
• 2.15.0
• 2.14.2
• 2.13.2
• 2.12.0
• 2.11.0
• 2.10.1
• 2.9.1
• 2.8.0
• 2.7.0
• 2.6.3
• 2.5.0
• 2.4.0
• 2.3.0
• 2.2.1
• 2.1.0
• 2.0.1
• 1.1.2
• 1.0.0
• 0.8.0
• 0.7.0
• 0.6.0

ReadRowsStream(wrapped, client, read_position, read_rows_kwargs)

A stream of results from a read rows request.

This stream is an iterable of ReadRowsResponse. Iterate over it to fetch all row messages.

If the fastavro library is installed, use the rows() method to parse all messages into a stream of row dictionaries.

If the pandas and fastavro libraries are installed, use the to_dataframe() method to parse all messages into a pandas.DataFrame.

Inheritance

builtins.object > ReadRowsStream

Methods

ReadRowsStream

ReadRowsStream(wrapped, client, read_position, read_rows_kwargs)

Construct a ReadRowsStream.

Parameters

Name	Description
wrapped	Iterable[ ReadRowsResponse ] The ReadRows stream to read.
client	google.cloud.bigquery_storage_v1beta1.gapic big_query_storage_client.BigQueryStorageClient A GAPIC client used to reconnect to a ReadRows stream. This must be the GAPIC client to avoid a circular dependency on this class.
read_position	Union[ dict, google.cloud.bigquery_storage_v1beta1.gapicStreamPosition ] Required. Identifier of the position in the stream to start reading from. The offset requested must be less than the last row read from ReadRows. Requesting a larger offset is undefined. If a dict is provided, it must be of the same form as the protobuf message google.cloud.bigquery_storage_v1beta1.gapicStreamPosition
read_rows_kwargs	dict Keyword arguments to use when reconnecting to a ReadRows stream.

Returns

Type	Description
Iterable[ ReadRowsResponse ]	A sequence of row messages.

__iter__

__iter__()

An iterable of messages.

Returns

Type	Description
Iterable[ ReadRowsResponse ]	A sequence of row messages.

rows

rows(read_session)

Iterate over all rows in the stream.

This method requires the fastavro library in order to parse row messages.

Warning: DATETIME columns are not supported. They are currently parsed as strings in the fastavro library.

Parameter

Name	Description
read_session	ReadSession The read session associated with this read rows stream. This contains the schema, which is required to parse the data messages.

Returns

Type	Description
Iterable[Mapping]	A sequence of rows, represented as dictionaries.

to_arrow

to_arrow(read_session)

Create a pyarrow.Table of all rows in the stream.

This method requires the pyarrow library and a stream using the Arrow format.

Parameter

Name	Description
read_session	ReadSession The read session associated with this read rows stream. This contains the schema, which is required to parse the data messages.

Returns

Type	Description
pyarrow.Table	A table of all rows in the stream.

to_dataframe

to_dataframe(read_session, dtypes=None)

Create a pandas.DataFrame of all rows in the stream.

This method requires the pandas libary to create a data frame and the fastavro library to parse row messages.

Warning: DATETIME columns are not supported. They are currently parsed as strings.

Parameters

Name	Description
read_session	ReadSession The read session associated with this read rows stream. This contains the schema, which is required to parse the data messages.
dtypes	Map[str, Union[str, pandas.Series.dtype]] Optional. A dictionary of column names pandas dtypes. The provided dtype is used when constructing the series for the column specified. Otherwise, the default pandas behavior is used.

Returns

Type	Description
pandas.DataFrame	A data frame of all rows in the stream.