Class StreamingDataFrame (1.29.0)

Version latest

• 1.29.0 (latest)
• 1.28.0
• 1.27.0
• 1.26.0
• 1.25.0
• 1.24.0
• 1.22.0
• 1.21.0
• 1.20.0
• 1.19.0
• 1.18.0
• 1.17.0
• 1.16.0
• 1.15.0
• 1.14.0
• 1.13.0
• 1.12.0
• 1.11.1
• 1.10.0
• 1.9.0
• 1.8.0
• 1.7.0
• 1.6.0
• 1.5.0
• 1.4.0
• 1.3.0
• 1.2.0
• 1.1.0
• 1.0.0
• 0.26.0
• 0.25.0
• 0.24.0
• 0.23.0
• 0.22.0
• 0.21.0
• 0.20.1
• 0.19.2
• 0.18.0
• 0.17.0
• 0.16.0
• 0.15.0
• 0.14.1
• 0.13.0
• 0.12.0
• 0.11.0
• 0.10.0
• 0.9.0
• 0.8.0
• 0.7.0
• 0.6.0
• 0.5.0
• 0.4.0
• 0.3.0
• 0.2.0

StreamingDataFrame(df: bigframes.dataframe.DataFrame, *, create_key=0)

Two-dimensional, size-mutable, potentially heterogeneous tabular data.

Data structure also contains labeled axes (rows and columns). Arithmetic operations align on both row and column labels. Can be thought of as a dict-like container for Series objects. The primary pandas data structure.

Note: The bigframes.streaming module is a preview feature, and subject to change.

Methods

__getitem__

__getitem__(*args, **kwargs)

Gets the specified column(s) from the StreamingDataFrame.

Parameter
Name	Description
key	index Index or list of indices. It can be a column label, a list of column labels, a Series of booleans or a pandas Index of desired column labels

Returns
Type	Description
bigframes.pandas.Series or Any	Value(s) at the requested index(es).

__repr__

__repr__(*args, **kwargs)

Converts a StreamingDataFrame to a string. Calls to_pandas.

Only represents the first <xref uid="bigframes.options">bigframes.options</xref>.display.max_rows.

__setitem__

__setitem__(*args, **kwargs)

Modify or insert a column into the StreamingDataFrame.

Note: This does not modify the original table the StreamingDataFrame was derived from.

Parameters
Name	Description
key	column index It can be a new column to be inserted, or an existing column to be modified.
value	scalar or Series Value to be assigned to the column

rename

rename(*args, **kwargs)

Rename columns.

Dict values must be unique (1-to-1). Labels not contained in a dict will be left as-is. Extra labels listed don't throw an error.

Parameter
Name	Description
columns	Mapping Dict-like from old column labels to new column labels.

Exceptions
Type	Description
KeyError	If any of the labels is not found.

Returns
Type	Description
bigframes.pandas.DataFrame	StreamingDataFrame with the renamed axis labels.

to_bigtable

to_bigtable(
    *,
    instance: str,
    table: str,
    service_account_email: typing.Optional[str] = None,
    app_profile: typing.Optional[str] = None,
    truncate: bool = False,
    overwrite: bool = False,
    auto_create_column_families: bool = False,
    bigtable_options: typing.Optional[dict] = None,
    job_id: typing.Optional[str] = None,
    job_id_prefix: typing.Optional[str] = None
) -> google.cloud.bigquery.job.query.QueryJob

Export the StreamingDataFrame as a continue job and returns a QueryJob object for some management functionality.

This method requires an existing bigtable preconfigured to accept the continuous query export statement. For instructions on export to bigtable, see https://cloud.google.com/bigquery/docs/export-to-bigtable.

Parameters
Name	Description
instance	str The name of the bigtable instance to export to.
table	str The name of the bigtable table to export to.
service_account_email	str Full name of the service account to run the continuous query. Example: [e-mail] If not provided, the user account will be used, but this limits the lifetime of the continuous query.
app_profile	str, default None The bigtable app profile to export to. If None, no app profile will be used.
truncate	bool, default False The export truncate option, see https://cloud.google.com/bigquery/docs/reference/standard-sql/other-statements#bigtable_export_option
overwrite	bool, default False The export overwrite option, see https://cloud.google.com/bigquery/docs/reference/standard-sql/other-statements#bigtable_export_option
auto_create_column_families	bool, default False The auto_create_column_families option, see https://cloud.google.com/bigquery/docs/reference/standard-sql/other-statements#bigtable_export_option
bigtable_options	dict, default None The bigtable options dict, which will be converted to JSON using json.dumps, see https://cloud.google.com/bigquery/docs/reference/standard-sql/other-statements#bigtable_export_option If None, no bigtable_options parameter will be passed.
job_id	str, default None If specified, replace the default job id for the query, see job_id parameter of https://cloud.google.com/python/docs/reference/bigquery/latest/google.cloud.bigquery.client.Client#google_cloud_bigquery_client_Client_query
job_id_prefix	str, default None If specified, a job id prefix for the query, see job_id_prefix parameter of https://cloud.google.com/python/docs/reference/bigquery/latest/google.cloud.bigquery.client.Client#google_cloud_bigquery_client_Client_query

Returns
Type	Description
google.cloud.bigquery.QueryJob	See https://cloud.google.com/python/docs/reference/bigquery/latest/google.cloud.bigquery.job.QueryJob The ongoing query job can be managed using this object. For example, the job can be cancelled or its error status can be examined.

to_pubsub

to_pubsub(
    *,
    topic: str,
    service_account_email: str,
    job_id: typing.Optional[str] = None,
    job_id_prefix: typing.Optional[str] = None
) -> google.cloud.bigquery.job.query.QueryJob

Export the StreamingDataFrame as a continue job and returns a QueryJob object for some management functionality.

This method requires an existing pubsub topic. For instructions on creating a pubsub topic, see https://cloud.google.com/pubsub/docs/samples/pubsub-quickstart-create-topic?hl=en

Note that a service account is a requirement for continuous queries exporting to pubsub.

Parameters
Name	Description
topic	str The name of the pubsub topic to export to. For example: "taxi-rides"
service_account_email	str Full name of the service account to run the continuous query. Example: [e-mail]
job_id	str, default None If specified, replace the default job id for the query, see job_id parameter of https://cloud.google.com/python/docs/reference/bigquery/latest/google.cloud.bigquery.client.Client#google_cloud_bigquery_client_Client_query
job_id_prefix	str, default None If specified, a job id prefix for the query, see job_id_prefix parameter of https://cloud.google.com/python/docs/reference/bigquery/latest/google.cloud.bigquery.client.Client#google_cloud_bigquery_client_Client_query

Returns
Type	Description
google.cloud.bigquery.QueryJob	See https://cloud.google.com/python/docs/reference/bigquery/latest/google.cloud.bigquery.job.QueryJob The ongoing query job can be managed using this object. For example, the job can be cancelled or its error status can be examined.