Package com.google.cloud.bigquery.storage.v1beta1 (3.3.1)

A client to BigQuery Storage API

The interfaces provided are listed below, along with usage samples.

BaseBigQueryStorageClient

Service Description: BigQuery storage API.

The BigQuery storage API can be used to read data stored in BigQuery.

The v1beta1 API is not yet officially deprecated, and will go through a full deprecation cycle (https://cloud.google.com/products#product-launch-stages) before the service is turned down. However, new code should use the v1 API going forward.

Sample for BaseBigQueryStorageClient:


 // This snippet has been automatically generated and should be regarded as a code template only.
 // It will require modifications to work:
 // - It may require correct/in-range values for request initialization.
 // - It may require specifying regional endpoints when creating the service client as shown in
 // https://cloud.google.com/java/docs/setup#configure_endpoints_for_the_client_library
 try (BaseBigQueryStorageClient baseBigQueryStorageClient = BaseBigQueryStorageClient.create()) {
   TableReferenceProto.TableReference tableReference =
       TableReferenceProto.TableReference.newBuilder().build();
   ProjectName parent = ProjectName.of("[PROJECT]");
   int requestedStreams = 1017221410;
   Storage.ReadSession response =
       baseBigQueryStorageClient.createReadSession(tableReference, parent, requestedStreams);
 }

Classes

ArrowProto

ArrowProto.ArrowRecordBatch

Arrow RecordBatch.

Protobuf type google.cloud.bigquery.storage.v1beta1.ArrowRecordBatch

ArrowProto.ArrowRecordBatch.Builder

Arrow RecordBatch.

Protobuf type google.cloud.bigquery.storage.v1beta1.ArrowRecordBatch

ArrowProto.ArrowSchema

Arrow schema.

Protobuf type google.cloud.bigquery.storage.v1beta1.ArrowSchema

ArrowProto.ArrowSchema.Builder

Arrow schema.

Protobuf type google.cloud.bigquery.storage.v1beta1.ArrowSchema

AvroProto

AvroProto.AvroRows

Avro rows.

Protobuf type google.cloud.bigquery.storage.v1beta1.AvroRows

AvroProto.AvroRows.Builder

Avro rows.

Protobuf type google.cloud.bigquery.storage.v1beta1.AvroRows

AvroProto.AvroSchema

Avro schema.

Protobuf type google.cloud.bigquery.storage.v1beta1.AvroSchema

AvroProto.AvroSchema.Builder

Avro schema.

Protobuf type google.cloud.bigquery.storage.v1beta1.AvroSchema

BaseBigQueryStorageClient

Service Description: BigQuery storage API.

The BigQuery storage API can be used to read data stored in BigQuery.

This class provides the ability to make remote calls to the backing service through method calls that map to API methods. Sample code to get started:


 // This snippet has been automatically generated and should be regarded as a code template only.
 // It will require modifications to work:
 // - It may require correct/in-range values for request initialization.
 // - It may require specifying regional endpoints when creating the service client as shown in
 // https://cloud.google.com/java/docs/setup#configure_endpoints_for_the_client_library
 try (BaseBigQueryStorageClient baseBigQueryStorageClient = BaseBigQueryStorageClient.create()) {
   TableReferenceProto.TableReference tableReference =
       TableReferenceProto.TableReference.newBuilder().build();
   ProjectName parent = ProjectName.of("[PROJECT]");
   int requestedStreams = 1017221410;
   Storage.ReadSession response =
       baseBigQueryStorageClient.createReadSession(tableReference, parent, requestedStreams);
 }

Note: close() needs to be called on the BaseBigQueryStorageClient object to clean up resources such as threads. In the example above, try-with-resources is used, which automatically calls close().

Methods
Method	Description	Method Variants
CreateReadSession	Creates a new read session. A read session divides the contents of a BigQuery table into one or more streams, which can then be used to read data from the table. The read session also specifies properties of the data to be read, such as a list of columns or a push-down filter describing the rows to be returned. A particular row can be read by at most one stream. When the caller has reached the end of each stream in the session, then all the data in the table has been read. Read sessions automatically expire 6 hours after they are created and do not require manual clean-up by the caller.	Request object method variants only take one parameter, a request object, which must be constructed before the call. createReadSession(TableReferenceProto.TableReference tableReference, ProjectName parent, int requestedStreams) createReadSession(TableReferenceProto.TableReference tableReference, String parent, int requestedStreams) createReadSession(Storage.CreateReadSessionRequest request) Callable method variants take no parameters and return an immutable API callable object, which can be used to initiate calls to the service. createReadSessionCallable()
ReadRows	Reads rows from the table in the format prescribed by the read session. Each response contains one or more table rows, up to a maximum of 10 MiB per response; read requests which attempt to read individual rows larger than this will fail. Each request also returns a set of stream statistics reflecting the estimated total number of rows in the read stream. This number is computed based on the total table size and the number of active streams in the read session, and may change as other streams continue to read data.	Callable method variants take no parameters and return an immutable API callable object, which can be used to initiate calls to the service. readRowsCallable()
BatchCreateReadSessionStreams	Creates additional streams for a ReadSession. This API can be used to dynamically adjust the parallelism of a batch processing task upwards by adding additional workers.	Request object method variants only take one parameter, a request object, which must be constructed before the call. batchCreateReadSessionStreams(Storage.ReadSession session, int requestedStreams) batchCreateReadSessionStreams(Storage.BatchCreateReadSessionStreamsRequest request) Callable method variants take no parameters and return an immutable API callable object, which can be used to initiate calls to the service. batchCreateReadSessionStreamsCallable()
FinalizeStream	Causes a single stream in a ReadSession to gracefully stop. This API can be used to dynamically adjust the parallelism of a batch processing task downwards without losing data. This API does not delete the stream -- it remains visible in the ReadSession, and any data processed by the stream is not released to other streams. However, no additional data will be assigned to the stream once this call completes. Callers must continue reading data on the stream until the end of the stream is reached so that data which has already been assigned to the stream will be processed. This method will return an error if there are no other live streams in the Session, or if SplitReadStream() has been called on the given Stream.	Request object method variants only take one parameter, a request object, which must be constructed before the call. finalizeStream(Storage.FinalizeStreamRequest request) "Flattened" method variants have converted the fields of the request object into function parameters to enable multiple ways to call the same method. finalizeStream(Storage.Stream stream) Callable method variants take no parameters and return an immutable API callable object, which can be used to initiate calls to the service. finalizeStreamCallable()
SplitReadStream	Splits a given read stream into two Streams. These streams are referred to as the primary and the residual of the split. The original stream can still be read from in the same manner as before. Both of the returned streams can also be read from, and the total rows return by both child streams will be the same as the rows read from the original stream. Moreover, the two child streams will be allocated back to back in the original Stream. Concretely, it is guaranteed that for streams Original, Primary, and Residual, that Original[0-j] = Primary[0-j] and Original[j-n] = Residual[0-m] once the streams have been read to completion. This method is guaranteed to be idempotent.	Request object method variants only take one parameter, a request object, which must be constructed before the call. splitReadStream(Storage.SplitReadStreamRequest request) "Flattened" method variants have converted the fields of the request object into function parameters to enable multiple ways to call the same method. splitReadStream(Storage.Stream originalStream) Callable method variants take no parameters and return an immutable API callable object, which can be used to initiate calls to the service. splitReadStreamCallable()

See the individual methods for example code.

Many parameters require resource names to be formatted in a particular way. To assist with these names, this class includes a format method for each type of name, and additionally a parse method to extract the individual identifiers contained within names that are returned.

This class can be customized by passing in a custom instance of BaseBigQueryStorageSettings to create(). For example:

To customize credentials:


 // This snippet has been automatically generated and should be regarded as a code template only.
 // It will require modifications to work:
 // - It may require correct/in-range values for request initialization.
 // - It may require specifying regional endpoints when creating the service client as shown in
 // https://cloud.google.com/java/docs/setup#configure_endpoints_for_the_client_library
 BaseBigQueryStorageSettings baseBigQueryStorageSettings =
     BaseBigQueryStorageSettings.newBuilder()
         .setCredentialsProvider(FixedCredentialsProvider.create(myCredentials))
         .build();
 BaseBigQueryStorageClient baseBigQueryStorageClient =
     BaseBigQueryStorageClient.create(baseBigQueryStorageSettings);

To customize the endpoint:


 // This snippet has been automatically generated and should be regarded as a code template only.
 // It will require modifications to work:
 // - It may require correct/in-range values for request initialization.
 // - It may require specifying regional endpoints when creating the service client as shown in
 // https://cloud.google.com/java/docs/setup#configure_endpoints_for_the_client_library
 BaseBigQueryStorageSettings baseBigQueryStorageSettings =
     BaseBigQueryStorageSettings.newBuilder().setEndpoint(myEndpoint).build();
 BaseBigQueryStorageClient baseBigQueryStorageClient =
     BaseBigQueryStorageClient.create(baseBigQueryStorageSettings);

Please refer to the GitHub repository's samples for more quickstart code snippets.

BaseBigQueryStorageSettings

Settings class to configure an instance of BaseBigQueryStorageClient.

The default instance has everything set to sensible defaults:

The default service address (bigquerystorage.googleapis.com) and default port (443) are used.
Credentials are acquired automatically through Application Default Credentials.
Retries are configured for idempotent methods but not for non-idempotent methods.

The builder of this class is recursive, so contained classes are themselves builders. When build() is called, the tree of builders is called to create the complete settings object.

For example, to set the total timeout of createReadSession to 30 seconds:


 // This snippet has been automatically generated and should be regarded as a code template only.
 // It will require modifications to work:
 // - It may require correct/in-range values for request initialization.
 // - It may require specifying regional endpoints when creating the service client as shown in
 // https://cloud.google.com/java/docs/setup#configure_endpoints_for_the_client_library
 BaseBigQueryStorageSettings.Builder baseBigQueryStorageSettingsBuilder =
     BaseBigQueryStorageSettings.newBuilder();
 baseBigQueryStorageSettingsBuilder
     .createReadSessionSettings()
     .setRetrySettings(
         baseBigQueryStorageSettingsBuilder
             .createReadSessionSettings()
             .getRetrySettings()
             .toBuilder()
             .setTotalTimeout(Duration.ofSeconds(30))
             .build());
 BaseBigQueryStorageSettings baseBigQueryStorageSettings =
     baseBigQueryStorageSettingsBuilder.build();

BaseBigQueryStorageSettings.Builder

Builder for BaseBigQueryStorageSettings.

BigQueryStorageClient

Service Description: BigQuery storage API.

The BigQuery storage API can be used to read data stored in BigQuery.

This class provides the ability to make remote calls to the backing service through method calls that map to API methods. Sample code to get started:

 
 try (BigQueryStorageClient bigQueryStorageClient = BigQueryStorageClient.create()) {
   TableReference tableReference = TableReference.newBuilder().build();
   String parent = "";
   int requestedStreams = 0;
   ReadSession response = bigQueryStorageClient.createReadSession(tableReference, parent, requestedStreams);
 }

Note: close() needs to be called on the bigQueryStorageClient object to clean up resources such as threads. In the example above, try-with-resources is used, which automatically calls close().

The surface of this class includes several types of Java methods for each of the API's methods:

A "flattened" method. With this type of method, the fields of the request type have been converted into function parameters. It may be the case that not all fields are available as parameters, and not every API method will have a flattened method entry point.
A "request object" method. This type of method only takes one parameter, a request object, which must be constructed before the call. Not every API method will have a request object method.
A "callable" method. This type of method takes no parameters and returns an immutable API callable object, which can be used to initiate calls to the service.

See the individual methods for example code.

This class can be customized by passing in a custom instance of BigQueryStorageSettings to create(). For example:

To customize credentials:

 
 BigQueryStorageSettings bigQueryStorageSettings =
     BigQueryStorageSettings.newBuilder()
         .setCredentialsProvider(FixedCredentialsProvider.create(myCredentials))
         .build();
 BigQueryStorageClient bigQueryStorageClient =
     BigQueryStorageClient.create(bigQueryStorageSettings);

To customize the endpoint:

 
 BigQueryStorageSettings bigQueryStorageSettings =
     BigQueryStorageSettings.newBuilder().setEndpoint(myEndpoint).build();
 BigQueryStorageClient bigQueryStorageClient =
     BigQueryStorageClient.create(bigQueryStorageSettings);

BigQueryStorageGrpc

BigQuery storage API. The BigQuery storage API can be used to read data stored in BigQuery. The v1beta1 API is not yet officially deprecated, and will go through a full deprecation cycle (https://cloud.google.com/products#product-launch-stages) before the service is turned down. However, new code should use the v1 API going forward.

BigQueryStorageGrpc.BigQueryStorageBlockingStub

A stub to allow clients to do synchronous rpc calls to service BigQueryStorage.

BigQueryStorageGrpc.BigQueryStorageFutureStub

A stub to allow clients to do ListenableFuture-style rpc calls to service BigQueryStorage.

BigQueryStorageGrpc.BigQueryStorageImplBase

Base class for the server implementation of the service BigQueryStorage.

BigQueryStorageGrpc.BigQueryStorageStub

A stub to allow clients to do asynchronous rpc calls to service BigQueryStorage.

BigQueryStorageSettings

Settings class to configure an instance of BigQueryStorageClient.

The default instance has everything set to sensible defaults:

The default service address (bigquerystorage.googleapis.com) and default port (443) are used.
Credentials are acquired automatically through Application Default Credentials.
Retries are configured for idempotent methods but not for non-idempotent methods.

The builder of this class is recursive, so contained classes are themselves builders. When build() is called, the tree of builders is called to create the complete settings object. For example, to set the total timeout of createReadSession to 30 seconds:

 
 BigQueryStorageSettings.Builder settingsBuilder = BaseBigQueryStorageSettings.newBuilder();
 settingsBuilder.createReadSessionSettings().getRetrySettings().toBuilder()
     .setTotalTimeout(Duration.ofSeconds(30));
 BaseBigQueryStorageSettings settings = settingsBuilder.build();

BigQueryStorageSettings.Builder

Builder for BigQueryStorageSettings.

ProjectName

ProjectName.Builder

Builder for projects/{project}.

ReadOptions

ReadOptions.TableReadOptions

Options dictating how we read a table.

Protobuf type google.cloud.bigquery.storage.v1beta1.TableReadOptions

ReadOptions.TableReadOptions.Builder

Options dictating how we read a table.

Protobuf type google.cloud.bigquery.storage.v1beta1.TableReadOptions

Storage

Storage.BatchCreateReadSessionStreamsRequest

Information needed to request additional streams for an established read session.

Protobuf type google.cloud.bigquery.storage.v1beta1.BatchCreateReadSessionStreamsRequest

Storage.BatchCreateReadSessionStreamsRequest.Builder

Information needed to request additional streams for an established read session.

Protobuf type google.cloud.bigquery.storage.v1beta1.BatchCreateReadSessionStreamsRequest

Storage.BatchCreateReadSessionStreamsResponse

The response from BatchCreateReadSessionStreams returns the stream identifiers for the newly created streams.

Protobuf type google.cloud.bigquery.storage.v1beta1.BatchCreateReadSessionStreamsResponse

Storage.BatchCreateReadSessionStreamsResponse.Builder

The response from BatchCreateReadSessionStreams returns the stream identifiers for the newly created streams.

Protobuf type google.cloud.bigquery.storage.v1beta1.BatchCreateReadSessionStreamsResponse

Storage.CreateReadSessionRequest

Creates a new read session, which may include additional options such as requested parallelism, projection filters and constraints.

Protobuf type google.cloud.bigquery.storage.v1beta1.CreateReadSessionRequest

Storage.CreateReadSessionRequest.Builder

Creates a new read session, which may include additional options such as requested parallelism, projection filters and constraints.

Protobuf type google.cloud.bigquery.storage.v1beta1.CreateReadSessionRequest

Storage.FinalizeStreamRequest

Request information for invoking FinalizeStream.

Protobuf type google.cloud.bigquery.storage.v1beta1.FinalizeStreamRequest

Storage.FinalizeStreamRequest.Builder

Request information for invoking FinalizeStream.

Protobuf type google.cloud.bigquery.storage.v1beta1.FinalizeStreamRequest

Storage.Progress

Protobuf type google.cloud.bigquery.storage.v1beta1.Progress

Storage.Progress.Builder

Protobuf type google.cloud.bigquery.storage.v1beta1.Progress

Storage.ReadRowsRequest

Requesting row data via ReadRows must provide Stream position information.

Protobuf type google.cloud.bigquery.storage.v1beta1.ReadRowsRequest

Storage.ReadRowsRequest.Builder

Requesting row data via ReadRows must provide Stream position information.

Protobuf type google.cloud.bigquery.storage.v1beta1.ReadRowsRequest

Storage.ReadRowsResponse

Response from calling ReadRows may include row data, progress and throttling information.

Protobuf type google.cloud.bigquery.storage.v1beta1.ReadRowsResponse

Storage.ReadRowsResponse.Builder

Response from calling ReadRows may include row data, progress and throttling information.

Protobuf type google.cloud.bigquery.storage.v1beta1.ReadRowsResponse

Storage.ReadSession

Information returned from a CreateReadSession request.

Protobuf type google.cloud.bigquery.storage.v1beta1.ReadSession

Storage.ReadSession.Builder

Information returned from a CreateReadSession request.

Protobuf type google.cloud.bigquery.storage.v1beta1.ReadSession

Storage.SplitReadStreamRequest

Request information for SplitReadStream.

Protobuf type google.cloud.bigquery.storage.v1beta1.SplitReadStreamRequest

Storage.SplitReadStreamRequest.Builder

Request information for SplitReadStream.

Protobuf type google.cloud.bigquery.storage.v1beta1.SplitReadStreamRequest

Storage.SplitReadStreamResponse

Response from SplitReadStream.

Protobuf type google.cloud.bigquery.storage.v1beta1.SplitReadStreamResponse

Storage.SplitReadStreamResponse.Builder

Response from SplitReadStream.

Protobuf type google.cloud.bigquery.storage.v1beta1.SplitReadStreamResponse

Storage.Stream

Information about a single data stream within a read session.

Protobuf type google.cloud.bigquery.storage.v1beta1.Stream

Storage.Stream.Builder

Information about a single data stream within a read session.

Protobuf type google.cloud.bigquery.storage.v1beta1.Stream

Storage.StreamPosition

Expresses a point within a given stream using an offset position.

Protobuf type google.cloud.bigquery.storage.v1beta1.StreamPosition

Storage.StreamPosition.Builder

Expresses a point within a given stream using an offset position.

Protobuf type google.cloud.bigquery.storage.v1beta1.StreamPosition

Storage.StreamStatus

Progress information for a given Stream.

Protobuf type google.cloud.bigquery.storage.v1beta1.StreamStatus

Storage.StreamStatus.Builder

Progress information for a given Stream.

Protobuf type google.cloud.bigquery.storage.v1beta1.StreamStatus

Storage.ThrottleStatus

Information on if the current connection is being throttled.

Protobuf type google.cloud.bigquery.storage.v1beta1.ThrottleStatus

Storage.ThrottleStatus.Builder

Information on if the current connection is being throttled.

Protobuf type google.cloud.bigquery.storage.v1beta1.ThrottleStatus

TableReferenceProto

TableReferenceProto.TableModifiers

All fields in this message optional.

Protobuf type google.cloud.bigquery.storage.v1beta1.TableModifiers

TableReferenceProto.TableModifiers.Builder

All fields in this message optional.

Protobuf type google.cloud.bigquery.storage.v1beta1.TableModifiers

TableReferenceProto.TableReference

Table reference that includes just the 3 strings needed to identify a table.

Protobuf type google.cloud.bigquery.storage.v1beta1.TableReference

TableReferenceProto.TableReference.Builder

Table reference that includes just the 3 strings needed to identify a table.

Protobuf type google.cloud.bigquery.storage.v1beta1.TableReference

Package com.google.cloud.bigquery.storage.v1beta1 (3.3.1)

BaseBigQueryStorageClient

Classes

Interfaces

Enums