Class PslScanBuilder (0.4.5) 

public class PslScanBuilder implements ScanBuilder, Scan, SupportsReportStatistics

Both Scan and ScanBuilder implementation, otherwise estimateStatistics() is not called due to bug in DataSourceV2Relation.

Information from https://github.com/GoogleCloudDataproc/spark-bigquery-connector/blob/b0b2a8add37bd044e5a9976ce635bb4230957f29/spark-bigquery-dsv2/spark-3.1-bigquery/src/main/java/com/google/cloud/spark/bigquery/v2/BigQueryScanBuilder.java#L30

Inheritance

java.lang.Object > PslScanBuilder

Implements

org.apache.spark.sql.connector.read.ScanBuilder, org.apache.spark.sql.connector.read.Scan, org.apache.spark.sql.connector.read.SupportsReportStatistics

Constructors

PslScanBuilder(PslReadDataSourceOptions options)

public PslScanBuilder(PslReadDataSourceOptions options)
Parameter
NameDescription
optionsPslReadDataSourceOptions

Methods

build()

public Scan build()
Returns
TypeDescription
org.apache.spark.sql.connector.read.Scan

description()

public String description()
Returns
TypeDescription
String

estimateStatistics()

public Statistics estimateStatistics()
Returns
TypeDescription
org.apache.spark.sql.connector.read.Statistics

readSchema()

public StructType readSchema()
Returns
TypeDescription
org.apache.spark.sql.types.StructType

toContinuousStream(String checkpointLocation)

public ContinuousStream toContinuousStream(String checkpointLocation)
Parameter
NameDescription
checkpointLocationString
Returns
TypeDescription
org.apache.spark.sql.connector.read.streaming.ContinuousStream

toMicroBatchStream(String checkpointLocation)

public MicroBatchStream toMicroBatchStream(String checkpointLocation)
Parameter
NameDescription
checkpointLocationString
Returns
TypeDescription
org.apache.spark.sql.connector.read.streaming.MicroBatchStream