A StreamWriter that can write JSON data to BigQuery tables. The JSONWriter is built on top of a Writer, and it simply converts all JSON data to protobuf messages then calls Writer's appendRows() method to write to BigQuery tables. It maintains all Writer functions, but also provides an additional feature: schema update support, where if the BigQuery table schema is updated, users will be able to ingest data on the new schema after some time (in order of minutes).
managedwriter.Writer managedwriter
Package
@google-cloud/bigquery-storageConstructors
(constructor)(params)
constructor(params: WriterOptions);Creates a new JSONWriter instance.
| Parameter | |
|---|---|
| Name | Description |
params |
WriterOptions
The parameters for the JSONWriter. See WriterOptions docs for more information. |
Methods
appendRows(rows, offsetValue)
appendRows(rows: JSONList, offsetValue?: IInt64Value['value']): PendingWrite;Writes a JSONList that contains objects to be written to the BigQuery table by first converting the JSON data to protobuf messages, then using Writer's appendRows() to write the data at current end of stream. If there is a schema update, the current Writer is closed and reopened with the updated schema.
| Parameters | |
|---|---|
| Name | Description |
rows |
JSONList
The list of JSON rows. |
offsetValue |
IInt64Value_3['value']
? - The offset value. |
| Returns | |
|---|---|
| Type | Description |
PendingWrite | {managedwriter.PendingWrite} The pending write. |
close()
close(): void;| Returns | |
|---|---|
| Type | Description |
void | |
setDefaultMissingValueInterpretation(defaultMissingValueInterpretation)
setDefaultMissingValueInterpretation(defaultMissingValueInterpretation: MissingValueInterpretation): void;Update how missing values are interpreted for the given stream.
| Parameter | |
|---|---|
| Name | Description |
defaultMissingValueInterpretation |
MissingValueInterpretation
|
| Returns | |
|---|---|
| Type | Description |
void | |
setMissingValueInterpretations(missingValueInterpretations)
setMissingValueInterpretations(missingValueInterpretations: MissingValueInterpretationMap): void;Update how missing values are interpreted for individual columns.
| Parameter | |
|---|---|
| Name | Description |
missingValueInterpretations |
MissingValueInterpretationMap
|
| Returns | |
|---|---|
| Type | Description |
void | |
setProtoDescriptor(protoDescriptor)
setProtoDescriptor(protoDescriptor: IDescriptorProto): void;Update the proto descriptor for the Writer. Internally a reconnection event is gonna happen to apply the new proto descriptor.
| Parameter | |
|---|---|
| Name | Description |
protoDescriptor |
IDescriptorProto
The proto descriptor. |
| Returns | |
|---|---|
| Type | Description |
void | |