- 3.27.0 (latest)
- 3.26.0
- 3.25.0
- 3.24.0
- 3.23.1
- 3.22.0
- 3.21.0
- 3.20.1
- 3.19.0
- 3.18.0
- 3.17.2
- 3.16.0
- 3.15.0
- 3.14.1
- 3.13.0
- 3.12.0
- 3.11.4
- 3.4.0
- 3.3.6
- 3.2.0
- 3.1.0
- 3.0.1
- 2.34.4
- 2.33.0
- 2.32.0
- 2.31.0
- 2.30.1
- 2.29.0
- 2.28.1
- 2.27.1
- 2.26.0
- 2.25.2
- 2.24.1
- 2.23.3
- 2.22.1
- 2.21.0
- 2.20.0
- 2.19.0
- 2.18.0
- 2.17.0
- 2.16.1
- 2.15.0
- 2.14.0
- 2.13.1
- 2.12.0
- 2.11.0
- 2.10.0
- 2.9.0
- 2.8.0
- 2.7.0
- 2.6.2
- 2.5.0
- 2.4.0
- 2.3.1
- 2.2.0
- 2.1.0
- 2.0.0
- 1.28.2
- 1.27.2
- 1.26.1
- 1.25.0
- 1.24.0
- 1.23.1
- 1.22.0
- 1.21.0
- 1.20.0
- 1.19.0
- 1.18.0
- 1.17.0
- 1.16.0
LoadJobConfig(**kwargs)Configuration options for load jobs.
Set properties on the constructed configuration by using the property name
as the name of a keyword argument. Values which are unset or :data:None
use the BigQuery REST API default values. See the BigQuery REST API
reference documentation
<https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad>_
for a list of default values.
Required options differ based on the
source_format value.
For example, the BigQuery API's default value for
source_format is "CSV".
When loading a CSV file, either
schema must be set or
autodetect must be set to
:data:True.
Properties
allow_jagged_rows
Optional[bool]: Allow missing trailing optional columns (CSV only).
allow_quoted_newlines
Optional[bool]: Allow quoted data containing newline characters (CSV only).
autodetect
Optional[bool]: Automatically infer the schema from a sample of the data.
See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.autodetect
clustering_fields
Optional[List[str]]: Fields defining clustering for the table
(Defaults to :data:None).
Clustering fields are immutable after table creation.
column_name_character_map
Optional[google.cloud.bigquery.job.ColumnNameCharacterMap]: Character map supported for column names in CSV/Parquet loads. Defaults to STRICT and can be overridden by Project Config Service. Using this option with unsupported load formats will result in an error.
connection_properties
Connection properties.
.. versionadded:: 3.7.0
create_disposition
Optional[google.cloud.bigquery.job.CreateDisposition]: Specifies behavior for creating tables.
create_session
[Preview] If :data:True, creates a new session, where
session_info will contain a
random server generated session id.
If :data:False, runs load job with an existing session_id passed in
connection_properties,
otherwise runs load job in non-session mode.
.. versionadded:: 3.7.0
decimal_target_types
Possible SQL data types to which the source decimal values are converted.
.. versionadded:: 2.21.0
destination_encryption_configuration
Optional[google.cloud.bigquery.encryption_configuration.EncryptionConfiguration]: Custom encryption configuration for the destination table.
Custom encryption configuration (e.g., Cloud KMS keys) or :data:None
if using default encryption.
destination_table_description
Optional[str]: Description of the destination table.
destination_table_friendly_name
Optional[str]: Name given to destination table.
encoding
Optional[google.cloud.bigquery.job.Encoding]: The character encoding of the data.
See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.encoding
field_delimiter
Optional[str]: The separator for fields in a CSV file.
hive_partitioning
Optional[.external_config.HivePartitioningOptions]: [Beta] When set, it configures hive partitioning support.
ignore_unknown_values
Optional[bool]: Ignore extra values not represented in the table schema.
job_timeout_ms
Optional parameter. Job timeout in milliseconds. If this time limit is exceeded, BigQuery might attempt to stop the job. https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfiguration.FIELDS.job_timeout_ms e.g.
job_config = bigquery.QueryJobConfig( job_timeout_ms = 5000 )
or
job_config.job_timeout_ms = 5000
| Exceptions | |
|---|---|
| Type | Description |
ValueError |
If value type is invalid. |
json_extension
Optional[str]: The extension to use for writing JSON data to BigQuery. Only supports GeoJSON currently.
labels
Dict[str, str]: Labels for the job.
This method always returns a dict. Once a job has been created on the server, its labels cannot be modified anymore.
| Exceptions | |
|---|---|
| Type | Description |
ValueError |
If value type is invalid. |
max_bad_records
Optional[int]: Number of invalid rows to ignore.
null_marker
Optional[str]: Represents a null value (CSV only).
See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.null_marker
parquet_options
Optional[google.cloud.bigquery.format_options.ParquetOptions]: Additional
properties to set if sourceFormat is set to PARQUET.
preserve_ascii_control_characters
Optional[bool]: Preserves the embedded ASCII control characters when sourceFormat is set to CSV.
projection_fields
Optional[List[str]]: If source_format is set to "DATASTORE_BACKUP", indicates which entity properties to load into BigQuery from a Cloud Datastore backup.
Property names are case sensitive and must be top-level properties. If no properties are specified, BigQuery loads all properties. If any named property isn't found in the Cloud Datastore backup, an invalid error is returned in the job result.
quote_character
Optional[str]: Character used to quote data sections (CSV only).
See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.quote
range_partitioning
Optional[google.cloud.bigquery.table.RangePartitioning]: Configures range-based partitioning for destination table.
Only specify at most one of xref_time_partitioning or xref_range_partitioning.| Exceptions | |
|---|---|
| Type | Description |
ValueError |
If the value is not RangePartitioning or :data:None. |
reference_file_schema_uri
Optional[str]: When creating an external table, the user can provide a reference file with the table schema. This is enabled for the following formats:
AVRO, PARQUET, ORC
schema
Optional[Sequence[Union[ SchemaField, Mapping[str, Any] ]]]: Schema of the destination table.
See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.schema
schema_update_options
Optional[List[google.cloud.bigquery.job.SchemaUpdateOption]]: Specifies updates to the destination table schema to allow as a side effect of the load job.
skip_leading_rows
Optional[int]: Number of rows to skip when reading data (CSV only).
source_format
Optional[google.cloud.bigquery.job.SourceFormat]: File format of the data.
time_partitioning
Optional[google.cloud.bigquery.table.TimePartitioning]: Specifies time-based partitioning for the destination table.
Only specify at most one of time_partitioning or range_partitioning.
use_avro_logical_types
Optional[bool]: For loads of Avro data, governs whether Avro logical types are converted to their corresponding BigQuery types (e.g. TIMESTAMP) rather than raw types (e.g. INTEGER).
write_disposition
Optional[google.cloud.bigquery.job.WriteDisposition]: Action that occurs if the destination table already exists.
Methods
__setattr__
__setattr__(name, value)Override to be able to raise error if an unknown property is being set
from_api_repr
from_api_repr(resource: dict) -> google.cloud.bigquery.job.base._JobConfigFactory: construct a job configuration given its API representation
| Parameter | |
|---|---|
| Name | Description |
resource |
Dict
A job configuration in the same representation as is returned from the API. |
| Returns | |
|---|---|
| Type | Description |
google.cloud.bigquery.job._JobConfig |
Configuration parsed from resource. |
to_api_repr
to_api_repr() -> dictBuild an API representation of the job config.
| Returns | |
|---|---|
| Type | Description |
Dict |
A dictionary in the format used by the BigQuery API. |