Class LoadJobConfig

LoadJobConfig(**kwargs)

Configuration options for load jobs.

Set properties on the constructed configuration by using the property name as the name of a keyword argument. Values which are unset or :data:None use the BigQuery REST API default values. See the BigQuery REST API reference documentation <https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad>_ for a list of default values.

Required options differ based on the source_format value. For example, the BigQuery API's default value for source_format is "CSV". When loading a CSV file, either schema must be set or autodetect must be set to :data:True.

Inheritance

builtins.object > google.cloud.bigquery.job.base._JobConfig > LoadJobConfig

Properties

allow_jagged_rows

Optional[bool]: Allow missing trailing optional columns (CSV only).

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.allow_jagged_rows

allow_quoted_newlines

Optional[bool]: Allow quoted data containing newline characters (CSV only).

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.allow_quoted_newlines

autodetect

Optional[bool]: Automatically infer the schema from a sample of the data.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.autodetect

clustering_fields

Optional[List[str]]: Fields defining clustering for the table

(Defaults to :data:None).

Clustering fields are immutable after table creation.

.. note::

BigQuery supports clustering for both partitioned and non-partitioned tables.

create_disposition

Optional[google.cloud.bigquery.job.CreateDisposition]: Specifies behavior for creating tables.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.create_disposition

decimal_target_types

Possible SQL data types to which the source decimal values are converted.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.decimal_target_types

.. versionadded:: 2.21.0

destination_encryption_configuration

Optional[google.cloud.bigquery.encryption_configuration.EncryptionConfiguration]: Custom encryption configuration for the destination table.

Custom encryption configuration (e.g., Cloud KMS keys) or :data:None if using default encryption.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.destination_encryption_configuration

destination_table_description

Optional[str]: Description of the destination table.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#DestinationTableProperties.FIELDS.description

destination_table_friendly_name

Optional[str]: Name given to destination table.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#DestinationTableProperties.FIELDS.friendly_name

encoding

Optional[google.cloud.bigquery.job.Encoding]: The character encoding of the data.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.encoding

field_delimiter

Optional[str]: The separator for fields in a CSV file.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.field_delimiter

hive_partitioning

Optional[.external_config.HivePartitioningOptions]: [Beta] When set, it configures hive partitioning support.

.. note:: Experimental. This feature is experimental and might change or have limited support.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.hive_partitioning_options

ignore_unknown_values

Optional[bool]: Ignore extra values not represented in the table schema.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.ignore_unknown_values

labels

Dict[str, str]: Labels for the job.

This method always returns a dict. Once a job has been created on the server, its labels cannot be modified anymore.

Exceptions

Type	Description
ValueError	If ``value`` type is invalid.

max_bad_records

Optional[int]: Number of invalid rows to ignore.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.max_bad_records

null_marker

Optional[str]: Represents a null value (CSV only).

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.null_marker

parquet_options

Optional[google.cloud.bigquery.format_options.ParquetOptions]: Additional properties to set if sourceFormat is set to PARQUET.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.parquet_options

projection_fields

Optional[List[str]]: If source_format is set to "DATASTORE_BACKUP", indicates which entity properties to load into BigQuery from a Cloud Datastore backup.

Property names are case sensitive and must be top-level properties. If no properties are specified, BigQuery loads all properties. If any named property isn't found in the Cloud Datastore backup, an invalid error is returned in the job result.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.projection_fields

quote_character

Optional[str]: Character used to quote data sections (CSV only).

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.quote

range_partitioning

Optional[google.cloud.bigquery.table.RangePartitioning]: Configures range-based partitioning for destination table.

.. note:: Beta. The integer range partitioning feature is in a pre-release state and might change or have limited support.

Only specify at most one of xref_time_partitioning or xref_range_partitioning.

Exceptions

Type	Description
ValueError	If the value is not RangePartitioning or :data:`None`.

schema

Optional[Sequence[Union[ SchemaField, Mapping[str, Any] ]]]: Schema of the destination table.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.schema

schema_update_options

Optional[List[google.cloud.bigquery.job.SchemaUpdateOption]]: Specifies updates to the destination table schema to allow as a side effect of the load job.

skip_leading_rows

Optional[int]: Number of rows to skip when reading data (CSV only).

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.skip_leading_rows

source_format

Optional[google.cloud.bigquery.job.SourceFormat]: File format of the data.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.source_format

time_partitioning

Optional[google.cloud.bigquery.table.TimePartitioning]: Specifies time-based partitioning for the destination table.

Only specify at most one of time_partitioning or range_partitioning.

use_avro_logical_types

Optional[bool]: For loads of Avro data, governs whether Avro logical types are converted to their corresponding BigQuery types (e.g. TIMESTAMP) rather than raw types (e.g. INTEGER).

write_disposition

Optional[google.cloud.bigquery.job.WriteDisposition]: Action that occurs if the destination table already exists.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.write_disposition

Methods

setattr

__setattr__(name, value)

Override to be able to raise error if an unknown property is being set

from_api_repr

from_api_repr(resource: dict)

Factory: construct a job configuration given its API representation

Parameter

Name	Description
resource	`Dict` A job configuration in the same representation as is returned from the API.

Returns

Type	Description
google.cloud.bigquery.job._JobConfig	Configuration parsed from ``resource``.

to_api_repr

to_api_repr()

Build an API representation of the job config.

Returns

Type	Description
Dict	A dictionary in the format used by the BigQuery API.

init

__init__(**kwargs)

Initialize self. See help(type(self)) for accurate signature.

LoadJobConfig

LoadJobConfig(**kwargs)

Configuration options for load jobs.