Changelog

PyPI History

3.11.4 (2023-07-19)

Bug Fixes

3.11.3 (2023-06-27)

Bug Fixes

  • Type annotations include Optional when None is accepted (#1554) (6c1ab80)

3.11.2 (2023-06-21)

Bug Fixes

  • Updates tests based on revised hacker_news tables (#1591) (d73cf49)

3.11.1 (2023-06-09)

Documentation

  • Add/reformat return types for cloud RAD docs (#1582) (6efdce1)

3.11.0 (2023-06-01)

Features

Bug Fixes

  • Filter None values from OpenTelemetry attributes (#1567) (9ea2e21)

  • Handle case when expirationMs is None (#1553) (fa6e13d)

  • Raise most recent exception when not able to fetch query job after starting the job (#1362) (09cc1df)

3.10.0 (2023-04-18)

Features

  • Add date, datetime, time, timestamp dtype to to_dataframe (#1547) (64e913d)

3.9.0 (2023-03-28)

Features

Bug Fixes

  • Keyerror when the load_table_from_dataframe accesses a unmapped dtype dataframe index (#1535) (a69348a)

3.8.0 (2023-03-24)

Features

  • Add bool, int, float, string dtype to to_dataframe (#1529) (5e4465d)

  • Add default LoadJobConfig to Client (#1526) (a2520ca)

  • Expose configuration property on CopyJob, ExtractJob, LoadJob, QueryJob (#1521) (8270a10)

Bug Fixes

  • Loosen ipywidgets restrictions further to address ipython compatibility issues (#1531) (50e5026)

3.7.0 (2023-03-06)

Features

  • Add connection_properties and create_session to LoadJobConfig (#1509) (cd0aaa1)

  • Add default_query_job_config property and property setter to BQ client (#1511) (a23092c)

Documentation

3.6.0 (2023-02-22)

Features

  • Adding preserveAsciiControlCharacter to CSVOptions (#1491) (f832e7a)

Bug Fixes

  • Annotate optional integer parameters with optional type (#1487) (a190aaa)

  • Loosen ipywidget dependency (#1504) (20d3276)

  • Removes scope to avoid unnecessary duplication (#1503) (665d7ba)

Dependencies

  • Update minimum google-cloud-core to 1.6.0 (a190aaa)

3.5.0 (2023-01-31)

Features

  • Add str method to DatasetReference (#1477) (f32df1f)

  • Add preserveAsciiControlCharacter to LoadJobConfig (#1484) (bd1da9a)

Documentation

  • Adds snippet for creating table with external data config (#1420) (f0ace2a)

  • Revise delete label table code sample, add TODO to clean up sni… (#1466) (0dab7d2)

  • samples: Table variable fix (#1287) (a71888a)

3.4.2 (2023-01-13)

Bug Fixes

  • Add support for python 3.11 (#1463) (730a1de)

  • Require grpcio >= 1.49.1 for python 3.11 (72b25c5)

Dependencies

  • Remove upper bound on packaging dependency (#1440) (6088129)

Documentation

  • Create sample to write schema file from table (#1439) (093cc68)

  • Created samples for load table and create table from schema file (#1436) (8ad2e5b)

  • Revise create table cmek sample (#1452) (57740e4)

  • Revise get table labels code sample, add TODO to clean up snipp… (#1464) (b5ccbfe)

  • Revise label table code samples (#1451) (14ae1f2)

  • Revise sample for nested schema (#1446) (a097631)

3.4.1 (2022-12-09)

Documentation

  • Add info about streaming quota limits to insert_rows\* methods (#1409) (0f08e9a)

Dependencies

  • make pyarrow and BQ Storage optional dependencies (e1aa921)

3.4.0 (2022-11-17)

Features

  • Add reference_file_schema_uri to LoadJobConfig, ExternalConfig (#1399) (931285f)

  • Add default value expression (#1408) (207aa50)

  • Add More Specific Type Annotations for Row Dictionaries (#1295) (eb49873)

3.3.6 (2022-11-02)

Features

  • Reconfigure tqdm progress bar in %%bigquery magic (#1355) (506f781)

Bug Fixes

Documentation

Miscellaneous Chores

3.3.4 (2022-09-29)

Bug Fixes

3.3.3 (2022-09-28)

Bug Fixes

  • Refactors code to account for a tdqm code deprecation (#1357) (1369a9d)

  • Validate opentelemetry span job attributes have values (#1327) (8287af1)

Documentation

  • samples: uses function (create_job) more appropriate to the described sample intent (5aeedaa)

3.3.2 (2022-08-16)

Bug Fixes

  • deps: require proto-plus >= 1.22.0 (1de7a52)

  • deps: require protobuf >=3.19, < 5.0.0 (#1311) (1de7a52)

3.3.1 (2022-08-09)

Bug Fixes

3.3.0 (2022-07-25)

Features

  • add destination_expiration_time property to copy job (#1277) (728b07c)

Bug Fixes

Documentation

  • samples: add table snapshot sample (#1274) (e760d1b)

  • samples: explicitly add bq to samples reqs, upgrade grpc to fix bug on m1 (#1290) (9b7e3e4)

3.2.0 (2022-06-06)

Features

Bug Fixes

  • deps: proto-plus >= 1.15.0, <2.0.0dev (ba58d3a)

  • deps: require packaging >= 14.3, <22.0.0dev (ba58d3a)

  • deps: require protobuf>= 3.12.0, <4.0.0dev (#1263) (ba58d3a)

Documentation

3.1.0 (2022-05-09)

Features

  • add str method to table (#1199) (8da4fa9)

  • refactor AccessEntry to use _properties pattern (#1125) (acd5612)

  • support using BIGQUERY_EMULATOR_HOST environment variable (#1222) (39294b4)

Bug Fixes

Documentation

  • updated variable typo in comment in code sample (#1239) (e420112)

3.0.1 (2022-03-30)

Bug Fixes

  • deps: raise exception when pandas is installed but db-dtypes is not (#1191) (4333910)

  • deps: restore dependency on python-dateutil (#1187) (212d7ec)

3.0.0 (2022-03-29)

⚠ BREAKING CHANGES

  • BigQuery Storage and pyarrow are required dependencies (#776)

  • use nullable Int64 and boolean dtypes in to_dataframe (#786)

  • destination tables are no-longer removed by create_job (#891)

  • In to_dataframe, use dbdate and dbtime dtypes from db-dtypes package for BigQuery DATE and TIME columns (#972)

  • automatically convert out-of-bounds dates in to_dataframe, remove date_as_object argument (#972)

  • mark the package as type-checked (#1058)

  • default to DATETIME type when loading timezone-naive datetimes from Pandas (#1061)

  • remove out-of-date BigQuery ML protocol buffers (#1178)

Features

  • add api_method parameter to Client.query to select INSERT or QUERY API (#967) (76d88fb)

  • default to DATETIME type when loading timezone-naive datetimes from Pandas (#1061) (76d88fb)

  • destination tables are no-longer removed by create_job (#891) (76d88fb)

  • In to_dataframe, use dbdate and dbtime dtypes from db-dtypes package for BigQuery DATE and TIME columns (#972) (76d88fb)

  • mark the package as type-checked (#1058) (76d88fb)

  • use StandardSqlField class for Model.feature_columns and Model.label_columns (#1117) (76d88fb)

Bug Fixes

  • automatically convert out-of-bounds dates in to_dataframe, remove date_as_object argument (#972) (76d88fb)

  • improve type annotations for mypy validation (#1081) (76d88fb)

  • remove out-of-date BigQuery ML protocol buffers (#1178) (76d88fb)

  • use nullable Int64 and boolean dtypes in to_dataframe (#786) (76d88fb)

Documentation

  • Add migration guide from version 2.x to 3.x (#1027) (76d88fb)

Dependencies

  • BigQuery Storage and pyarrow are required dependencies (#776) (76d88fb)

2.34.3 (2022-03-29)

Bug Fixes

2.34.2 (2022-03-05)

Bug Fixes

  • deps: require google-api-core>=1.31.5, >=2.3.2 (#1157) (0c15790)

  • deps: require proto-plus>=1.15.0 (0c15790)

2.34.1 (2022-03-02)

Dependencies

2.34.0 (2022-02-18)

Features

  • support BI Engine statistics in query job (#1144) (7482549)

2.33.0 (2022-02-16)

Features

  • add --no_query_cache option to %%bigquery magics to disable query cache (#1141) (7dd30af)

Bug Fixes

  • return 403 when VPC-SC violation happens (#1131) (f5daa9b)

Documentation

  • reference BigQuery REST API defaults in LoadJobConfig descrip… (#1132) (18d9580)

  • show common job properties in get_job and cancel_job samples (#1137) (8edc10d)

2.32.0 (2022-01-12)

Features

Bug Fixes

  • remove query text from exception message, use exception.debug_message instead (#1105) (e23114c)

2.31.0 (2021-11-24)

Features

  • allow cell magic body to be a $variable (#1053) (3a681e0)

  • promote RowIterator.to_arrow_iterable to public method (#1073) (21cd710)

Bug Fixes

  • apply timeout to all resumable upload requests (#1070) (3314dfb)

Dependencies

2.30.1 (2021-11-04)

Bug Fixes

Documentation

  • show gcloud command to authorize against sheets (#1045) (20c9024)

  • use stable URL for pandas intersphinx links (#1048) (73312f8)

2.30.0 (2021-11-03)

Features

Documentation

  • add code samples for Jupyter/IPython magics (#1013) (61141ee)

  • samples: add create external table with hive partitioning (#1033) (d64f5b6)

2.29.0 (2021-10-27)

Features

  • add QueryJob.schema property for dry run queries (#1014) (2937fa1)

  • add session and connection properties to QueryJobConfig (#1024) (e4c94f4)

  • add support for INTERVAL data type to list_rows (#840) (e37380a)

  • allow queryJob.result() to be called on a dryRun (#1015) (685f06a)

Documentation

  • document ScriptStatistics and other missing resource classes (#1023) (6679109)

  • fix formatting of generated client docstrings (#1009) (f7b0ee4)

Dependencies

2.28.1 (2021-10-07)

Bug Fixes

  • support ARRAY data type when loading from DataFrame with Parquet (#980) (1e59083)

2.28.0 (2021-09-30)

Features

  • add AvroOptions to configure AVRO external data (#994) (1a9431d)

Documentation

2.27.1 (2021-09-27)

Bug Fixes

  • remove py.typed since package fails mypy check (#988) (39030f2)

2.27.0 (2021-09-24)

Features

  • Add py.typed for PEP 561 compliance (#976) (96e6bee)

  • include key metadata in Job representation (#964) (acca1cb)

Bug Fixes

  • Arrow extension-type metadata was not set when calling the REST API or when there are no rows (#946) (864383b)

  • disambiguate missing policy tags from explicitly unset policy tags (#983) (f83c00a)

  • remove default timeout (#974) (1cef0d4)

Documentation

  • simplify destination table sample with f-strings (#966) (ab6e76f)

2.26.0 (2021-09-01)

Features

  • set the X-Server-Timeout header when timeout is set (#927) (ba02f24)

Bug Fixes

  • guard imports against unsupported pyarrow versions (#934) (b289076)

2.25.2 (2021-08-31)

Bug Fixes

  • error inserting DataFrame with REPEATED field (#925) (656d2fa)

  • underscores weren’t allowed in struct field names when passing parameters to the DB API (#930) (fcb0bc6)

Documentation

  • update docstring for bigquery_create_routine sample (#883) (#917) (e2d12b7)

2.25.1 (2021-08-25)

Bug Fixes

  • populate default timeout and retry after client-side timeout (#896) (b508809)

  • use REST API in cell magic when requested (#892) (1cb3e55)

2.25.0 (2021-08-24)

Features

  • Support using GeoPandas for GEOGRAPHY columns (#848) (16f65e6)

2.24.1 (2021-08-13)

Bug Fixes

  • remove pytz dependency and require pyarrow>=3.0.0 (#875) (2cb3563)

2.24.0 (2021-08-11)

Features

  • add support for transaction statistics (#849) (7f7b1a8)

  • make the same Table\* instances equal to each other (#867) (c1a3d44)

  • retry failed query jobs in result() (#837) (519d99c)

  • support ScalarQueryParameterType for type_ argument in ScalarQueryParameter constructor (#850) (93d15e2)

Bug Fixes

  • make unicode characters working well in load_table_from_json (#865) (ad9c802)

2.23.3 (2021-08-06)

Bug Fixes

  • increase default retry deadline to 10 minutes (#859) (30770fd)

2.23.2 (2021-07-29)

Dependencies

  • expand pyarrow pins to support 5.x releases (#833) (80e3a61)

2.23.1 (2021-07-28)

Bug Fixes

  • insert_rows() accepts float column values as strings again (#824) (d9378af)

2.23.0 (2021-07-27)

Features

  • Update proto definitions for bigquery/v2 to support new proto fields for BQML. (#817) (fe7a902)

Bug Fixes

  • no longer raise a warning in to_dataframe if max_results set (#815) (3c1be14)

  • retry ChunkedEncodingError by default (#802) (419d36d)

Documentation

  • correct docs for LoadJobConfig.destination_table_description (#810) (da87fd9)

2.22.1 (2021-07-22)

Bug Fixes

  • issue a warning if buggy pyarrow is detected (#787) (e403721)

  • use a larger chunk size when loading data (#799) (b804373)

Documentation

2.22.0 (2021-07-19)

Features

  • add LoadJobConfig.projection_fields to select DATASTORE_BACKUP fields (#736) (c45a738)

  • add standard sql table type, update scalar type enums (#777) (b8b5433)

  • add support for more detailed DML stats (#758) (36fe86f)

  • add support for user defined Table View Functions (#724) (8c7b839)

Bug Fixes

  • avoid possible job already exists error (#751) (45b9308)

Dependencies

  • allow 2.x versions of google-api-core, google-cloud-core, google-resumable-media (#770) (87a09fa)

Documentation

  • add loading data from Firestore backup sample (#737) (22fd848)

2.21.0 (2021-07-12)

Features

  • Add max_results parameter to some of the QueryJob methods. (#698) (2a9618f)

  • Add support for decimal target types. (#735) (7d2d3e9)

  • Add support for table snapshots. (#740) (ba86b2a)

  • Enable unsetting policy tags on schema fields. (#703) (18bb443)

  • Make it easier to disable best-effort deduplication with streaming inserts. (#734) (1246da8)

  • Support passing struct data to the DB API. (#718) (38b3ef9)

Bug Fixes

  • Inserting non-finite floats with insert_rows(). (#728) (d047419)

  • Use pandas function to check for NaN. (#750) (67bc5fb)

Documentation

  • Add docs for all enums in module. (#745) (145944f)

  • Omit mention of Python 2.7 in CONTRIBUTING.rst. (#706) (27d6839)

2.20.0 (2021-06-07)

Features

  • support script options in query job config (#690) (1259e16)

2.19.0 (2021-06-06)

Features

  • list_tables, list_projects, list_datasets, list_models, list_routines, and list_jobs now accept a page_size parameter to control page size (#686) (1f1c4b7)

2.18.0 (2021-06-02)

Features

2.17.0 (2021-05-21)

Features

  • detect obsolete BQ Storage extra at runtime (#666) (bd7dbda)

  • Support parameterized NUMERIC, BIGNUMERIC, STRING, and BYTES types (#673) (45421e7)

Bug Fixes

2.16.1 (2021-05-12)

Bug Fixes

  • executemany rowcount only reflected the last execution (#660) (aeadc8c)

2.16.0 (2021-05-05)

Features

  • add with_name() to ScalarQueryParameterType (#644) (6cc6876)

Dependencies

  • expand supported pyarrow versions to v4 (#643) (9e1d386)

2.15.0 (2021-04-29)

Features

  • Extended DB API parameter syntax to optionally provide parameter types (#626) (8bcf397)

Bug Fixes

  • add DECIMAL and BIGDECIMAL as aliases for NUMERIC and BIGNUMERIC (#638) (aa59023)

  • The DB API Binary function accepts bytes data (#630) (4396e70)

2.14.0 (2021-04-26)

Features

  • accept DatasetListItem where DatasetReference is accepted (#597) (c8b5581)

  • accept job object as argument to get_job and cancel_job (#617) (f75dcdf)

  • add Client.delete_job_metadata method to remove job metadata (#610) (0abb566)

  • add max_queue_size argument to RowIterator.to_dataframe_iterable (#575) (f95f415)

  • add type hints for public methods (#613) (f8d4aaa)

  • DB API cursors are now iterable (#618) (e0b373d)

  • retry google.auth TransportError by default (#624) (34ecc3f)

  • use pyarrow stream compression, if available (#593) (dde9dc5)

Bug Fixes

  • consistent percents handling in DB API query (#619) (6502a60)

  • missing license headers in new test files (#604) (df48cc5)

  • unsetting clustering fields on Table is now possible (#622) (33a871f)

Documentation

  • add sample to run DML query (#591) (ff2ec3a)

  • update the description of the return value of _QueryResults.rows() (#594) (8f4c0b8)

2.13.1 (2021-03-23)

Bug Fixes

  • add ConnectionError to default retry (#571) (a3edb8b)

2.13.0 (2021-03-22)

Features

  • add ExternalConfig.connection_id property to connect to external sources (#560) (d93986e)

Bug Fixes

  • avoid overly strict dependency on pyarrow 3.x (#564) (97ee6ec)

  • avoid policy tags 403 error in load_table_from_dataframe (#557) (84e646e)

2.12.0 (2021-03-16)

Features

  • make QueryJob.done() method more performant (#544) (a3ab9ef)

Bug Fixes

  • remove DB-API dependency on pyarrow with decimal query parameters (#551) (1b946ba)

2.11.0 (2021-03-09)

Features

  • add context manager support to client (#540) (d5c7e11)

2.10.0 (2021-02-25)

Features

Bug Fixes

  • error using empty array of structs parameter (#474) (c1d15f4)

  • QueryJob.exception() returns the errors, not raises them (#467) (d763279)

Documentation

  • bigquery: Add alternative approach to setting credentials (#517) (60fbf28)

  • explain retry behavior for DONE jobs (#532) (696c443)

2.9.0 (2021-02-18)

Features

  • add determinism level for javascript UDFs (#522) (edd3328)

  • expose reservation usage stats on jobs (#524) (4ffb4e0)

Documentation

  • clarify `%%bigquery`` magics and fix broken link (#508) (eedf93b)

  • update python contributing guide (#514) (01e851d)

2.8.0 (2021-02-08)

Features

Bug Fixes

  • Don’t try to close closed cursors. (#498) (bf44e7b)

2.7.0 (2021-01-27)

Bug Fixes

  • invalid conversion of timezone-aware datetime values to JSON (#480) (61b4385)

  • reading the labels attribute on Job instances (#471) (80944f0)

  • use explicitly given project over the client’s default project for load jobs (#482) (530e1e8)

Dependencies

2.6.2 (2021-01-11)

Bug Fixes

  • add minimum timeout to getQueryResults API requests (#444) (015a73e)

  • use debug logging level for OpenTelemetry message (#442) (7ea6b7c)

Documentation

2.6.1 (2020-12-09)

Bug Fixes

  • handle null values in array query parameters (#426) (78fde4a)

Documentation

  • add examples of fields argument to update methods (#418) (8c7e02b)

2.6.0 (2020-12-07)

Features

  • add support for materialized views (#408) (57ffc66), closes #407

  • convert BIGNUMERIC values to decimal objects (#414) (d472d2d), closes #367

  • support CSV format in load_table_from_dataframe pandas connector (#399) (0046742)

Bug Fixes

  • preserve timestamp microsecond precision with rows from REST API (#402) (04510a7)

Documentation

2.5.0 (2020-12-02)

Features

  • add TableReference.__str__ to get table ID in standard SQL (#405) (53dff2a), closes #354

  • add progress bar for magics (#396) (04d0273)

  • add support for unrecognized model types (#401) (168f035)

Bug Fixes

  • avoid floating point for timestamp in insert_rows (#393) (a1949ae)

Performance Improvements

Documentation

Dependencies

  • update required version of opentelementry for opentelemetry-exporter-google-cloud (#398) (673a9cb)

2.4.0 (2020-11-16)

Features

  • add progress bar to QueryJob.to_dataframe and to_arrow (#352) (dc78edd)

  • allow routine references (#378) (f9480dc)

Bug Fixes

  • dbapi: allow rows to be fetched from scripts (#387) (b899ad1), closes #377

Performance Improvements

  • avoid extra API calls from to_dataframe if all rows are cached (#384) (c52b317)

  • cache first page of jobs.getQueryResults rows (#374) (86f6a51)

  • use getQueryResults from DB-API (#375) (30de15f)

Dependencies

  • expand pyarrow dependencies to include version 2 (#368) (cd9febd)

2.3.1

11-05-2020 09:27 PST

Internal / Testing Changes

  • update google.cloud.bigquery.__version__

2.3.0 (2020-11-04)

Features

  • add reload argument to \*Job.done() functions (#341) (e51fd45)

  • pass retry from Job.result() to Job.done() (#41) (284e17a)

Bug Fixes

  • add missing spaces in opentelemetry log message (#360) (4f326b1)

  • dbapi: avoid running % format with no query parameters (#348) (5dd1a5e)

  • create_job method accepts dictionary arguments (#300) (155bacc)

Performance Improvements

  • use jobs.getQueryResults to download result sets (#363) (0c3476d)

Documentation

  • add documents for QueryPlanEntry and QueryPlanEntryStep (#344) (dca2e4c)

2.2.0 (2020-10-19)

Features

  • add method api_repr for table list item (#299) (07c70f0)

  • add support for listing arima, automl, boosted tree, DNN, and matrix factorization models (#328) (502a092)

  • add timeout paramter to load_table_from_file and it dependent methods (#327) (b0dd892)

  • add to_api_repr method to Model (#326) (fb401bd)

  • allow client options to be set in magics context (#322) (5178b55)

Bug Fixes

  • make TimePartitioning repr evaluable (#110) (20f473b), closes #109

  • use version.py instead of pkg_resources.get_distribution (#307) (b8f502b)

Performance Improvements

  • add size parameter for load table from dataframe and json methods (#280) (3be78b7)

Documentation

  • update clustering field docstrings (#286) (5ea1ece), closes #285

  • update snippets samples to support version 2.0 (#309) (61634be)

Dependencies

2.1.0 (2020-10-08)

Features

  • add constants for MONTH and YEAR time partitioning types (#283) (9090e1c)

Bug Fixes

  • remove unnecessary dependency on libcst (#308) (c055930)

Performance Improvements

Documentation

  • samples: add create_table_clustered code snippet (#291) (d1eb8b3)

2.0.0

09-30-2020 14:51 PDT

Implementation Changes

  • Transition the library to microgenerator. (#278) This is a breaking change that drops support for Python 2.7 and 3.5 and brings a few other changes. See migration guide for more info.

Internal / Testing Changes

  • Update protoc-generated comments (via synth). (#270)

  • Add CI secrets manager (via synth). (#271)

1.28.0 (2020-09-22)

Features

  • add custom cell magic parser to handle complex --params values (#213) (dcfbac2)

  • add instrumentation to list methods (#239) (fa9f9ca)

  • add opentelemetry tracing (#215) (a04996c)

  • expose require_partition_filter for hive_partition (#257) (aa1613c)

Bug Fixes

  • fix dependency issue in fastavro (#241) (2874abf)

  • update minimum dependency versions (#263) (1be66ce)

  • validate job_config.source_format in load_table_from_dataframe (#262) (6160fee)

Documentation

  • recommend insert_rows_json to avoid call to tables.get (#258) (ae647eb)

1.27.2 (2020-08-18)

Bug Fixes

  • rationalize platform constraints for ‘pyarrow’ extra (#235) (c9a0567)

1.27.1 (2020-08-18)

Bug Fixes

1.27.0 (2020-08-15)

Features

  • add support and tests for struct fields (#146) (fee2ba8)

  • add support for getting and setting table IAM policy (#144) (f59fc9a)

  • bigquery: add client_options to base class (#216) (478597a)

Bug Fixes

  • converting to dataframe with out of bounds timestamps (#209) (8209203), closes #168

  • raise error if inserting rows with unknown fields (#163) (8fe7254)

1.26.1 (2020-07-25)

Documentation

Bug Fixes

  • RowIterator.to_arrow() error when BQ Storage client cannot be created (#181) (7afa3d7)

Dependencies

  • Updated version constraints on grmp dependency in anticipation of 1.0.0 release (#189)

1.26.0 (2020-07-20)

Features

  • use BigQuery Storage client by default (if dependencies available) (#55) (e75ff82), closes #91

  • bigquery: add eq method for class PartitionRange and RangePartitioning (#162) (0d2a88d)

  • bigquery: expose date_as_object parameter to users (#150) (a2d5ce9)

  • bigquery: expose date_as_object parameter to users (#150) (cbd831e)

Bug Fixes

  • dry run queries with DB API cursor (#128) (bc33a67)

  • omit NaN values when uploading from insert_rows_from_dataframe (#170) (f9f2f45)

Documentation

  • bigquery: add client thread-safety documentation (#132) (fce76b3)

  • bigquery: add docstring for conflict exception (#171) (9c3409b)

  • bigquery: consistent use of optional keyword (#153) (79d8c61)

  • bigquery: fix the broken docs (#139) (3235255)

1.25.0 (2020-06-06)

Features

  • add BigQuery storage client support to DB API (#36) (ba9b2f8)

  • bigquery: add create job method (#32) (2abdef8)

  • bigquery: add support of model for extract job (#71) (4a7a514)

  • add HOUR support for time partitioning interval (#91) (0dd90b9)

  • add support for policy tags (#77) (38a5c01)

  • make AccessEntry objects hashable (#93) (23a173b)

  • bigquery: expose start index parameter for query result (#121) (be86de3)

  • bigquery: unit and system test for dataframe with int column with Nan values (#39) (5fd840e)

Bug Fixes

  • allow partial streaming_buffer statistics (#37) (645f0fd)

  • distinguish server timeouts from transport timeouts (#43) (a17be5f)

  • improve cell magic error message on missing query (#58) (6182cf4)

  • bigquery: fix repr of model reference (#66) (26c6204)

  • bigquery: fix start index with page size for list rows (#27) (400673b)

1.24.0

02-03-2020 01:38 PST

Implementation Changes

  • Fix inserting missing repeated fields. (#10196)

  • Deprecate client.dataset() in favor of DatasetReference. (#7753)

  • Use faster to_arrow + to_pandas in to_dataframe() when pyarrow is available. (#10027)

  • Write pandas datetime[ns] columns to BigQuery TIMESTAMP columns. (#10028)

New Features

  • Check rows argument type in insert_rows(). (#10174)

  • Check json_rows arg type in insert_rows_json(). (#10162)

  • Make RowIterator.to_dataframe_iterable() method public. (#10017)

  • Add retry parameter to public methods where missing. (#10026)

  • Add timeout parameter to Client and Job public methods. (#10002)

  • Add timeout parameter to QueryJob.done() method. (#9875)

  • Add create_bqstorage_client parameter to to_dataframe() and to_arrow() methods. (#9573)

Dependencies

  • Fix minimum versions of google-cloud-core and google-resumable-media dependencies. (#10016)

Documentation

  • Fix a comment typo in job.py. (#10209)

  • Update code samples of load table file and load table URI. (#10175)

  • Uncomment Client constructor and imports in samples. (#10058)

  • Remove unused query code sample. (#10024)

  • Update code samples to use strings for table and dataset IDs. (#9974)

Internal / Testing Changes

  • Bump copyright year to 2020, tweak docstring formatting (via synth). #10225

  • Add tests for concatenating categorical columns. (#10180)

  • Adjust test assertions to the new default timeout. (#10222)

  • Use Python 3.6 for the nox blacken session (via synth). (#10012)

1.23.1

12-16-2019 09:39 PST

Implementation Changes

  • Add iamMember entity type to allowed access classes. (#9973)

  • Fix typo in import error message (pandas -> pyarrow). (#9955)

Dependencies

  • Add six as an explicit dependency. (#9979)

Documentation

  • Add sample to read from query destination table. (#9964)

1.23.0

12-11-2019 13:31 PST

New Features

  • Add close() method to client for releasing open sockets. (#9894)

  • Add support of use_avro_logical_types for extract jobs. (#9642)

  • Add support for hive partitioning options configuration. (#9626)

  • Add description for routine entities. (#9785)

Documentation

  • Update code samples to use strings for table and dataset IDs. (#9495)

Internal / Testing Changes

  • Run unit tests with Python 3.8. (#9880)

  • Import Mapping from collections.abc not from collections. (#9826)

1.22.0

11-13-2019 12:23 PST

Implementation Changes

  • Preserve job config passed to Client methods. (#9735)

  • Use pyarrow fallback for improved schema detection. (#9321)

  • Add TypeError if wrong job_config type is passed to client job methods. (#9506)

  • Fix arrow deprecation warning. (#9504)

New Features

  • Add --destination_table parameter to IPython magic. (#9599)

  • Allow passing schema as a sequence of dicts. (#9550)

  • Implement defaultEncryptionConfiguration on datasets. (#9489)

  • Add range partitioning to tables, load jobs, and query jobs. (#9477)

Dependencies

  • Pin google-resumable-media to includ 0.5.x. (#9572)

Documentation

  • Fix link anchors in external config docstrings. (#9627)

  • Add python 2 sunset banner to documentation. (#9036)

  • Add table create sample using integer range partitioning. (#9478)

  • Document how to achieve higher write limit and add tests. (#9574)

  • Add code sample for scripting. (#9537)

  • Rewrite docs in Google style, part 2. (#9481)

  • Use multi-regional key path for CMEK in snippets. (#9523)

Internal / Testing Changes

  • Fix undelete table system test to use milliseconds in snapshot decorator. (#9649)

  • Format code with latest version of black. (#9556)

  • Remove duplicate test dependencies. (#9503)

1.21.0

10-16-2019 10:33 PDT

New Features

  • add ability to pass in a table ID instead of a query to the %%bigquery magic (#9170)

  • add support for custom QueryJobConfig in BigQuery.cursor.execute method (#9278)

  • store QueryJob to destination var on error in %%bigquery magic (#9245)

  • add script statistics to job resource (#9428)

  • add support for sheets ranges (#9416)

  • add support for listing jobs by parent job (#9225)

  • expose customer managed encryption key for ML models (#9302)

  • add Dataset.default_partition_expiration_ms and Table.require_partition_filter properties (#9464)

Dependencies

  • restrict version range of google-resumable-media (#9243)

Documentation

  • document how to load data as JSON string (#9231)

  • standardize comments and formatting in existing code samples (#9212)

  • rewrite docstrings in Google style (#9326)

  • fix incorrect links to REST API in reference docs (#9436)

Internal / Testing Changes

  • add code samples to lint check (#9277)

  • update code samples to use strings for table and dataset IDs (#9136)

  • simplify scripting system test to reduce flakiness (#9458)

1.20.0

09-13-2019 11:22 PDT

Implementation Changes

  • Change default endpoint to bigquery.googleapis.com (#9213)

  • Change the default value of Cursor instances’ arraysize attribute to None (#9199)

  • Deprecate automatic schema conversion. (#9176)

  • Fix list_rows() max results with BQ storage client (#9178)

New Features

  • Add Model.encryption_config. (via synth) (#9214)

  • Add Client.insert_rows_from_dataframe() method (#9162)

  • Add support for array parameters to Cursor.execute(). (#9189)

  • Add support for project IDs with org prefix to Table.from_string() factory. (#9161)

  • Add --max_results option to Jupyter magics (#9169)

  • Autofetch table schema on load if not provided. (#9108)

  • Add max_results parameter to QueryJob.result(). (#9167)

Documentation

Internal / Testing Changes

  • Revert “Disable failing snippets test (#9156).” (#9220)

1.19.0

09-03-2019 14:33 PDT

Implementation Changes

  • Raise when unexpected fields are present in the LoadJobConfig.schema when calling load_table_from_dataframe. (#9096)

  • Determine the schema in load_table_from_dataframe based on dtypes. (#9049)

  • Raise helpful error when loading table from dataframe with STRUCT columns. (#9053)

  • Fix schema recognition of struct field types. (#9001)

  • Fix deserializing None in QueryJob for queries with parameters. (#9029)

New Features

  • Include indexes in table written by load_table_from_dataframe, only if fields corresponding to indexes are present in LoadJobConfig.schema. (#9084)

  • Add client_options to constructor. (#8999)

  • Add --dry_run option to %%bigquery magic. (#9067)

  • Add load_table_from_json() method to create a table from a list of dictionaries. (#9076)

  • Allow subset of schema to be passed into load_table_from_dataframe. (#9064)

  • Add support for unsetting LoadJobConfig.schema. (#9077)

  • Add support to Dataset for project IDs containing an org prefix. (#8877)

  • Add enum with SQL type names allowed to be used in SchemaField. (#9040)

Documentation

  • Fix the reference URL for Client.create_dataset(). (#9149)

  • Update code samples to use strings for table names instead of client.dataset(). (#9032)

  • Remove compatability badges from READMEs. (#9035)

  • Fix Pandas DataFrame load example under Python 2.7. (#9022)

Internal / Testing Changes

  • Disable failing snippets test for copying CMEK-protected tables. (#9156)

  • Fix BigQuery client unit test assertions (#9112)

  • Replace avro with arrow schemas in test_table.py (#9056)

1.18.0

08-08-2019 12:28 PDT

New Features

  • Add bqstorage_client param to QueryJob.to_arrow() (#8693)

  • Include SQL query and job ID in exception messages. (#8748)

  • Allow using TableListItem to construct a Table object. (#8738)

  • Add StandardSqlDataTypes enum to BigQuery (#8782)

  • Add to_standard_sql() method to SchemaField (#8880)

  • Add debug logging statements to track when BQ Storage API is used. (#8838)

  • Hide error traceback in BigQuery cell magic (#8808)

  • Allow choice of compression when loading from dataframe (#8938)

  • Additional clustering metrics for BQML K-means models (via synth). (#8945)

Documentation

  • Add compatibility check badges to READMEs. (#8288)

  • Link to googleapis.dev documentation in READMEs. (#8705)

  • Remove redundant service account key code sample. (#8891)

Internal / Testing Changes

  • Fix several pytest “skip if” markers (#8694)

  • Update tests to support conversion of NaN as NULL in pyarrow 0.14.\*. (#8785)

  • Mock external calls in one of BigQuery unit tests (#8727)

  • Set IPython user agent when running queries with IPython cell magic (#8713)

  • Use configurable bucket name for GCS samples data in systems tests. (#8783)

  • Move maybe_fail_import() to top level test utils (#8840)

  • Set BQ Storage client user-agent when in Jupyter cell (#8734)

1.17.0

07-12-2019 07:56 PDT

New Features

  • Support faster Arrow data format in to_dataframe when using BigQuery Storage API. (#8551)

  • Add to_arrow to get a pyarrow.Table from query results. (#8609)

Dependencies

  • Exclude bad 0.14.0 pyarrow release. (#8551)

1.16.0

07-01-2019 10:22 PDT

New Features

  • Add Routines API. (#8491)

  • Add more stats to Models API, such as optimization_strategy (via synth). (#8344)

Documentation

  • Add docs job to publish to googleapis.dev. (#8464)

  • Add sample demonstrating how to create a job. (#8422)

  • Use autodetected location in code samples. (#8340, #8341)

Internal / Testing Changes

  • Refactor to_dataframe to deterministicly update progress bar. (#8303)

1.15.0

06-14-2019 10:10 PDT

Implementation Changes

  • Fix bug where load_table_from_dataframe could not append to REQUIRED fields. (#8230)

New Features

  • Add page_size parameter to QueryJob.result. (#8206)

1.14.0

06-04-2019 11:11 PDT

New Features

  • Add maximum_bytes_billed argument and context.default_query_job_config property to magics. (#8179)

Dependencies

  • Don’t pin google-api-core in libs using google-cloud-core. (#8213)

1.13.0

05-31-2019 10:22 PDT

New Features

  • Use job_config.schema for data type conversion if specified in load_table_from_dataframe. (#8105)

Internal / Testing Changes

  • Adds private _connection object to magics context. (#8192)

  • Fix coverage in ‘types.py’ (via synth). (#8146)

1.12.1

05-21-2019 11:16 PDT

Implementation Changes

  • Don’t raise error when encountering unknown fields in Models API. (#8083)

Documentation

  • Use alabaster theme everwhere. (#8021)

Internal / Testing Changes

  • Add empty lines (via synth). (#8049)

1.12.0

05-16-2019 11:25 PDT

Implementation Changes

  • Remove duplicates from index on pandas DataFrames returned by to_dataframe(). (#7953)

  • Prevent error when time partitioning is populated with empty dict (#7904)

  • Preserve order in to_dataframe with BQ Storage from queries containing ORDER BY (#7793)

  • Respect progress_bar_type in to_dataframe when used with BQ Storage API (#7697)

  • Refactor QueryJob.query to read from resource dictionary (#7763)

  • Close the to_dataframe progress bar when finished. (#7757)

  • Ensure that KeyboardInterrupt during to_dataframeno longer hangs. (#7698)

  • Raise ValueError when BQ Storage is required but missing (#7726)

  • Make total_rows available on RowIterator before iteration (#7622)

  • Avoid masking auth errors in to_dataframe with BQ Storage API (#7674)

New Features

  • Add support for passing client_info. (#7849 and (#7806)

  • Phase 1 for storing schemas for later use. (#7761)

  • Add destination and related properties to LoadJob. (#7710)

  • Add clustering_fields property to TableListItem (#7692)

  • Add created and expires properties to TableListItem (#7684)

Dependencies

  • Pin google-cloud-core >= 1.0.0, < 2.0dev. (#7993)

  • Add [all] extras to install all extra dependencies (#7610)

Documentation

  • Move table and dataset snippets to samples/ directory (#7683)

Internal / Testing Changes

  • Blacken unit tests. (#7960)

  • Cleanup client tests with method to create minimal table resource (#7802)

1.11.2

04-05-2019 08:16 PDT

Dependencies

  • Add dependency on protobuf. (#7668)

1.11.1

04-04-2019 09:19 PDT

Internal / Testing Changes

  • Increment version number in setup.py.

1.11.0

04-03-2019 19:33 PDT

Implementation Changes

  • Remove classifier for Python 3.4 for end-of-life. (#7535)

New Features

  • Enable fastparquet support by using temporary file in load_table_from_dataframe (#7545)

  • Allow string for copy sources, query destination, and default dataset (#7560)

  • Add progress_bar_type argument to to_dataframe to use tqdm to display a progress bar (#7552)

  • Call get_table in list_rows if the schema is not available (#7621)

  • Fallback to BQ API when there are problems reading from BQ Storage. (#7633)

  • Add methods for Models API (#7562)

  • Add option to use BigQuery Storage API from IPython magics (#7640)

Documentation

  • Remove typo in Table.from_api_repr docstring. (#7509)

  • Add docs session to nox configuration for BigQuery (#7541)

Internal / Testing Changes

  • Refactor table() methods into shared implementation. (#7516)

  • Blacken noxfile and setup file in nox session (#7619)

  • Actually use the progress_bar_type argument in QueryJob.to_dataframe(). (#7616)

1.10.0

03-06-2019 15:20 PST

Implementation Changes

  • Harden ‘ArrayQueryParameter.from_api_repr’ against missing ‘parameterValue’. (#7311)

  • Allow nested records w/ null values. (#7297)

New Features

  • Add exists_ok and not_found_ok options to ignore errors when creating/deleting datasets/tables. (#7491)

  • Accept a string in Table and Dataset constructors. (#7483)

Documentation

  • Update docstring of RowIterator’s to_dataframe (#7306)

  • Updated client library documentation URLs. (#7307)

Internal / Testing Changes

1.9.0

02-04-2019 13:28 PST

New Features

  • Add arguments to select dtypes and use BQ Storage API to QueryJob.to_dataframe(). (#7241)

Documentation

  • Add sample for fetching total_rows from query results. (#7217)

1.8.1

12-17-2018 17:53 PST

Documentation

  • Document Python 2 deprecation (#6910)

  • Normalize docs for ‘page_size’ / ‘max_results’ / ‘page_token’ (#6842)

1.8.0

12-10-2018 12:39 PST

Implementation Changes

  • Add option to use BQ Storage API with to_dataframe (#6854)

  • Fix exception type in comment (#6847)

  • Add to_bqstorage to convert from Table[Reference] google-cloud-bigquery-storage reference (#6840)

  • Import iam.policy from google.api_core. (#6741)

  • Add avro logical type control for load jobs. (#6827)

  • Allow setting partition expiration to ‘None’. (#6823)

  • Add retry argument to _AsyncJob.result. (#6302)

Dependencies

  • Update dependency to google-cloud-core (#6835)

Documentation

  • Add avro load samples (#6832)

Internal / Testing Changes

  • Blacken libraries (#6794)

  • Fix copy/paste typos in noxfile comments (#6831)

1.7.0

11-05-2018 16:41 PST

Implementation Changes

  • Add destination table properties to LoadJobConfig. (#6202)

  • Allow strings or references in create_dataset and create_table (#6199)

  • Fix swallowed error message (#6168)

New Features

  • Add --params option to %%bigquery magic (#6277)

  • Expose to_api_repr method for jobs. (#6176)

  • Allow string in addition to DatasetReference / TableReference in Client methods. (#6164)

  • Add keyword arguments to job config constructors for setting properties (#6397)

Documentation

  • Update README service links in quickstart guides. (#6322)

  • Move usage guides to their own docs. (#6238)

  • Normalize use of support level badges (#6159)

Internal / Testing Changes

  • Deprecation cleanups (#6304)

  • Use _get_sub_prop helper so missing load stats don’t raise. (#6269)

  • Use new Nox (#6175)

  • Harden snippets against transient GCS errors. (#6184)

1.6.0

New Features

  • Add support for GEOGRAPHY type (#6147)

  • Add default QueryJobConfig to Client (#6088)

Documentation

  • Remove unused “append” samples (#6100)

Internal / Testing Changes

  • Address dataset leaks, conflicts in systests (#6099)

  • Harden bucket teardown against 429 Too Many Requests. (#6101)

1.5.1

Implementation Changes

  • Retry ‘502 Bad Gateway’ errors by default. (#5930)

  • Avoid pulling entire result set into memory when constructing dataframe. (#5870)

  • Add support for retrying unstructured 429 / 500 / 502 responses. (#6011)

  • Populate the jobReference from the API response. (#6044)

Documentation

  • Prepare documentation for repo split (#5955)

  • Fix leakage of bigquery/spanner sections into sidebar menu. (#5986)

Internal / Testing Changes

  • Test pandas support under Python 3.7. (#5857)

  • Nox: use inplace installs (#5865)

  • Update system test to use test data in bigquery-public-data. (#5965)

1.5.0

Implementation Changes

  • Make ‘Table.location’ read-only. (#5687)

New Features

  • Add ‘clustering_fields’ properties. (#5630)

  • Add support for job labels (#5654)

  • Add ‘QueryJob.estimated_bytes_processed’ property (#5655)

  • Add support/tests for loading tables from ‘gzip.GzipFile’. (#5711)

  • Add ‘ExternalSourceFormat’ enum. (#5674)

  • Add default location to client (#5678)

Documentation

  • Fix typo in CopyJob sources docstring (#5690)

Internal / Testing Changes

  • Add/refactor snippets for managing BigQuery jobs (#5631)

  • Reenable systests for ‘dataset.update’/’table.update’. (#5732)

1.4.0

Implementation Changes

  • Add ‘internalError’ to retryable error reasons. (#5599)

  • Don’t raise exception if viewing CREATE VIEW DDL results (#5602)

New Features

  • Add Orc source format support and samples (#5500)

  • Move ‘DEFAULT_RETRY’ (w/ its predicate) to a new public ‘retry’ module. (#5552)

  • Allow listing rows on an empty table. (#5584)

Documentation

  • Add load_table_from_dataframe() to usage docs and changelog and dedents snippets in usage page (#5501)

  • Add samples for query external data sources (GCS & Sheets) (#5491)

  • Add BigQuery authorized view samples (#5515)

  • Update docs to show pyarrow as the only dependency of load_table_from_dataframe() (#5582)

Internal / Testing Changes

  • Add missing explict coverage for ‘_helpers’ (#5550)

  • Skip update_table and update_dataset tests until etag issue is resolved. (#5590)

1.3.0

New Features

  • NUMERIC type support (#5331)

  • Add timeline and top-level slot-millis to query statistics. (#5312)

  • Add additional statistics to query plan stages. (#5307)

  • Add client.load_table_from_dataframe() (#5387)

Documentation

  • Use autosummary to split up API reference docs (#5340)

  • Fix typo in Client docstrings (#5342)

Internal / Testing Changes

  • Prune systests identified as reduntant to snippets. (#5365)

  • Modify system tests to use prerelease versions of grpcio (#5304)

  • Improve system test performance (#5319)

1.2.0

Implementation Changes

  • Switch list_partitions helper to a direct metatable read (#5273)

  • Fix typo in Encoding.ISO_8859_1 enum value (#5211)

New Features

  • Add UnknownJob type for redacted jobs. (#5281)

  • Add project parameter to list_datasets and list_jobs (#5217)

  • Add from_string factory methods to Dataset and Table (#5255)

  • Add colu