BigQuery release notes

This page documents production updates to BigQuery. We recommend that BigQuery developers periodically check this list for any new announcements.

You can see the latest product updates for all of Google Cloud on the Google Cloud page, browse and filter all release notes in the Google Cloud console, or you can programmatically access release notes in BigQuery.

To get the latest product updates delivered to you, add the URL of this page to your feed reader, or add the feed URL directly: https://cloud.google.com/feeds/bigquery-release-notes.xml

August 10, 2022

You can now set default configurations at a project or organization level. This feature is now generally available (GA).

August 09, 2022

August 08, 2022

A weekly digest of client library updates from across the Cloud SDK.

Go

Changes for bigquery/storage/apiv1beta1

1.37.0 (2022-08-04)

Features
  • bigquery/connection: Add service_account_id output field to CloudSQL properties (1d6fbcc)
  • bigquery/storage/managedwriter: refactor AppendResponse (#6402) (c07bca2)
  • bigquery: support JSON as a data type (#5986) (835fe4f)
Bug Fixes
  • bigquery: include user_email field when requesting job information (#6256) (da42b4e)
Documentation
  • bigquery/storage: clarify size limitations for AppendRowsRequest chore: add preferred_min_stream_count to CreateReadSessionRequest chore: add write_stream to AppendRowsResponse (1d6fbcc)

August 03, 2022

The max_staleness materialized view option helps you achieve consistently high performance with controlled costs when processing large, frequently changing datasets. This feature is now in preview.

August 01, 2022

A weekly digest of client library updates from across the Cloud SDK.

Python

Changes for google-cloud-bigquery

3.3.0 (2022-07-25)

Features
  • add destination_expiration_time property to copy job (#1277) (728b07c)
Bug Fixes
Documentation
  • samples: add table snapshot sample (#1274) (e760d1b)
  • samples: explicitly add bq to samples reqs, upgrade grpc to fix bug on m1 (#1290) (9b7e3e4)

The trigonometric SQL function CBRT is now generally available (GA). With this function, you can compute the cube root of a value.

The LOAD DATA statement is now available for Preview in Google Standard SQL for BigQuery. You can use the LOAD DATA statement to load data from one or more files into a table.

July 28, 2022

You can now create BigQuery subscriptions in Pub/Sub to write messages directly to an existing BigQuery table.

July 27, 2022

Inverse trigonometric SQL functions are now generally available (GA). These functions include:

  • COT: Compute the cotangent for an angle.
  • COTH: Compute the hyperbolic cotangent for an angle.
  • CSC: Compute the cosecant for an angle.
  • CSCH: Compute the hyperbolic cosecant for an angle.
  • SEC: Compute the secant for an angle.
  • SECH: Compute the hyperbolic secant for an angle.

July 25, 2022

BigLake is now generally available (GA). You can now create BigQuery ML models using data in Cloud Storage by using BigLake and publish BigLake tables as Analytics Hub listings.

The new Migrate section in the BigQuery documentation helps you migrate to BigQuery. This includes high-level guidance with a migration overview, an introduction to free-to-use tools that help you with each phase of migration, and platform-specific migration guides.

July 20, 2022

Analytics Hub is now available in additional regions across the Americas, Asia Pacific, and Europe. For more information, see Analytics Hub supported regions.

July 14, 2022

Previously, the Storage Write API had a maximum concurrent connection limit of 100 connections for non-multi-regions such as Montreal (northamerica-northeast1). This limit has now been increased to 1,000 connections across all non-multi-regions. For more information, see Storage Write API quotas and limits.

July 12, 2022

You can now select a job type when assigning a folder, organization, or project to a reservation in the Google Cloud console. This feature is now generally available (GA).

July 11, 2022

The google.cloud.bigquery.reservation.v1beta1.api package is deprecated and will be removed on September 27, 2022. After that date, requests to that package will fail. Data created by using google.cloud.bigquery.reservation.v1beta1.api are accessible by using the google.cloud.bigquery.reservation.v1.api package.

Next steps:

July 07, 2022

Azure workload identity federation is now available in preview for BigQuery Omni connections. This feature helps you secure data by allowing you to grant Google access to an application you manage in your Azure tenant so that neither you nor Google must manage application client secrets.

July 06, 2022

The APPENDS change history TVF is now in preview. This table-valued function provides a history of table appends over a window of time.

July 01, 2022

An updated version of JDBC driver for BigQuery is now available. This version includes a fix for an issue with connector returning stack overflow in some cases when executing complex long queries.

June 29, 2022

You can now set the view field in the tables.get() API method to indicate which table information is returned. Setting the value to BASIC reduces latency by omitting some storage statistics.

Previously, all BigQuery BI Engine projects had a maximum reservation size per project per location limit of 100 GB. This limit is now 250 GB. For more information, see BI Engine quotas and limits.

June 23, 2022

The BI Engine preferred tables feature lets you limit BI Engine acceleration to a specified set of tables. This feature is now in preview.

June 21, 2022

Query queues are now available in preview for on-demand and flat-rate customers. When query queues are enabled, BigQuery automatically determines the query concurrency rather than setting a fixed limit. Flat-rate customers can override this setting with a custom concurrency target. Additional queries beyond the concurrency target are queued until processing resources become available.

June 15, 2022

Deterministic encryption SQL functions are now generally available (GA). New AEAD encryption functions include DETERMINISTIC_ENCRYPT, DETERMINISTIC_DECRYPT_BYTES, and DETERMINISTIC_DECRYPT_STRING. These functions allow column-level encryption and decryption of data while supporting aggregation and table joins.

June 14, 2022

You can now use the Cloud console to set up VPC service control perimeters to restrict access from BigQuery Omni to external clouds. You can also specify whether you want to grant read or write permission on your external resource. This feature is now generally available (GA).

You can now explore data in Data Studio by using links from your BigQuery query results in the Google Cloud Console. This feature is now generally available (GA).

June 13, 2022

A new system variable, @@dataset_project_id, is now generally available. @@dataset_project_id allows you to set a default project where one is not specified for a dataset in your query. This variable is also available as a Connection Property.

June 10, 2022

Quotas for multi-statement queries have changed. The cumulative time limit for a multi-statement query has increased from 6 hours to 24 hours.

June 08, 2022

Batch and interactive translation services are now generally available (GA), and include support for most major SQL dialects. This release also includes preview availability of SQL object name mapping and metadata extraction tools that you can use to increase the accuracy of your batch translation jobs.

June 06, 2022

You can now attach Resource Manager tags to datasets. This feature is supported in Preview. Tags let you conditionally apply Identity and Access Management (IAM) policies to resources.

The following Storage Read API quotas and limits have changed:

  • There is now a limit of 2,000 concurrent ReadRows calls per project in the US and EU multi-regions and 400 concurrent ReadRows calls in other regions.
  • The number of data plane requests per user per project per minute has increased from 5,000 to 25,000.

For more information, see Storage Read API quotas and limits.

June 03, 2022

BigQuery Omni now supports Reservation and Access Control DCL. This feature is in Preview.

May 31, 2022

Column-level data masking is now available in preview. You can use data masking to selectively obscure column data for groups of users, while still allowing access to the column. When you use data masking in combination with column-level access control, you can configure a range of access to column data, from full access to no access, based on the requirements of different groups of users.

May 24, 2022

You can now load data into BigQuery using Informatica Data Loader. This feature is generally available. Informatica provides connectors that can ingest data into BigQuery.

May 23, 2022

Metrics for query/statement_scanned_bytes and query/statement_scanned_bytes_billed are no longer delayed for 6 hours in order to smooth reporting over the duration of the job. Values are now reported every 180 seconds without smoothing. For more information about metrics, see Google Cloud metrics.

May 18, 2022

Updated versions of ODBC and JDBC drivers for BigQuery are now available that include enhancements.

May 05, 2022

The new format element %J is generally available (GA) for DATE, TIME, DATETIME, and TIMESTAMP functions. This format element lets you use the ISO 8601 1-based day of the year.

PARSE_DATE, PARSE_TIME, PARSE_DATETIME, and PARSE_TIMESTAMP now support the following date and time format elements: %a, %A, %g, %G, %j, %u, %U, %V, %w, and %W.

May 02, 2022

Case-insensitive collation support for BigQuery is now available for Preview. Collation determines how strings are sorted and compared in collation-supported operations. If case-insensitive collation is used, case is ignored in comparison and sorting operations.

These operations support collation:

The COLLATE function is now available for Preview in Google Standard SQL for BigQuery. With the COLLATE function, you can pass in a STRING and return a STRING with a collation specification.

The DEFAULT COLLATE clause is now available for Preview. With this clause, the default collation specification is applied to all column data types supporting collation. You can use the DEFAULT COLLATE clause in the following DDL statements:

The COLLATE clause is now available for Preview. With this clause, a collation specification is applied to a specific column in a table. You can use the COLLATE clause in the following DDL statements:

April 25, 2022

The ability to configure the time travel window is now in Preview. You can specify the duration of the time travel window, from a minimum of two days to a maximum of seven days.

Three new INFORMATION_SCHEMA views that show table storage metadata are now in Preview.

BigQuery Admin Resource Charts are now generally available (GA) for on-demand users, enabling administrators to monitor key metrics and troubleshoot issues across the entire organization. Previously, it was only available for reservation users. A new permission, bigquery.jobs.listExecutionMetadata, has been added to make it easier to gain access to the full UI.

April 11, 2022

Starting in July 2022, the projects.list API method will return results in unsorted order. Currently, the API returns the results in sorted order, although this is not a documented behavior of the API.

April 07, 2022

BigLake is now available in Preview. BigLake is a storage engine that allows you to query and unify cross-cloud data lakes and warehouses. Additionally, it provides fine-grained access controls to your tables, allowing you to set access policies on a column or row basis.

BigQuery now supports the creation of search indexes and a SEARCH function. This feature is in Preview. This enables you to use Google Standard SQL to efficiently find data elements in unstructured text and semi-structured data.

April 06, 2022

Analytics Hub is now available in Preview. Analytics Hub is a new service in BigQuery that lets you create secure data exchanges and share analytics assets within and across organizations. This platform allows data providers to publish listings that reference shared datasets. Analytics Hub subscribers can then view and subscribe to these listings.

April 05, 2022

BigQuery Omni now supports cross-cloud transfer. This feature is in Preview. For more information, see Cross-cloud transfer (AWS) and Cross-cloud transfer (Azure).

March 31, 2022

The international public dataset for Data Signals for Google Search Trends is now available in Preview and available in the Google Cloud Marketplace and Analytics Hub.

March 17, 2022

The BigQuery slot recommender is now available in Preview. The slot recommender creates recommendations for customers using on-demand billing. These recommendations help you to understand the cost and performance tradeoffs of purchasing different amounts of slot capacity.

March 16, 2022

You can now explicitly specify a schema for BigQuery external tables created over Parquet, ORC, and Avro file formats. Previously, the schema was always auto-detected using the last lexicographic file.

March 04, 2022

February 16, 2022

Remote functions are now available for preview. Remote functions allow you to implement your function in other languages than SQL and Javascript, or with libraries or services which are not allowed in BigQuery user-defined functions.

February 15, 2022

The table clones feature in BigQuery is now in Preview. A table clone is a lightweight, writable copy of a table. You are only charged for storing the data in a table clone that differs from its base table.

February 14, 2022

The QUALIFY clause, which lets you filter the results of analytic functions in Google Standard SQL, is now generally available (GA).

BigQuery reliability guide is now available. This guide describes how to build solutions with BigQuery that meet your application's needs for availability, durability, consistency, and data recovery. Topics include the following:

February 10, 2022

BigQuery Omni now supports INFORMATION_SCHEMA.JOBS_* and INFORMATION_SCHEMA.RESERVATION* views. This feature is in Preview. For more information, see View resource metadata (AWS) and View resource metadata (Azure).

February 03, 2022

The BigQuery migration assessment is now available in Preview. Use this feature to assess the complexity of migrating from your current data warehouse to BigQuery.

February 02, 2022

The WITH RECURSIVE feature has been added to Google Standard SQL for BigQuery and is now in Preview. This feature allows a query in a WITH clause to refer to either itself or to queries defined later in the WITH clause.

January 31, 2022

January 06, 2022

BigQuery standard SQL now supports the JSON data type for storing JSON data. The JSON data type is in Preview. For more information, see Working with JSON data in Standard SQL.

December 23, 2021

Documentation now includes a series of introductory topics to orient you to BigQuery including:

  • What is BigQuery? - Product overview, available tools, and learning resources
  • Storage - Infrastructure, ingestion, and optimization
  • Analytics - Strategies, SQL queries, and BI tools
  • Administration - Resources, workload management, security, and monitoring

In addition, the table of contents is updated to guide you through your staged BigQuery deployment with stages including: Discovery, Get started, Design, Ingest, Analyze, Administer, Secure, and Develop.

December 16, 2021

The row-level security feature now supports administrator access to historical data for tables with row-level access policies.

December 14, 2021

November 19, 2021

Updated versions of ODBC and JDBC drivers for BigQuery are now available that include enhancements.

November 16, 2021

BigQuery is now available in the Santiago (southamerica-west1) region.

BigQuery BI Engine is now available in the Santiago (southamerica-west1) region.

November 12, 2021

BigQuery now supports authorized datasets (General Availability).

November 09, 2021

The following scripting statements have been added to Google Standard SQL for BigQuery.

  • CASE: Executes the first list of SQL statements where a boolean expression is TRUE.
  • CASE search_expression: Executes the first list of SQL statements where the search expression matches a WHEN expression.
  • LABELS: Provides an unconditional jump to the end of the block or loop associated with a label.
  • REPEAT: Repeatedly executes a list of SQL statements until the boolean condition at the end of the list is TRUE.
  • FOR...IN: Loops over every row in a table expression.

These features are generally available (GA).

November 08, 2021

The following INFORMATION_SCHEMA views now support a DDL column. The value of the column is the DDL statement that can be used to create the resource.

This feature is generally available (GA).

November 02, 2021

BigQuery now supports parameterized types. The following parameterized types are supported:

This feature is generally available GA.

October 28, 2021

The table snapshots feature is now generally available (GA). It includes a Cloud Console interface and support for creating a table snapshot in a different project from its base table.

October 27, 2021

SQL column-level encryption using Cloud Key Management Service (KMS) is now generally available (GA), letting you encrypt keysets within AEAD encryption functions.

October 25, 2021

BigQuery Omni, a multi-cloud analytics solution, is now generally available.

October 12, 2021

The BigQuery Storage Write API is now generally available (GA). The Storage Write API combines the functionality of high-throughput streaming ingestion and batch loading into a single API.

October 04, 2021

October 01, 2021

BigQuery pricing has changed as follows:

  1. BigQuery Storage Read API has moved from a single regional SKU to a set of regional SKUs for bytes scanned. All BigQuery Storage Read API users can now read up to 300 TB of data per month at no charge. For more information, see BigQuery data extraction pricing.

  2. BigQuery now charges BigQuery Storage Read API users for network egress. For more information, see BigQuery Storage Read API Network Egress Within Google Cloud.

BigQuery now supports the following geospatial data functions:

  • ST_BUFFER: Returns a GEOGRAPHY that represents the buffer around the input GEOGRAPHY. You specify the number of segments to determine how much the resulting geography can deviate from the ideal buffer radius.

  • ST_BUFFERWITHTOLERANCE: Returns a GEOGRAPHY that represents the buffer around the input GEOGRAPHY. You specify the tolerance to determine how much the resulting geography can deviate from the ideal buffer radius.

These functions are available as a preview.

September 28, 2021

Table functions are now generally available (GA). With the GA release, authorized table functions are now supported.

September 27, 2021

BigQuery now supports the following geospatial data functions:

  • ST_BOUNDINGBOX: Returns a STRUCT that represents the bounding box for a geography.

  • ST_EXTENT: Returns a STRUCT that represents the bounding box for a set of geographies.

  • S2_COVERINGCELLIDS: Returns an array of S2 cell IDs that cover a geography.

  • S2_CELLIDFROMPOINT: Returns the S2 cell ID covering a point geography.

These functions are generally available (GA).

September 21, 2021

When saving query results from the Cloud Console to a CSV file, the available download size is now 10 MB. Previously the limit was 16,000 rows. Also, you can now download tables with nested and repeated data to CSV files.

September 17, 2021

BigQuery now supports the following geospatial data functions:

  • ST_EXTERIORRING: Returns a linestring geography that corresponds to the outermost ring of a polygon geography.

  • ST_INTERIORRINGS: Returns an array of linestring geographies that corresponds to the interior rings of a polygon geography.

  • ST_ANGLE: Returns the angle between two intersecting lines.

  • ST_AZIMUTH: Returns the azimuth of a line segment formed by two points.

  • ST_NUMGEOMETRIES: Returns the number of geometries in a geography.

  • ST_GEOMETRYTYPE: Returns the Open Geospatial Consortium (OGC) geometry type that describes a geography as a string.

These functions are generally available (GA).

September 08, 2021

Deleting the metadata for a specific job using the bq command-line tool is now generally available (GA).

Session support for BigQuery is now in Preview. With sessions:

  • You can associate your SQL activities in a session across scripts and multi-statement transactions in BigQuery with a unique session identifier.
  • You can use session variables (for example, default timezone or dataset) and temporary tables throughout the life of the session and also across scripts and transactions
  • When you enable sessions, all actions performed across multiple sessions can be viewed using the SESSION_ID column now available in jobs INFORMATION_SCHEMA views.

August 31, 2021

An updated version of ODBC driver for BigQuery is now available that includes enhancements.

August 30, 2021

Exporting table data in Parquet format is now generally available (GA).

August 23, 2021

BigQuery Admin Resource Charts are now generally available (GA) for reservation users, enabling administrators to more easily monitor and troubleshoot their BigQuery environment. They provide visibility into key metrics such as slot consumption, job concurrency, job execution time, job errors, and bytes processed across the entire organization.

BigQuery Slot Estimator is now in Preview for reservation users. This tool analyzes slot utilization data to help administrators estimate the right number of slots to purchase, and provides insights on how job performance might be impacted by adding or reducing slot capacity for the entire organization or specific reservations.

August 19, 2021

Cloud Spanner federated queries are now generally available (GA).

August 03, 2021

BigQuery now supports the ALTER COLUMN SET DATA TYPE data definition language (DDL) statement. This feature is generally available (GA).

BigQuery now supports the following data definition language (DDL) statement:

This feature is generally available (GA).

BigQuery is now available in the Toronto (northamerica-northeast2) region.

BigQuery BI Engine is now available in the Toronto (northamerica-northeast2) region.

August 02, 2021

An updated version of JDBC driver for BigQuery is now available that includes Enhancements & New Features.

July 27, 2021

BigQuery now supports the INTERVAL type, which represents a duration or an amount of time. This type is in Preview.

July 26, 2021

DML query jobs now return statistics about the number of rows that were inserted, deleted, or updated. For more information, see DmlStats in the Job resource type. In addition, DML statistics are now available in the INFORMATION_SCHEMA.JOBS_BY_* views. This feature is generally available (GA).

July 19, 2021

BigQuery now supports workload management data control language (DCL) statements:

This feature is generally available GA.

BigQuery now supports the following SQL query operators:

This feature is generally available (GA).

BigQuery standard SQL now supports the CONTAINS_SUBSTR function. This feature is generally available (GA).

July 07, 2021

July 01, 2021

An updated version of ODBC driver for BigQuery is now available that includes bug fixes, parameterized data type support, and metadata retrieval performance improvements.

An updated version of JDBC driver for BigQuery is now available that includes bug fixes, parameterized data type support, and job retry improvements.

June 29, 2021

BigQuery is now available in the Delhi (asia-south2) region.

BigQuery BI Engine is now available in the Delhi (asia-south2) region.

BigQuery now supports multi-statement transactions. These allow you to perform mutating operations, such as inserting or deleting rows, on one or more tables, and either commit or roll back the changes atomically. This feature is in Preview.

June 28, 2021

BigQuery now supports access management data control language (DCL) statements and corresponding views:

GRANT and REVOKE statements are generally available (GA). OBJECT_PRIVILEGES table is available in Preview.

BigQuery now supports the following casting features:

These features are generally available (GA).

BigQuery now supports the ALTER COLUMN SET OPTIONS data definition language (DDL) statement. This feature is generally available (GA).

Table functions are now available in Preview. These user-defined functions, commonly known as table-valued functions (TVFs), return a table value.

The Google Trends dataset is now available in Preview and available in the Google Cloud Marketplace.

June 25, 2021

BigQuery table snapshots are now in Preview. A table snapshot is a low-cost, read-only copy of a table's data as it was at a particular time. For more information, see Introduction to table snapshots.

June 21, 2021

Row-level security on table data is now generally available in BigQuery.

BigQuery is now available in the Melbourne (australia-southeast2) region.

BigQuery BI Engine is now available in the Melbourne (australia-southeast2) region.

June 07, 2021

BigQuery now supports parameterized types. The following parameterized types are supported:

This feature is in Preview.

May 25, 2021

The free trial period for BigQuery BI Engine's SQL interface has been extended to July 15th, 2021. You must enroll to participate in the preview. With this feature, BI Engine now interacts with popular BI tools such as Looker, Tableau, and more, by means of an interactive SQL interface.

May 20, 2021

BigQuery GIS now supports loading geography data from newline-delimited GeoJSON files. This feature is generally available (GA). For more information, see Loading GeoJSON data.

BigQuery GIS now supports the following functions. These functions are generally available (GA).

These functions return a point of a linestring geography as a point geography.

May 19, 2021

BigQuery now supports the ability to rename tables using SQL. See ALTER TABLE RENAME TO. This feature is generally available (GA).

May 11, 2021

Updated version of ODBC driver for BigQuery includes bug fixes and install guide improvements.

Updated version of JDBC driver for BigQuery includes bug fixes, service account keyfile support, connection property enhancements, and BigQuery client library updates.

May 10, 2021

BigQuery now supports the following SQL query clauses and operators:

This feature is in Preview.

April 30, 2021

BigQuery now supports the following data definition language (DDL) statements:

This feature is in GA.

April 21, 2021

BigQuery supports changing an existing non-clustered table to a clustered table and vice versa. You can also update the set of clustered columns of a clustered table. This feature was first documented in October 2020 but was not included in a release note. For more information, see Modifying clustering specification.

April 12, 2021

The BigQuery Admin Resource Charts Preview is now available for Reservation users, enabling administrators to more easily monitor and troubleshoot their BigQuery environment. It provides visibility into key metrics such as slot consumption, job concurrency, and job execution time across the entire organization.

April 09, 2021

BigQuery now has better support for loading ENUM and LIST types in Parquet files.

  • ENUM logical types can be converted to STRING or BYTES.
  • Schema inference is supported for LIST logical types.

For more information, see Loading Parquet data from Cloud Storage.

April 07, 2021

Beginning in early Q3 2021, BigQuery Storage Read API will start charging for network egress. In addition, BigQuery Storage Read API will become available in all locations, with appropriate pricing. Another release note will be issued when these changes take effect.

April 06, 2021

The BigQuery Storage Write API is now in Preview. The Storage Write API is a stream-based API for ingesting data into BigQuery at low cost and high throughput. It provides exactly-once delivery semantics with real-time latency. For more information, see Using the BigQuery Storage Write API.

April 02, 2021

BigQuery standard SQL now supports the ALTER TABLE DROP COLUMN. This feature is in Preview.

The maximum length has been increased from 128 characters to 300 characters for the following BigQuery fields: table column names, column alias names, and user-defined function names.

March 31, 2021

BigQuery standard SQL now supports the following statements for creating, configuring, and deleting datasets:

These statements are generally available (GA).

BigQuery standard SQL now supports the TABLESAMPLE operator, which lets you query random subsets of data from large BigQuery tables. For more information, see Table sampling. This feature is in Preview.

BigQuery standard SQL now supports the following JSON functions:

These statements are generally available (GA).

INFORMATION_SCHEMA views for table partitions are now available. This feature is in Preview.

The INFORMATION_SCHEMA.TABLES view now includes a DDL column that can be used to recreate the table. This feature is in Preview.

Support for the BigNumeric type in BigQuery standard SQL is now generally available (GA).

March 24, 2021

BigQuery is now available in the Warsaw (europe-central2) region.

BigQuery BI Engine is now available in the Warsaw (europe-central2) region.

March 02, 2021

Updated version of Magnitude Simba ODBC driver includes bug fixes, performance improvements, and enhancements such as support for dynamic SQL and additional DDL and DML keywords.

Updated version of Magnitude Simba JDBC driver includes bug fixes and performance improvements.

February 25, 2021

BigQuery BI Engine now interacts with popular BI tools such as Looker, Tableau, and more, by means of an SQL interface. You must enroll to participate in the preview.

BigQuery materialized views are now generally available (GA). BigQuery materialized views are now generally available (GA). Materialized views are precomputed views that periodically cache the results of a query, enhancing performance and efficiency, and reducing costs, particularly for aggregated queries. For more information, see Introduction to materialized views.

February 16, 2021

BigQuery now supports exporting table data in Parquet format. This feature is in Preview. For more information, see Parquet export details.

February 12, 2021

BigQuery standard SQL queries on Google Cloud Storage data are now supported by cached query results.

February 02, 2021

Updated version of Magnitude Simba JDBC driver includes bug and security fixes and enhancements for additional DDL keywords and dynamic SQL.

January 29, 2021

Clustered tables now support the DATETIME type for clustering columns. For more information, see Creating and using clustered tables. This feature is generally available.

January 19, 2021

BigQuery is now available in the Iowa (us-central1) region.

BigQuery BI Engine is now available in the Iowa (us-central1) region.

January 14, 2021

BigQuery's Cloud Console UI has been updated with many usability improvements for analyzing data, including multi-tab navigation, a new resource panel, and a new SQL editor. These updates are in Preview. For more information, see Using the Cloud Console.

December 21, 2020

BigQuery standard SQL now supports the BigNumeric data type for high-precision computations. The BigNumeric data type is in Preview.

December 10, 2020

Updated version of Magnitude Simba ODBC driver includes bug fixes and enhancements such as support for BigNumeric data and improved driver logic.

Updated version of Magnitude Simba JDBC driver includes bug and security fixes and enhancements such as support for Java 11, SSL trust store, BigNumeric data, and version-agnostic fully-qualified class names.

November 17, 2020

BigQuery column-level security is now generally available. Policy tags can be replicated across locations. For more information, see Introduction to BigQuery column-level security.

October 26, 2020

BigQuery standard SQL now supports the SUBSTRING function. This function is generally available (GA).

October 22, 2020

The ST_GEOGFROMGEOJSON and ST_GEOGFROMTEXT geographic functions support a new make_valid parameter. If set to TRUE, the function attempts to correct polygon issues when importing geography data.

The ST_GEOGFROMTEXT function also supports a new planar parameter. If set to TRUE, the function treats imported WKT geometries as having planar edges.

These new function parameters are in Beta.

Updated version of Magnitude Simba ODBC driver includes improvements to performance, logging, OpenSSL support, and bug fixes.

October 19, 2020

BigQuery Audit Logs stopped using the following checks for redacting resource names for cross-project access and caller identities: The bigquery.jobs.create permission check and the internal setting for a project domain. Please review the documentation at Caller identities and resource names.

October 14, 2020

Dynamic SQL is now generally available (GA). Dynamic SQL lets you generate and execute SQL statements dynamically at runtime. For more information, see EXECUTE IMMEDIATE.

BigQuery standard SQL now supports the following new functions. These functions are generally available (GA).

BigQuery now supports the following new statements. These statements are generally available (GA).

BigQuery standard SQL now supports DATE arithmetics operators.

BigQuery now supports Unicode table names. For more information, see Table naming.

Queries can now have duplicate column names.

October 06, 2020

Authorized user-defined functions (UDFs) are now generally available (GA). Authorized UDFs let you share query results without giving access to the underlying tables. For more information, see Authorized UDFs.

The Cloud Console now lets you opt in to search and autocomplete powered by Data Catalog. This feature is in beta.

October 01, 2020

BigQuery table-level access control is now generally available. For more information, see Introduction to table access controls .

September 21, 2020

The following time-unit partitioning features are now Generally Available (GA):

  • Creating partitions using hourly, monthly, and yearly time-unit granularities.
  • Using a DATETIME column as the partitioning column.

September 18, 2020

Beginning in late 2020, all BigQuery predefined roles with permission to create jobs, including queries, will also be able to get information about their own jobs and cancel their own jobs. This change does not include permission to see jobs created by other users.

September 17, 2020

September 11, 2020

You can now use the BQ.JOBS.CANCEL system procedure to cancel a job. For more information, see Canceling jobs.

August 31, 2020

Updated version of Magnitude Simba JDBC driver includes bug fixes and enhancements such as automatically turning on the BigQuery Storage API for anonymous table reads (no charge for temporary table reads).

August 26, 2020

INFORMATION_SCHEMA views for BigQuery reservations are now Generally Available (GA).

August 25, 2020

INFORMATION_SCHEMA views for jobs metadata by timeslice are now available.

August 24, 2020

BigQuery support for using service account credentials with scheduled queries is Generally Available (GA).

August 19, 2020

When using consecutive ON/USING clauses, parentheses are now optional and can be omitted. For example, you can use either of the following statements:

  • FROM A JOIN (B JOIN C ON B.x = C.y) USING (z)
  • FROM A JOIN B JOIN C ON B.x = C.y USING (z)

August 18, 2020

You can now explicitly assign projects to use on-demand pricing with Reservations. See Assign a project to None.

August 13, 2020

The exports per day (Extract Bytes) default quota has been raised from 10 TB to 50 TB per day.

August 11, 2020

For flat-rate pricing, the minimum slot purchase is now 100 slots. Slots can be purchased in 100-slot increments.

August 06, 2020

BigQuery is now available in the following regions: Oregon (us-west1), Belgium (europe-west1), and Netherlands (europe-west4).

BigQuery BI Engine is now available following regions: Oregon (us-west1), Belgium (europe-west1), and Netherlands (europe-west4).

July 31, 2020

Updated version of Magnitude Simba ODBC driver includes performance improvements and bug fixes.

July 27, 2020

INFORMATION_SCHEMA views for streaming metadata are now in alpha. You can use these views to retrieve historical and real-time information about streaming data into BigQuery.

July 24, 2020

BigQuery Data Transfer Service is now available in the following regions: Montréal (northamerica-northeast1), Frankfurt (europe-west3), Mumbai (asia-south1), and Seoul (asia-northeast3).

July 16, 2020

BigQuery GIS now supports two new functions, ST_CONVEXHULL and ST_DUMP:

  • ST_CONVEXHULL returns the smallest convex GEOGRAPHY that covers the input.
  • ST_DUMP returns an ARRAY of simple GEOGRAPHYs where each element is a component of the input GEOGRAPHY.

For more information, see the ST_CONVEXHULL and ST_DUMP reference pages.

July 13, 2020

The Standard SQL statement ASSERT is now supported. You can use ASSERT to validate that data matches specified expectations.

July 06, 2020

Updated version of Magnitude Simba ODBC driver. This version includes some performance improvements and bug fixes, and it catches up with the JDBC driver by adding support for user defined functions and variable time zones using the connection string.

June 29, 2020

The BigQuery SLA has been updated to >= 99.99% Monthly Uptime Percentage for all users.

June 26, 2020

Starting in mid-August, unqualified INFORMATION_SCHEMA queries for SCHEMATA and SCHEMATA_OPTIONS views will default to returning metadata from the US multi-region. For information about how to specify a region, see region qualifier syntax.

June 16, 2020

June 09, 2020

Clustering for non-partitioned tables is now supported. For more information about clustered tables, see Introduction to clustered tables.

June 08, 2020

BigQuery is now available in the Jakarta (asia-southeast2) region.

BigQuery BI Engine is now available in the Jakarta (asia-southeast2) region.

June 04, 2020

BigQuery Table ACL is now available as a beta release. For more information, see Introduction to table access controls .

June 02, 2020

You can now purchase BigQuery slots using the bq command-line tool. BigQuery Reservations allows you to purchase slots to take advantage of BigQuery flat-rate pricing and allocate slots for workload management.

A new GIS function, ST_Simplify, is available. ST_Simplify returns a simplified version of the input GEOGRAPHY by replacing sections with straight lines.

Standard SQL view definition bodies can now contain references without project qualifiers, as long as the view is created by the tables.insert REST API or is in the same project used to run the CREATE VIEW DDL query.

Standard SQL user-defined function definition bodies can now contain references to tables and views.

May 21, 2020

The BigQuery Storage API now supports reading small anonymous (cached) tables without any limitations.

May 20, 2020

Happy 10th birthday, BigQuery!

Cloud SQL federated queries are now generally available (GA).

Dynamic SQL is now available as a beta release in all BigQuery regions. Dynamic SQL lets you generate and execute SQL statements dynamically at runtime. For more information, see EXECUTE IMMEDIATE.

BigQuery Trial slots are now available in US and EU multi-regions. Trial slots are a limited promotion for qualified customers.

May 13, 2020

Updated versions of Magnitude Simba JDBC drivers have been released.

May 12, 2020

Updated versions of Magnitude Simba ODBC drivers have been released.

May 08, 2020

May 01, 2020

A new function, JSON_EXTRACT_ARRAY, has been added to the list of JSON functions. This function allows you to extract the contents of a JSON document as a string array. For more information, see the JSON_EXTRACT_ARRAY reference section.

The ORDER BY clause now supports the NULLS FIRST and NULLS LAST clauses. These clauses allow you to specify the sort order of null and non-null values. For more information, see the ORDER BY reference section.

April 30, 2020

The BigQuery Data Transfer Service is now available in the Taiwan (asia-east1) region.

April 27, 2020

BigQuery is now available in the Las Vegas (us-west4) region.

BigQuery BI Engine is now available in the Las Vegas (us-west4) region.

April 16, 2020

BigQuery Reservations is now Generally Available (GA). BigQuery Reservations allows you to purchase BigQuery slots to take advantage of BigQuery flat-rate pricing and allocate slots for workload management.

Around the end of April 2020, INFORMATION_SCHEMA (Beta) views for dataset metadata will return metadata about all datasets in a region. Currently, these views return metadata about all datasets in the project across all regions. This upcoming change will also provide support for querying a specific region's metadata (for example, region-us.INFORMATION_SCHEMA.SCHEMATA instead of INFORMATION_SCHEMA.SCHEMATA).

You can replicate this future behavior now by filtering on the SCHEMATA view's LOCATION column (for example, LOCATION = 'US').

April 09, 2020

Scheduling queries no longer requires the bigquery.transfers.update permission. The bigquery.jobs.create permission can now be used to schedule queries. See Scheduling queries for details.

April 08, 2020

BigQuery materialized views are now available as a beta release. For more information, see Introduction to materialized views .

April 02, 2020

BigQuery Reservations is now available in all BigQuery regions.

March 31, 2020

INFORMATION_SCHEMA views for BigQuery reservations are now in public alpha.

March 30, 2020

March 27, 2020

BigQuery Column-level security is now available as a beta release. For more information, see Introduction to BigQuery Column-level security.

March 02, 2020

The BigQuery Storage API is now generally available. For more information, see the Storage API overview.

Support for querying Cloud Storage data in ORC and Parquet format is now Generally Available.

Support for loading and querying externally partitioned data that is stored on Cloud Storage using a default hive partitioning layout is now Generally Available. The data must be in Avro, Parquet, ORC, CSV, or JSON format.

Personally identifiable information (PII) in audit log records for successful reads of resources with restricted access is no longer redacted. For more information, see Caller identities and resource names.

February 26, 2020

BigQuery Flex slots are now available in beta in all BigQuery Reservations regions. See Workload management and Commitment plans for more information.

BigQuery DML statements no longer have quota limits.

February 24, 2020

BigQuery is now available in the Salt Lake City (us-west3) region.

BI Engine is now available in the Salt Lake City (us-west3) region.

February 20, 2020

Added support for custom queries and views from Data Studio.

Increased reservation size to 100 GB.

February 18, 2020

Support for tables partitioned on an integer column is now GA. For more information, see Creating and using integer range partitioned tables.

February 14, 2020

INFORMATION_SCHEMA views for live and historical jobs metadata are now in beta.

February 13, 2020

The BigQuery sandbox is now GA.

February 11, 2020

Updated versions of Magnitude Simba ODBC drivers are now available.

January 24, 2020

BigQuery is now available in the Seoul (asia-northeast3) region.

January 22, 2020

The BigQuery Storage API is now available in all BigQuery regional and multi-regional locations. For more information, see Dataset locations.

January 16, 2020

Updated versions of Magnitude Simba JDBC and ODBC drivers have been released.

January 14, 2020

BigQuery Reservations is now available in the London (europe-west2) region.

December 20, 2019

The ALTER TABLE and ALTER VIEW Data Definition Language statements are now Generally Available (GA). These statements let you set options on tables and views.

BigQuery Reservations is now available in the Tokyo (asia-northeast1) region.

December 13, 2019

BigQuery Reservations is now in beta in US and EU regions. BigQuery Reservations allows you to purchase BigQuery slots to take advantage of BigQuery flat-rate pricing and allocate slots for workload management.

December 11, 2019

Support for tables partitioned on an integer column is now in Beta. For more information, see Creating and using integer range partitioned tables.

December 10, 2019

For users billed on the on-demand pricing model, BigQuery can now burst beyond the default slot capacity to accelerate queries. There is no extra charge associated with query acceleration.

November 26, 2019

INFORMATION_SCHEMA now has updated permissions checks. INFORMATION_SCHEMA is currently in beta.

Previously, queries against INFORMATION_SCHEMA views filtered the results based on the IAM permissions needed to get metadata about the resource. Now, queries return accessDenied (403) errors if users have insufficient IAM permissions to get metadata.

November 22, 2019

You can now use INFORMATION_SCHEMA views to get routine metadata in BigQuery. INFORMATION_SCHEMA is currently in beta.

November 21, 2019

BigQuery now supports using service account credentials with scheduled queries. This feature is in Beta.

November 20, 2019

BigQuery is now available in the South Carolina (us-east1) region.

November 12, 2019

Support for omitting column names in INSERT and MERGE statements is now Generally Available (GA).

October 31, 2019

Support for querying Cloud Storage data in ORC and Parquet format is now in beta.

Support for loading and querying externally partitioned data that is stored on Cloud Storage using a default hive partitioning layout is now in beta. The data must be in Avro, Parquet, ORC, CSV, or JSON format.

October 23, 2019

The BigQuery service is being renamed from:

bigquery-json.googleapis.com

to

bigquery.googleapis.com.

bigquery-json.googleapis.com will become an alias of bigquery.googleapis.com going forward.

This change is largely transparent to users and does not affect runtime traffic or management operations. Risks and mitigation are discussed below.

Calls to the BigQuery API will not be affected.

Timing

The change is a two phase process that we plan to release during November and December 2019.

Impact when using Service Management and Service Usage APIs

Responses from the Service Management API and the Service Usage API for information about the BigQuery service will contain information about bigquery.googleapis.com, even when you request data for bigquery-json.googleapis.com. Other than the name, the responses from the Service Management API and the Service Usage API will be the same. For example, listing enabled services will now return bigquery.googleapis.com, instead of bigquery-json.googleapis.com.

Risks and Mitigation

  • During the week of the second phase, alerts on Stackdriver Transparent Service Level Indicators (SLIs) for the bigquery-json.googleapis.com will break. Users should add a duplicate alert for bigquery.googleapis.com. The following are the serviceruntime SLIs where you now need to use bigquery.googleapis.com:

    • serviceruntime.googleapis.com/api/request_count
    • serviceruntime.googleapis.com/api/request_latencies
    • serviceruntime.googleapis.com/api/request_latencies_backend
    • serviceruntime.googleapis.com/api/request_latencies_overhead
    • serviceruntime.googleapis.com/api/request_sizes
    • serviceruntime.googleapis.com/api/response_sizes

  • The Google provider for Terraform has been updated to use the new name, bigquery.googleapis.com. Terraform users should upgrade to a Google provider with version >= 2.17.0 before November 4, 2019.

October 09, 2019

The RANGE_BUCKET mathematical function has been added. This function scans through a sorted array and returns the 0-based position of a given point's upper bound. This function can be useful if you need to group your data to build partitions, histograms, business-defined rules, and more.

October 04, 2019

Updated versions of Magnitude Simba JDBC and ODBC drivers have been released.

October 03, 2019

The ability to use scripting, stored procedures, and temporary tables is now in Beta.

September 27, 2019

  • The maximum size of a BI Engine reservation is now 50 GB per project per location. For more information, see Capacity limits.
  • We added more options for BI Engine capacity that is bundled with flat-rate pricing. For more information, see Flat-rate pricing.

September 17, 2019

BigQuery is now available in the Frankfurt (europe-west3) region.

BigQuery BI Engine is now available in the Frankfurt (europe-west3) region.

September 11, 2019

Billing is now enabled for BigQuery BI Engine. You are charged according to the information on the Pricing page.

August 30, 2019

Support for copying datasets, including across regions, is now in Beta.

August 27, 2019

When you create an external table that is linked to a Google Sheets file stored in Google Drive, you can now specify an individual sheet or a cell range. You are no longer limited to the first sheet only.

August 22, 2019

Updated version of Magnitude Simba ODBC driver.

You no longer have to manually re-cluster data in clustered tables. BigQuery now performs automatic re-clustering on all clustered tables.

August 19, 2019

If you stream data into BigQuery without populating the insertId field, you get the following higher quotas in the US multi-region location. These higher quotas are currently in beta.

  • Maximum rows per second: 1,000,000
  • Maximum bytes per second: 1 GB

For more information, see the Quotas and limits page.

August 07, 2019

You can now query data in BigQuery and Cloud SQL with a federated query. Cloud SQL federated queries are now in beta.

August 05, 2019

The BigQuery Storage API now supports reading data from BigQuery Storage in ARROW format. The BigQuery Storage API is currently in beta.

July 25, 2019

Updated version of Magnitude Simba ODBC driver. This version adds support for the BigQuery Storage API. The structure of the Linux ODBC tar package has also been changed to contain both 32-bit and 64-bit driver versions.

July 19, 2019

The scheduled query feature is now Generally Available.

July 01, 2019

Support for persistent user-defined functions (UDFs) is now in Beta.

June 13, 2019

Updated version of Magnitude Simba JDBC driver.

June 10, 2019

BigQuery is moving to the Google Cloud Platform Console. The classic web UI remains available through June 2020. For information on migrating to the GCP Console, see Moving to the Google Cloud Platform Console.

June 03, 2019

Support for omitting column names in INSERT and MERGE statements is now in Beta.

May 28, 2019

The maximum response size for query results has increased from 128 MB to 10 GB.

May 15, 2019

Support for clustering a table by using a GEOGRAPHY column is now Generally Available (GA).

May 13, 2019

The concurrent rate limit for on-demand, interactive queries has increased from 50 to 100 concurrent queries.

April 29, 2019

BigQuery BI Engine is now available in several additional regions:

  • Los Angeles
  • Montréal
  • Northern Virginia
  • São Paulo
  • Zürich
  • Hong Kong
  • Mumbai
  • Taiwan
  • Singapore
  • Sydney

For a complete list of BigQuery BI Engine locations, see Supported regions

April 22, 2019

The following quotas have changed:

  • The number of copy jobs per project per day has increased from 10,000 to 100,000.
  • The number of load jobs per project per day has increased from 50,000 to 100,000.
  • The number of export jobs per project per day has increased from 50,000 to 100,000.

April 18, 2019

BigQuery is now available in the Osaka (asia-northeast2) region. For upcoming regional launches, see the Google Cloud Platform blog.

April 10, 2019

BigQuery standard SQL now supports AEAD encryption functions.

April 05, 2019

BigQuery now allows you to purchase as few as 500 slots when you enroll in a flat-rate pricing plan. You are also now able to choose either a monthly or a yearly flat-rate pricing plan. The new flat-rate pricing plans are currently in alpha.

April 04, 2019

March 12, 2019

Updated version of Simba ODBC and JDBC driver. The Simba JDBC driver now includes support for the BigQuery Storage API.

March 11, 2019

BigQuery is now available in the Zürich (europe-west6) region. For upcoming regional launches, see the Google Cloud Platform blog.

March 01, 2019

The BigQuery web UI in the GCP Console now supports assigning access controls to BigQuery datasets.

February 25, 2019

Support for DML MERGE statements is now generally available (GA).

You can now save query results to Google Drive by using the BigQuery web UI in the GCP Console. Saving query results to Google Drive is currently in beta.

February 22, 2019

The BigQuery Storage API is now in beta.

February 15, 2019

BigQuery is now available in the São Paulo (southamerica-east1) region. For upcoming regional launches, see the Google Cloud Platform blog.

February 13, 2019

The following load job quota has changed:

  • The 16 MB maximum block size limit for Avro files has been removed.

February 11, 2019

The following DML quotas have changed:

  • The maximum number of combined UPDATE, DELETE, and MERGE statements per day per table has increased from 200 to 1,000.
  • The limit of 10,000 combined UPDATE, DELETE, and MERGE statements per day per project has been removed. There is no longer a project-level limit on DML statements.

February 04, 2019

You can now use INFORMATION_SCHEMA views to retrieve resource metadata in BigQuery. INFORMATION_SCHEMA is currently in beta.

The Data Definition Language now supports ALTER TABLE statements to set options on tables and views. ALTER TABLE statements are currently in beta.

January 31, 2019

You can now try BigQuery without providing a credit card or attaching a billing account to your project by using the BigQuery sandbox. The sandbox is now in beta and is available to all GCP customers.

The daily export quota has been changed to measure the actual export file size in bytes, when counted toward your daily export quota. Usage is no longer measured by rounding up to the nearest GB.

January 28, 2019

BigQuery is now available in the Los Angeles (us-west2) and Hong Kong (asia-east2) regions. For upcoming regional launches, see the Google Cloud Platform blog.

January 22, 2019

BigQuery audit logging now writes enhanced messages using the BigQueryAuditMetadata format.

December 18, 2018

BigQuery is now available in the Mumbai (asia-south1), and Montréal (northamerica-northeast1) regions. For upcoming regional launches, see the Google Cloud Platform blog.

December 12, 2018

December 07, 2018

BigQuery is now available in the Taiwan (asia-east1), Finland (europe-north1), and Northern Virginia (us-east4) regions. For upcoming regional launches, see the Google Cloud Platform blog.

December 05, 2018

BigQuery now supports converting Avro logical types when you load Avro data.

You can now set or update a default partition expiration time on a dataset.

December 04, 2018

Updated versions of the Simba ODBC and JDBC drivers.

November 26, 2018

Updated version of the Simba ODBC drivers.

November 13, 2018

BigQuery is now available in the Singapore (asia-southeast1) region. For upcoming regional launches, see the Google Cloud Platform blog.

October 29, 2018

BigQuery now supports the bigquery.metadataViewer predefined IAM role.

October 22, 2018

BigQuery is now available in the Sydney (australia-southeast1) region. For upcoming regional launches, see the Google Cloud Platform blog.

October 17, 2018

You can now try BigQuery without providing a credit card or attaching a billing account to your project by using the BigQuery sandbox for Firebase. The sandbox is now in beta.

October 10, 2018

BigQuery is now available in the London (europe-west2) region. For upcoming regional launches, see the Google Cloud Platform blog.

October 09, 2018

The location is no longer required when executing jobs that reference resources outside the US and EU regions.

September 25, 2018

BigQuery standard SQL now supports the GENERATE_UUID function.

September 20, 2018

Updated versions of Simba ODBC and JDBC drivers to support regional resources.

September 11, 2018

September 10, 2018

August 27, 2018

Updated version of the Simba JDBC driver.

August 20, 2018

BigQuery now supports loading Cloud Firestore export data and querying Cloud Firestore data stored in Cloud Storage.

July 30, 2018

BigQuery support for clustered tables is now Beta.

July 25, 2018

BigQuery has achieved approximately 5x improvement in TPC-DS 10T power run geomean over the past 18 months.

July 19, 2018

The BigQuery web UI in the GCP Console is now Beta.

BigQuery support for clustered tables is now Alpha.

June 28, 2018

BigQuery standard SQL now supports the functions FROM_BASE32 and TO_BASE32.

BigQuery standard SQL now supports templated SQL UDFs.

BigQuery standard SQL now supports the function GENERATE_TIMESTAMP_ARRAY.

June 18, 2018

Loading data in Parquet format is now GA (generally available).

June 06, 2018

Updated version of the Simba ODBC drivers.

May 25, 2018

The following quotas have changed for load jobs:

May 15, 2018

Support for the NUMERIC data type is now in Beta.

May 04, 2018

The following quotas have changed:

May 01, 2018

Updated version of the Simba JDBC driver.

April 17, 2018

BigQuery is now available in the Tokyo (asia-northeast1) region.

BigQuery now includes bigquery.jobs.listAllpermissions. bigquery.jobs.list still allows users to list all jobs in a project, but details and metadata are redacted for jobs submitted by other users.

April 13, 2018

By default, the number of export jobs per project per day has increased from 1,000 to 50,000.

April 10, 2018

Support for DML MERGE statements is now in Beta.

March 30, 2018

BigQuery standard SQL now supports: the ERROR function to return errors and the SAFE prefix to suppress errors.

BigQuery now allows you to monitor slot usage by job type in Stackdriver.

March 05, 2018

BigQuery data definition language now supports creating a table from the result of a query.

BigQuery support for using DML statements with partitioned tables is now in Beta.

March 01, 2018

Support for loading Parquet files is now in Beta.

Updated version of Simba JDBC driver.

February 08, 2018

Several API methods will begin including a location field in their responses: jobs.insert, jobs.get, jobs.cancel, jobs.query, jobs.getQueryResults, datasets.list, and jobs.list. The location field is currently not required in requests.

For example, the id property will include a location in the following format: "id": "[PROJECT_ID]:[LOCATION].[JOB_ID]". The "id" property should continue to be treated as opaque. Also, the "jobReference" property will begin including a location field in the following format:

"jobReference": {
  "jobId": "[JOB_ID]",
  "location": "[LOCATION]",
  ...

Added documentation on manually changing table schemas.

January 17, 2018

BigQuery data definition language support is now in Beta.

January 05, 2018

ISOWEEK and ISOYEAR are now supported by the following functions: DATE_TRUNC, DATETIME_TRUNC, TIMESTAMP_TRUNC, DATE_DIFF, and DATETIME_DIFF.

December 08, 2017

Updated versions of Simba ODBC and JDBC releases.

December 04, 2017

The BigQuery web UI supports additional dimensions for filtering your query history.

November 30, 2017

INTERSECT DISTINCT and EXCEPT DISTINCT are now supported set operators in standard SQL.

FARM_FINGERPRINT is now a supported function in legacy SQL.

WEEK(<WEEKDAY>) is now a valid date part for the EXTRACT function.

There is now a quota of 300 concurrent API requests per user.

IAM custom roles are now available for BigQuery.

The _PARTITIONTIME pseudo column is now supported when querying data using a wildcard table over partitioned tables.

November 14, 2017

High compute queries are being phased out so you are no longer required to opt-in. All high compute queries under tier 100 are billed as tier 1. Running a high compute query does not require setting a threshold flag.

All queries above tier 100 will fail with a RESOURCES_EXCEEDED_PER_BYTE error unless the query is running in a reserved instance. Existing users who have run a high compute query higher than tier 100 in the last 30 days retain the ability to run queries at the higher billing tier until October 31, 2018.

November 02, 2017

The IGNORE NULLS and RESPECT NULLS modifiers are now supported in the FIRST_VALUE, LAST_VALUE, and NTH_VALUE analytic functions.

The FOR SYSTEM_TIME AS OF expression is now available in the FROM clause.

October 16, 2017

Updated versions of Simba ODBC and JDBC releases.

September 29, 2017

GROUP BY ROLLUP is now supported in standard SQL.

September 28, 2017

The following string functions are now supported in standard SQL:

Added support for creating ingestion-time partitioned tables by specifying partitioning options when you write query results or load data into a new destination table.

September 01, 2017

The following analytic functions are now supported in standard SQL:

August 18, 2017

Added best practices documentation for BigQuery:

August 17, 2017

Updated versions of Simba ODBC and JDBC releases.

August 11, 2017

The following standard SQL statistical aggregate functions now support the OVER clause — COVAR_SAMP, COVAR_POP, and CORR:

August 02, 2017

Announced support for WEEK, ISOWEEK, and ISOYEAR with the standard SQL EXTRACT function:

July 27, 2017

Announced availability of snappy codec for Avro compressed data blocks:

July 26, 2017

Announced availability of standard SQL TO_HEX and FROM_HEX functions:

July 13, 2017

Announced support for using DML statements to modify tables with required fields:

  • DML
  • You can also update a table with required fields by using a SELECT statement and writing to a destination table.

June 19, 2017

June 07, 2017

Announced availability of new public dataset:

June 06, 2017

Announced availability of new public dataset:

June 05, 2017

The following standard SQL statistical aggregate functions now support the OVER clause — STDDEV, STDDEV_POP, STDDEV_SAMP, VAR_POP, VAR_SAMP, VARIANCE:

June 02, 2017

Standard SQL now supports the TO_JSON_STRING function.

May 23, 2017

Added troubleshooting documentation for the BigQuery Data Transfer Service:

May 22, 2017

Announced availability of new public dataset:

May 17, 2017

May 12, 2017

Announced availability of new public datasets:

May 10, 2017

Announced availability of new IAM role:

May 05, 2017

Announced availability of new public dataset:

May 02, 2017

Announced availability of new public datasets:

April 20, 2017

Announced availability of new public dataset:

April 05, 2017

Updated versions of JDBC releases.

March 31, 2017

Announced availability of new public dataset:

March 16, 2017

Updated limits for maximum rows per second when streaming data into BigQuery.

Standard SQL now supports REVERSE and ARRAY_REVERSE functions for reversing STRING, BYTES, and ARRAY types.

March 15, 2017

Standard SQL now supports several functions that use HyperLogLog++ to estimate the number of unique values in a large dataset.

Standard SQL now supports window, or analytic, mode for ARRAY_AGG and STRING_AGG using the OVER clause.

Updated limits for Avro headers and maximum file size for load jobs.

March 14, 2017

Updated versions of JDBC and ODBC releases.

March 08, 2017

Announced general availability of Data Manipulation Language for nonpartitioned tables in standard SQL.

Announced general availability of IGNORE NULLS and RESPECT NULLS modifiers for ARRAY_AGG and APPROX_QUANTILES in standard SQL.

Announced daily limits for copying a table in BigQuery.

February 01, 2017

Standard SQL now supports:

Announced support for copying non-partitioned tables to partitioned tables. For more information, see Copying to partitioned tables.

Announced availability of a new public dataset: New York City Tree Census data.

Announced availability of a new public dataset: US Bureau of Labor Statistics data.

December 22, 2016

Announced availability of custom null markers for CSV load jobs.

Standard SQL now supports the following code point functions: TO_CODE_POINTS, CODE_POINTS_TO_BYTES, CODE_POINTS_TO_STRING.

Standard SQL now supports ORDER BY and LIMIT clauses within ARRAY_AGG, ARRAY_CONCAT_AGG, and STRING_AGG.

Hash functions in standard SQL.

URL functions in standard SQL.

Standard SQL now supports parameterized queries, which help prevent SQL injection when queries are constructed using user input.

Announced availability of a new public dataset: New York City Citi Bike Trips data.

Announced availability of a new public dataset: NYC 311 Service Requests data.

Announced availability of a new public dataset: NYPD Motor Vehicle Collisions data.

If you use OMIT...IF to exclude a portion of a record in a repeating field, and the query also selects other independently repeating fields, BigQuery now returns an error. To avoid this error, migrate your query to standard SQL.

November 30, 2016

Announced availability of JSON functions in standard SQL.

Announced availability of a new public dataset: IRS Form 990 Data.

Announced publication of a new tutorial:

November 07, 2016

Announced availability of a new public dataset: Medicare Data.

November 02, 2016

Announced availability of a new public dataset: Open Images Data.

November 01, 2016

Announced JDBC drivers.

You can set a SQL variant, either legacy SQL or standard SQL, by setting a query prefix.

Announced availability of a new public dataset: Major League Baseball Data.

Announced availability of standard SQL safe casting functions, which return a null value in cases where normal casting generates an error.

The BigQuery Client Libraries page now links exclusively to one set of libraries: the Google Cloud Client Libraries. Work is underway to update all sample code throughout the BigQuery documentation to use the Google Cloud Client Libraries.

Bracket notation is now supported for the legacy SQL JSON functions.

Additional documentation is available for access control using Identity and Access Management (IAM):

  • A table that describes the permissions available in BigQuery.
  • A table that describes the permissions required for methods.
  • A table that lists all BigQuery IAM roles and the permissions included in each role.
  • A section that describes the methods to use for programmatic manipulation of roles.

September 29, 2016

Announced general availability of standard SQL, which is based on the SQL 2011 standard, with extensions for querying nested and repeated data.

New documentation includes:

Announced Data Manipulation Language (DML) functionality for standard SQL, which enables you to insert, update, and delete rows and columns in your BigQuery datasets.

For syntax details and examples, see DML Syntax.

Announced BigQuery Stackdriver Monitoring functionality, which enables you to monitor your BigQuery resources, including creating charts and alerts based on BigQuery metrics.

Announced ODBC drivers.

Flat-Rate Pricing gives you unlimited querying for a fixed monthly price.

You can now save and share queries.

August 26, 2016

The updated beta IAM predefined roles for BigQuery are now accessible to all.

August 01, 2016

When loading Avro files, BigQuery no longer creates an unnamed root-level column named "root." To ease the transition to this new schema, projects with the following contents are exempted from this change through a project override:

  • Projects that contained federated tables based on Avro files as of August 1, 2016
  • Projects that have queried or loaded Avro files in the 30 days prior to August 1, 2016

All affected project owners will be notified and provided the option to remove the project override.

July 27, 2016

Announced support for labeling datasets using the BigQuery API or command-line tool.

June 02, 2016

Announced support for time-partitioned tables.

Announced Beta availability of standard SQL, which is based on the SQL 2011 standard, with extensions for querying nested and repeated data.

Announced a Beta preview of Product Identity and Access Management (IAM) Roles.

May 06, 2016

Query reference updates:

March 23, 2016

Announced availability of BigQuery Public Datasets.

Added support for Avro source format for load operations and as an federated data source in the BigQuery API or command-line tool.

Added support for CSV and JSON schema auto-detection for federated data sources in the BigQuery API or command-line tool.

Added support for the BYTES data type.

Updated the interface for creating tables in the BigQuery web UI.

December 15, 2015

Added Custom quotas, which allow you to set project-wide billing alerts that help prevent runaway costs.

Added Query explain, which shows, stage by stage, how BigQuery executes your queries.

Added improvements to the Streaming API:

  • You can use the Streaming API in EU locations.
  • You can use Template tables to manage related tables used for streaming.
  • No more warm-up delay. You can now query your data within a few seconds of the first streaming insertion.

Added support for analyzing Google Cloud Audit logs that enable you to track usage and access.

August 25, 2015

Added support for user-defined functions (UDFs).

Added the ability to query CSV and JSON data directly from Google Cloud Storage. For more information, see federated data sources.

Added ability to cancel a job

Pricing updates:

  • Introduced BigQuery Slots, which lets you guarantee resources, regardless of demand on the overall multi-tenant pool.
  • Introduced query pricing tiers.

Quota updates: * Increased the concurrent rate limit to 50 queries. * Increased the daily query limit to 100,000 queries.

BigQuery web UI improvements:

Added streaming buffer statistics for tables that are being actively modified by streaming inserts. These statistics offer better information about table size and availability for copy/extract, and they can also help with debugging.

April 16, 2015

Increased the API requests per second, per user quota from 10 to 100 requests per second.

Query reference updates:

Added ability to undelete a table using table decorators.

March 25, 2014

Added views. A view is a virtual table defined by a SQL query.

Added table wildcard functions, for a cost-effective way to query data from a specific set of tables.

Improved JOIN support by adding CROSS JOIN and multiple JOIN functionality. For more information, see joins.

Increased quota limits for streaming data into BigQuery.

Added a description property for datasets, tables and fields.

Updated the concurrent rate limit and export request quota in the quota policy.

Added parallel export capability.

Changed the pricing model, lowering rates and adding reserved capacity pricing.

Updated batch query functionality. Batch queries no longer have a 30 minute execution delay. Instead, batch jobs now begin as soon as idle resources are available. If the job hasn't started within 3 hours, the job priority changes to interactive.

September 18, 2013

Added support for streaming inserts.

Added table decorators for time-based table snapshots.

Added new aggregate functions.

June 11, 2013

Added support for package-based pricing.

Added query caching.

Updated quota policies.

Added support for returning large query results.

March 14, 2013

Added TIMESTAMP data type functionality.

Added GROUP EACH BY functionality for aggregations of more distributed values.

Added JOIN EACH functionality for joining tables of any size.

Added ability to add columns to table schema after the data is already loaded into BigQuery. To add columns to a table schema, pass the new schema to the schema parameter of Tables.update().

October 01, 2012

API: New support for importing JSON files and nested and repeated fields

API: New quota limits

August 28, 2012

API: New batch queries feature allows for queries to be executed between 30 minutes to 3 hours later, but are charged at a lower cost than interactive queries which execute immediately

New tool release: BigQuery Connector for Excel now allows you to make queries to Google BigQuery from Microsft Excel!

August 17, 2012

API: CSV chunking - when importing, you can now specify the allowQuotedNewlines flag to be FALSE in the bq command-line tool, or in the API, and the import will be chunked, or processed in parallel. If you have quoted new lines in your data, be sure to set this flag to be TRUE.

API: Deletion of large tables has been fixed

API: Improved validation of project IDs

July 16, 2012

BigQuery Browser Tool: Added syntax highlighting

BigQuery Browser Tool: Add a drag handle to the separator below the query editor

BigQuery Browser Tool: New favicon

BigQuery Browser Tool: Fixed some keyboard shortcuts in the query editor

API: Project owners can now call bigquery.jobs.get for jobs created by other users

May 30, 2012

API: Introduced new quota enforcement schema. User can now have one (1) large query and one (1) pipeline job (import, export, streaming, etc.) concurrently.

API: Added support for leading underscores in field name

API: Improved error handling

API: Added token based pagination support for list operations. Clients can use token based pagination to page through a single table snapshot. Subsequent changes to the table are not reflected when paging through list results.

API: Improved list operations for tables and datasets

May 11, 2012

API: Table expiration time is now exposed

API: Better errors for field/rows being too long

API: Custom delimiters added

April 20, 2012

BigQuery Browser Tool: New welcome page and more helpful messages when there aren't any queries or jobs to display

API

  • Clearer quota error messages
  • Copy jobs are now atomic

April 13, 2012

BigQuery Browser Tool

  • Add retries for connection errors
  • Table view: Field names in schemas are left-aligned
  • Table view: Removed "Actions" menu (use the nav panel instead)
  • Table view: Added "Query Table" menu, which prefills query builder with a table query
  • Query builder has simpler prefill logic and a better help bubble

API: Remove bigquery.jobs.delete

April 05, 2012

BigQuery Browser Tool

  • 10MB limit to uploads done through the BigQuery browser tool
  • Bug fix for zero row results

API

  • API Discovery docs have typed configuration
  • Bug fix limiting the amount of data returned in a TableData.List
  • Atomic updates of tables for all import jobs
  • Join behavior tightened (no outer join, only left most table can be large)

March 09, 2012

BigQuery Browser Tool

  • Pressing the tab key in the query box now adds a tab instead of tabbing to the next UI element
  • Pressing the enter key in the query box now preserves leading whitespace to the next lines
  • Pressing the enter key no longer starts a query but will always add a newline to the box, and CTRL+ENTER runs the query
  • Navigation panel no longer forgets scroll location and submenu state when jobs are running
  • BigQuery browser tool now has a new URL at http://bigquery.cloud.google.com; it is still possible to access the browser tool using the old URL

March 01, 2012

BigQuery Browser Tool

  • Faster BigQuery browser tool load time
  • "Max invalid records" support in the BigQuery browser tool

API

  • Enable support for VARIANCE, STDDEV, and AVG aggregate functions
  • Add job.kind to Jobs:list API response
  • Better error messages for quota-related errors

February 01, 2012

BigQuery Browser Tool

  • Show previous query results option in the query panel
  • The current project view is now highlighted
  • "Refresh" option in the projects drop-down menu

API

  • Support for loading gzip compressed files from Google Cloud Storage
  • Error handling
    • Faster detection of unreadable files
    • Improvements in handline multibyte UTF-8 characters

November 01, 2011

BigQuery Browser Tool

The Web UI now allows JSON schema format to be used in the "Create Table" dialog.

This is useful in situations where developers may want to use the same schema format for API calls, the bq command-line client, and the Web UI.

API

  • The default configuration.load.createDisposition for load jobs is now CREATE_IF_NEEDED
  • "load" jobs now allow users to specify a number of rows to skip
    • This is useful for skipping header rows or other metadata that is not parseable when you import CSV files.
    • This value applies to all files in the source list.
  • BigQuery can now load data from files that contain delimiters other than the comma
    • Delimiters may be any character expressable as a single Latin-1 byte other than ", , and .
    • Example (using the bq command line tool): bq load --field_delimiter=: newtable gs://bucket/input.txt

October 01, 2011

BigQuery V2 Release

  • A new REST API
  • A new web user interface
  • Support for JOIN statements
  • Ability to export a table or query result to a CSV file in Google Cloud Storage
  • Support for ACLs on groups of tables (datasets)
  • A new object architecture describing tables, groups of tables, and queries