Dataform release notes

This page documents production updates to Dataform. Check this page for announcements about new or updated features, bug fixes, known issues, and deprecated functionality.

You can see the latest product updates for all of Google Cloud on the Google Cloud page, browse and filter all release notes in the Google Cloud console, or programmatically access release notes in BigQuery.

To get the latest product updates delivered to you, add the URL of this page to your feed reader, or add the feed URL directly: https://cloud.google.com/feeds/dataform-release-notes.xml

April 02, 2024

You can now use Customer-Managed Encryption Keys (CMEK) to protect repositories in Dataform. CMEK in Dataform is available in preview. For more information, see Use customer-managed encryption keys.

March 13, 2024

The 3.0.0-beta.0 version of the open-source Dataform framework is available. This update introduces significant changes, including, but not limited to, the following:

  • Deprecation of dataform.json in favor of workflow_settings.yaml
  • Stateless package installation by @dataform/cli
  • Warehouse-agnostic compilation output

You don't need to take immediate action to update your Dataform code.

For more information, see the 3.0.0-beta.0 release on GitHub.

March 12, 2024

Granting repository access to all authenticated users is available. For more information, see Grant public access to a repository.

March 11, 2024

Dataform is available in the africa-south1 region. For more information, see Locations.

February 26, 2024

Dataform is available in the us-south1 region. For more information, see Locations.

February 21, 2024

Support for VPC Service Controls is generally available (GA).

February 15, 2024

Dataform is available in the following regions:

  • asia-east2
  • asia-northeast3
  • asia-southeast2
  • europe-southwest1
  • europe-west12
  • me-central1
  • me-central2
  • northamerica-northeast
  • us-east4
  • us-east5
  • us-west2
  • us-west4

For more information, see Locations.

January 29, 2024

January 17, 2024

The Releases & Scheduling tab replaces the Release configurations and Workflow configurations tabs in repositories. Configuration and monitoring of release configurations and workflow configurations is available in the Releases & Scheduling tab.

For more information, see Create a release configuration, and Schedule executions with workflow configurations.

December 11, 2023

December 07, 2023

The 2.8.0 version of the open-source Dataform framework is available. This update introduces performance improvements. For more information, see the 2.8.0: Packaging and compilation performance improvements release on GitHub.

December 05, 2023

December 04, 2023

November 16, 2023

Dataform is compliant with VPAT.

For more information, see Dataform compliance.

October 27, 2023

Batch workspace deletion is available.

October 26, 2023

File search is available in workspaces.

October 23, 2023

The 2.7.0 version of the open-source Dataform framework is available. This update introduces explicitly listed column names in incremental insert statements. For more information, see the 2.7.0: Updates for Dataform GCP incremental SQL release on GitHub.

October 20, 2023

October 17, 2023

ssh_authentication_config and service_account fields are available in the google_dataform_repository Dataform Terraform resource.

October 13, 2023

Formatting of Dataform core and JavaScript code is available.

September 07, 2023

Service account delegation for workflow execution is now available. You can configure service account delegation for whole repositories, or for individual workflow configurations.

September 06, 2023

August 31, 2023

Legacy Dataform will be deprecated on February 26, 2024, after which you will not be able to access legacy projects.

We recommend migration to Dataform in Google Cloud Platform.

To learn more about Dataform in Google Cloud Platform, see Overview of Dataform.

To discover how to migrate legacy projects, see Migrate from legacy Dataform.

August 22, 2023

Dataform is compliant with SOC 2 and SOC 3.

For more information, see Dataform compliance.

August 18, 2023

SSH authentication for GitHub, GitLab, and Bitbucket Cloud repositories is supported.

August 09, 2023

Dataform is compliant with HIPAA, ISO/IEC 27001, and SOC 1.

For more information, see Dataform compliance.

August 02, 2023

August 01, 2023

The following Dataform API methods for version control are available:

  • CommitRepositoryChanges()
  • ReadRepositoryFile()
  • QueryRepositoryDirectoryContents()
  • FetchRepositoryHistory()

The following fields of Repository are available:

  • labels
  • set_authenticated_user_admin

July 27, 2023

June 19, 2023

Filtering the compiled graph in a workspace is available.

May 04, 2023

Dataform is generally available (GA).

Dataform release configurations are available. Release configurations let you configure execution environments, for example, staging and production.

Dataform workflow configurations are available. Workflow configurations let you execute SQL workflows on a schedule.

April 18, 2023

Cloud Logging is available for Dataform in Preview.

March 24, 2023

Workspace compilation overrides are available in Preview.

March 21, 2023

Dataform in Preview is available in the following regions:

  • australia-southeast1
  • southamerica-east1

March 10, 2023

Dataform in Preview is available in the following regions:

  • asia-south1
  • europe-west6

March 09, 2023

Query preview in a workspace is available in Preview.

March 06, 2023

Dataform in Preview is available in the following regions:

  • asia-northeast1
  • europe-west2
  • europe-west3
  • us-east1

February 14, 2023

Dataform in Preview is available in the following regions:

  • asia-southeast1
  • europe-west1
  • us-west1

January 27, 2023

Dataform in Preview is available in the asia-east1 region.

January 25, 2023

Workspace navigation tabs are available in Preview.

January 09, 2023

November 25, 2022

Definition of BigQuery variables in the pre_operations block of type: incremental queries that contain a uniqueKey is available in Preview.

October 21, 2022

August 25, 2022

Dataform is available in Preview.