This page documents production updates to Dataplex. Check this page for announcements about new or updated features, bug fixes, known issues, and deprecated functionality.
You can see the latest product updates for all of Google Cloud on the Google Cloud page, browse and filter all release notes in the Google Cloud console, or programmatically access release notes in BigQuery.
To get the latest product updates delivered to you, add the URL of this page to your
feed
reader, or add the feed URL directly: https://cloud.google.com/feeds/dataplex-release-notes.xml
March 27, 2024
Data insights in Dataplex is available in Preview. Data insights offers an automated and intuitive way to explore and understand your data. It uses Gemini large language models to generate queries based on the metadata of a table, and lets you uncover patterns, assess data quality, and perform statistical analysis.
March 25, 2024
Automated cataloging of Vertex AI feature store is available in Preview. With this integration, you can discover Vertex AI feature groups and features across projects and regions using the Console or Dataplex API. Dataplex fully automates the process of ingesting and indexing metadata, while performing source IAM permission checks, providing a governed single-pane-of-glass experience for data and AI artifacts across Cloud services.
December 17, 2023
Automated cataloging of Spanner is generally available (GA) in Dataplex. With this integration, can discover Spanner instances, databases, and tables across projects and regions using the Console or the Dataplex API. The metadata ingestion and indexing operations are fully automated, with IAM permissions set at the data source, providing a critical foundation for data management and governance.
December 01, 2023
Automated cataloging of Vertex AI models and datasets is generally available (GA) in Dataplex. With this integration, you are able to discover Vertex AI models and datasets across projects and regions using the Dataplex Console and API. Dataplex fully automates the process of ingesting and indexing metadata, while performing source IAM permission checks, providing a governed single-pane-of-glass experience for data and AI artifacts across Cloud services.
October 06, 2023
Automated cataloging of Bigtable is generally available (GA) in Dataplex. With this integration, you can discover Bigtable tables and instances across projects and regions using the Console or theDataplex API. The metadata ingestion and indexing operations are fully automated, with IAM permissions set at the data source, providing a critical foundation for data management and governance.
October 03, 2023
Dataplex BigLake integration is generally available (GA). Dataplex BigLake integration lets you upgrade a Cloud Storage bucket to managed, creating BigLake tables and Object tables instead of external tables. This allows the application of column-level, row-level, and table-level policies, enabling fine-grained security and dynamic data masking.
September 29, 2023
Dataplex is available in the following regions:
- Delhi (
asia-south2) - Melbourne (
australia-southeast2) - Toronto (
northamerica-northeast2)
August 21, 2023
Dataplex automatic data quality and data profiling are generally available.
- Data profiling
- Jump start your data analytics with statistical insights, such as average values, unique values, data bounds, and top-N.
- Understand drifts and build anomaly models with the generated metadata.
- Publish data quality and data profiling information in the BigQuery console. Learn more.
- Profile data in BigQuery tables, views, BigLake, and external tables.
- Ease deployment through a managed, serverless, and zero-copy execution.
- Take advantage of advanced features like filtering, sampling, and saving results to a central BigQuery table.
- Automatic data quality
- Deliver trusted data by building an end-to-end data quality monitoring pipeline.
- View rule recommendations, enhance with business rules, monitor on a routine or in a pipeline, generate reports, get alerted on failures, and troubleshoot the issues.
- View quality information in the BigQuery UI for every table user to see. Learn more.
- Improve data quality in BigQuery tables, views, BigLake, and external tables.
- Ease deployment through managed, serverless, and zero-copy execution.
- Take advantage of advanced features like filtering, sampling, and saving results to a central BigQuery table.
August 14, 2023
Data lineage at entry level for spark jobs executed in Dataproc is GA.
August 01, 2023
Dataplex is available in the following regions:
- Los Angeles (
us-west2) - Salt Lake City (
us-west3) - Las Vegas (
us-west4) - Columbus (
us-east5) - Santiago (
southamerica-west1) - Finland (
europe-north1) - Warsaw (
europe-central2) - Madrid (
europe-southwest1) - Milan (
europe-west8) - Paris (
europe-west9) - Jakarta (
asia-southeast2)
July 24, 2023
Configuring the retention period for data lineage metadata is available in Preview. You can extend the retention period for lineage metadata from the default 30 days to a custom duration.
May 18, 2023
- Dataplex auto data quality (AutoDQ) and data profiling can be used on any BigQuery tables, including tables that aren't part of a Dataplex lake. You don't need to create a Dataplex lake to run Dataplex AutoDQ and data profiling.
- Dataplex AutoDQ and data profiling support BigQuery views, BigLake tables, and BigQuery external tables.
- Dataplex AutoDQ and data profiling support sampling your data to reduce time and cost.
April 21, 2023
Export of data lineage metadata is available in Preview. You can export all lineage metadata in a project and location in bulk.
March 13, 2023
Dataplex data lineage is generally available (GA). Data lineage lets you track how data moves through your systems: where it comes from, where it is passed to, and what transformations are applied to it.
January 30, 2023
Dataplex business glossary is now available in Preview. Dataplex business glossary lets you manage business related terminologies and definitions across the organization, and use them for describing and discovering data entries.
Dataplex Attribute Store is now available in Preview. Dataplex Attribute Store lets you associate attributes (with behavior specifications, such as resource access and column access) with tables and columns.
December 22, 2022
Dataplex data lineage is now available in Preview. Data lineage lets you track how data moves through your systems: where it comes from, where it is passed to, and what transformations are applied to it.
December 16, 2022
Dataplex BigLake integration is now available in Preview. Dataplex BigLake integration allows upgrading a Cloud Storage bucket to managed, creating BigLake tables instead of external tables. This allows the manual application of column-level, row-level, and table-level policies.
December 12, 2022
Dataplex auto data quality (AutoDQ) is now available in Preview. Dataplex auto data quality helps data users build trust in their data with a turnkey and automated product that encapsulates the entire process of data quality.
Dataplex data profiling is now available in Preview. Dataplex data profiling helps data users build deeper understanding about their data by identifying common data characteristics. Dataplex utilizes this information to recommend the data quality rules as well.
December 01, 2022
Dataplex Source and Sink plugins are generally available (GA) in Cloud Data Fusion for ingesting and processing data.
October 20, 2022
Data exploration workbench (Explore) is generally available (GA). Explore provides a fully-managed, serverless data exploration experience powered by fully-governed collaboration, one-click scheduling, and interactive querying using Spark SQL scripts and Jupyter notebooks.
July 20, 2022
Dataplex is now unified with Data Catalog to provide a complete data management and governance experience with built-in data intelligence and automation capabilities. See Dataplex product overview.
May 23, 2022
The Dataplex Source and Sink plugins are available in Public Preview for ingesting and processing data in Cloud Data Fusion versions 6.6.0 and later.
May 02, 2022
Added support for scheduling Google-provided and custom Dataflow templates from the Dataplex page in the Cloud Console. Monitor these templates from the Dataplex page in the Cloud Console.
April 15, 2022
Dataplex Data Quality tasks support running data quality validations on BigQuery tables that may not be part of a Dataplex lake, and on GCS data that's available as a BigQuery external table.
March 25, 2022
February 17, 2022
Dataplex Explore is available in Preview. Explore provides a fully-managed, serverless data exploration experience that enables you to query your data using Apache SparkSQL queries and Jupyter notebooks.