Datastream release notes

Datastream release notes

This page documents production updates to Datastream. Check this page for announcements about new or updated features, bug fixes, known issues, and deprecated functionality.

You can see the latest product updates for all of Google Cloud on the Google Cloud page, browse and filter all release notes in the Google Cloud console, or programmatically access release notes in BigQuery.

To get the latest product updates delivered to you, add the URL of this page to your feed reader, or add the feed URL directly: https://cloud.google.com/feeds/datastream-release-notes.xml

January 26, 2024

You can now recover a stream for an Oracle source. You can also start your Oracle source stream from a specific position. For more information, see Recover a stream and Manage streams.

January 23, 2024

Datastream now supports the JA16SJIS character encoding for Oracle sources.

January 16, 2024

The maximum event size that Datastream supports when streaming data to BigQuery is now increased to 20 MB.

November 22, 2023

Datastream now supports SSL/TLS encryption for connections to PostgreSQL sources that don't require client certificates.

November 10, 2023

You can now stream the following large object data types for Oracle sources:

  • BLOB
  • CLOB
  • NCLOB

November 09, 2023

You can now recover a permanently failed stream. For more information, see Recover a stream.

You can now start a stream from a specific binary log position for MySQL sources using the Datastream API. For more information, see Start a stream from a specific binary log position.

October 20, 2023

Support for the PostgreSQL ARRAY data type is now added in Datastream.

September 10, 2023

Datastream now supports SSL/TLS encryption for Oracle sources. For more information, see the Datastream API reference documentation.

September 07, 2023

The Datastream BigQuery Migration Toolkit is now launched. The toolkit is an open-source software that simplifies migrating from the Dataflow Datastream to BigQuery template to the Datastream native BigQuery replication solution. For more information, see the toolkit documentation.

June 30, 2023

Support for ENUM and CITEXT data types is now added for PostgreSQL sources.

June 07, 2023

The maximum event size that Datastream supports is now increased. The new limit is 10 MB when streaming data to BigQuery and 30 MB when streaming to Cloud Storage.

May 11, 2023

Datastream now supports backfill for PostgreSQL tables of any size. For more information, click here.

April 17, 2023

Datastream now supports Oracle multi-tenant (CDB/PDB) architecture. For information about configuring pluggable databases for use with Datastream, click here.

April 04, 2023

Datastream support for BigQuery as destination is now generally available (GA). For more information, click here.

Datastream support for PostgreSQL as source is now generally available (GA). For more information, click here.

February 27, 2023

Terraform now supports Datastream private connectivity, connection profile, and stream resources. For more information, see Getting started with Terraform and Datastream.

February 21, 2023

You can now set the number of maximum concurrent backfill tasks for a stream using the Datastream API. To learn more, see Manage streams.

January 11, 2023

The validate_only and force parameters were added to the projects.locations.connectionProfiles resource in the Datastream API. To learn more, see the Datastream API reference documentation.

September 15, 2022

Datastream now supports BigQuery as a destination. Click here to access the documentation.

Datastream now supports PostgreSQL as source. Click here to access the documentation.

We have introduced changes to Datastream pricing. For more information, see the Pricing page.

August 30, 2022

You can now set the number of maximum concurrent CDC tasks for a stream using the Datastream API. To learn more, see Manage streams.

June 16, 2022

Datastream now supports the use of tags on its resources, which include private connectivity configurations, connection profiles, and streams. Tags are key-value pairs that you can apply to your Datastream resources for fine-grained access control. To learn more, see Access control with tags. To use tags, see Manage tags.

June 10, 2022

Datastream is now HIPAA compliant.

May 13, 2022

Datastream now supports backfilling Oracle database tables that have more than 100 million rows. Click here to access the documentation.

December 14, 2021

Datastream now supports customer-managed encryption keys (CMEK). Click here to access the documentation.

November 30, 2021

Datastream is now generally available (GA) in all Google Cloud regions.

As part of this launch, the Stream details page now has an OBJECTS tab with detailed information about each object included in the stream (backfill status, object-level errors, and so on).

Click here to access the documentation.

May 26, 2021

Datastream is a serverless and easy-to-use change data capture (CDC) and replication service. It allows you to synchronize data across heterogeneous databases and applications reliably, and with minimal latency and downtime.

Datastream supports streaming from Oracle and MySQL databases into Cloud Storage. The service offers streamlined integration with Dataflow templates to power up-to-date materialized views in BigQuery for analytics, replicate your databases into Cloud SQL or Spanner for database synchronization, or leverage the event stream directly from Cloud Storage to realize event-driven architectures.

Benefits of Datastream include:

  • Being serverless so there are no resources to provision or manage, and the service scales up and down automatically, as needed, with minimal downtime.
  • Easy-to-use setup and monitoring experiences that achieve super-fast time-to-value.
  • Integration across the best of Google Cloud data services' portfolio for data integration across Datastream, Dataflow, Data Fusion, Pub/Sub, BigQuery, and more.
  • Synchronizing and unifying data streams across heterogeneous databases and applications.
  • Security, with private connectivity options and the security you expect from Google Cloud.
  • Being accurate and reliable, with transparent status reporting and robust processing flexibility in the face of data and schema changes.
  • Supporting multiple use cases, including analytics, database replication, and synchronization for migrations and hybrid-cloud configurations, and for building event-driven architectures.

Documentation for Datastream includes a quickstart, conceptual content, how to use this service through the user interface, REST API calls, and gcloud, an API tutorial, and reference, support, and resource-related information. Click here to access the documentation.