Datastream

Serverless and easy-to-use change data capture and replication service.

Try Datastream free View documentation Go to console

Access to streaming, low-latency data from MySQL, PostgreSQL, AlloyDB, and Oracle databases
Near real-time analytics in BigQuery with Datastream for BigQuery
Easy-to-use setup with built-in secure connectivity for faster time to value
Serverless platform that automatically scales, with no resources to provision or manage
Secure foundation for database synchronization and event-driven architectures

VIDEO

Introduction to Datastream for BigQuery

5:39

Benefits

Replicate and synchronize data with minimal latency

Synchronize data across heterogeneous databases, storage systems, and applications reliably, with low latency, while minimizing impact on source performance.

Scale up or down with a serverless architecture

Get up and running fast with a serverless and easy-to-use service that seamlessly scales up or down, and has no infrastructure to manage.

Unmatched flexibility with Google Cloud services

Connect and integrate data across your organization with the best of Google Cloud services like BigQuery, Cloud Spanner, Dataflow, and Data Fusion.

Key features

Datastream's differentiated approach

Streaming data from relational databases

Datastream reads and delivers every change—insert, update, and delete—from your MySQL, PostgreSQL, AlloyDB and Oracle databases to load data into BigQuery, Cloud SQL, Cloud Storage, and Cloud Spanner. Agentless and Google-native, it reliably streams every event as it happens. Datastream processes over half a trillion events per month.

Normalized data types across sources

Datastream normalizes every event’s data type from the source database’s type into a unified Datastream type. This lossless data type normalization across sources means easier downstream processing in a source-agnostic way, regardless of where the data originated.

Schema drift resolution

As source schemas change, Datastream allows for fast and seamless schema drift resolution. Datastream rotates files, creating a new file in the destination bucket, on every schema change. Original source data types are just an API-call away with an up-to-date, versioned Schema Registry.

Secure by design

Datastream supports multiple secure, private connectivity methods to protect data in transit. In addition, data is encrypted in transit and at rest so you can rest easy knowing your data is protected as it streams.

BLOG

Unlock the power of change data capture and replication

Customers

Learn from customers replicating data using Datastream

Blog post

Datastream helps Falabella get much quicker insights on their operational data.

5-min read

Blog post

Cogeco Communications, Inc. brings data from 100's of Oracle tables at 10X time and effort efficiency.

5-min read

Blog post

Schnuck Markets, Inc. replaced batch processing with Datastream for faster insights in BigQuery.

5-min read

See all customers

"As we’re modernizing our applications, we're replicating data reliably out of Oracle and into Cloud SQL for PostgreSQL with Datastream which has resulted in minimal downtime to our application."

Shawn O’Rourke, Manager of Technology, MLB

Blog Link

What's new

News and events about Datastream

Video

Google I/O 2023: Using Google Cloud's latest services to build data-driven apps Watch video

Blog post

Introducing Datastream for BigQuery Read the blog

Blog post

Datastream is now generally available - read the launch announcement Read the blog

Video

Introducing Datastream Watch video

Blog post

Migrate from Oracle to PostgreSQL with Datastream Read the blog

Blog post

Unlock the power of CDC and replication with Datastream Read the blog

Documentation

Find resources and documentation for Datastream

Datastream overview

Learn how Datastream helps you replicate and synchronize data across heterogeneous databases, storage systems, and applications.

Datastream quickstart using the Cloud Console

Learn how to use the Google Cloud Console as a visual interface to start streaming data.

Configure Datastream using the API

Learn how to use the API to configure Datastream to transfer data from a source Oracle database into Cloud Storage.

How to migrate from Oracle to Cloud SQL for PostgreSQL

Use Google’s Datastream-based, open-source toolkit to migrate from Oracle to Cloud SQL for PostgreSQL.

Replicate data into BigQuery with Datastream and Dataflow

Learn how to combine Datastream with Dataflow templates to replicate data from a relational database.

How to replicate data from an Oracle database into BigQuery

Watch this video to learn how to replicate data in real time from Oracle into BigQuery using Data Fusion’s replication accelerator that's integrated with Datastream.

Use cases

Continuous change data capture: replicate every event as it happens

Use case

Real time, anytime change streams

Change data capture integrates data by reading change events (inserts, updates, and deletes) from source databases and writing them to a data destination, so action can be taken. Datastream supports change streams from Oracle and MySQL databases into BigQuery, Cloud SQL, Cloud Storage, and Cloud Spanner, enabling real-time analytics, database replication, and other use cases. Additional sources and destinations are coming in the future.