This page describes how to create, modify, and view Cloud Spanner change streams. To learn more about change streams, please see About change streams.
Because change streams are schema objects, you create and manage them through the same DDL-driven schema updates used for any other kind of database-definition work, such as creating tables or adding indexes.
Spanner begins a long-running operation after you submit a schema-changing DDL statement, including those used to create, alter, or delete change streams. A new or altered change stream will begin to watch the columns or tables specified by its new configuration upon the completion of this long-running operation.
Create a change stream
To create a change stream, you need to supply its name and the schema objects it will watch: either the entire database, or a list of specific tables and columns. You can optionally specify a retention period if you want to override the one-day default retention period.
The DDL syntax for creating a change stream looks like this:
CREATE CHANGE STREAM change_stream_name [FOR column_or_table_watching_definition[, ... ] ] [ OPTIONS ( retention_period = timespan, value_capture_type = type ) ]
A new change stream begins to watch its assigned schema objects as soon as the long-running operation that created it completes.
The following examples illustrate the creation of change streams with various configurations.
Watch an entire database
To create a change stream that watches every data change performed
throughout a database's tables, use the
CREATE CHANGE STREAM EverythingStream FOR ALL;
ALL configuration implicitly
includes all of the database's future data
tables and columns, as soon as they get created. It does not
include views, information schema tables, or
other objects aside from normal data tables.
Watch specific tables
To limit a change stream's scope to specific tables, rather than an entire database, specify a list of one or more tables:
CREATE CHANGE STREAM SingerAlbumStream FOR Singers, Albums;
Spanner automatically updates change streams that watch whole tables to reflect any schema changes affecting those tables, such as added or dropped columns.
Watch specific columns
...]) syntax to watch changes to one or more specific,
non-key columns within tables that you name:
CREATE CHANGE STREAM NamesAndTitles FOR Singers(FirstName, LastName), Albums(Title);
You can't specify primary key columns here because every change stream always tracks the primary keys of every table it watches. This allows every data change record to identify the changed row by its primary key.
Watch tables and columns in a single stream
You can combine table-watching and column-watching syntax from the previous two examples in a single change stream:
CREATE CHANGE STREAM NamesAndAlbums FOR Singers(FirstName, LastName), Albums;
Specify a longer retention period
To specify a change stream data retention period
longer than the default of one day, use the
OPTIONS clause to specify a period of up to one week,
expressed as either hours (
h) or days (
d). Two examples:
CREATE CHANGE STREAM LongerDataRetention FOR ALL OPTIONS( retention_period = '36h' );
CREATE CHANGE STREAM MaximumDataRetention FOR ALL OPTIONS( retention_period = '7d' );
Modify a change stream
To modify a change stream's configuration, use an
ALTER CHANGE STREAM DDL
statement. It uses syntax similar to
CREATE CHANGE STREAM. You can
change which columns a stream watches, or the length of its data
retention period. You can also suspend its watching altogether while
retaining its data change records.
Modify what a change stream watches
This example adds the entire
Songs table to the
change stream configured earlier:
ALTER CHANGE STREAM NamesAndAlbums SET FOR Singers(FirstName, LastName), Albums, Songs;
Spanner replaces the named change stream's behavior with the new configuration upon the completion of the long-running operation that updates the change stream's definition within the database's schema.
Modify a change stream's data retention policy
To modify the length of time that a change stream retains its internal
records, include a
SET OPTIONS clause in an
ALTER CHANGE STREAMS DDL
This example adjusts the data retention period to the
change stream created previously:
ALTER CHANGE STREAM NamesAndAlbums SET OPTIONS( retention_period = '36h' );
Suspend a change stream
If you wish to have a change stream halt its activity but retain its internal records—at least for the life of its data retention period—you can alter it so that it watches nothing.
To do this, issue an
ALTER CHANGE STREAM DDL statement that replaces
the change stream's definition with the special phrase
DROP FOR ALL.
ALTER CHANGE STREAM MyStream DROP FOR ALL;
The stream continues to exist in the database, but watches no objects, and generates no further data change records. Its existing change records remain intact, subject to the stream's data retention policy.
To resume a suspended stream, issue another
ALTER CHANGE STREAM
statement with its previous configuration.
Delete a change stream
To permanently delete a change stream, issue a
DROP CHANGE STREAM
statement that includes the stream's name:
DROP CHANGE STREAM NamesAndAlbums;
Spanner immediately stops the stream, removes it from the database's schema, and deletes its data change records.
List and view change streams
The Google Cloud console provides a web interface for listing and reviewing a database's change stream definitions. You can also view the structure of change streams as their equivalent DDL statements, or by querying the database's information schema.
View change streams with Google Cloud console
To see a list of a database's change streams and review their definitions:
Visit the Spanner instances page of the Google Cloud console.
Navigate to the appropriate instance and database.
Click Change streams in the navigational menu.
This displays a list of all of that database's change streams, and summarizes the configuration of each one. Clicking a stream's name reveals further details about the tables and columns that it watches.
View definitions of change streams as DDL
Viewing a database's schema as DDL includes descriptions of all its
change streams, where they appear as
CREATE CHANGE STREAM statements.
To do this from the console, click The Show equivalent DDL link on the database's page in the Google Cloud console.
To do this from the command line, use the
ddl describecommand of Google Cloud CLI.
Query the information schema about change streams
You can directly query a database's information schema about its change streams. The following tables contain the metadata that defines change streams' names, the tables and columns that they watch, and their retention periods:
Learn how to build change streams pipelines with Dataflow.