Troubleshoot a stream

If a problem occurs while a stream is transferring data from a source database into a destination, the stream can enter a Failed or Failed permanently state. In both cases, you can rectify the problem.

Troubleshoot a stream

  1. Go to the Streams page in the Google Cloud Console.

    Go to the Streams page

  2. Click the Column display options icon in the upper-right corner of the page. The icon appears as three vertical columns.

  3. If it isn't selected, select the Status checkbox, and then click OK. Datastream displays the following statuses:

    • Failed: for an error that occurs on a Running stream. Such errors imply that the stream is still active or continuously attempting to run.
    • Failed permanently: for a stream that can't continue to run. Such errors might cause data loss.

  4. Click the stream that you want to troubleshoot. Any errors associated with the stream appear on the Stream details page.

    For example, if Datastream can't connect to the source database, then the We can't use the credentials that you provided to connect to the data source. error message appears on this page.

  5. Address the errors. You can resolve errors for either the stream or the connection profile.

    For example, if errors are associated with either the source data objects of the stream or its destination configuration information, then modify the stream.

    If errors are associated with the connectivity information of the stream, then update the configuration information about the source database or the destination for any connection profiles being used by the stream.

  6. Fix the Failed stream so that it can automatically resume, or recover the Failed permanently stream.

Recover a stream

The first thing to try when recovering a stream is to recover it from the current position. For more information about stream recovery options, see Stream recovery overview.

If recovering a stream from the current position fails, then try the following:

  1. Drop or truncate the affected tables in the destination. You need to do this because while the stream was down, Datastream might have missed some DELETE events. DELETE events can't be recovered if you don't truncate the table before performing the backfill.
  2. Recover the stream from the most recent position. For PostgreSQL, recreate the replication slot or create a new replication slot.
  3. Once the stream is running, trigger backfills to restore all historical data. For information about how to trigger a backfill, see Initiate backfill.

What's next