Work with Oracle database redo log files

Overview

Datastream leverages Oracle LogMiner, which is part of Oracle Database, to query archived redo log files. These files contain information about the history of activity on a database.

Each Oracle database has a set of online redo log files. All transaction records on the database are recorded in the files.

When the current redo log file is rotated (or switched), the archive process copies this file into an archive storage. Meanwhile, the database promotes another file to serve as the current file.

Datastream's Oracle connector extracts change data capture (CDC) events from archived Oracle redo log files.

Set configuration parameters for Oracle's redo log files

This design has profound implications on Datastream's potential latency. If Oracle's redo log files are switched frequently or kept to a smaller size (for example, < 256MB), Datastream can replicate changes faster.

There are configuration parameters that you can set to control the log file rotation frequency:

• Size: Online redo log files have a minimum size of 4 MB, and the default size is dependent on your operating system. You can modify the size of the log files by creating new online log files and dropping the older log files.

  To find the size of the online redo log files, run the following query:

  SELECT GROUP#, STATUS, BYTES/1024/1024 MB FROM V$LOG

  Generating very large log files might cause Datastream to time out, which can lead to stream failure. The recommended redo log file size is below 1GB.

  For information about resizing redo logs for a self-hosted Oracle instance, see How to resize redo logs in Oracle.

  For information about resizing redo logs for an Amazon RDS Oracle instance, see Resizing online redo logs.

• Time: The ARCHIVE_LAG_TARGET parameter provides an upper limit of how long (in seconds) the current log of the primary database can span.

  This isn't the exact log switch time, because it takes into account how long it will take to archive the log. The default value is 0 (no upper bound), and a reasonable value of 1800 (or 30 minutes) or less is suggested.

  You can use the following commands to set the ARCHIVE_LAG_TARGET parameter, either during initialization or while the database is up:

  • SHOW PARAMETER ARCHIVE_LAG_TARGET; This command displays how many seconds it will take for the current log to span.

  • ALTER SYSTEM SET ARCHIVE_LAG_TARGET = number-of-seconds; Use this command to change the upper limit.

    For example, to set the upper limit to 10 minutes (or 600 seconds), enter ALTER SYSTEM SET ARCHIVE_LAG_TARGET = 600;

You can switch the redo log files manually by running the following command:

ALTER SYSTEM SWITCH LOGFILE;

Although using this command is effective for testing purposes, we don't recommend it for production use-cases because of the privileges it requires and the significant performance impact on the database.