Pass a runtime argument to a Replication job

This page describes how to pass runtime arguments in Cloud Data Fusion replication jobs.

Before you begin

• Create a replication job in a Cloud Data Fusion instance. For more information, see Get started with Replication.

Pass a Debezium argument to a replication job

To pass a Debezium argument to a MySQL or SQL Server replication job in Cloud Data Fusion, specify a runtime argument prefixed with source.connector.

Console

1. Go to your instance:

   1. In the Google Cloud console, go to the Cloud Data Fusion page.

   2. To open the instance in the Cloud Data Fusion web interface, click Instances, and then click View instance.

      Go to Instances

2. Click Menu > Control Center.

3. Locate the Application for the replication job and click Preferences. The Preferences window opens.

4. In the Key field, specify a runtime argument for the replication job by prefixing it with source.connector.

   

5. Click Save & Close.

REST API

To set a runtime argument using the REST API, see the CDAP Preferences Microservices reference.

Configure JDBC parameters

To pass a JDBC parameter to a MySQL or SQL Server replication job, specify the runtime argument prefixed with source.connector.database.

For example, to configure the JDBC parameter sessionVariables to MAX_EXECUTION_TIME=43200000, set a runtime argument with the key source.connector.database.sessionVariables and the value MAX_EXECUTION_TIME=43200000.

To configure multiple JDBC parameters, set a runtime argument for each parameter. For example, to configure the JDBC parameters encrypt=true&trustServerCertificate=true, pass the following arguments:

Key	Value
source.connector.database.encrypt	true
source.connector.database.trustServerCertificate	true

Note: Cloud Data Fusion doesn't support setting JDBC arguments during the setup or assessment of a replication job. If you must set JDBC parameters to resolve assessment issues, bypass the assessment by deploying the replication job through the Replication API. Then, before you start the job, configure the JDBC arguments as described previously in this section.

Configure the primary key parameter

The source table that gets replicated must have a primary key. This is a strict requirement only when Oracle is the source database. For SQL Server and MySQL sources, you can specify a custom primary key, even if the source table doesn't have one.

Set the key parameter with the following runtime argument:

source.connector.message.key.columns = SCHEMA.TABLE:KEY_COLUMN

Replace the following:

• SCHEMA: The name of the source schema.
• TABLE: The source table name.
• KEY_COLUMN: The column that contains the secure key.

You can set the primary key for multiple tables with the key property. The following example shows how to set the key for the tables inventory.customers and purchase.orders:

source.connector.message.key.columns = inventory.customers:pk1,pk2;purchase.orders:pk3,pk4

Configure isolation mode for a snapshot in SQL Server replication

For more information about the runtime argument for isolation mode, see Isolation levels in SQL Server replication.

What's next

• Learn more about Replication in Cloud Data Fusion.
• Refer to the Replication API reference.
• Work through the tutorial for Replicating data from MySQL to BigQuery.
• Work through the tutorial for Replicating data from SQL Server to BigQuery.
• Work through the tutorial Replicating data from Oracle to BigQuery.