Verify a migration

This page provides instructions to confirm that the migration job migrated data from your source database instance to the destination Cloud SQL database instance correctly.

At a minimum, verify that the database instance and its tables exist in the Cloud SQL instance. You can also verify the tables' row counts or exact contents.

Use an open-source Data Validation Tool to validate that data matches between the source and the destination.

To validate a minimal example, follow these steps:

  1. Deploy or use a virtual machine with access to both the source and the destination.

  2. In the virtual machine, create a folder in which to install the Data Validation Tool.

  3. Navigate to the folder that you created.

  4. Use pip to install the Data Validation Tool:

    pip install google-pso-data-validator
    
  5. Create connections to the source Oracle database and the destination Cloud SQL for PostgreSQL database:

    data-validation connections add -c source Oracle --host 'ip-address' --port port --user username --password pswd --database database-name
    data-validation connections add -c target Postgres --host 'ip-address' --port port --user username --password pswd --database database-name
    

    For example:

    data-validation connections add -c source Oracle --host '10.10.10.11' --port 1521 --user system --password pswd --database XE
    data-validation connections add -c target Postgres --host '10.10.10.12' --port 5432 --user postgres --password pswd --database postgres
    
  6. Create or generate a list of tables to compare data between the source and destination databases:

    export TABLES_LIST=$(data-validation find-tables --source-conn source --target-conn target --allowed-schemas schema-name)
    

    For example:

    export TABLES_LIST=$(data-validation find-tables --source-conn source --target-conn target --allowed-schemas public)
    
  7. Run full validation against all tables:

    data-validation validate column --source-conn source --target-conn target --tables-list "${TABLES_LIST}"
    

We suggest that you run this validation during replication to ensure relative consistency. Large table queries may take too long to run during a small promotion window. In such cases, use the Data Validation Tool to add filters to reduce runtime or prepare the table list to include a subset of tables for the final validation.