prosync
Prev UpChapter 6. ReferenceHome Next

prosync

prosync — utility to replay changes to a source database on a destination database of a different type

Synopsis

prosync config generate [option...]

prosync init [option...]

prosync run [option...]

prosync complete [option...]

Description #

prosync is a utility to perform the change data capture (CDC) of a source database and replay the changes on a destination database of a different type.

prosync is not responsible for:

  • Database schema transfer.

  • Transferring data that is missing from log files.

  • Data verification after the data transfer.

Supported Operations #

  • DML: INSERT, UPDATE, DELETE

  • DDL: TRUNCATE

Usage #

Migrating the Database #

prosync participates in the hot DB migration (see Section 4.5 for more details).

Preparing for a Migration of a Hot DB #

To prepare for the migration of a hot DB, perform the following steps:

  1. Prepare and set up the destination DB for the migration:

    • Create the necessary schemas and tables.

    • Remove or turn off triggers.

  2. Generate the configuration file: 

    prosync config generate -o filename

  3. Update the configuration file with appropriate values.

  4. Initialize prosync with the command: 

    prosync init -f filename

    For an Oracle source, prosync will store the current SCN to start the subsequent replication with this value. For a PostgreSQL/Postgres Pro source, prosync will create a logical replication slot for subsequent reading of changes.

Performing the Migration of a Hot DB #

Perform the migration as follows:

  1. Launch the procopy load: 

    procopy load -f filename

  2. When it finishes, launch the prosync replay of changes: 

    prosync run -f filename

  3. Wait until the difference between Parsing SCN and Oracle latest SCN decreases to an admissible value. For the migration of a hot DB, it is critical for this difference to decrease: 

    Parsing SCN          3671057
Oracle latest SCN    4995972

  4. Disable the load on the source DB and wait until Parsing SCN and Oracle latest SCN virtually stop changing. Stop prosync.

    Note

    Changes always occur in Oracle even if nobody changes the data, so Oracle latest SCN is continuously growing. Therefore, the difference between Parsing SCN and Oracle latest SCN never reaches zero.

Excluding Columns from Tasks of the Table Type #

When executing tasks of the TABLE type (see the section called “Types of Tasks” for details) during data migration involving prosync, do not exclude (through exclude_columns or include_columns) columns that are used by LogMiner as a row key. Otherwise, executing DELETE or UPDATE commands can change more than one row.

For tables without keys (heap), do not exclude any columns as LogMiner uses the whole row for identification.

Note

These rules do not apply for data migrations that are only performed by procopy.

Notes for PostgreSQL/Postgres Pro Sources #

When migrating the PostgreSQL/Postgres Pro DB, a replication slot is needed for running prosync replay of changes. Therefore:

  • Before launching the replay of changes by prosync run, execute the init command, which will create the replication slot.

  • When run finishes, execute complete command to clear the replication slot.

Command-Line Reference #

This section describes prosync commands. Optional parameters are enclosed in square brackets.

config generate #

prosync config generate [-f|--format json|yaml]
[-o|--output filename] [-c|--source_config]

Generates a prosync configuration file.

-f json|yaml
--format json|yaml

Specifies the format of the configuration file: JSON or YAML. The default is YAML.

-o filename
--output filename

Specifies the filename of the configuration file. By default, the configuration file will be output to stdout.

-c
--source-config

Use the procopy configuration file as the basis of the prosync configuration file and add missing options there.

init #

prosync init -f|--file filename [-u|--update-config]

Gets the current SCN from the source database.

-f filename
--file filename

Name of the configuration file, where the connection options are specified.

-u
--update-config

Use the procopy configuration file as the basis and add missing options there.

run #

prosync run -f|--file filename
[--log-level error|warn|info|debug] [--dry-run|--read-only]
[-c|--clear] [--with-table-stats]

Launches the replay of changes to the source DB on the destination DB.

-f filename
--file filename

Name of the configuration file, where the connection options are specified.

--log-level error|warn|info|debug

Logging level. The default is info.

--dry-run

Turns off writing to the destination. prosync will perform all the operations except writing. This flag is useful for checking the performance of writing.

--read-only

Turns off all the prosync logic except reading from the source. This flag is useful for checking the performance of reading.

-c
--clear

Clear the screen before each next output of the statistics. When enabled, it is recommended to output logs to a file, as follows: 

prosync run ... -c 2>prosync.log

--with-table-stats

Output the extended statistics on tables.

complete #

prosync complete [-f filename]

Clears the replication slot used by run. The slot name is determined from the configuration file, whose default name is config.yaml.

Prev Up Next
procopy Home procheck