The following is an example of the prosync configuration file fragment in the YAML format that lists prosync-specific configuration parameters:
prosync_options:
export_batch_size: 10000000
max_export_batch_size: 100000000
process_batch_size: 10000
batch_save_after_timeout: 5m0s
import_batch_size: 1000000
prefetch_rows: 250
use_online_logs_unsafe: false
validate_log_sequence: true
lob_write_size: 30000
operation_count_for_external_storage: 5000
For how to set values for time configuration parameters, refer to Section 4.3.6.
Table 5.3. prosync Configuration Parameters
| Name | Description | Default Value | Example |
|---|---|---|---|
prosync_options | prosync-specific parameters | ||
prosync_options.export_batch_size | Batch size for prosync reading from Oracle LogMiner | ||
prosync_options.max_export_batch_size | Maximum size to which the batch size can be increased when reading. If the maximum size is reached, prosync terminates with an error. This can happen if there is an open transaction that would not close. The user must resolve this issue. | ||
prosync_options.process_batch_size | Batch size for processing a transaction. This parameter is for internal tuning of the application. | 10000 | |
prosync_options.batch_save_after_timeout | Time interval after which a batch will be written to the destination
DB if its size is still less than max_export_batch_size.
The value of zero disables time-based
flush to the destination DB. | 5s | 5m |
prosync_options.import_batch_size | Batch size for inserting data. Zero value means that the data will be inserted as new data occurs. | ||
prosync_options.prefetch_rows | Deprecated. Number of rows that the driver retrieves from the databases at a time. | ||
prosync_options.use_online_logs_unsafe | Read the changes from archive and online REDO LOGS.
The flag enables getting the changes as soon as they occur in online
logs. Not recommended to use for reliability. Because of how
Oracle writes logs, some operations can be missed. | false | |
prosync_options.validate_log_sequence | Validate a sequence of log files. If a log file is missing,
prosync terminates with an error. If losing
some data is OK, set this parameter to false. | true | |
prosync_options.lob_write_size | Maximum number of bytes to write at a time when writing LOBs | ||
prosync_options.operation_count_for_external_storage | Number of operations in one transaction that causes writing of intermediate data to disk | ||
prosync_options.transfer_id | Data transfer identifier. Used to create names of slots for replication and publications. This parameter is required. | d87708a4-c35e-4379-8845-d173683e5385 | |
prosync_options.null_char_replace | String to replace the null character, that is, the symbol with the
zero code. This is a global setting, which can be overridden for each
column with the value of tasks.transform.column_name.null_char_replace.
|
nil, but for the Oracle source and PostgreSQL/Postgres Pro destination: null_char_replace: replace: "" destination_types: ["varchar", "char", "bpchar", "text", "json", "jsonb", "xml"]
| |
prosync_options.null_char_replace.replace | Character or string to replace \0x0000. If not specified, no replacement will be done. | nil | replace: "\n" |
prosync_options.null_char_replace.destination_types | Specifies the types in the destination database to replace null characters for. Character or string to replace \0x0000. If not specified, no replacement will be done. | [] | destination_types: ["varchar", "char", "text", "json", "jsonb", "xml"] |
prosync_options.readers | Number of readers from LogMiner. If set
to a positive value, the setting of use_online_logs_unsafe
is ignored. | 0 — one-threaded slow reading, which is, however, compatible
with use_online_logs_unsafe | |
prosync_options.use_failover_slots | Whether to use failover replication slots. Only applicable to the PostgreSQL/Postgres Pro 17.0 or higher source. | false | |
prosync_options.use_replica_auto_sync_slots | Enables prosync logic for propagation of replication slots. Use if you need failover replication slots on the cluster, but have PostgreSQL/Postgres Pro lower than 17.0 or if you need to read from a replica (alone or not). | false | |
prosync_options.postgres_slot_advance_interval | How often a slot will be moved on the whole cluster if
prosync_options.use_replica_auto_sync_slots is enabled.
Must be greater than 100 milliseconds.
| 5s | 500ms |
prosync_options.loaders | Number of workers that write to the destination. If not specified or equals zero, one worker will be used. | 0 | |
prosync_options.truncate_process | Whether to process the truncate operations.
If true, truncate operations will
be processed for the destination table in the destination DB.
| false | |
prosync_options.get_replication_message_timeout | Timeout for receiving messages from the
PostgreSQL/Postgres Pro
source DB. Must be greater than or equal to wal_sender_timeout.
| 60s | |
prosync_options.dump_mode | Dump mode. If enabled, operations are saved to a dump table rather than performed. | false |