pgpro_bindump is a plugin designed to implement additional commands for the Postgres Pro replication protocol, specifically for use with the pg_probackup3 utility. pgpro_bindump offers the following benefits:
Simplified communication: pgpro_bindump does not require the SSH protocol and associated configurations to connect to a database server.
Integrated WAL transfer: WAL files are transferred together with data files, reducing the risk of data loss during backups.
Optimized storage: Files are not copied as they are, but are archived
using the pg_probackup3 own format,
CBOR (RFC 8949 Concise Binary Object Representation),
which enhances the storage efficiency.
Here are some key points of how pgpro_bindump works:
A dedicated backend process is created to handle connections from external applications. You can use the pg_probackup3 utility or a custom application to send requests to this backend.
Multiple connections can be established for the same application if needed.
Since the pgpro_bindump plugin has direct access to the database files, it does not require any additional permissions.
pgpro_bindump uses the libpgprobackup library, along with its built-in encoder module, to send data to the application.
The walsender_plugin_libraries configuration
parameter simplifies the process of adding and utilizing new
replication commands, including custom ones.
pgpro_bindump currently has the following limitations and requirements:
libpgprobackup.so and
libpb3_encoder.so are required for
pgpro_bindump operation.
The pgpro_bindump plugin is included in
Postgres Pro as part of the
postgrespro-std-18-contrib package (for
the installation instructions, see Chapter 16).
Once you complete Postgres Pro installation,
follow the steps below.
Specify the following parameters in the postgresql.conf
file:
shared_preload_libraries = 'pgpro_bindump' wal_level = 'replica' # or 'logical' walsender_plugin_libraries = 'pgpro_bindump'
Restart the Postgres Pro instance.
Since pgpro_bindump is a plugin and not an
extension, the CREATE EXTENSION command is not needed.
The pgpro_bindump plugin implements the following replication commands:
just-noop — a do-nothing command.
start_backup — starts the backup process.
copy_files — copies files in the
CBOR format.
stop_backup — stops the backup process.
The backup process runs in the PRO mode — an advanced backup data source mode that incorporates all available optimizations and enhancements, including the proprietary replication protocol and specialized replication commands.
To send commands from a user application to pgpro_bindump and to receive the output data from pgpro_bindump by the application, use libpq.
An established connection to a running Postgres Pro server is required.
start_backup LABELlabel, INCR_MODEincr_mode, START_LSNstart_lsn, TRANSFER_MODEtransfer_mode, [TRANSFER_BUFFER_SIZEbuffer_size]
Starts the backup process. Sends the following data to the application:
start_lsn — LSN at the start of the backup.
start_tli — the timeline ID at the start of
the backup.
min_xid — the minimal transaction ID at the
start of the backup.
min_multixact — the minimal
multixact ID at the start of the backup.
tablespace_list — a list of tablespaces
included in the backup.
database_list — a list of databases included
in the backup.
LABEL labelThe name of the backup.
INCR_MODE incr_mode
The type of incremental backup. Possible values:
PTRACK, DELTA.
START_LSN start_lsn
An XLogRecPtr value representing LSN at the start of
the backup.
TRANSFER_MODE transfer_modeThe method to be used for sending data from a server to an application. Possible values:
raw — unpacked data is sent in random
blocks of arbitrary size.
packed — packed data is sent in
blocks of 128 KB with a common header.
packed is the default value.
TRANSFER_BUFFER_SIZE buffer_sizeA numeric value representing the buffer size. The preset (unchanged) value is 131072 (128 KB).
copy_files VERIFY_CHECKSUMS [TRUE | FALSE], COMPRESS_ALGcompress_alg, COMPRESS_LVLcompress_lvl, TRANSFER_MODEtransfer_mode, [INCR_MODEincr_mode], [START_LSNstart_lsn], [TRANSFER_BUFFER_SIZEbuffer_size], [WORKER_INFOnumber_of_workers:worker_id]
Forwards blocks of data files and WAL files packaged in the CBOR format.
VERIFY_CHECKSUMS
A boolean value (true or false)
indicating whether to verify checksums.
COMPRESS_ALG compress_alg
The compression algorithm to be used. Possible values:
zlib, zstd,
lz4, none.
COMPRESS_LVL compress_lvlThe compression level. Possible values: 1, 2, 3, 4, 5.
TRANSFER_MODE transfer_modeThe method to be used for sending data from a server to an application. Possible values:
raw — unpacked data is sent in random
blocks of arbitrary size.
packed — packed data is sent in
blocks of 128 KB with a common header.
packed is the default value.
INCR_MODE incr_mode
The type of incremental backup. Possible values:
PTRACK, DELTA.
START_LSN start_lsn
An XLogRecPtr value representing LSN at the start of
the backup.
TRANSFER_BUFFER_SIZE buffer_sizeA numeric value representing the buffer size. The preset (unchanged) value is 131072 (128 KB).
WORKER_INFO number_of_workers:worker_idThe number of workers and the worker ID.
stop_backup STREAM [TRUE | FALSE], COMPRESS_ALGcompress_alg, COMPRESS_LVLcompress_lvl, [EXTERNALSexternals] [TRANSFER_BUFFER_SIZEbuffer_size], [TRANSFER_MODEtransfer_mode]
Completes the copy process and closes the existing connection. Sends the following data to the application:
stop_lsn — LSN at the end of the backup.
stop_tli — the timeline ID at the end of
the backup.
The backup.control file.
A set of WAL files (only if WAL streaming is enabled).
STREAM
A boolean value (true or false)
indicating whether WAL streaming is enabled.
COMPRESS_ALG compress_alg
The compression algorithm to be used. Possible values:
zlib, zstd,
lz4, none.
COMPRESS_LVL compress_lvlThe compression level. Possible values: 1, 2, 3, 4, 5.
EXTERNALS externalsA list of external directories.
TRANSFER_BUFFER_SIZE buffer_sizeA numeric value representing the buffer size. The preset (unchanged) value is 131072 (128 KB).
TRANSFER_MODE transfer_modeThe method to be used for sending data from a server to an application. Possible values:
raw — unpacked data is sent in random
blocks of arbitrary size.
packed — packed data is sent in blocks
of 128 KB with a common header.
packed is the default value.
Postgres Professional, Moscow, Russia.