Release date: 2024-03-14
This release provides new features, optimizations and bug fixes. Major changes are as follows:
Enhanced the pgpro_stats
extension to give a better understanding of what system resources are used for distributed queries.
Now the regular pgpro_stats_statements view shows the gathered
statistics for individual statements on the current Shardman
node (they can be part of some distributed query), while the
pgpro_stats_sdm_statements view shows the gathered
statistics for the distributed queries originating from the current node, that is,
aggregated from all the participating nodes.
Added the pgpro_pwr package compatible with Shardman. This allows Shardman users to build workload reports.
Improved the EXPLAIN output. If a query plan contains ForeignScan
nodes, the EXPLAIN output for queries executed on the remote server can now be included.
Added a new configuration parameter enable_partition_pruning_extra
that enables extended subplan pruning logic when building and executing generic plans where the set
of useful partitions depends on the prepared query parameters. This allows Shardman
to do initial pruning of complex subplans, joins and partial aggregates, in particular.
Added metrics to the shardman.pg_stat_csn
view that show delays of the global horizon and the transaction that may cause that delay. They may be useful
to research autovacuum issues.
Considerably improved backup and restore with the shardmanctl probackup
command. Notable changes are as follows:
Added support of backups to an S3-compatible object storage.
Implemented selective WAL archiving on the specified shards by the
probackup subcommand.
Added two new commands shardmanctl probackup delete and
shardmanctl probackup merge. The delete
command deletes a backup with a specified ID and the archived WAL files
that are no longer in use. The merge command merges the
backups that belong to a common incremental backup chain.
Added new shardmanctl probackup set-config command that
adds the specified settings to the pg_probackup.conf or
modifies the existing ones.
Added a new option log-to-console for the validate
subcommand. Set the log rotation file size to 20 MB. If this value is reached,
the log file is rotated once a validate or backup
subcommand is launched.
Increased the number of retries for some subcommands to avoid backup failures caused by large database sizes.
Added topology compatibility checks between the current Shardman
cluster and the one in the backup directory to
backup and restore subcommands.
Set the default value for the number of concurrent pg_probackup processes to the number of logical CPUs of the system.
Fixed data cleanup after a failure of a backup subcommand.
Previously, some data of a failed backup could still remain in the repository.
Fixed hanging that could occur during metadata-only restore of a Shardman cluster.
Fixed the pg_probackup issue that could occur during the schema recovery process.
Changed the behavior of metadata-only restore to avoid losing a cluster. Now the cluster is stopped before such a restore and restarted after it, a cluster that has no nodes cannot be restored from the etcd dump, and if cluster IDs of the dump and the current cluster are different, the user is asked whether restoring the cluster with the changed ID is OK.
Added new options for archive-command:
--compress, --compress-algorithm,
--compress-level, --batch-size,
and -j|--jobs. This helps to reduce the WAL size.
Improved the show subcommand output.
Added new flags -archive to output the log information,
-instance and -i|--backup-id to output
information for the specified backups and instances.
Updated the getconnstr
and cluster topology
commands so that they do not issue a lock on other processes. Previously, some
commands failed to receive a connection string because of the locks.
Fixed a panic that could occur on a Shardman cluster
configured with PlacementPolicy = manual
when a user executed the command shardmanctl cluster repfactor set.
Hid uninformative warnings that pg_dump displayed
during execution of shardmanctl
nodes add and shardmanctl probackup
backup commands.
Removed a lock that was required by the shardmanctl status command.
Previously shardmanctl status did not provide any useful information
in case a process hung as it was waiting for the lock from that process.
Added the forceSuUserLocalPeerAuth configuration parameter.
When enabled, it sets a peer authentication via unix socket for the postgres user unless strictUserHBA
is set to true. See sdmspec.json for details.
Added a URL for Prometheus automatic service discovery metrics to shardmand.