During execution of distributed queries, Shardman sends derived SQL queries to remote nodes that hold data partitions involved in the query execution. Let's call these SQL queries query fragments. Shardman sends such queries using the postgres_fdw extension. The node that queries the sharded table is called the coordinator, while the nodes that accept query fragments are called shards.

When the pgpro_stats extension is enabled on a Shardman cluster node, it collects statistics about local and distributed queries. The information about distributed queries initiated by this node is incomplete because it misses data about remote query fragments. The statistics concerning queries initiated by other nodes is also ambiguous because there is no simple way for a user to determine the distributed query to which the fragment corresponds.

To address these issues, pgpro_stats for Shardman introduces an aggregation of statistics for the distributed queries. These aggregated statistics can be accesses with the pgpro_stats_sdm_statements view. However, each Shardman node collects statistics for all the statements, so that the pgpro_stats_statements view can work the way it did before.

When a node receives a query fragment, it saves its statistics to a separate shared hash table. Periodically and asynchronously, each node sends this information from a separate table to the coordinator corresponding to the query. The coordinator aggregates the statistical data obtained from the query fragments with the statistics of its parent query, which is the query initiated by the client.

The pgpro_stats extension starts a separate background worker. This worker is responsible for sending the accumulated statistics to the coordinator nodes either every 5 seconds or when triggered by the guard latch. The collecting function sets this latch when the hash table is almost full.

To reduce the network traffic initiated by a statistics sender, compression is applied to the statistics data sent. The compression method can be selected by the pgpro_stats.transport_compression configuration parameter.

Each node stores the total number of statistics entries received from the shard node and the timestamp of when they were last received. When a coordinator node receives a statistics message, it updates the appropriate values, which are accessible using the SQL interface.

There are additional pgpro_stats SQL functions introduced by Shardman additions described in Section 6.2 and configuration parameters described in the section called “pgpro_stats parameters”.