Shardman cluster configuration is stored in etcd. Shardman cluster services are organized as systemd services. The Shardman configuration daemon shardmand monitors the cluster configuration and manages stolon clusters and services. Each node has one shardmand service, whose typical name is shardmand@CLUSTER_NAME.service. Here CLUSTER_NAME is a Shardman cluster name, cluster0 by default. Each node has several stolon services.

Each registered DBMS instance has an associated stolon keeper service that directly manages this PostgreSQL instance. The keeper starts, stops, initializes and resyncs PostgreSQL instances according to the desired stolon cluster state.

Each registered DBMS instance has an associated stolon sentinel service. For each replication group, stolon sentinels elect the leader among existing sentinels. This leader makes decisions about the desired cluster state (for example, which keeper should become a new master when the existing one fails). When the new master in a replication group is selected, the leader selects the keeper with the minimal lag. When all replicas are synchronous, the keeper with the maximal priority is selected to become a new master even when the master in the replication group is alive. Shardman only uses synchronous replicas (otherwise, there is a chance to lose data when a node fails ). The keeper service for which the NODE_NAME part of the systemd instance name matches the hostname of the node it is running on always has higher priority (1) than the other keepers (0).

Since Shardman services are organized as systemd units, their logs are written to journald. You can use journalctl to examine it. For example, to get all logs since 2021-02-09 18:22 for the keeper service clover-1-n1-0 on node n1 of cluster cluster0, you can use the following command:

$  journalctl -u shardman-keeper@cluster0-clover-1-n1-0 --since '2021-02-09 18:22'

To control the log verbosity for all Shardman services, set SDM_LOG_LEVEL in the shardmand configuration file.