## 18.11. RDMA Connections

Postgres Pro Enterprise provides support for client/server connections that use remote direct memory access (RDMA) technology. You must have an RDMA implementation set up on both client and server systems.

With RDMA, data can be sent directly into the memory of the remote system, bypassing the operating system kernel. As a result, this reduces the CPU load, ensures low network latency in distributed systems, and yields better performance.

### 18.11.1. Enabling Postgres Pro Enterprise Client/Server RDMA Connections

For client/server RDMA connections, Postgres Pro Enterprise uses libpq with `rsocket` API. To enable RDMA connections for your Postgres Pro Enterprise database via `rsocket`, you must configure both client and server systems, as follows:

1. On the server side, add the listen_rdma_addresses GUC variable to the `postgresql.conf` configuration file to specify TCP/IP address(es) on which the server is to listen for new RDMA connections via `rsocket` from client applications. For example:

       listen_rdma_addresses = 'server1,172.17.3.21'

   You can specify a comma-separated list of host names and/or numeric IP addresses. If set to `*`, this variable will enable RDMA connections via `rsocket` to all the available IP interfaces.

   ### Important

   Since all connections use the same port specified in the `port` variable, make sure to specify different IP addresses in `listen_addresses` and `listen_rdma_addresses` to avoid conflicts.

2. On the client side, modify the libpq connection parameters to use `rsocket` API. This can be done in one of the following ways:

   - Add the following `rsocket` settings to the libpq connection service file `pg_service.conf`:

         # rsocket configuration
         [rsocket]
         host=hostname
         port=5433
         user=username
         with_rsocket=true

     When the client is connecting to the server via libpq, the name of the service provided in square brackets in the `pg_service.conf` file must be specified in the `service` parameter keyword, or in the corresponding `PGSERVICE` environment variable. This approach enables you to dynamically control client/server connection type.

   - Alternatively, you can set the `WITH_RSOCKET` environment variable:

         export WITH_RSOCKET=true

     In this case, all the client/server connections will use `rsocket` API.

### 18.11.2. Using RDMA with pg_dump

To connect pg_dump to the server over RDMA via `rsocket`:

1. Set up the server as described in Section 18.11.1.

2. Set the `WITH_RSOCKET` environment variable:

       export WITH_RSOCKET=true

Once the setup is complete, launch pg_dump as usual. For example:

    pg_dump dbname -h hostname > db.sql

### 18.11.3. Using RDMA for Master-Standby Replication

To enable master-standby replication over RDMA connections via `rsocket`, you need to configure master and standby nodes. For example, to allow client connections to the server 172.17.3.21 from 172.17.3.0/24 addresses on behalf of the `postgres` user, follow the steps below:

To set up the master node for replication over RDMA connections via `rsocket`:

1. Edit the `pg_hba.conf` file to allow client connections and replication from the specified IP addresses:

       host    all            postgres     172.17.3.0/24         md5
       host    replication    postgres     172.17.3.0/24         md5

2. Edit the `postgresql.conf` file:

       listen_rdma_addresses = '172.17.3.21'
       wal_level = hot_standby
       hot_standby = on
       max_wal_senders = 1

   The `listen_rdma_addresses` variable specifies the TCP/IP address(es) on which the server is to listen for new RDMA connections via `rsocket`.

3. Restart the master node.

### Tip

If you set `log_connections` and `log_disconnections` GUC variables, the log file will include detailed information on each client connection and disconnection, respectively. For example:

    LOG:  connection received: host=172.17.3.22 port=37709 with_rsocket=true
    LOG:  disconnection: session time: 0:00:00.622 user=postgres database= host=172.17.3.22 port=37709 with_rsocket=true

To set up the standby node:

1. Set the `WITH_RSOCKET` environment variable:

       export WITH_RSOCKET=true

2. Copy the data from the master node using pg_basebackup:

       pg_basebackup -D datadir -x -R -h 172.17.3.21 -U postgres

   All the data appears on the standby node under the specified `datadir` directory.

3. Make sure the ``datadir`/recovery.conf` contains the `with_rsocket` parameter.

       standby_mode = 'on'
       primary_conninfo = 'user=postgres host=172.17.3.21 port=5432 with_rsocket=true'

4. In the `postgresql.conf` file, clear the `listen_rdma_addresses` parameter:

       listen_rdma_addresses = ''

Once the setup is complete, start the standby node. The streaming replication is now performed over the RDMA connections using `rsocket` API.