This section describes how to configure secure communications between the etcd store and Shardman services and tools.
etcd is a critical component for a Shardman cluster. If an intruder gets access to the etcd store, it gains full control over the whole cluster, including DBMS access with DBA privileges. To protect your cluster, it is recommended that you configure TLS authentication between etcd daemons and Shardman services.
To this end, you can use HTTPS transport with certificates signed by your local certification authority (CA) to encrypt traffic between the etcd cluster and Shardman services and restrict etcd access. To do this, perform the steps described in the next sections.
To generate SSL certificates, perform the following steps:
If the CA does not exist, generate a self-signed root certificate. Generate all certificates on one trusted host. Here certificates that expire in 10000 days are generated (you can choose a more suitable interval):
#openssl genrsa -out rootCA.key 4096#openssl req -x509 -new -key rootCA.key -days 10000 -out rootCA.crt
Prepare the following openssl configuration file for each etcd host:
[ req ] default_bits = 4096 distinguished_name = req_distinguished_name req_extensions = req_ext [ req_distinguished_name ] countryName = Country Name (2 letter code) stateOrProvinceName = State or Province Name (full name) localityName = Locality Name (eg, city) organizationName = Organization Name (eg, company) commonName = Common Name (e.g. server FQDN or YOUR name) [ req_ext ] subjectAltName = @alt_names [alt_names] DNS.1 = n1 IP.1 = 192.168.1.1 IP.2 = 127.0.0.1
Under [alt_names],
specify alternative subject names for the etcd host.
These names must include the etcd server hostname, IP
address and local IP. Including the local IP is convenient rather than required.
Save the file. For example, the names of configuration files for nodes
n1 — n3 can be n1.san.conf
— n3.san.conf.
Using the configuration files prepared, generate private keys and certificate requests for etcd hosts:
#openssl genrsa -out n1.etcd.key 4096#openssl req -config n1.san.conf -new -key n1.etcd.key -out n1.etcd.csr -subj "/C=RU/ST=Moscow Region/L=Moscow/O=Test/CN=n1"
Here “
/C=RU/ST=Moscow Region/L=Moscow/O=Test/CN=n1
”
means that the certificate request is generated with the country name
RU, state Moscow Region, locality Moscow,
organization Test and common name n1. The
common name must match the DNS name of your etcd
server.
Sign the certification request:
#openssl x509 -extfile san.conf -extensions req_ext -req -in n1.etcd.csr -CA rootCA.crt -CAkey rootCA.key -CAcreateserial -out n1.etcd.crt -days 10000
Check the certificates to ensure they contain correct X509v3 Subject
Alternative Name fields. The fields must contain the list of DNS names
and IP addresses that you added to the openssl configuration file:
#openssl x509 -in n1.etcd.crt -noout -text
Generate client certificates for Shardman services
and client tools. These certificates do not need to contain the
subjectAltName header, and CN is not important in
the example below. It generates one common certificate-key pair for services and
one — for tools:
#openssl x509 genrsa -out shardman_services.key 4096#openssl req -new -key shardman_services.key -out shardman_services.csr -subj "/C=RU/ST=Moscow Region/L=Moscow/O=Test/CN=shardman_services"#openssl x509 -req -in shardman_services.csr -CA rootCA.crt -CAkey rootCA.key -CAcreateserial -out shardman_services.crt -days 10000#openssl x509 genrsa -out shardman_tools.key 4096#openssl req -new -key shardman_tools.key -out shardman_tools.csr -subj "/C=RU/ST=Moscow Region/L=Moscow/O=Test/CN=shardman_tools"#openssl x509 -req -in shardman_tools.csr -CA rootCA.crt -CAkey rootCA.key -CAcreateserial -out shardman_tools.crt -days 10000
Now configure services ( etcd and shardmand ) to use the generated certificates. To do this, perform the following steps:
On each etcd node, put rootCA.crt,
nX.etcd.crt and nX.etcd.key in the
location accessible to the etcd daemon (for example,
create /etc/etcd directory and put files there). Ensure
that the nX.etcd.key file is only accessible to the
etcd daemon user.
Specify the following configuration for etcd
daemons in /etc/default/etcd:
# unqualified first name ETCD_NAME=n1 # where we actually listen for peers ETCD_LISTEN_PEER_URLS=https://0.0.0.0:2380 # where we actually listen for clients ETCD_LISTEN_CLIENT_URLS=https://0.0.0.0:2379 # advertise where this machine is listening for clients ETCD_ADVERTISE_CLIENT_URLS=https://n1:2379 # --initial flags are used during bootstrapping and ignored afterwards, so it is # ok to specify them always # advertise where this machine is listening for peer ETCD_INITIAL_ADVERTISE_PEER_URLS=https://n1:2380 ETCD_INITIAL_CLUSTER_TOKEN=etcd-cluster # ansible_nodename is fqdn ETCD_INITIAL_CLUSTER=n1=https://n1:2380,n2=https://n2:2380,n3=https://n3:2380 ETCD_INITIAL_CLUSTER_STATE=new ETCD_DATA_DIR=/var/lib/etcd/default/member ETCD_AUTO_COMPACTION_RETENTION=1 ETCD_KEY_FILE=/etc/etcd/n1.etcd.key ETCD_CERT_FILE=/etc/etcd/n1.etcd.crt ETCD_TRUSTED_CA_FILE=/etc/etcd/rootCA.crt ETCD_CLIENT_CERT_AUTH=true ETCD_PEER_CERT_FILE=/etc/etcd/n1.etcd.crt ETCD_PEER_KEY_FILE=/etc/etcd/n1.etcd.key ETCD_PEER_TRUSTED_CA_FILE=/etc/etcd/rootCA.crt ETCD_PEER_CLIENT_CERT_AUTH=true
Replace n1 here with the appropriate node name.
Restart etcd services on all etcd cluster nodes:
#systemctl restart etcd
To check the new configuration, use the following command:
#etcdctl --endpoints=https://n1:2379,https://n2:2379,https://n3:2379 --cacert /etc/etcd/rootCA.crt --cert /etc/etcd/n1.etcd.crt --key /etc/etcd/n1.etcd.key member list -w table+------------------+---------+------+-----------------+-----------------+------------+ | ID | STATUS | NAME | PEER ADDRS | CLIENT ADDRS | IS LEARNER | +------------------+---------+------+-----------------+-----------------+------------+ | 66ebe06d7302c3f0 | started | n2 | https://n2:2380 | https://n2:2379 | false | | b1080bf5ff059980 | started | n1 | https://n1:2380 | https://n1:2379 | false | | d98323257249fefb | started | n3 | https://n3:2380 | https://n3:2379 | false | +------------------+---------+------+-----------------+-----------------+------------+
On each Shardman cluster node, put
rootCA.crt, shardman_services.crt and
shardman_services.key in a location accessible to the
postgres user (for example, create the /etc/shardman
directory and put files there). Ensure that the shardman_services.key
file is only accessible to the postgres user.
Edit the
shardmand
configuration file
/etc/shardman/shardmand-cluster0.env as follows:
SDM_STORE_ENDPOINTS=https://n1:2379,https://n2:2379,https://n3:2379 SDM_STORE_CA_FILE=/etc/shardman/rootCA.crt SDM_STORE_CERT_FILE=/etc/shardman/shardman_services.crt SDM_STORE_KEY=/etc/shardman/shardman_services.key
Restart shardmand@cluster0 services on all
Shardman nodes:
#systemctl restart shardmand@cluster0
Before using Shardman tools, copy rootCA.crt,
shardman_tools.crt and shardman_tools.key to
some location on the Shardman management node where they are
accessible to the management user. Here, any node with installed Shardman
utilities that is used to manage the Shardman cluster is
meant by management node. This can also be one of the
Shardman cluster nodes (or all of them). By management user,
a user is meant who runs
shardmanctl
tool. It is assumed that the
certificates and key are located in the /etc/shardman directory.
When using Shardman tools, be sure to add
--store-ca-file, --store-cert-file and --store-key
options to shardmanctl command. For example, the following command
gets the cluster status:
#shardmanctl --store-ca-file /etc/shardman/rootCA.crt --store-cert-file /etc/shardman/shardman_tools.crt --store-key /etc/shardman/shardman_tools.key --store-endpoints https://n1:2379,https://n2:2379,https://n3:2379 status