pg_probackup3 supports S3 interface for storing backups. Backup data is transferred to and from S3 without saving it in intermediate locations thus eliminating the need of having a large temporary storage.
S3 is available only when pg_probackup3 is used with Postgres Pro Enterprise.
If you are going to use pg_probackup3 with S3 interface, complete the following steps:
Create a bucket with a unique and meaningful name in the S3 storage for you future backups.
Create ACCESS_KEY and SECRET_ACCESS_KEY tokens to be used for secure connectivity instead of your username and password.
For communication between pg_probackup3 and S3 server, set values of environment variables corresponding to your S3 server. For example:
export PG_PROBACKUP_S3_HOST=127.0.0.1 export PG_PROBACKUP_S3_PORT=9000 export PG_PROBACKUP_S3_REGION=ru-msk export PG_PROBACKUP_S3_BUCKET_NAME=test1 export PG_PROBACKUP_S3_ACCESS_KEY=admin export PG_PROBACKUP_S3_SECRET_ACCESS_KEY=password export PG_PROBACKUP_S3_HTTPS=ON
Alternatively, you can provide S3 server settings in the
configuration file or by using the command-line options. For more
details, refer to the --config-file option in the
section Common Options
and to the section S3 Options.
It makes sense to specify S3 server settings if
--s3=minio, as described in the
section S3 Options.
The following environment variables can be specified:
PG_PROBACKUP_S3_HOST
Address of the S3 server. Can include the port number, separated by
a colon. If the port number is not specified in a host string, the
value of PG_PROBACKUP_S3_PORT is assumed. Do not add
a colon if the port number is not specified.
For example:
export PG_PROBACKUP_S3_PORT=80 export PG_PROBACKUP_S3_HOST="127.0.0.1:9000"
In this example, for the “127.0.0.1” address, the
port 9000 is explicitly specified and will be used instead of
value 80, specified through PG_PROBACKUP_S3_PORT.
PG_PROBACKUP_S3_PORTThe port of the S3 server.
PG_PROBACKUP_S3_REGION
The region of the S3 server. The default value is us-east-1.
PG_PROBACKUP_S3_BUCKET_NAMEThe name of the bucket on the S3 server.
PG_PROBACKUP_S3_ACCESS_KEYPG_PROBACKUP_S3_SECRET_ACCESS_KEYSecure tokens on the S3 server.
PG_PROBACKUP_S3_HTTPSThe protocol to be used. Possible values:
ON or HTTPS — use HTTPS
Other than ON or HTTPS — use HTTP
PG_PROBACKUP_S3_BUFFER_SIZE
The size of the read/write buffer for communicating with S3, in MiB.
The default is 16.
PG_PROBACKUP_S3_RETRIES
The maximum number of attempts to execute an S3 request
in case of failures. The default is 3.
PG_PROBACKUP_S3_TIMEOUT
The maximum amount of time to execute an HTTP request to the S3
server, in seconds. The default is 300.
PG_PROBACKUP_S3_IGNORE_CERT_VER
Don't verify the certificate host and peer.
The default is OFF.
PG_PROBACKUP_S3_CA_CERTIFICATESpecify the path to file with trust Certificate Authority (CA) bundle.
PG_PROBACKUP_S3_CA_PATHSpecify the directory with trust CA certificates.
PG_PROBACKUP_S3_CLIENT_CERTSetup SSL client certificate.
PG_PROBACKUP_S3_CLIENT_KEYSetup private key file for TLS and SSL client certificate.
The following minimal permissions to the target S3 bucket should be granted for the access key used by pg_probackup3 with versioning disabled:
for the init command:
s3:GetBucketVersioning s3:ListBucket
for the add-instance, set-config, and set-backup commands:
s3:GetBucketVersioning s3:ListBucket s3:PutObject
for the del-instance command:
s3:GetBucketVersioning s3:ListBucket s3:DeleteObject
for the backup, archive-push, and file-map commands:
s3:ListBucket s3:PutObject s3:GetBucketVersioning s3:AbortMultipartUpload s3:GetObject
for the restore, fuse, show, show-config, and send-backup commands:
s3:GetBucketVersioning s3:GetObject s3:ListBucket
for the validate command:
s3:GetBucketVersioning s3:GetObject s3:ListBucket s3:PutObject
for the merge and retention commands:
s3:ListBucket s3:PutObject s3:GetBucketVersioning s3:AbortMultipartUpload s3:GetObject s3:DeleteObject
for the delete command:
s3:GetBucketVersioning s3:GetObject s3:ListBucket s3:DeleteObject
When versioning is enabled, the following additional permissions are required:
s3:ListBucketVersions for all commands that require
s3:ListBucket
s3:DeleteObjectVersion for all commands that require
s3:DeleteObject