support/notes
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

2026-03-12 21:01:38

Browse Source
This commit is contained in:
Valeriu PLESNILA
2026-03-12 22:01:38 +01:00
parent 3bd1db26cc
commit 26296b6d6a
336 changed files with 27507 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
postgresql/draft_01.txt Normal file
View File

@@ -0,0 +1,60 @@
en mode archive-wal
- pg_wal recycle les WAL (et garde la taille <= max_wal_size) tant que l'archivage des WAL peut se faire, sinon il grosssi au delà de max_wal_size
- si pb d'espace dans le FS d'archivage des WAL, le process d'archivage s'arrête mais il reprend si le pb d'espace est résolu
- on peut envoyer les archives à /dev/null
barman
barman list-files aquaris_inst_5501 --target standalone 20240221T194242
=> tous les fichiers + WAL nécessaires à la resturation du backup en môde consistant
barman list-files aquaris_inst_5501 --target full 20240221T194242
=> tous les fichiers + WAL nécessaires à la resturation du backup en môde consistant + les autres WAL streamés depuis
barman list-files aquaris_inst_5501 --target wal 20240221T194242
=> seulement les WAL nécessaires à la resturation du backup en môde consistant + les autres WAL streamés depuis
barman list-backup aquaris_inst_5501
aquaris_inst_5501 20240221T194242 - Wed Feb 21 19:47:27 2024 - Size: 1.2 GiB - WAL Size: 1.9 GiB
=> les 1,9Gib de WAL on été streamé après le backup
si on fait un autre backup:
barman backup aquaris_inst_5501 --wait
barman list-backup aquaris_inst_5501
aquaris_inst_5501 20240221T204618 - Wed Feb 21 20:50:56 2024 - Size: 1.2 GiB - WAL Size: 0 B
aquaris_inst_5501 20240221T194242 - Wed Feb 21 19:47:27 2024 - Size: 1.2 GiB - WAL Size: 1.9 GiB
Si on fait encore 2 backup:
arman list-backup aquaris_inst_5501
aquaris_inst_5501 20240221T205658 - Wed Feb 21 20:57:08 2024 - Size: 1.2 GiB - WAL Size: 0 B
aquaris_inst_5501 20240221T205623 - Wed Feb 21 20:56:32 2024 - Size: 1.2 GiB - WAL Size: 48.0 MiB
aquaris_inst_5501 20240221T204618 - Wed Feb 21 20:50:56 2024 - Size: 1.2 GiB - WAL Size: 32.0 MiB
aquaris_inst_5501 20240221T194242 - Wed Feb 21 19:47:27 2024 - Size: 1.2 GiB - WAL Size: 1.9 GiB - OBSOLETE
=> vu qu'on a une policy RETENSION=3, le premier backup est devenu obsolete
=> 2 minutes plus tard il a été automatiquement purgé par barman cron
barman list-backup aquaris_inst_5501
aquaris_inst_5501 20240221T205658 - Wed Feb 21 20:57:08 2024 - Size: 1.2 GiB - WAL Size: 0 B
aquaris_inst_5501 20240221T205623 - Wed Feb 21 20:56:32 2024 - Size: 1.2 GiB - WAL Size: 48.0 MiB
aquaris_inst_5501 20240221T204618 - Wed Feb 21 20:50:56 2024 - Size: 1.2 GiB - WAL Size: 32.0 MiB
===========
Barman va streamer dans un premier temps les WAL dans le répertiore streaming
Ensuite il les déplace (+compress) dans le répertiore wal
To do:
- aquaris => recréer 2 instances:
archivelog
noarchivelog
exegol:
- add /backup FS => OK
- réinstall barman dans une autre arborescence => OK

91
postgresql/draft_02.txt Normal file
View File

@@ -0,0 +1,91 @@
barman recover \
--remote-ssh-command 'ssh sembla' \
--target-time="2024-02-25 18:07:00" \
aquaris_5501 20240223T180120 \
/data/restore
/app/postgres/product/16.2/bin/pg_ctl \
--pgdata=/data/restore \
-l /tmp/restore.log \
start
/app/postgres/product/16.2/bin/pg_ctl \
--pgdata=/data/restore \
-l /tmp/restore.log \
stop
barman list-backups aquaris_5501
aquaris_5501 20240227T150757 - Tue Feb 27 15:12:54 2024 - Size: 237.0 MiB - WAL Size: 191.7 MiB
aquaris_5501 20240227T145343 - Tue Feb 27 14:58:54 2024 - Size: 239.6 MiB - WAL Size: 242.8 MiB
aquaris_5501 20240227T143931 - Tue Feb 27 14:44:54 2024 - Size: 239.9 MiB - WAL Size: 242.9 MiB
après le premier backup: Tue Feb 27 02:52:33 PM CET 2024
après le 2-ème backup: Tue Feb 27 03:06:47 PM CET 2024
pendant le 3-ème backup: Tue Feb 27 03:09:10 PM CET 2024
après le 3-ème backup: Tue Feb 27 03:20:17 PM CET 2024
arrêt des inserts: Tue Feb 27 03:23:48 PM CET 2024
1. TARGET = Tue Feb 27 03:06:47 PM CET 2024
- on utilise 20240227T145343
- target time: "2024-02-27 15:06:47"
barman recover \
--remote-ssh-command 'ssh sembla' \
--target-time="2024-02-27 15:06:47" \
aquaris_5501 20240227T145343 \
/data/restore
barman recover \
--remote-ssh-command 'ssh sembla' \
--target-time="2024-02-27 15:06:47" \
aquaris_5501 20240227T143931 \
/data/restore
2. TARGET = le 1-er backup
barman \
recover --remote-ssh-command 'ssh sembla' \
aquaris_5501 20240227T143931 \
/data/restore
=> surprise! le recovr s'est fait jusqu'au dernier WAL backupé par BARMAN
barman \
recover --remote-ssh-command 'ssh sembla' \
aquaris_5501 latest \
/data/restore
jawa=> select min(t),max(t) from timeline;
min | max
----------------------------+----------------------------
2024-02-27 14:33:12.373606 | 2024-02-27 15:23:51.508241
(1 row)
Avec la clause "get-wal" il y a eu uhje restauration COMPLETE, WAL current (partiel streamé) inclus
barman \
recover --remote-ssh-command 'ssh sembla' \
--get-wal \
aquaris_5501 latest \
/data/restore
jawa=> select min(t),max(t) from timeline;
min | max
----------------------------+----------------------------
2024-02-27 14:33:12.373606 | 2024-02-27 15:24:19.508331
(1 row)

36
postgresql/pg_install_01.txt Normal file
View File

@@ -0,0 +1,36 @@
dnf install -y gcc.x86_64 make.x86_64 readline.x86_64 readline-devel.x86_64 zlib-devel.x86_64 zlib.x86_64 openssl-devel.x86_64
./configure \
--prefix=/app/postgres/15.3 \
--datarootdir=/data \
--with-ssl=openssl
make
make install
useradd postgres-G postgres -g postgres
useradd postgres -G postgres -g postgres
chown -R postgres:postgres /app/postgres /data /backup
pg_ctl -D /data/postgresql/dbf -l logfile start
# add to .bash_profile
export PS1="\u@\h:\w> "
alias listen='lsof -i -P | grep -i "listen"'
export POSTGRES_HOME=/app/postgres/15.3
export PGDATA=/data/postgresql/dbf
export LD_LIBRARY_PATH=$POSTGRES_HOME/lib:$LD_LIBRARY_PATH
export PATH=$POSTGRES_HOME/bin:$PATH
# init database and startup
initdb
pg_ctl -D /data/postgresql/dbf -l /home/postgres/postgres.log start

136
postgresql/pg_install_suse_01.txt Normal file
View File

@@ -0,0 +1,136 @@
# install packages
zypper install -y gcc
zypper install -y make
zypper install -y automake
zypper install -y readline-devel
zypper install -y zlib-devel
zypper install -y openssl-devel
# compile from sources
mkdir -p /app/kit/postgresql
wget https://ftp.postgresql.org/pub/source/v15.3/postgresql-15.3.tar.gz
cd /app/kit/postgresql/postgresql-15.3
mkdir -p /app/postgres/15.3
./configure \
--prefix=/app/postgres/15.3 \
--datarootdir=/data \
--with-ssl=openssl
make
make install
# create user postres and change owner from binaries, data and backup directories
groupadd postgres
useradd postgres -G postgres -g postgres
# create/opdate .bash_profile for postgres user:
------------------------------------------------------------
alias listen='lsof -i -P | grep -i "listen"'
export POSTGRES_HOME=/app/postgres/15.3
export PGDATA=/data/postgresql/dbf
export LD_LIBRARY_PATH=$POSTGRES_HOME/lib:$LD_LIBRARY_PATH
export PATH=$POSTGRES_HOME/bin:$PATH
------------------------------------------------------------
chown -R postgres:postgres /app/postgres /data /backup
# sinitialize and start PostgreSQL server
mkdir -p $PGDATA
pg_ctl -D /data/postgresql/dbf -l /home/postgresql.log start
# add to .bash_profile
export PS1="\u@\h:\w> "
alias listen='lsof -i -P | grep -i "listen"'
export POSTGRES_HOME=/app/postgres/15.3
export PGDATA=/data/postgresql/dbf
export LD_LIBRARY_PATH=$POSTGRES_HOME/lib:$LD_LIBRARY_PATH
export PATH=$POSTGRES_HOME/bin:$PATH
# init database and startup
initdb
pg_ctl -D /data/postgresql/dbf -l /home/postgres/postgres.log start
# test local connection
psql
\db
# define listening interfaces and ports in $PGDATA/postgresql.conf
listen_addresses = '127.0.0.1,192.168.0.101,192.168.1.101'
port = 5432
# change postgres user password
psql
alter user postgres password 'secret';
# activate password authentification on all interfaces, from any host, to any database, using any user
# add lines in $PGDATA/pg_hba.conf
# TYPE DATABASE USER ADDRESS METHOD
host all all 127.0.0.1/24 md5
host all all 192.168.0.101/24 md5
host all all 192.168.1.101/24 md5
# tst a remote connection:
psql -h aquaris -U postgres
# create systemd service
# it was not possible for me to use environement variable do define the path of pg_ctl binary
cat /usr/lib/systemd/system/postgresql.service
[Unit]
Description=PostgreSQL database server
After=network.target
[Service]
Type=forking
User=postgres
Group=postgres
Environment=PGDATA=/data/postgresql/dbf
Environment=PGLOG=/home/postgres/postgresql.log
ExecStart=/app/postgres/15.3/bin/pg_ctl -D ${PGDATA} -l ${PGLOG} start
ExecStop=/app/postgres/15.3/bin/pg_ctl stop
# Give a reasonable amount of time for the server to start up/shut down.
# Ideally, the timeout for starting PostgreSQL server should be handled more
# nicely by pg_ctl in ExecStart, so keep its timeout smaller than this value.
TimeoutSec=300
[Install]
WantedBy=multi-user.target
# start/stop/status and enable service for automatic startup
systemctl start postgresql
systemctl stop postgresql
systemctl status postgresql
systemctl enable postgresql
# to enable WAL archiving, set following parameters in $PGDATA/postgresql.conf
wal_level = replica
archive_mode = on
archive_command = ''test ! -f /backup/postgresql/wal/%f && cp %p /backup/postgresql/wal/%f''
archive_timeout = 3600 # optional, to force a switch every 1 hour
# https://public.dalibo.com/exports/formation/manuels/modules/i2/i2.handout.html

37
postgresql/pg_online_backup_01.txt Normal file
View File

@@ -0,0 +1,37 @@
# Online backup script using pg_basebackup
##########################################
BACKUP_DIR=/backup/postgresql/pgdata
BACKUP_COMPRESSED_DIR=/backup/postgresql/daily
if [ -z "${BACKUP_DIR}" ]
then
echo "\${BACKUP_DIR} variable is empty"
exit 1
else
rm -rf ${BACKUP_DIR}/*
fi
if [ -z "${BACKUP_COMPRESSED_DIR}" ]
then
echo "\${BACKUP_COMPRESSED_DIR} variable is empty"
exit 1
fi
pg_basebackup -h localhost -P -D ${BACKUP_DIR}
pg_verifybackup ${BACKUP_DIR}
if [ "$?" != "0" ]
then
echo "Verify backup failed"
exit 1
fi
NOW=$(date '+%Y-%m-%d__%H_%M_%S')
cd ${BACKUP_DIR}
tar -cvf - * | pigz > ${BACKUP_COMPRESSED_DIR}/${NOW}.tar.gz
##########################################
# https://public.dalibo.com/exports/formation/manuels/modules/i2/i2.handout.html

71
postgresql/pitr_example_01.txt Normal file
View File

@@ -0,0 +1,71 @@
create table players (id int, about text, age int);
insert into players (id, about, age)
values (generate_series(1, 5000),
repeat('A cool player. ', 2) || 'My number is ' || trunc(random()*1000),
trunc(random()*10 * 2 + 10));
*******************************************
dbaquaris=> select count(*) from players;
5000
dbaquaris=> select current_timestamp;
2023-07-09 17:13:00.860309+02
*******************************************
insert into players (id, about, age)
values (generate_series(1, 100000),
repeat('A cool player. ', 2) || 'My number is ' || trunc(random()*1000),
trunc(random()*10 * 2 + 10));
*******************************************
dbaquaris=> select count(*) from players;
105000
dbaquaris=> select current_timestamp;
2023-07-09 17:36:08.502146+02
*******************************************
insert into players (id, about, age)
values (generate_series(1, 1000000),
repeat('A cool player. ', 2) || 'My number is ' || trunc(random()*1000),
trunc(random()*10 * 2 + 10));
*******************************************
dbaquaris=> select count(*) from players;
1105000
dbaquaris=> select current_timestamp;
2023-07-09 17:37:32.076851+02
*******************************************
# PITR to 2023-07-09 17:36:08
- stop PostgreSQL
- take one of the base backup before ther PITR and put it ion a temporary folder
mkdir /backup/postgresql/tmp
cd /backup/postgresql/tmp
gunzip -c /backup/postgresql/daily/2023-07-09__16_43_59_emptydb.tar.gz | tar -xvf -
- add in modify postgresql.conf
restore_command = 'cp /backup/postgresql/wal/%f %p'
recovery_target_time = '2023-07-09 17:36:08'
recovery_target_inclusive = true
- create recovery.signal file
touch recovery.signal
- start PostgreSQL server with the data in the temporary directory
pg_ctl start -D /backup/postgresql/tmp -l /tmp/reco.log
- check logfile; at the end of the recovery you will be asked to execute the following fonction in order to open the instance
select pg_wal_replay_resume();
- stop PostgreSQL server with the data in the temporary directory
pg_ctl stop -D /backup/postgresql/tmp -l /tmp/reco.log

33
postgresql/postghresql_in_docker_01.txt Normal file
View File

@@ -0,0 +1,33 @@
# pul docker image
docker pull postgres
# create persistent data directory
mkdir -p /app/persistent_docker/postgresql_17/data
# start without docker-compose
docker run -d \
--name postgresql \
-e POSTGRES_PASSWORD=secret \
-e PGDATA=/var/lib/postgresql/data/pgdata \
-v /app/persistent_docker/postgresql_17/data:/var/lib/postgresql/data \
-p 5432:5432 \
postgres
# run psql in interactive mode
docker run -it --rm postgres psql -h kamino -U postgres
# docker-compose.yaml
services:
postgresql:
image: postgres
restart: always
shm_size: 128mb
container_name: postgresql
environment:
- POSTGRES_PASSWORD=secret
- PGDATA=/var/lib/postgresql/data/pgdata
volumes:
- /app/persistent_docker/postgresql_17/data:/var/lib/postgresql/data
ports:
- 5432:5432

365
postgresql/postgres_16_on_Rocky_9_setup_01.txt Normal file
View File

@@ -0,0 +1,365 @@
# create VM with Rocky Linux 9
dd if=/dev/zero of=/vm/ssd0/aquaris/boot_01.img bs=1G count=1
dd if=/dev/zero of=/vm/ssd0/aquaris/root_01.img bs=1G count=8
dd if=/dev/zero of=/vm/ssd0/aquaris/swap_01.img bs=1G count=2
dd if=/dev/zero of=/vm/ssd0/aquaris/app_01.img bs=1G count=8
virt-install \
--graphics vnc,password=secret,listen=0.0.0.0 \
--name=aquaris \
--vcpus=2 \
--memory=4096 \
--network bridge=br0 \
--network bridge=br0 \
--cdrom=/vm/hdd0/_kit_/Rocky-9.3-x86_64-minimal.iso \
--disk /vm/ssd0/aquaris/boot_01.img \
--disk /vm/ssd0/aquaris/root_01.img \
--disk /vm/ssd0/aquaris/swap_01.img \
--disk /vm/ssd0/aquaris/app_01.img \
--os-variant=rocky9
# VM network setup after creation
nmcli connection show
nmcli connection show --active
nmcli connection modify enp1s0 ipv4.address 192.168.0.101/24
nmcli connection modify enp1s0 ipv4.method manual ipv6.method ignore
nmcli connection modify enp1s0 ipv4.gateway 192.168.0.1
nmcli connection modify enp1s0 ipv4.dns 192.168.0.8
nmcli connection modify enp1s0 ipv4.dns-search swgalaxy
nmcli connection modify enp2s0 ipv4.address 192.168.1.101/24 ipv4.method manual ipv6.method ignore
# list host interfaces
hostname -I
# set host name
hostnamectl hostname aquaris.swgalaxy
# install packages
dnf install -y gcc make automake readline-devel zlib-devel openssl-devel libicu-devel.x86_64
dnf install -y zip.x86_64 tar.x86_64 libzip.x86_64 unzip.x86_64 bzip2.x86_64 bzip2-devel.x86_64 pigz.x86_64
dnf install -y wget.x86_64 lsof.x86_64 bind-utils tree.x86_64 python3-devel.x86_64 rsync.x86_64
# add data and backup disks
# on VM get next letter for devices
lsblk
# on Dom0 create and attach the disk to VM
dd if=/dev/zero of=/vm/ssd0/aquaris/data_01.img bs=1G count=8
dd if=/dev/zero of=/vm/ssd0/aquaris/backup_01.img bs=1G count=4
virsh attach-disk aquaris /vm/ssd0/aquaris/data_01.img vde --driver qemu --subdriver raw --targetbus virtio --persistent
virsh attach-disk aquaris /vm/ssd0/aquaris/backup_01.img vdf --driver qemu --subdriver raw --targetbus virtio --persistent
# to list the disk of VM
virsh domblklist aquaris --details
# on VM create partitions, format and mount devices
fdisk /dev/vde
fdisk /dev/vdf
lsblk
pvs
pvcreate /dev/vde1
pvcreate /dev/vdf1
vgs
vgcreate vgdata /dev/vde1
vgcreate vgbackup /dev/vdf1
vgs
lvs
lvcreate -n data -l 100%FREE vgdata
lvcreate -n backup -l 100%FREE vgbackup
lvs
mkfs.xfs /dev/mapper/vgdata-data
mkfs.xfs /dev/mapper/vgbackup-backup
mkdir -p /data /backup
echo "/dev/mapper/vgdata-data /data xfs defaults 1 1" >> /etc/fstab
echo "/dev/mapper/vgbackup-backup /backup xfs defaults 1 1" >> /etc/fstab
systemctl daemon-reload
mount -a
df -hT
# build PostgreSQL from sources
mkdir -p /app/postgres/product/16.2
mkdir -p /app/staging_area
cd /app/staging_area
wget https://ftp.postgresql.org/pub/source/v16.2/postgresql-16.2.tar.gz
gunzip -c postgresql-16.2.tar.gz | tar -xvf -
cd postgresql-16.2
./configure \
--prefix=/app/postgres/product/16.2 \
--with-ssl=openssl
make
make install
# create user postres and change owner from binaries, data and backup directories
groupadd postgres
useradd postgres -G postgres -g postgres
chown -R postgres:postgres /app /data /backup
# create/opdate .bash_profile for postgres user:
export PS1="\u@\h:\w> "
alias listen='lsof -i -P | grep -i "listen"'
alias pgenv='source /app/postgres/admin/scripts/pgenv'
# create PostgreSQL instance on port 5501
mkdir -p /app/postgres/admin
mkdir -p scripts
cd /app/postgres/admin
mkdir -p aquaris_5501/divers
mkdir -p aquaris_5501/log
mkdir -p aquaris_5501/scripts
# create a script to source PostgeSQL instance varaiables
cat <<'EOF' > /app/postgres/admin/scripts/pgenv
export PGPORT=$1
export MYHOST=$(hostname -s)
export PGHOME=/app/postgres/product/16.2
export PGDATA=/data/${MYHOST}_${PGPORT}
export PGBACKUP=/backup/${MYHOST}_${PGPORT}
export PGLOG=/app/postgres/admin/${MYHOST}_${PGPORT}/log/${MYHOST}_${PGPORT}.log
export LD_LIBRARY_PATH=$PGHOME/lib:$LD_LIBRARY_PATH
export PATH=$PGHOME/bin:$PATH
EOF
# sinitialize and start PostgreSQL server
pgenv 5501
initdb -D $PGDATA
# update PostgreSQL instance configuration file
# $PGDATA/postgresql.conf
listen_addresses = '*'
port = 5501
# update $PGDATA/pg_hba.conf in order to allow remote connections using a password
cat <<'EOF' >> $PGDATA/pg_hba.conf
host all all all md5
EOF
# start PostgreSQL instance
pg_ctl start -D $PGDATA -l $PGLOG
or
pg_ctl start --pgdata $PGDATA --log $PGLOG
# stop PostgreSQL instance
pg_ctl stop -m immediate
# create a database + an owner from this database
psql
postgres=# create role jawa login password 'secret';
postgres=# create database jawa;
postgres=# alter database jawa owner to jawa;
postgres=# \l
postgres=# \du
# test connection
psql -p 5501 -h aquaris -U jawa
# create users for barman: barman(superuser) and streaming_barman(replication)
createuser --superuser --replication -P barman
createuser --replication -P streaming_barman
# update $PGDATA/pg_hba.conf in order to allow replication for streaming_barman user
cat <<'EOF' >> $PGDATA/pg_hba.conf
host replication streaming_barman all md5
EOF
# ensure that following parameter are >10
postgres=# Show max_wal_senders;
postgres=# Show max_replication_slots;
# otherwise update
postgres=# ALTER SYSTEM SET max_wal_senders = 10;
postgres=# ALTER SYSTEM SET max_replication_slots = 10;.
# Barman can be installed on a remote machine where PosgeSQL binaries are installed
# customoze ~/.bashrc on remote machine
cat <<'EOF' >> ~/.bashrc
export PS1="\u@\h:\w> "
alias listen='lsof -i -P | grep -i "listen"'
export POSTGRES_HOME=/app/postgres/product/16.2
export LD_LIBRARY_PATH=$POSTGRES_HOME/lib:$LD_LIBRARY_PATH
export PATH=$POSTGRES_HOME/bin:$PATH
EOF
# barman install
mkdir /backup/barman
mkdir /app/barman
cd /app/barman
mkdir product conf log run scripts
mkdir conf/barman.d
mkdir /app/barman/product/barman_3.10.0
python -m venv /app/barman/product/barman_3.10.0
source /app/barman/product/barman_3.10.0/bin/activate
python -m pip install --upgrade pip
pip install psycopg2
pip install barman
barman -v
# optinally, activate Barman in .bash_profile
cat <<'EOF' >> ~/.bash_profile
# Activate Barman
source /app/barman/product/barman_3.10.0/bin/activate
EOF
# store passwords
cat <<'EOF' >>~/.pgpass
aquaris:5501:*:barman:secret
aquaris:5501:*:streaming_barman:secret
EOF
chmod 0600 ~/.pgpass
# test connection
psql -h aquaris -p 5501 -U barman -d postgres
psql -h aquaris -p 5501 -U streaming_barman -d postgres
# create barman global configuration file
cat <<'EOF' > /app/barman/conf/barman.conf
[barman]
; System user
barman_user = postgres
; Directory of configuration files. Place your sections in separate files with .conf extension
; For example place the 'main' server section in /etc/barman.d/main.conf
configuration_files_directory = /app/barman/conf/barman.d
; Main directory
barman_home = /backup/barman
; Locks directory - default: %(barman_home)s
;barman_lock_directory = /app/barman/run
; Log location
log_file = /app/barman/log/barman.log
; Log level (see https://docs.python.org/3/library/logging.html#levels)
log_level = INFO
; Default compression level: possible values are None (default), bzip2, gzip, pigz, pygzip or pybzip2
compression = pigz
EOF
# for the global configuration file, create a symlync .barman.conf in the home directory
ln -s /app/barman/conf/barman.conf ~/.barman.conf
# target postgres instance example
cat <<'EOF' > /app/barman/conf/barman.d/aquaris_5501.conf
[aquaris_5501]
description = "PostgreSQL instance on aquaris, port 5501"
conninfo = host=aquaris port=5501 user=barman dbname=postgres
streaming_conninfo = host=aquaris port=5501 user=streaming_barman dbname=postgres
backup_method = postgres
streaming_archiver = on
slot_name = barman
create_slot = auto
retention_policy = REDUNDANCY 4
EOF
# create replication slot
barman receive-wal --create-slot aquaris_5501
# create barman CRON script
cat <<'EOF' > /app/barman/scripts/barman_cron
# Setup environement
source ~/.bash_profile
# Run barmab CRON tasks
barman cron
EOF
chmod +x /app/barman/scripts/barman_cron
# scedule CRON script in crontab every 1 minute
crontab -l
* * * * * /app/barman/scripts/barman_cron > /app/barman/log/barman_cron.log
# force a switch wal on target PostgreSQL instance
barman switch-wal --force --archive aquaris_5501
# backup PostgreSQL instance and wait for all the required WAL files to be archived
barman backup aquaris_5501 --wait
# list registered PostgeSQL servers
barman list-servers
# list backup of one of all servers
barman list-backups all
barman list-backups aquaris_5501
# list files of backup required to restore a minimal consistent image
# one or more WAL will be included
barman list-files aquaris_5501 --target standalone 20240223T165330
# list only WAL that can be used in addition with the backup
# that will include necessary WAL to restore a consistent image (as in previous command) + all streamed (and automatically compressed) since the backup
barman list-files aquaris_5501 --target wal 20240223T165330
# list all files (base + WAL) availablle for restiore since the backup
barman list-files aquaris_5501 --target full 20240223T165330
# show backup informations
barman show-backup aquaris_5501 20240223T165330
# verify cecksums of backup files
barman verify-backup aquaris_5501 20240220T174149
***************************************************
barman \
recover --remote-ssh-command 'ssh sembla' \
aquaris_5501 latest \
/data/restore
barman \
recover --remote-ssh-command 'ssh sembla' \
--get-wal \
aquaris_5501 latest \
/data/restore
/app/postgres/product/16.2/bin/pg_ctl \
--pgdata=/data/restore \
-l /tmp/restore.log \
start
/app/postgres/product/16.2/bin/pg_ctl \
--pgdata=/data/restore \
-l /tmp/restore.log \
stop
barman-wal-restore -U postgres exegol aquaris_5501 --test Bugs Bunny
**************************************************

24
postgresql/postgres_17_compile_01.txt Normal file
View File

@@ -0,0 +1,24 @@
dnf install bison.x86_64
dnf install flex.x86_64
rpm -qa | grep -i bison
bison-3.7.4-5.el9.x86_64
rpm -qa | grep -i flex
flex-2.6.4-9.el9.x86_64
dnf install perl-FindBin.noarch
rpm -qa | grep -i FindBin
perl-FindBin-1.51-481.el9.noarch
dnf install perl-lib.x86_64
rpm -qa | grep -i perl-lib-
perl-lib-0.65-481.el9.x86_64
# single command
dnf install -y bison.x86_64 flex.x86_64 perl-lib.x86_64 perl-FindBin.noarch
./configure \
--prefix=/app/postgres/rdbms/17.2 \
--with-ssl=openssl

305
postgresql/postgres_TLS_01.md Normal file
View File

@@ -0,0 +1,305 @@
## Generate PostgreSQL server certificate
PostgreSQL server generate **certificate private key** and a **certificate request** for the `CN = PostgreSQL server`
```bash
openssl genrsa -out postgres_server.key 4096
openssl req -new -key postgres_server.key -out postgres_server.csr
```
The CA root put the **certificate request** in a temporary location (ex. `generated` directory), generate a **signed certificate** and optionally create a **certificate full chain**.
Configuration file used by root CA:
```
[ req ]
default_bits = 4096
prompt = no
default_md = sha256
distinguished_name = dn
req_extensions = req_ext
[ dn ]
CN = PostgreSQL server
[ req_ext ]
subjectAltName = @alt_names
[ alt_names ]
DNS.1 = raxus.swgalaxy
DNS.2 = mobus.swgalaxy
DNS.3 = pgsql.swgalaxy
```
> **_NOTE:_** `CN` in CA configuration file can be diffrent than the `CN` of CSR. The CA does not replace it unless explicitly configured to do so.
```bash
openssl x509 -req \
-in generated/postgres_server.csr \
-CA rootCA.pem -CAkey rootCA.key -CAserial rootCA.srl \
-out generated/postgres_server.crt \
-days 3650 \
-sha256 \
-extensions req_ext -extfile generated/postgres_server.cnf
cat generated/postgres_server.crt rootCA.pem > generated/postgres_server.fullchain.crt
```
To inspect a certificate in Linux command line:
```bash
openssl x509 -in rootCA.pem -text -noout
openssl x509 -in generated/postgres_server.crt -text -noout
```
## Generate PostgreSQL client certificate(s)
We will generate 2 certificates for:
- `CN=PostgreSQL client1`
- `CN=PostgreSQL client2`
```bash
openssl genrsa -out postgres_client1.key 4096
openssl req -new -key postgres_client1.key -out postgres_client1.csr
openssl genrsa -out postgres_client2.key 4096
openssl req -new -key postgres_client2.key -out postgres_client2.csr
```
Configuration file used by root CA will be the same for both certificates:
```
[ req ]
default_bits = 4096
prompt = no
default_md = sha256
distinguished_name = dn
req_extensions = req_ext
[ dn ]
CN = Generic Client PostgreSQL
[ req_ext ]
subjectAltName = @alt_names
[ alt_names ]
DNS.1 = anyhost.anydomain
```
```bash
openssl x509 -req \
-in generated/postgres_client1.csr \
-CA rootCA.pem -CAkey rootCA.key -CAserial rootCA.srl \
-out generated/postgres_client1.crt \
-days 3650 \
-sha256 \
-extensions req_ext -extfile generated/postgres_client.cnf
cat generated/postgres_client1.crt rootCA.pem > generated/postgres_client1.fullchain.crt
openssl x509 -req \
-in generated/postgres_client2.csr \
-CA rootCA.pem -CAkey rootCA.key -CAserial rootCA.srl \
-out generated/postgres_client2.crt \
-days 3650 \
-sha256 \
-extensions req_ext -extfile generated/postgres_client.cnf
cat generated/postgres_client2.crt rootCA.pem > generated/postgres_client2.fullchain.crt
```
## On PostgreSQL server, add the root CA certificate as trusted certificate.
Put root CA certificate (`.crt` ou `.pem`)under `/etc/pki/ca-trust/source/anchors`, then update the system trust store.
```bash
cd /etc/pki/ca-trust/source/anchors
chmod 644 rootCA.pem
update-ca-trust extract
```
```bash
openssl verify -CAfile /etc/pki/tls/certs/ca-bundle.crt <path_to>/postgres_server.crt
# Inspect the root CA certificate to get the exact CN
grep -R "<CN_name>" /etc/pki/ca-trust/extracted/
```
## Set up SSL on PostgreSQL server
Place server certificate, server private key and root CA certificate (optional but recommended) in the PostgreSQL data directory.
Default paths:
- `$PGDATA/server.key`
- `$PGDATA/server.crt`
Or override with:
```
ssl_cert_file = '/path/to/server.crt'
ssl_key_file = '/path/to/server.key'
ssl_ca_file = '/path/to/root.crt'
```
Set file pemisions:
```bash
chmod 640 $PGDATA/postgres_server.crt
chmod 600 $PGDATA/postgres_server.key
chmod 640 $PGDATA/rootCA.pem
```
> root CA certificate is required **only for mTLS** mode when the server will validate the client certificate authenticity
> Add (concatebate) all CA intermediate (if any) in `ssl_cert_file`
Enable TLS in `$PGDATA/postgresql.conf`.
```
ssl_cert_file = 'postgres_server.crt'
ssl_key_file = 'postgres_server.key'
ssl_ca_file = 'rootCA.pem'
ssl = on
ssl_ciphers = 'HIGH:!aNULL:!MD5'
ssl_prefer_server_ciphers = on
ssl_min_protocol_version = 'TLSv1.2'
ssl_max_protocol_version = 'TLSv1.3'
```
PostgreSQL will now listen for both encrypted and unencrypted connections on the **same port**.
## Server side SSL modes
### Request TLS mode
Config in `$PGDATA/pg_hba.conf`:
```
host all all all md5
```
### Require TLS mode
Config in `$PGDATA/pg_hba.conf`:
```
hostssl all all 0.0.0.0/0 md5
```
### Require TLS mode + client certificate
Config in `$PGDATA/pg_hba.conf`:
```
hostssl all all 0.0.0.0/0 cert
```
## Client side SSL modes
| Mode | Encrypts | Validates CA | Validates Hostname | Typical Use |
| --- | --- | --- | --- | --- |
| `require` | Yes | No | No | Basic encryption |
| `verify-ca` | Yes | Yes | No | Internal/IP-based |
| `verify-full` | Yes | Yes | Yes | Production |
Examples:
```bash
# mode: require
psql "host=raxus.swgalaxy port=5501 user=vplesnila dbname=postgres sslmode=require"
# mode: verify-ca
psql "host=raxus.swgalaxy port=5501 user=vplesnila dbname=postgres sslmode=verify-ca sslrootcert=rootCA.pem"
```
For `verify-full` mode the client need client certificate, client private key and root CA certificate on client side.
In our example we will use previously generated certificats for:
- `CN=PostgreSQL client1`
- `CN=PostgreSQL client2`
Set file pemisions:
```bash
chmod 640 postgres_client1.crt
chmod 600 postgres_client1.key
chmod 640 postgres_client2.crt
chmod 600 postgres_client2.key
chmod 640 rootCA.pem
```
In `verify-full` mode we can get ride of client password by mapping the `CN` in the certificate to a **local PostgreSQL role** (aka **local user**).
Create local roles (with no password) in PostgreSQL instance:
```sql
CREATE ROLE app1 LOGIN;
CREATE ROLE app2 LOGIN;
```
Add in `$PGDATA/pg_ident.conf`:
```
# MAPNAME SYSTEM-IDENTITY PG-USERNAME
certmap_app1 "PostgreSQL client1" app1
certmap_app2 "PostgreSQL client2" app2
```
Add in `$PGDATA/pg_hba.conf`:
```
hostssl all app1 0.0.0.0/0 cert map=certmap_app1
hostssl all app2 0.0.0.0/0 cert map=certmap_app2
```
> Restart PostgreSQL instance after modifying `$PGDATA/pg_ident.conf` and `$PGDATA/pg_hba.conf`.
Connect in `verify-full` mode using certificate for authentification:
```bash
# mode: verify-full
psql "host=raxus.swgalaxy port=5501 user=app1 dbname=postgres sslmode=verify-full sslrootcert=rootCA.pem sslcert=postgres_client1.crt sslkey=postgres_client1.key"
psql "host=raxus.swgalaxy port=5501 user=app2 dbname=postgres sslmode=verify-full sslrootcert=rootCA.pem sslcert=postgres_client2.crt sslkey=postgres_client2.key"
```
As `SAN` **(Subject Alternative Name)** of the **server** certificate match to:
- `raxus.swgalaxy`
- `mobus.swgalaxy`
- `pgsql.swgalaxy`
it is possible to connect to all theses DNS enteries using the same **server** certificate.
Example:
```bash
psql "host=pgsql.swgalaxy port=5501 user=app1 dbname=postgres sslmode=verify-full sslrootcert=rootCA.pem sslcert=postgres_client1.crt sslkey=postgres_client1.key"
```
> ⭐ Server certificates → client checks SAN/CN for hostname
> ⭐ Client certificates → server checks only CN for user identity
> **IMPORTANT**: we can mix TLS authentification by certificate and password.
PostgreSQL processes pg_hba.conf top‑to‑bottom, and the first matching rule wins.
Once a connection matches a line (based on type, database, user, address, etc.), PostgreSQL stops and applies that rule’s authentication method.
The folowing `$PGDATA/pg_hba.conf` allows authentification using certificates for PostgreSQL users app1 and app2 and authentification using password for all other users.
```
hostssl all app1 0.0.0.0/0 cert map=certmap_app1
hostssl all app2 0.0.0.0/0 cert map=certmap_app2
hostssl all all 0.0.0.0/0 md5
```
Check if connections are using TLS:
```sql
SELECT * FROM pg_stat_ssl;
```
Check for connection username:
```sql
select pid as process_id,
usename as username,
datname as database_name,
client_addr as client_address,
application_name,
backend_start,
state,
state_change
from pg_stat_activity;
```

32
postgresql/usefull_commands_01.txt Normal file
View File

@@ -0,0 +1,32 @@
# create user
create user wombat password 'secret';
# create database
create database dbaquaris;
# grant ALL privileges on a database to a user
grant all on database dbaquaris to wombat;
\c dbaquaris
grant all on schema public to wombat;
# connect
psql -h aquaris -U wombat -d dbaquaris;
# create schema
create schema bank;
# WAL archive status
select * from pg_stat_archiver;
select * from pg_ls_dir ('pg_wal/archive_status') ORDER BY 1;
# switch current WAL
select pg_switch_wal();
# show data directory
show data_directory;
# list databases with details
\l+