Barman Installation and configuration
Hi Friends,
In this block we will get information regarding installation and configuration of 2nd Quadrant Open source tool barman. Barman is distributed under GNU GPL 3
Barman stands for backup and recovery manager, it is one of the advanced python based tools which can take the Full and Incremental backup of PostgreSQL database remotely.
At the end of this block you will be able to Install, take backup and perform recovery of database.
Step 1 : Install PostgreSQL-9.4 or higher.
Step 2 : Download pgbarman from below link.
https://excellmedia.dl.sourceforge.net/project/pgbarman/2.1/barman-2.1.tar.gz
Step3: Install prerequisite before installing pgbarman below are necessary prerequisite.
OS: Unix based.
pre-requisites:
python 2.6 or 2.7
Python modules:
argcomplete
argh >= 0.21.2
psycopg2
python-dateutil < 2.0 (since version 2.0 requires python3)
distribute (optional)
How to install?
https://yum.postgresql.org/9.6/redhat/rhel-7-x86_64/
rpm -ivh http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
yum install python-pip // sudo easy_install pip
pip install package name here
Step4: untar the pgbarman file and install it as given below
untar barman:
python2.6 setup.py build
python2.6 setup.py install
Step5: copy barman.conf from doc to /etc/
create user:
useradd barman -p barman
cp doc/barman.conf /etc/
chown -R barman:barman /etc/barman.conf
Step 6: Make password less authentication between barman server and postgres server .I have
created on same machine so I did password less authentication.
--ssh-keygen between barman and postgres:
[root@localhost barman-2.1]# su - barman
[barman@localhost ~]$ ssh-keygen
[barman@localhost ~]$ ssh-copy-id -i .ssh/id_rsa.pub postgres@127.0.0.1
Repeated above steps from postgres user
[root@localhost barman-1.5.1]# su - postgres
[postgres@localhost ~]$ ssh-keygen
[postgres@localhost ~]$ ssh-copy-id -i .ssh/id_rsa.pub barman@127.0.0.1
Step 7: Edit barman.conf:
[barman]
barman_home = /home/barman
barman_user = barman
log_file = /home/barman/barman.log
compression = gzip
reuse_backup = link
minimum_redundancy = 1
[main]
description = "Main DB Server"
ssh_command = ssh postgres@127.0.0.1
conninfo = host=127.0.0.1 user=postgres
archiver = on
Step 8: Edit the postgresql.conf and enable the archiving .
There is one last configuration to be made on the main, to switch on backup (or archive) mode.First,
we need to locate the value of the incoming backup directory from the barman, switch to the user
barman:
su - barman
Run this command to locate the incoming backup directory:
barman show-server main | grep incoming_wals_directory
incoming_wals_directory: /home/barman/main/incoming
Note down the value of incoming_wals_directory; in my setup it's /home/barman/main/incoming
Now switch to the user postgres on postgres server.
Open the postgresql.conf and make the following changes to the file:
Uncomment the wal_level parameter and set its value to archive.
Uncomment the archive_mode parameter and set its value to on.
Uncomment the archive_command parameter and set its value to 'rsync -a %p barman@127.0.0.1:/home/barman/main/incoming/%f'.
Use the IP address of the Barman server. If you got a different value for incoming_wals_directory.
Step 9: restart postgresql server
Step10: Restart postgres server or instance .
[barman@localhost ~]$ barman check main
Server main:
PostgreSQL: OK
archive_mode: OK
wal_level: OK
archive_command: OK
continuous archiving: OK
directories: OK
retention policy settings: OK
backup maximum age: OK (no last_backup_maximum_age provided)
compression settings: OK
minimum redundancy requirements: FAILED (have 0 backups, expected at least 1)
ssh: OK (PostgreSQL server)
not in recovery: OK
wal shipping failed:
[barman@localhost ~]$ barman switch-xlog --force --archive main
Step11 : Below command gives the information of the postgres server
[barman@localhost ~]$ barman show-server main
Server main:
active: True
archive_command: false
archive_mode: on
archived_count: 0
backup_directory: /home/barman/main
backup_options: BackupOptions(['exclusive_backup'])
bandwidth_limit: None
basebackup_retry_sleep: 30
basebackup_retry_times: 0
basebackups_directory: /home/barman/main/base
compression: None
config_file: /home/postgres/data/postgresql.conf
conninfo: host=127.0.0.1 user=postgres port=5432
copy_method: rsync
current_archived_wals_per_second: 0.0
current_xlog: 000000010000000000000043
custom_compression_filter: None
custom_decompression_filter: None
data_directory: /home/postgres/data
description: Main PostgreSQL Database
disabled: False
failed_count: 0
Step12 : Lets take first full backup of main using barman
[barman@localhost ~]$ barman backup main
Starting backup for server main in /home/barman/main/base/20160226T134115
Backup start at xlog location: 0/48000028 (000000010000000000000048, 00000028)
Copying files.
Copy done.
Asking PostgreSQL server to finalize the backup.
Backup size: 480.8 MiB. Actual size on disk: 480.8 MiB (-0.00% deduplication ratio).
Backup end at xlog location: 0/480000C0 (000000010000000000000048, 000000C0)
Backup completed
Processing xlog segments for main
Older than first backup. Trashing file 000000010000000000000047 from server
main
000000010000000000000048
000000010000000000000048.00000028.backup
--barman commands:
barman check main
barman show-server main
barman backup main
barman backup --reuse=link main
barman list-backup main
barman recover main 20160226T134400 /tmp/data
chown -R postgres:postgres /tmp/data
============Taking backup from standby
For taking backup from replica few modifications are required
1. First create extension pgespresso on master
Pre-requisites:
No pre-requisites required.
Verify pg_config path:
which pg_config
/usr/pgsql-9.6/bin/pg_config
Download and install pgespresso
wget https://codeload.github.com/2ndquadrant-it/pgespresso/zip/master
unzip master
cd pgespresso-master/
make
make install
2. Create extension::
-bash-4.2$ psql
psql (9.6.3)
Type "help" for help.
postgres=# create extension pgespresso;
CREATE EXTENSION
Note: once the extension is created on master it will be replicated to slave, provided the same package should be installed on server where replica is present.
3. Create streaming replication using pg_basebackup or repmgr.
Refer to my block of repmgr and streaming replication for more details.
4. Setup barman installation using above steps
5. Setup barman.conf file
[barman@localhost ~]$ cat /etc/barman.conf
[barman]
barman_home = /home/barman
barman_user = barman
log_file = /home/barman/barman.log
compression = gzip
reuse_backup = link
retention_policy = REDUNDANCY 4
bandwidth_limit = 4000
network_compression = true
backup_options = concurrent_backup
last_backup_maximum_age = 1 DAY
[standby]
description = "PostgreSQL Database Hot Standby"
ssh_command = ssh postgres@127.0.0.1
conninfo = host=127.0.0.1 user=postgres port=5433 ; PORT OF REPLICA
minimum_redundancy = 1
archiver = on
streaming_archiver = on
6. Setup archiver:
This step is important in order to perform the check without fail else will receive archiver/transaction log error.
[barman@localhost ~]$ barman receive-wal standby --reset
Starting receive-wal for server standby
Resetting receive-wal directory status
Removing status file /home/barman/standby/streaming/000000010000000000000019.partial
[barman@localhost ~]$ barman receive-wal standby &
[1] 13065
Starting receive-wal for server standby
standby: pg_receivexlog: starting log streaming at 0/19000000 (timeline 1)
[barman@localhost ~]$ barman check standby
Server standby:
PostgreSQL: OK
superuser: OK
PostgreSQL streaming: OK
wal_level: OK
directories: OK
retention policy settings: OK
backup maximum age: OK (interval provided: 1 day, latest backup age: 51 minutes, 29 seconds)
compression settings: OK
failed backups: OK (there are 0 failed backups)
minimum redundancy requirements: OK (have 5 backups, expected at least 1)
ssh: OK (PostgreSQL server)
archive_mode: OK
archive_command: OK
continuous archiving: OK
pg_receivexlog: OK
pg_receivexlog compatible: OK
receive-wal running: OK
archiver errors: OK
7. Start the backup from standby
[barman@localhost ~]$ barman backup standby
Starting backup using rsync-concurrent method for server standby in /home/barman/standby/base/20170823T154137
Backup start at xlog location: 0/19000220 (000000010000000000000019, 00000220)
Copying files.
Copy done.
Asking PostgreSQL server to finalize the backup.
Backup size: 108.8 MiB. Actual size on disk: 156.8 KiB (-99.86% deduplication ratio).
Backup end at xlog location: 0/19000300 (000000010000000000000019, 00000300)
Backup completed
Your comments or errors are always welcome.
--
Thanks
In this block we will get information regarding installation and configuration of 2nd Quadrant Open source tool barman. Barman is distributed under GNU GPL 3
Barman stands for backup and recovery manager, it is one of the advanced python based tools which can take the Full and Incremental backup of PostgreSQL database remotely.
At the end of this block you will be able to Install, take backup and perform recovery of database.
Step 1 : Install PostgreSQL-9.4 or higher.
Step 2 : Download pgbarman from below link.
https://excellmedia.dl.sourceforge.net/project/pgbarman/2.1/barman-2.1.tar.gz
Step3: Install prerequisite before installing pgbarman below are necessary prerequisite.
OS: Unix based.
pre-requisites:
python 2.6 or 2.7
Python modules:
argcomplete
argh >= 0.21.2
psycopg2
python-dateutil < 2.0 (since version 2.0 requires python3)
distribute (optional)
How to install?
https://yum.postgresql.org/9.6/redhat/rhel-7-x86_64/
rpm -ivh http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
yum install python-pip // sudo easy_install pip
pip install package name here
Step4: untar the pgbarman file and install it as given below
untar barman:
python2.6 setup.py build
python2.6 setup.py install
Step5: copy barman.conf from doc to /etc/
create user:
useradd barman -p barman
cp doc/barman.conf /etc/
chown -R barman:barman /etc/barman.conf
Step 6: Make password less authentication between barman server and postgres server .I have
created on same machine so I did password less authentication.
--ssh-keygen between barman and postgres:
[root@localhost barman-2.1]# su - barman
[barman@localhost ~]$ ssh-keygen
[barman@localhost ~]$ ssh-copy-id -i .ssh/id_rsa.pub postgres@127.0.0.1
Repeated above steps from postgres user
[root@localhost barman-1.5.1]# su - postgres
[postgres@localhost ~]$ ssh-keygen
[postgres@localhost ~]$ ssh-copy-id -i .ssh/id_rsa.pub barman@127.0.0.1
Step 7: Edit barman.conf:
[barman]
barman_home = /home/barman
barman_user = barman
log_file = /home/barman/barman.log
compression = gzip
reuse_backup = link
minimum_redundancy = 1
[main]
description = "Main DB Server"
ssh_command = ssh postgres@127.0.0.1
conninfo = host=127.0.0.1 user=postgres
archiver = on
Step 8: Edit the postgresql.conf and enable the archiving .
There is one last configuration to be made on the main, to switch on backup (or archive) mode.First,
we need to locate the value of the incoming backup directory from the barman, switch to the user
barman:
su - barman
Run this command to locate the incoming backup directory:
barman show-server main | grep incoming_wals_directory
incoming_wals_directory: /home/barman/main/incoming
Note down the value of incoming_wals_directory; in my setup it's /home/barman/main/incoming
Now switch to the user postgres on postgres server.
Open the postgresql.conf and make the following changes to the file:
Uncomment the wal_level parameter and set its value to archive.
Uncomment the archive_mode parameter and set its value to on.
Uncomment the archive_command parameter and set its value to 'rsync -a %p barman@127.0.0.1:/home/barman/main/incoming/%f'.
Use the IP address of the Barman server. If you got a different value for incoming_wals_directory.
Step 9: restart postgresql server
Step10: Restart postgres server or instance .
[barman@localhost ~]$ barman check main
Server main:
PostgreSQL: OK
archive_mode: OK
wal_level: OK
archive_command: OK
continuous archiving: OK
directories: OK
retention policy settings: OK
backup maximum age: OK (no last_backup_maximum_age provided)
compression settings: OK
minimum redundancy requirements: FAILED (have 0 backups, expected at least 1)
ssh: OK (PostgreSQL server)
not in recovery: OK
wal shipping failed:
[barman@localhost ~]$ barman switch-xlog --force --archive main
Step11 : Below command gives the information of the postgres server
[barman@localhost ~]$ barman show-server main
Server main:
active: True
archive_command: false
archive_mode: on
archived_count: 0
backup_directory: /home/barman/main
backup_options: BackupOptions(['exclusive_backup'])
bandwidth_limit: None
basebackup_retry_sleep: 30
basebackup_retry_times: 0
basebackups_directory: /home/barman/main/base
compression: None
config_file: /home/postgres/data/postgresql.conf
conninfo: host=127.0.0.1 user=postgres port=5432
copy_method: rsync
current_archived_wals_per_second: 0.0
current_xlog: 000000010000000000000043
custom_compression_filter: None
custom_decompression_filter: None
data_directory: /home/postgres/data
description: Main PostgreSQL Database
disabled: False
failed_count: 0
Step12 : Lets take first full backup of main using barman
[barman@localhost ~]$ barman backup main
Starting backup for server main in /home/barman/main/base/20160226T134115
Backup start at xlog location: 0/48000028 (000000010000000000000048, 00000028)
Copying files.
Copy done.
Asking PostgreSQL server to finalize the backup.
Backup size: 480.8 MiB. Actual size on disk: 480.8 MiB (-0.00% deduplication ratio).
Backup end at xlog location: 0/480000C0 (000000010000000000000048, 000000C0)
Backup completed
Processing xlog segments for main
Older than first backup. Trashing file 000000010000000000000047 from server
main
000000010000000000000048
000000010000000000000048.00000028.backup
--barman commands:
barman check main
barman show-server main
barman backup main
barman backup --reuse=link main
barman list-backup main
barman recover main 20160226T134400 /tmp/data
chown -R postgres:postgres /tmp/data
============Taking backup from standby
For taking backup from replica few modifications are required
1. First create extension pgespresso on master
Pre-requisites:
No pre-requisites required.
Verify pg_config path:
which pg_config
/usr/pgsql-9.6/bin/pg_config
Download and install pgespresso
wget https://codeload.github.com/2ndquadrant-it/pgespresso/zip/master
unzip master
cd pgespresso-master/
make
make install
2. Create extension::
-bash-4.2$ psql
psql (9.6.3)
Type "help" for help.
postgres=# create extension pgespresso;
CREATE EXTENSION
Note: once the extension is created on master it will be replicated to slave, provided the same package should be installed on server where replica is present.
3. Create streaming replication using pg_basebackup or repmgr.
Refer to my block of repmgr and streaming replication for more details.
4. Setup barman installation using above steps
5. Setup barman.conf file
[barman@localhost ~]$ cat /etc/barman.conf
[barman]
barman_home = /home/barman
barman_user = barman
log_file = /home/barman/barman.log
compression = gzip
reuse_backup = link
retention_policy = REDUNDANCY 4
bandwidth_limit = 4000
network_compression = true
backup_options = concurrent_backup
last_backup_maximum_age = 1 DAY
[standby]
description = "PostgreSQL Database Hot Standby"
ssh_command = ssh postgres@127.0.0.1
conninfo = host=127.0.0.1 user=postgres port=5433 ; PORT OF REPLICA
minimum_redundancy = 1
archiver = on
streaming_archiver = on
6. Setup archiver:
This step is important in order to perform the check without fail else will receive archiver/transaction log error.
[barman@localhost ~]$ barman receive-wal standby --reset
Starting receive-wal for server standby
Resetting receive-wal directory status
Removing status file /home/barman/standby/streaming/000000010000000000000019.partial
[barman@localhost ~]$ barman receive-wal standby &
[1] 13065
Starting receive-wal for server standby
standby: pg_receivexlog: starting log streaming at 0/19000000 (timeline 1)
[barman@localhost ~]$ barman check standby
Server standby:
PostgreSQL: OK
superuser: OK
PostgreSQL streaming: OK
wal_level: OK
directories: OK
retention policy settings: OK
backup maximum age: OK (interval provided: 1 day, latest backup age: 51 minutes, 29 seconds)
compression settings: OK
failed backups: OK (there are 0 failed backups)
minimum redundancy requirements: OK (have 5 backups, expected at least 1)
ssh: OK (PostgreSQL server)
archive_mode: OK
archive_command: OK
continuous archiving: OK
pg_receivexlog: OK
pg_receivexlog compatible: OK
receive-wal running: OK
archiver errors: OK
7. Start the backup from standby
[barman@localhost ~]$ barman backup standby
Starting backup using rsync-concurrent method for server standby in /home/barman/standby/base/20170823T154137
Backup start at xlog location: 0/19000220 (000000010000000000000019, 00000220)
Copying files.
Copy done.
Asking PostgreSQL server to finalize the backup.
Backup size: 108.8 MiB. Actual size on disk: 156.8 KiB (-99.86% deduplication ratio).
Backup end at xlog location: 0/19000300 (000000010000000000000019, 00000300)
Backup completed
Your comments or errors are always welcome.
--
Thanks
Ajinkya
Comments
Post a Comment