- rsyncrsnapshot
- Jonathan Haack
- Haack's Networking
- netcmnd@jonathanhaack.com
Since I am backing up multiple remote hosts, I chose to create separate configs for each remote host instead of putting those destinations in the /etc/rsnapshot.conf together with the localhost/backup server. I have the config set to retain 90 days worth of backups called “alpha” and it is set to sync first. The localhost configuration is as follows:
#base config config_version 1.2 snapshot_root /mnt/backups/rsnapback/localhost.domain/ cmd_cp /bin/cp cmd_rm /bin/rm cmd_rsync /usr/bin/rsync cmd_ssh /usr/bin/ssh cmd_logger /usr/bin/logger verbose 2 loglevel 3 logfile /var/log/rsnapshot.log sync_first 1 use_lazy_deletes 1 retain alpha 90 #localhost backup /home/ home/ backup /etc/ etc/ backup /usr/local/bin/ bin/ backup /root/ root/ backup /var/www/ www/
Then, to run this, I use a simple script I created. Again, this script is just for running the localhost, so it can also be used for someone's single desktop:
#!/bin/bash
service="/usr/bin/rsnapshot"
logfile="/home/logs/tempa.log"
START1="$(date +%s)"
touch $logfile
$service -V sync > $logfile
END1="$(date +%s)"
DURATION1=$[ ${END1} - ${START1} ]
MINUTES=$[ ${DURATION1} / 60 ]
sed -i "1s/^/Jonathan, at $(date), the rsnapshot sync took exactly ${DURATION1} seconds which is approximately ${MINUTES} minutes to complete.\n/" $logfile
if
tail -n -5 $logfile | grep "completed"
then
START1="$(date +%s)"
echo "Jonathan, at $(date), the sync has completed and I am now running alpha."
$service -V alpha >> $logfile
END1="$(date +%s)"
DURATION1=$[ ${END1} - ${START1} ]
MINUTES=$[ ${DURATION1} / 60 ]
sed -i "1s/^/Jonathan, at $(date), the rsnapshot alpha took exactly ${DURATION1} seconds which is approximately ${MINUTES} minutes to complete.\n/" $logfile
mail -s "[$(hostname -f)]-rsnap-success-$(date)" alerts@alerts.com < $logfile
rm $logfile
chown user:user -R /mnt/backups/rsnapback/localhost.domain/
chmod 770 -R /mnt/backups/rsnapback/localhost.domain/
else
echo "Jonathan, at $(date), the sync has failed and I am now notifying you."
mail -s "[$(hostname -f)]-rsnap-failure-$(date)" alerts@alerts.com < $logfile
rm $logfile
fi
Then, in /etc/cron.d/rsnapshot, I set up a cronjob to run this script on a schedule. For my particular use case, twice a day is sufficient, but this can obviously be adjusted to the user's needs:
0 0,14 * * * root /bin/bash /usr/local/bin/rsnap-alpha-netback.jonathanhaack.com.sh > /dev/null 2>&1
Now that the localhost/backup server is backing up it's own essential boot volume files to the backup zpool, it's time to create the configs and scripts for the remote hosts. In my case, I have over 15 remote hosts set up this way, so I will highlight the setup for one in order to demonstrate what I've done. First, create a directory for the configs such as /etc/rsnapshot/ and then create a config file within it such as /etc/rsnapshot/rsnapshot-domain.com.conf. The config for the remote host looks very similar to the localhost config, but it has syntax for connecting to the remote host. Of course, make sure that you have properly exchanged ssh keys and installed rsync on the remote host before proceeding:
#base config config_version 1.2 snapshot_root /mnt/backups/rsnapback/domain.com/ cmd_cp /bin/cp cmd_rm /bin/rm cmd_rsync /usr/bin/rsync cmd_ssh /usr/bin/ssh cmd_logger /usr/bin/logger verbose 2 loglevel 3 logfile /var/log/rsnapshot.log sync_first 1 use_lazy_deletes 1 retain alpha 90 #directories backup root@domain.com:/etc/ etc/ backup root@domain.com:/usr/local/bin/ bin/ backup root@domain.com:/var/www/ www/ backup root@domain.com:/home/ home/ backup root@domain.com:/root/ root/
Now that the config is setup, you use a script very similar to the localhost script above, but note the syntax changes for sync and alpha that will specify to rsnapshot to use this particular configuration file. Additionally, just like the localhost, I specified for rsnapshot to retain a snapshot called alpha for 90 days, which suits my use case. Here's the script for the remote hosts:
#!/bin/bash
service="/usr/bin/rsnapshot"
logfile="/home/sexa/logs/domain.com.log"
host="domain.com"
START1="$(date +%s)"
touch $logfile
$service -c /etc/rsnapshot/rsnapshot-$host.conf -V sync > $logfile
END1="$(date +%s)"
DURATION1=$[ ${END1} - ${START1} ]
MINUTES=$[ ${DURATION1} / 60 ]
sed -i "1s/^/Jonathan, at $(date), the rsnapshot sync took exactly ${DURATION1} seconds which is approximately ${MINUTES} minutes to complete.\n/" $logfile
if
tail -n -5 $logfile | grep "completed"
then
START1="$(date +%s)"
echo "Jonathan, at $(date), $(hostname -f) ran a sync that completed so I am now running alpha."
$service -c /etc/rsnapshot/rsnapshot-$host.conf -V alpha >> $logfile
END1="$(date +%s)"
DURATION1=$[ ${END1} - ${START1} ]
MINUTES=$[ ${DURATION1} / 60 ]
sed -i "1s/^/Jonathan, at $(date), the rsnapshot alpha took exactly ${DURATION1} seconds which is approximately ${MINUTES} minutes to complete.\n/" $logfile
#echo "Jonathan, at $(date), the rsnapshot alpha took exactly ${DURATION1} seconds which is approximately ${MINUTES} minutes to complete." | tee -a $logfile
mail -s "[${host}]-rsnap-success-$(date)" alerts@alerts.com < $logfile
rm $logfile
chown user:user -R /mnt/backups/rsnapback/$host/
chmod 770 -R /mnt/backups/rsnapback/$host/
else
echo "Jonathan, at $(date), $(hostname -f) ran a sync that failed and I am now notifying you."
mail -s "[${host}]-rsnap-failure-$(date)" alerts@alerts.com < $logfile
rm $logfile
fi
Similarly, I need to set up a cronjob to execute this script in /etc/cron.d/rsnapshot for which I once again chose twice a day:
0 00,14 * * * root /bin/bash /usr/local/bin/rsnap-alpha-comain.com.sh > /dev/null 2>&1
You would think the retain 90 would delete the old ones, but it does not seem to work for whatever reason, and so I setup a small script to delete anything past 90 which runs daily at 8am, well after the last snapshot completed. It's also a loop which will make sure to prune all the snapshots between 90-100. If the script fails, it's possible that some snapshots could run past 90, hence including a ten day buffer. Here's the backup scrip, and trigger warning, it's a kludge:
#!/bin/bash
date=`date +"%Y%m%d-%H:%M:%S"`
log="/home/logs/rsnap-delete.log"
img="domain.com domain.org domain.net"
for i in $img;
do
touch $log
START0="$(date +%s)"
rm -rf /mnt/backups/rsnapback/$i/alpha.90
rm -rf /mnt/backups/rsnapback/$i/alpha.91
rm -rf /mnt/backups/rsnapback/$i/alpha.92
rm -rf /mnt/backups/rsnapback/$i/alpha.93
rm -rf /mnt/backups/rsnapback/$i/alpha.94
rm -rf /mnt/backups/rsnapback/$i/alpha.95
rm -rf /mnt/backups/rsnapback/$i/alpha.96
rm -rf /mnt/backups/rsnapback/$i/alpha.97
rm -rf /mnt/backups/rsnapback/$i/alpha.98
rm -rf /mnt/backups/rsnapback/$i/alpha.99
END0="$(date +%s)"
DURATION0=$[ ${END0} - ${START0} ]
MINUTES0=$[ ${DURATION0} / 60 ]
echo "Jonathan, at $(date), the rsnapshot purge took exactly ${DURATION0} seconds which is approximately
${MINUTES0} minutes to complete." | tee -a $log
mail -s "[rsnap-delete-$i]-$(hostname -f)-$(date)" alerts@alerts.com < $log
rm $log
done
So far, everything is working great. In the future, I would like to debug why rsnapshot retains more than the 90 specified in alpha and/or polish up the delete script some more. I would also like to improve the rsnap-alpha script to check for a process id rather than checking the sync log for the conditional upon which it executes the script. I already have a colleague who has a good template for doing that and it's in progress. In the mean time, however, I've already used this for 120 days or so and had to use it to restore some versions I accidentally deleted. Very pleased for now.
I've decided to keep this older information below, as it includes a more standard setup for single host, including alpha, beta, gamma, and delta versions.
[Begin old tutorial] Before you begin, you must already know how to add a hard drive to your system, a mount point, and an appropriate fstab entry. This also assumes you have an MTA properly set-up in order to send email. To create a basic configuration, do the following:
sudo apt install rsnapshot nano /etc/rsnapshot.conf <rsnapshot_root /enter-mount-point-here>
Uncomment the alpha, beta, gamma, and delta sections under backup levels section, enable the sync-first option, and specify the directories you want to backup (remember, localhost, calls whatever you have above in rsnapshot_root from above):
<retain alpha 6> <retain beta 7> <retain gamma 4> <retain delta 3> <sync_first 1> <backup /home/ localhost> <backup /etc/ localhost>
Now, edit the default script inside /etc/cron.d/rsnapshot and for, simple configurations, just enable the four entries they have inside that template. The default cron entries look like this:
<0 */4 * * * root /usr/bin/rsnapshot alpha> <30 3 * * * root /usr/bin/rsnapshot beta> <0 3 * * 1 root /usr/bin/rsnapshot gamma> <30 2 1 * * root /usr/bin/rsnapshot delta>
Personally, I use a simple script to ensure that the sync_first command completed before I run any of the rsnapshot alpha, or rsnapshot beta, and so on. My script is simple and limited to what I could get up and working. There are reliability problems with this script in full disclosure, but for my use case, it works at present. Here it is:
Of course, I have one of these for alpha, another for beta, and gamma, and so on until delta. Each of these checks the verbose log for successful completion and emails me upon verification of that from the output of the rsnapshot sync_first command. If it fails, it stops the service and emails me. A colleague of mine has a much more advanced script that checks whether the drive is mounted and other various niceties that should be part of a more robust backup solution. I have included a version of the script he developed and shared with me here:
OK … that's pretty much it for getting started with rsnapshot. Now, let's look at rsync over ssh, which is a great way to pull an immediate mirror of a site with ease. I have included syntax for alternate port and newer syntax without alternate port.
From remote server to target backup localhost, with alt port:
rsync -av --delete -e 'ssh -p 59333 -i /home/user/.ssh/id_rsa' user@10.8.9.3:/home/user/Server/ /home/user/Server/
From localhost to target remote backup server, with alt port:
rsync -av --delete -e 'ssh -p 59333 -i /home/user/.ssh/id_rsa' /home/user/Server/ user@10.8.9.3:/home/user/Server/
From remote server to target backup localhost, with standard port:
sudo rsync -avi --delete root@server.com:/home/location/of/mysqldump.sql /home/user/backup/of/mysqldump.sql
From localhost to target remote backup server, with standard port:
sudo rsync -avi --delete /home/user/backup/of/mysqldump.sql root@server.com:/home/location/of/mysqldump.sql
Once you master these steps, checkout remote-upgrades, so you can use these backup scripts and tools with a greater remote vpn and backup solution.
This tutorial is a designated “Invariant Section” of the “Technotronic” section of Haack's Wiki as described on the Start Page.
— oemb1905 2023/04/10 15:16
