Duplicity backups
I’m running Buffalo Linkstation as an home storage for some files. I have previously rooted (to be able to install my own apps) that and installed some opt packages. I also have service from ISP where I can store several Gigabytes of data e.g. for backup purposes.
Recently I finally completed the Duplicity backups, here is how it basically went.
Prerequisites for me was:
– Linux server where you store your files to be backed up (I have Buffalo Linkstation)
– Remote place where you store (I have Kapsi ISP)
First install the duplicity
For me as I’m using rooted Buffalo Linkstation LS-QVL 8TB I’m using commands like
ipkg install py25-duplicity ipkg install py25-boto ipkg install py25-crypto ipkg install py25-paramiko
I had to install quite many packages but paramiko is the one that I’m actually using (ssh, sftp). This process wasn’t that straight forward and first tries didn’t end up in success, thats why implementing this took so long.
Also prerequisites for duplicity there is librsync GnuPG NcFTP Boto and Python 2.4 or later. Those you will have to install also.
Then I created the following scripts to do the backup.
fullbackup.sh
#!/bin/sh # duplicity full backup to kapsi #Email address of recipient. Adjust to suit. export Email="where_to@send_the_reports.com" # timestamp for email date=`date +"%d.%m.%Y %k:%M:%S"` # Export the PASSPHRASE variable export PASSPHRASE=Your_secret_password! export TMPDIR=/opt/tmp/duplicity # Get the date repDate=`date "+%Y%m%d_%H%M%S"` # Cleanup first (anything older than 60 days) /opt/bin/duplicity-py2.5 remove-older-than 60D -v9 --force scp://account@server.com:22//home/users/account/siilo/backup/duplicity >>/var/log/fullbackup_$repDate.log # Now do the backup /opt/bin/duplicity-py2.5 full /mnt/array1/path --asynchronous-upload --volsize 100 --include-globbing-filelist=/mnt/array1/path/cfg/backupaccount.txt scp://account@server.com:22//home/users/account/siilo/backup/duplicity >>/var/log/fullbackup_$repDate.log # Get the disk space echo "Availble Disk Space on Server" >> >>/var/log/fullbackup_$repDate.log echo >>/var/log/fullbackup_$repDate.log /opt/bin/ssh -p22 account@server.com quota -v -s >> >>/var/log/fullbackup_$repDate.log /opt/bin/ssh -p22 account@server.com du -h ///home/users/account/siilo/backup/duplicity >>/var/log/fullbackup_$repDate.log # Mail me the results cat /var/log/fullbackup_$repDate.log | nail -s "Backup Full [$date]" $Email
–volsize parameter changes the default 25MB filesize to 100MB and –asynchronous-upload starts sending the package immediately once it’s ready and packs the second package. This increases the temporary space needed but makes transfers faster.
I have similar script with incremental option to use in incremental backups.
Cron
00 4 1 * * /opt/local/bin/fullbackup.sh 00 4 2-31 * * /opt/local/bin/incrbackup.sh
backupaccount.txt
/mnt/array1/path/Dir1 /mnt/array1/path/Dir2 /mnt/array1/path/Dir3 - /mnt/array1/path/Dir4/misc - /mnt/array1/path/Dir4/not_important /mnt/array1/path/Dir4 - /mnt/array1/path/trashbox - **
On backupaccount.txt file I just say the directories under /mnt/array1/path what I want to be included and excluded. So the /mnt/array1/path/Dir4 gets copied but it excludes few directories. Also in the end there is to exclude trashbox and ** for rest from /mnt/array1/path.
This is basically how the things gets backed up.
Benefits
– Crypted backups, so you can send them where ever you want without fear of getting your files to wrong hands
– Space and Bandwith efficient. Incrementals are really clever and fast to do even for big file repositories.
– Adaptivity, it supports many different protocols and file format is standard. You can probably have it running same way in the future services also.
I think for my purposes duplicity backups is really good way to do backups to remote location safely.