Category: Crypto

Duplicity backups

Duplicity backups

I’m running Buffalo Linkstation as an home storage for some files. I have previously rooted (to be able to install my own apps) that and installed some opt packages. I also have service from ISP where I can store several Gigabytes of data e.g. for backup purposes.

Recently I finally completed the Duplicity backups,  here is how it basically went.

Prerequisites for me was:
– Linux server where you store your files to be backed up (I have Buffalo Linkstation)
– Remote place where you store (I have Kapsi ISP)

First install the duplicity

For me as I’m using rooted Buffalo Linkstation LS-QVL 8TB I’m using commands like

ipkg install py25-duplicity
ipkg install py25-boto
ipkg install py25-crypto
ipkg install py25-paramiko

I had to install quite many packages but paramiko is the one that I’m actually using (ssh, sftp). This process wasn’t that straight forward and first tries didn’t end up in success, thats why implementing this took so long.

Also prerequisites for duplicity there is librsync GnuPG NcFTP Boto and Python 2.4 or later. Those you will have to install also.

Then I created the following scripts to do the backup.

fullbackup.sh
#!/bin/sh
# duplicity full backup to kapsi

#Email address of recipient. Adjust to suit.
export Email="where_to@send_the_reports.com"

# timestamp for email
date=`date +"%d.%m.%Y %k:%M:%S"`

# Export the PASSPHRASE variable
export PASSPHRASE=Your_secret_password!
export TMPDIR=/opt/tmp/duplicity

# Get the date
repDate=`date "+%Y%m%d_%H%M%S"`

# Cleanup first (anything older than 60 days)
/opt/bin/duplicity-py2.5 remove-older-than 60D -v9 --force scp://account@server.com:22//home/users/account/siilo/backup/duplicity >>/var/log/fullbackup_$repDate.log

# Now do the backup
/opt/bin/duplicity-py2.5 full /mnt/array1/path --asynchronous-upload --volsize 100 --include-globbing-filelist=/mnt/array1/path/cfg/backupaccount.txt scp://account@server.com:22//home/users/account/siilo/backup/duplicity >>/var/log/fullbackup_$repDate.log

# Get the disk space
echo "Availble Disk Space on Server" >> >>/var/log/fullbackup_$repDate.log
echo >>/var/log/fullbackup_$repDate.log
/opt/bin/ssh -p22 account@server.com quota -v -s >> >>/var/log/fullbackup_$repDate.log
/opt/bin/ssh -p22 account@server.com du -h ///home/users/account/siilo/backup/duplicity >>/var/log/fullbackup_$repDate.log

# Mail me the results
cat /var/log/fullbackup_$repDate.log | nail -s "Backup Full [$date]" $Email

–volsize parameter changes the default 25MB filesize to 100MB and –asynchronous-upload starts sending the package immediately once it’s ready and packs the second package. This increases the temporary space needed but makes transfers faster.

I have similar script with incremental option to use in incremental backups.

Cron
00 4 1 * * /opt/local/bin/fullbackup.sh
00 4 2-31 * * /opt/local/bin/incrbackup.sh
backupaccount.txt
/mnt/array1/path/Dir1
/mnt/array1/path/Dir2
/mnt/array1/path/Dir3
- /mnt/array1/path/Dir4/misc
- /mnt/array1/path/Dir4/not_important
/mnt/array1/path/Dir4
- /mnt/array1/path/trashbox
- **

On backupaccount.txt file I just say the directories under /mnt/array1/path what I want to be included and excluded. So the /mnt/array1/path/Dir4 gets copied but it excludes few directories. Also in the end there is to exclude trashbox and ** for rest from /mnt/array1/path.

This is basically how the things gets backed up.

duplicity

Benefits

– Crypted backups, so you can send them where ever you want without fear of getting your files to wrong hands
– Space and Bandwith efficient. Incrementals are really clever and fast to do even for big file repositories.
– Adaptivity, it supports many different protocols and file format is standard. You can probably have it running same way in the future services also.

I think for my purposes duplicity backups is really good way to do backups to remote location safely.

SSH Key handling on multiple servers environment

To have fluent access to securely walk through you servers, I have collected here some helpful instructions. I’m not going to go too detailed and the simplest things but some that I have found useful.

First of the tools, on linux and other unix machines and mac, I use the basic OpenSSH tools that there are, so ssh, scp, ssh-keygen and so on. On Windows I use putty family tools + some extra tools like Connection Managers.

So first create keys like one with passphrase and one without:

ssh-keygen -t rsa -b 4096

Then I rename them some way like adding userid on the name and ppk extension for the private file to make it easier to handle in Windows machines. I also modify the comment string to have something useful there.

On Windows you can use puttygen.exe for the same.

Then of course make sure your private key is only as few places as possible and in trusted places. I keep mine only on my trusted personal devices.

Then with the following command I copied the files to servers where I need them.

cat ~/.ssh/id_rsa_some.pub | ssh userid@servername.fi "cat >> ~/.ssh/authorized_keys"

Now if you need to log on to many machines loops, you can do that with Agent forwarding. In ssh you can simply do this with -A switch, and in Putty you can enable that on the profiles with Connection – SSH – Auth – Allow agent forwarding.

After that you have on your favorite laptop or desktop private file once, and there you will use eg. in Windows pageant to load that key by giving password for authentication only once. Then you will have the agent program running and with those Agent forwarding enabled, you will get the authentication to all your machines.

There is some security considerations here to take in account so you should consider that also when doing this.

 

Recovering lost Windows Passwords

Sometimes you loose your passwords on windows machines or happend to use old machines so rarely that you really can’t remember your passwords. This happened for my ALC-8500-2 Battery Charging IBM Thinkpad 600E laptop recently.

There is really nice tools for solving this quickly. I did mine like this. Fetch the Ophcrack tools from their site. I have cd-rom on that thinkpad so I used one of my old Vista 2.1.0 version of the disk I already had. You can fetch the latest live cd eg. XP 3.4 from here. Then I booted that live cd to text only and it started doing it’s stuff. I let the computer there for a while but It didn’t manage to broke the passwords I needed only some weak ones. Then I copied ophcrack.txt file via fat32 drive to usb-stick and to my main computer with some Intel Core i7 power. Then fetched ophcrack for win and installed that. Microsoft Security Essentials will alarm on suspicious program but you can leave that untouched as its trusted software. Then I also had to pull xp free small tables file.

Then just extract that package and configure Tables – XP free small and install. And then feeded from usb-stick that ophcrack.txt and after pressing Crack and some 3mins later I had the long lost passwords. If you are in a hurry, you can just choose the one line from that file what you really need.