Backup Solution

This post is taking things in a slightly different direction. If you are not technically inclined, you will probably want to stop reading immediately :).

I have spent a while trying to figure out a good backup strategy for all my media, and I think I have come to a pretty solid solution.

At a high level, I use rsync to backup my computers to a D-Link DNS-323 RAID1 (mirroring) NAS, which runs cronjobs to backup select stuff to my Dreamhost account.

I use rsync for just about everything. Since Janel and I are mac users, I needed to get the latest and greatest release (3.0.5 right now) in order to handle as many of the OSX FS attributes as possible. There is a good write-up on how to get it installed here. I run rsync by hand from the command line to get it backed up to my NAS.

$RSYNC -e 'ssh -c blowfish -ax' -avx -S --delete --exclude-from $EXCLUDE $* $SRC $USERNAME@$IP:$DST

OK, that is about all there is on the Mac side. Moving to the NAS.

The DNS-323 has an embedded processor that runs linux. It is a pretty weak processor, but does the trick here. It is really easy to hack into it and enable ssh (and therefore whatever you want). D-Link appears to be supportive of the backdoor that gives you access. There is tons of documentation on it at wiki.dns3232.info. Using blowfish for your encryption (see command line above) will really help your transfer rate. The slow processor is only able to maintain ~3MB/s using rsync.

I tried a variety of other solutions, but found that the easiest thing to do was just install (and use) rsyncd. I have installed the latest release in order to support all the mac attributes.

In addition to keeping the data mirrored across the two drives, I run a cronjob (on the DNS 323) that keeps a monthly backup of my data.

So, that should get the data over to the NAS. However, if I were to get robbed or have a fire, I would lose everything. I decided it would be best to find an off-site backup solution. Most of the solutions out there do not support linux, or they are really expensive. I found a webhost -- dreamhost.com -- that provides 50GB of "backup" space. Most of the webhosts out there get pissy when you backup stuff (particularly music) to their servers (even if you set it's permissions so that people can't download). Dreamhost addresses this by giving you space on a separate server that you can use for backups. If you go beyond the 50GB, they charge you $0.10 per month per GB. Dreamhost also has ssh and rsyncd running on their servers for easy sync'ing.

If you decide to sign up for Dreamhost, you should google around for coupons. There are tons available. I was able to save $97 on my first year's service.

I run a weekly cronjob which pushes all my music into the backup space on Dreamhost. It also copies all my photography into the regular webspace (which I was told would be fine). It is really annoying to do off-site backups if you have a laptop because it takes so long to upload the files (and I tend to close my laptop when I am not using it). However, I can upload over my local network to my NAS relatively quickly ... and then I can let my NAS take it's sweet time uploading offsite.