This is just a short note on my experiences running backups with Duplicity.
Duplicity is an open source package that allows you to do incremental backups, complete with proper indexing, to remote storage. This can be a modern ‘cloud’ storage like S3, but I prefer to run it over a simple SSH link.
Next to properly working incremental backups, it also provides data security by using GPG to encrypt the data. And it has a lot of stuff you would expect : configurable full dump cycles, purging of old backups. There is a windows / C# implementation too (haven’t tried it though)
The only thing lacking may be deduplication, which is kinda hard given that all data is encrypted.
It took me some time to get all the parameters right, but after some initial fiddling, I wrapped it all in some puppet code that gets deployed to all new machines / nodes.
So every new machine is backup up automagicly using duplicity by only applying my basic puppet profile to the host.
I also did an extensive restore test during the implementation phase with went fine.
Highly recommended little know tool in some dark corner of the Internet : http://duplicity.nongnu.org/ . Don’t let the HTML 1.0 web design turn you off, this tool is maintained and stable.