I've been trying to set up a backup solution for my linode and ultimately got down to two basic choices.

S3

I've actually set this up using s3sync and this tutorial and using these scripts (modified so it requires no user input, without --public-read and with -s for SSL ). I've made a separate script for each directory I want to backup and if I decide to go with S3 I intend to tie them all up into a single script that will be cronned.

Pros

I'd say it's the price. By my calculations you can get 10GB backed up including monthly full re-uploads (rsyncing meanwhile) and up to 100K requests, for about $3 a month. And if you go beyond that, there are no barriers whatsoever.

Cons

1. For some reason it created a subdirectory in etc on my S3 bucket (so it's etc/etc) even though my prefix was kept at simply "etc". I don't get that. Clearly, I'm not yet familiar enough with this.

2. Even as briefly as I tried it I already saw an "internal server error" a few times as well as "With result 505 HTTP Version Not Supported" which doesn't really inspire my confidence in its reliability... to be honest.. especially after those february and june (2008) "massive outages" I've read about, which may have resulted in loss of data. Cloud computing of this scale still seems to have growing pains I suppose..

3. It's not quite a standard technology as it need special tools (like s3sync) for access and it's hard to find easy to use and efficient interfaces with which I could add or remove files and folders in my buckets. Most of the ones that exist are either half assed (s3fox can't delete), proprietary or don't work on Linux (which is my primary desktop system).

4. No real customer support, AFAIK.

FTP/SSH

By this I mean services like BQ Backup where you get a fixed amount of space for a fixed monthly fee (like $5 for 10GB) with SSH and FTP access.

Pros

1. More methods of backup, from rdiff to rsync.

2. A lot of available software for viewing and manipulating remote files.

3. Arguably simpler, due to familiarity..

Cons

1. Price. It's a little more expensive than S3. For example, I probably wouldn't even use enough to be charged only $3 a month on S3 let alone $5. With BQ Backup though I would probably pay at least $2 or $3 a month for space I'm not actually using, albeit this might be justified by the relative peace of mind and simplicity/convenience.

----

So, what do you think? Would you add any other pros or cons to the above options? I'm still trying to decide.

Thanks

[quote="memenode"]I've been trying to set up a backup solution for my linode and ultimately got down to two basic choices.

If you are a linux user as you say why don't you rsync to your local linux machine. You likely already pay for enough bandwidth at home to cover this amount of traffic and you are more then likely to have your <=10Gb of space on a local partition.

The biggest reason is my upload connection. It is only 256 kbit/s so in case I actually need to restore the full backup (which would likely involve gigabytes of stuff) I would need to wait a little too long just to upload everything to my server, and that makes for more downtime.

Other reasons are that my ISP, t-com, is reconnecting me daily (changing my IP) so I'd need to set up some dyndns domain and run an IP updater. Also, whenever I'd need to turn my computer off it'd be inaccessible for backup crons, though that's not such a big issue as above.

Check these guys out: http://www.rsync.net/

As of current, my install and all data is about 1.5GB in total, so for about 2.40USD I can have my entire system backed up, or for geo-redundant for about 5$ a month.

Honestly though, I could slide away with less than a GB of actual data and config files. My current backup scheme is to tar up my webroot directory and my mysql databases weekly and my home server reaches out and grabs them.

Thanks freedom_is_chaos, that looks nice, but the price per gigabyte is a little too high for me. Just my /home directory is more than 3GB so I'd probably easily fill up the minimum order of 4GB.

And if that was to ever grow to 10GB price would be a little too high. I've had a $5 a month backup on Slicehost for a full 10GB slice. They charge $10 for a 20GB slice. I'd like to keep it within those price limits.

I'll bookmark them though, just in case.

Well I found a couple of services within that price range which seem to be fairly reputable (aside from BQBackup which apparently had some issues recently, but are otherwise also in good standing): Rsync Palace (offering 15GB for $5.95) and WebbyCart (offering 15GB for $4.99). Pretty good.

Anyone had any experience with these?

I'll try to get some sleep and then hopefully make a decision. Meanwhile, any suggestions are welcome.

In the end the only thing that keeps S3 under consideration for me is the price flexibility and the fact that I might use it to host some images, CSS files etc. (to speed loading times) so having backups tied into that might be a more efficient way to spend bucks.

But other than that I'd say convenience, simplicity and familiarity are on the side of FTP/SSH/rsync based solutions..

Thanks

I use rsnapshot from a remote host to back up important directories on my linode. The backups are initiated from the remote host so the ISP changing the IP doesn't impact the backups.

--John

You could do an el chepo web hosting and toss everything up there

??? El Chepo is an auto parts company, no web hosting.

http://www.manta.com/coms2/dnbcompany_gwr26j

James

Hehe

True.. though I hear some hosts are prohibiting use for only backups.. and on shared hosting you don't usually get SSH.

Thanks for using the scripts. You can always remove prefix altoghether.

And it would not bug you again. My scripts were designed to setup the bucket so it always any uploaded data in a new sub folder.

Removing the prefix option entirely. Will solve your problem.

With respect to pricing. S3 is as open as you want it to be. You have to decide what's your safety zone. I store lots of GB worth of data on S3. It saves me $500 every month (comapred to the storage option i get from the Colo provider).

I see, thanks for the tip.

Pricing is definitely a plus, especially if you have extremely large amounts of data to back up I suppose. Otherwise the FTP/SSH option does seem fairly attractive, especially these $5 a month for 15GB services. That actually kinda comes close to S3 pricing.

Maybe I should just go with a classical backup solution for now until I am in need of enough backup space that S3 savings would really mean a lot (though I'd have to evaluate that when I get there). Meanwhile I might gain some more experience with S3 if I decide to host some static content to offload my server a bit.

Thanks

I do not know your use case. However, i would suggest the following:

1. Offload any static elements of your site/app to S3.

2. Call the static elements via Cloudfront.

3. Dump any excess data from your linode onto S3 so its safe. In the case of system backups keep the acl as private and you will have 100% security.

5. Linode offers Raid1 which is much better then whats offered for EC2 standalone. It would be wise to dump what ever extra data you have onto S3 and call it in whenever you need it.

Reallistically speaking you would have to work really hard in order to have a very high invoice from S3.

Best Regards
Hareem Haque

Have you seen http://www.tarsnap.com/? I haven't gotten around to use it but it seems pretty good.

Well since I'm so indecisive and S3 option was already half baked I ended up finishing it and using it for now, until I make a better decision (if there's one). My first backup is uploading as we speak and then I'll make sure ACL is private (I already removed --public-read and am using SSL).

Thanks Hareem for your suggestions. I still wonder though about all these "internal server error" messages that happen (a few of them already as I'm uploading a first backup). When it happens it waits 30 seconds and then retries so I assume it gets it the second time most of the time, but this sort of thing makes me wonder if I can be 100% sure that my data is up without any corruptions. It's just odd that these errors are so common on S3.

Atourino, thanks for suggesting tarsnap. It looks interesting, but is still in beta and the site is super-minimal so not sure.. It's pricing is in line with S3 and also uses some nonstandard tech so in that sense it's similar to S3... though with more security..

I'll bookmark it and then if I decide to use a non-S3 backup at some point I might consider it too.

Thanks