Linode Forum

[KipBond]

I'm the sysadmin for a small organization where we currently host our own servers. I'm wanting to migrate to a hosted service, and have used Linode for my own sites for quite a while. I love Linode… nobody beats it… and I'd love to use Linode for my organization's server needs.

But, we will soon have a few terabytes of genetic data that we need to store in our database. With Linode's current price of $1/GB, that would cost us thousands of dollars per month for storage. That's just way too much for us to even consider.

So, what are my options? Rackspace and/or Amazon S3 have storage from $0.10–$0.15/GB/month. That seems a lot more reasonable. Are there other/better options? I've never used Amazon S3 (or EBS), so I'm not sure it would work. Can we have the database server on Linode use S3 (or EBS) storage?

[caker]

Linode storage is local, hardware RAID, directly attached block storage.

Rackspace's CloudFiles and Amazon's S3 won't help you if your database needs terabytes of on-filesystem data. Both of those services use proprietary protocols to access objects over the network, and although there are hacks to 'mount' these, you either won't get POSIX semantics or it'll be incredibly slow, or both.

However, if you can stash the objects in a storage service like that, and only pull them back when you need them, then that may work for you.

If you do need massive amounts of attached storage, than perhaps EBS is a good solution for you.

Hope that helps,
-Chris

[KipBond]

Thanks, caker. I'll look into Amazon EBS.

PS: you guys rock! :D

[bji]

caker wrote:
Rackspace's CloudFiles and Amazon's S3 won't help you if your database needs terabytes of on-filesystem data. Both of those services use proprietary protocols to access objects over the network, and although there are hacks to 'mount' these, you either won't get POSIX semantics or it'll be incredibly slow, or both.


I don't know about rackspace, but with S3, the mounting solutions are also incredibly flaky and unreliable. I could deal with slow and not completely POSIX compliant; but the fact that my s3fs mount would screw itself up and have to be manually recovered multiple times per week was the overshadowing reason that I gave up on that solution.

[db3l]

At the risk of introducing another non-Linode vendor into the thread, you might also take a look at some of the SoftLayer dedicated offerings. Among other things they have storage-oriented plans with locally attached storage that should perform better than EBS, though I'm not positive how the costs compare. If you skip RAID (or at least go with basic SATA drives in RAID 5 or 10) it's actually pretty reasonable. I know I've considered this approach for myself if I hit a wall before something shows up as a Linode offering.

One possible other pro is that I think SoftLayer's Dallas DC is the same as Linode's, so you may even be able to mix and match a bit, while maintaining minimal latency between the two sets of servers, depending on what part of your design needs to be local to the storage (e.g., front end boxes more economically through Linode, with the backend database server at SoftLayer). Data transfer will accrue against public limits in both cases, but SL is 3GB by default and as long as you have a few Linode's, and given that I expect most data flow will be for data retrieval from SL to Linode (which is now free on the Linode side) it would probably work out fine.

-- David

[Guspaz]

caker wrote: If you do need massive amounts of attached storage, than perhaps EBS is a good solution for you.

Telling customers to go use a competitor's VPS product (EBS can only be attached to EC2 instances) isn't helping your business... You're losing business over this, that's clear, right? Some of the lost business is small stuff, like where we spent an extra ~$40-50 per month to pay for the extra parts to build a file server and get a faster internet connection in our office, because Linode couldn't sell us affordable storage to host our design department's files at Linode. Matt keeps running out of room for his photos. KipBond has to go with Amazon instead of Linode to get decent amounts of storage. I'm sure there are many other cases, large and small, where somebody had to spend money somewhere else because Linode couldn't meet their needs...

It's really frustrating that Linode fits all of our needs so well, except for this one area, which is a real sore spot. I'm really hoping that Linode's apparent policy of ignoring the problem is because you're working on a solution and can't talk about it, and not because you actually don't want to service this customer need...

[zunzun]

Guspaz wrote: Telling customers to go use a competitor's VPS product (EBS can only be attached to EC2 instances) isn't helping your business... You're losing business over this, that's clear, right?

I instantly imagined Mr. Aker thinking:

"I don't want to talk to you no more, you empty headed animal food trough wiper. I fart in your general direction. Your mother was a hamster and your father smelt of elderberries."

James