Linode Forum
Linode Community Forums
 FAQFAQ    SearchSearch    MembersMembers      Register Register 
 LoginLogin [ Anonymous ] 
Post new topic  Reply to topic
Author Message
 Post subject:
PostPosted: Sun Jun 20, 2010 10:58 pm 
Offline
Junior Member

Joined: Fri Feb 25, 2005 7:34 pm
Posts: 26
bji wrote:
I have thought about making a really good S3 bucket mounting filesystem for Linux. Then I could have lots of cheap off-Linode storage, and use the local Linode hard drive as a file cache for the most frequently accessed files and as a write cache. There exists already tools for mounting S3 buckets as volumes but the last time I looked (admittedly, a couple of years ago) they were not nearly high quality enough for my required confidence level.


You should check out s3backer. Admittedly, I've not used it, but I discovered it when researching S3 myself, and it seems to solve the problems you mention.

Edit: just noticed there was another page of posting where s3backer was already mentioned. Sorry.


Top
   
 Post subject:
PostPosted: Mon Jun 21, 2010 10:26 am 
Offline
Senior Member
User avatar

Joined: Sun Feb 08, 2004 7:18 pm
Posts: 562
Location: Austin
vonskippy wrote:
Although this thread has seen reasonable action for this forum, nobody else has chimed in with a "me too, I need more storage".


Me too, I need more storage!

The RAM upgrade is nice, in the sense that it's better to have it than to not have it. Honestly, I don't expect to notice the difference.

I've also been a customer since summer 2003, straight through, and over the past couple of years, disk space has been a constant headache.

The last bump was over a year ago. I'm having to seriously consider moving to a dedicated server. I've found a place which offers one for $50/month, with 500GB disk space. Think of it!

Of course it's only a dual-core Atom, but what I need is DISK.


Top
   
 Post subject:
PostPosted: Mon Jun 21, 2010 10:46 am 
Offline
Linode Staff
User avatar

Joined: Tue Apr 15, 2003 6:24 pm
Posts: 3090
Website: http://www.linode.com/
Location: Galloway, NJ
We offer more disk. It's called: upgrade your plan.

Cheers,
-Chris


Top
   
 Post subject:
PostPosted: Mon Jun 21, 2010 12:47 pm 
Offline
Senior Member
User avatar

Joined: Sat Aug 30, 2008 1:55 pm
Posts: 1739
Location: Rochester, New York
For those of an Ubuntu persuasion looking to use s3backer for bulk storage but not wanting to compile stuff, I've created an updated PPA of the latest source distribution (1.3.1):

https://launchpad.net/~rtucker/+archive/s3backer

If you're using lucid (10.04 LTS), just:

Code:
sudo add-apt-repository ppa:rtucker/s3backer


Or, if you're using hardy (8.04 LTS), add this to your sources.list:

Code:
deb http://ppa.launchpad.net/rtucker/s3backer/ubuntu hardy main 
deb-src http://ppa.launchpad.net/rtucker/s3backer/ubuntu hardy main


Once you've done either of the above, simply update and install as per usual:

Code:
sudo apt-get update
sudo apt-get install s3backer


You can then follow the nice instructions at this URL to create a 1 TB ReiserFS filesystem, adjusting parameters to taste. If you're using something other than hardy/lucid, let me know and I can wave a chicken accordingly.

No warranty, if it breaks you get to keep both halves, I may or may not update it if I hear of a new release, etc. I haven't actually done any performance testing with it yet, but I'm thinking of ways to do that.


Top
   
 Post subject:
PostPosted: Tue Jun 22, 2010 11:16 am 
Offline
Senior Member
User avatar

Joined: Tue May 26, 2009 3:29 pm
Posts: 1691
Location: Montreal, QC
My understanding is that with S3, you can't really be guaranteed that a read will return written data after you write it, because the data may not have propagated yet to the node that you end up reading from. So any S3 mounting app would need to have a rather large write buffer that is maintained long after data is written...

I've heard reports of S3 being used for POSIX operations to be quite slow, although my experience from home with Dropbox (which is S3 backed) is pretty fast... Still, the whole "uses your bandwidth" thing is a bit of an issue.


Top
   
 Post subject:
PostPosted: Tue Jun 22, 2010 12:39 pm 
Offline
Senior Member

Joined: Sun Mar 07, 2010 7:47 pm
Posts: 1970
Website: http://www.rwky.net
Location: Earth
Guspaz wrote:
I've heard reports of S3 being used for POSIX operations to be quite slow, although my experience from home with Dropbox (which is S3 backed) is pretty fast... Still, the whole "uses your bandwidth" thing is a bit of an issue.


You have to remember drop box keeps a copy of what you're syncing locally so it doesn't have to rely on contacting the s3 systems.

I use s3 for backup purposes only, it would work well as a CDN, but for a generic file system it is way too slow.


Top
   
 Post subject:
PostPosted: Tue Jun 22, 2010 7:08 pm 
Offline
Senior Member

Joined: Thu Aug 28, 2003 12:57 am
Posts: 273
caker wrote:
We offer more disk. It's called: upgrade your plan.

Cheers,
-Chris


Sorry, I thought this was clear from the discussion but you seem to have missed it somehow.

I am asking for more space at the same price as what I pay now, or at a reasonable incremental increase in price. Your offer that I should pay an extra $10 per month for 6 GB of space is lame.

Believe me, I know that if I don't like the price, then I can go elsewhere; so no one has to remind me, thanks.


Top
   
 Post subject:
PostPosted: Tue Jun 22, 2010 8:48 pm 
Offline
Senior Member
User avatar

Joined: Sat Aug 30, 2008 1:55 pm
Posts: 1739
Location: Rochester, New York
Donate today to the Council of Storage Technology Scientists, who are closer than ever to breaking the 600 GB threshold and allowing our disk allocations to increase by perhaps 4 to 6 GB within the next few decades!


Top
   
 Post subject:
PostPosted: Tue Jun 22, 2010 9:48 pm 
Offline
Senior Member

Joined: Fri Dec 07, 2007 1:37 am
Posts: 385
Location: NC, USA
bji wrote:
Your offer that I should pay an extra $10 per month for 6 GB of space is lame.

Not to be constantly correcting your numbers, but if you upgrade plans your $10/mo gets you 8GB of disk space.

And if you don't like the price, oh nevermind...


Top
   
 Post subject:
PostPosted: Wed Jun 23, 2010 3:10 pm 
Offline
Senior Member

Joined: Thu Aug 28, 2003 12:57 am
Posts: 273
Stever wrote:
bji wrote:
Your offer that I should pay an extra $10 per month for 6 GB of space is lame.

Not to be constantly correcting your numbers, but if you upgrade plans your $10/mo gets you 8GB of disk space.

And if you don't like the price, oh nevermind...


Thanks for the correction - I seem to be highly math impaired on these forums.


Top
   
 Post subject:
PostPosted: Wed Jun 23, 2010 3:51 pm 
Offline
Senior Member

Joined: Wed May 13, 2009 1:32 pm
Posts: 737
Location: Italy
post edited, talking in a civil way is always better than fight.


Top
   
 Post subject:
PostPosted: Wed Sep 22, 2010 3:31 am 
Offline
Junior Member

Joined: Sat Sep 18, 2010 7:49 pm
Posts: 24
:idea: I guess linode just has to make a profit don't they?
you could go with burst.net
I guess you get no support. At linode you are probably paying for support mostly.


Top
   
 Post subject:
PostPosted: Wed Sep 22, 2010 4:27 am 
Offline
Senior Member
User avatar

Joined: Fri Oct 24, 2003 3:51 pm
Posts: 965
Location: Netherlands
dmwilliams wrote:
:idea: I guess linode just has to make a profit don't they?
you could go with burst.net

Trolling much?

dmwilliams wrote:
I guess you get no support. At linode you are probably paying for support mostly.

And no overselling. If everyone fills up their 250 GB, there's gonna be trouble.

_________________
/ Peter


Top
   
 Post subject:
PostPosted: Wed Sep 22, 2010 5:17 am 
Offline
Senior Member

Joined: Sat Nov 15, 2008 4:24 pm
Posts: 55
Location: Czech Republic
I can understand why disk can be a problem. People tend to allocate all their share so there is no overlapping as it is with CPU or bandwidth.

Disk is more like RAM - everybody allocate all he can even if he doesn't use it and thus it is expensive. I, for example would be able to live with 8G of 16G most of the time but it is not possible to resize image on fly so I have allocated whole 16G.


Top
   
 Post subject:
PostPosted: Thu Sep 23, 2010 2:59 am 
Offline

Joined: Thu Sep 23, 2010 2:38 am
Posts: 1
Yeah, I'll throw in too for the call for improved disk space. It's the only thing that makes my Linodes feel underpowered.

Chris' answer to this was pretty flip, IMO. No matter how-which-way you upgrade your Linode, you're paying between $1.25 to $2/month/gigabyte.

By contrast, other folks have figured out how to build storage for $.12/GB, one-time cost (plus maintenance & initial development): http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/.

After looking at setups like that, Linode's price/gig hurts a bit.


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic


Who is online

Users browsing this forum: No registered users and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
RSS

Powered by phpBB® Forum Software © phpBB Group