Hello all. You may remember me as the grumpy guy who always complains about low disk space on Linode plans. You may also remember that I made a DISASTROUS attempt to switch from Linode to Amazon EC2 Micro to try to get more disk space for my money. And you may remember that I came back to Linode pretty quickly.

Well I am once again bumping up against the end of my disk space on Linode. I have removed every significant source of disk space usage that I can, and I am still at 95% usage. Basically, my wife keeps adding photos to our online photo gallery faster than Linode adds more disk space to their plans. And after trying every trick I can think of to reduce my disk space usage over the years, I am finally stuck with no other easy options.

Of course, I can upgrade my Linode plan; but it sucks to spend $10 more per month just for an additional 8 GB of disk space. Jeeze if only I could send Linode a 64 GB flash drive to plug into some extra USB port on the server just for my usage, my problems would be solved

So I may end up having to upgrade anyway, to a Linode 1024; I'd be way overpaying for unneeded CPU and RAM just to get the disk space I need. But before I do that ... are there any creative suggestions for maximizing my disk space availability on my existing Linode?

Has anyone played with compressed filesystems under Linux? I did a little searching but the results were not encouraging; it doesn't look like there are many 'seamless' options for doing this, and using a compressed filesystem has the added worry of possibly interacting badly with Linode's virtual disk management tools.

I also thought of running a 512 MB RAM disk which I would copy on boot up from my home server. I have a few static HTML directories that I don't expect will ever change and I could keep their permanent storage off site. But this only gains me 512 MB on my Linode 768 and would leave me with only 256 MB of RAM (which is probably enough for my needs, but still that's a little tight).

I did use s3fs with an S3 mounted bucket for a few months and that just sucked in so many ways that I will not consider that option. The main problem is that the photo gallery software I use (gallery) interacts really badly with S3 mounted buckets and had soo many performance and reliability problems. Also I was getting kernel panics not infrequently that I worry may have been caused by this.

Any other creative ideas? Anything with the bandwidth and latency problems similar to S3 is basically out of the question, so most if not all network based storage solutions are probably not feasable - unless the networked storage was local to the data center.

Is there a cheap-ish provider of networked data storage somewhere in the Dallas data center that I could pay to host my data?

I think your issue with s3 is that you're using it through s3fs. You should really find some gallery software that interacts directly with s3. I can't really make any suggestions because I couldn't find one when I last tried to look.

I'd suggest you supplement your linode with a smugmug account. The cheapest plan is $5 a month and you can store unlimited (*) number of jpegs. They're backed by s3 if that means anything. If you don't like smugmug there's also phanfare.


(*) pretty much unlimited

Thanks, those are good suggestions. I did try to use S3 in a better integrated manner with gallery but I found that the only instructions I found for doing so were a) really complicated (but doable), and b) required the use of mysql as the database backend. I already use postgresql for my gallery config so without *major* headache of switching databases, or hours upon hours of altering gallery to work with S3 and postgresql, I can't easily use that solution.

Perhaps despite the hours and hours of effort that will be required on my part, this is the best solution. But I would still be interested in learning about other possible solutions.

A compressed filesystem wouldn't help you much, not if most of your usage (especially the part that keeps growing) is photos. Assuming those are JPGs, further compression is not likely to be possible, and you'll just have a lot of complexity for little or no (or possibly negative!) return.

I finally gave up and switched to a dedicated Atom server for $50/month from Wholesale Internet. It has a 500GB hard drive. True, the CPU isn't the same as what I had with Linode, but that's okay with me. It may not be for you, but I do find that it handles everything I threw at my Linode much better than the Linode did, because the disk performance is so much better. And I never even have to _think_ about running out of disk space.

Given that most of your space is going to photos, a compressed filesystem is unlikely to help much, since your data is already compressed.


Is there any opportunity for you to shard your albums by frequency of access to separate frequently accessed stuff from less so? If I were in your shoes, I might consider storing the less frequently accessed stuff right at home (essentially free, unlimited storage), with a network mounted filesystem (preferably over a secure VPN tunnel from the home server to my Linode), and then push older or less accessed stuff down there.

My guess is that growth is slow enough that you could manage the division of photos manually, say selecting the top 'n' oldest albums once a month and replacing them on your Linode with symbolic links to the network mount from home. Were I doing this I'd probably have an automated sync script always keeping the home storage completely in sync with anything stored locally on the Linode so when you need to shift something, it's just removing the local copy and making the link to the matching directory already present over the network share.

I'm sure there must be some other more automated way (say a generic caching layer above the network filesystem), but I'd bet some simple manual management would be fine, and you'd still have the basic Linux filesystem cache for short term accesses.

Yes, this would mean that accesses to the older stuff would make your Linode pull the data from your house, and then push it to the client but much of that I/O should overlap so the timing need not be much worse than the basic retrieval from home, and it's for less frequently accessed stuff. And given that the images would have existed initially on the Linode before being shuffled off home, gallery's thumbnails/cache could still exist locally. And yes, if your home's link or server went down you'd lose access to those photos for the outage.

Now this may not be practical for any number of reasons (everything accessed frequently, unbearably slow or unstable link at home, etc...) but it may be worth considering or at least testing.

-- David

s3fs sucks hard, as was made pretty clear in the last thread you opened. That's not necessarily a fault in s3fs, since S3 itself was never designed for filesystem-like access patterns.

Have you tried buying a second VPS in the same geographical region, but from a cheap provider that has lots of disk space? Although NFS or SSHFS would also involve network latency and bandwidth limits, they usually perform much better than S3 because those protocols were designed to be filesystems in the first place.

Now, here's my $0.02 just in case you're considering any further modifications to your existing gallery app: Don't. Those hours would be much better spent looking for a new gallery app that either comes with decent S3 integration by default, or has a well-tested plug-in to the same effect. Besides, Google can find you ways to migrate data from most open-source apps to most other open-source apps, though it's not always easy. At any rate, you'd be investing your time in a semi-permanent solution rather than in just another stopgap measure. If you don't have a very compelling reason to stick to PostgreSQL, look into alternatives that use MySQL -- it's not that bad. Besides, you can run both on the same server if you need to.

Or just get coffee and/or lunch from a slightly cheaper place a couple of times a month, and use the extra cash to buy a bigger Linode

Using a compressed filesystem on images is not going to work. Give up on that solution.

You seem determined to use the Linode when it possibly isn't the best option. However if you're too paranoid to let some dedicated photo hosting service host it for you then upgrade that Linode.

I find it very hard to believe that your $50 Atom dedicated server has better disk performance than a Linode running on 15K RPM SAS drives in RAID10. Unless there was a heavy user on your linode, in which case a simple ticket would have solved that. For that matter, a Linode should trivially outperform an Atom in the CPU department too.

My $20 linode certainly outperforms *any* low-end dedicated server I've ever had, even when the dedicated server cost four times as much.

It's cheating like hell but has anyone tried gmailfs on a linode?

It's free, should provide plenty of space, and should be no slower than s3fs. It's an ugly solution but might just work.


The only other solution I can think of is put thumbnails and application code on the linode and bigger images on some form of cheap web hosting.

Where it was most noticeable was email. I keep large email folders in maildir format on the server. Doing anything with email, whether it's just listing a folder, moving messages, or whatever, is much faster on the dedicated box where I have my own hard drive. I don't think it was a problem with a rogue user; I was on Linode for a very long time, and did move from host to host on occasion.

Surely Linode is a better deal on CPU, but that wasn't my bottleneck. The Atom is more than enough for what I need to do. And that does include a photo gallery with server-side resizing.

I've had the opposite experience. Disk performance, both in terms of sequential transfer and IOPS, is far higher on my Linode than my dedicated server's 80GB 7200RPM drive was.

I am using Linode for this because we have 8+ years of photo albums stored in gallery and it would be a monumental task to convert all of that over to some other service. It has nothing to do with paranoia.

It's getting to the point where I have no more outs though and just have to pony up more $$$; because the only other options all involve hours upon hours of effort on my part.

What I was really hoping for was some networked storage service that is local in the Linode Dallas data center that I could host networked storage on for cheap-ish and still get good performance from my Linode. Barring that, it's going to have to be a Linode upgrade ...

Of course I could always hope for Linode to upgrade their disk space plans ...

Well since you seem to be open to suggestions. How reduced in size and compression/quality are the full size images?

Is this Gallery 2 you're using? If so, do you have more than 1 resized version? If you're open to changing what you host and how things are organized, you can eliminate the resized version and reduce the quality of the full size version so you just have a thumb and a "full-size" image that loads quickly. Of course if you're using this as your archive to that's not going to work, but I'd suggest since you're bumping up against disk space issues and if you're trying to use Linode/Gallery as your photo archive, that you reconsider that. It's cheaper to use another online backup service such as Mozy or Crashplan for backups.

This is exactly my experience too. I had a 5600+ dedicated server from hetzner and disk IO was always way faster at linode.

I also had a cheap dedicated server from bytemark and the linode won on everything but disk space.

If you use a program like Drupal, you can have it integrate with S3 without mounting a S3 bucket as a partition and thus move all of your photos your Linode.

See http://drupal.org/project/storage_api and http://drupal.org/project/media_mover to start