bji wrote:
I have thought about making a really good S3 bucket mounting filesystem for Linux. Then I could have lots of cheap off-Linode storage, and use the local Linode hard drive as a file cache for the most frequently accessed files and as a write cache. (...)
I periodically think of trying to incorporate S3 for some aspects of my storage, but each time I look I conclude that it's not really all that cheap, once I take into account costs for actually performing updates. Though a much more static read-only long term storage would probably still be good.
BTW, you may already have seen this, but the last time I considered it again, I was intrigued by s3backer (
http://code.google.com/p/s3backer/) and its design as a pure block storage device, rather than some of the other filesystem-approaches. And it can leverage local disk as a cache perhaps similarly to how you are thinking.
Of course, as a block device you may also risk heavier I/O charges compared to something that manages the S3 buckets differently, but there are some knobs to help with that as well.
-- David