obs wrote:
What kind of problems do you have with s3, seems pretty simple to me I use s3fs with fuse and just use the linux cp commands to copy my files over.
S3 has three fundamental problems:
1) It's expensive. Using it for anything but long-term archiving is *REALLY* expensive. You have to pay four times for any data; first for the space to store it, second for the requests to get/put it, third for the S3 bandwidth to access it, and fourth for the linode bandwidth to access it. Using it for anything but archival storage is probably cost-prohibitive.
2) It' slow. All the benchmarks I've seen show it as being too slow to use as active storage. Again, probably not useful for much beyond archival purposes.
3) It's not ACID, meaning it *can't* be used for reliable storage when mounted. This means that if you write a block/file and read it again, you're not guaranteed to get back the same thing as you just wrote.
Number 3 is the killer; S3 is designed as a distributed system to handle large load, not be read like a consistent filesystem. And since S3 *isn't* consistent (when you write data, it takes time to propagate among nodes, and there's no guarantee that your read will hit a node that has the same data), it's very risk to use as a filesystem. Again, probably not bad for archival purposes where you write a lot but almost never read, but not good for an active filesystem.