Hi Linode community -

I currently rent a dedicated server from another provider. The machine is a few years old, nothing special, but it has 2 Gigs of RAM and RAID-1. I run a few small Django (Python) applications (Apache / mod_wsgi / MySQL) and one medium-sized Django application (a community website with forums, etc). I also run a low traffic IRC server.

What are some of the things to consider when switching to a virtualized server like Linode? For example, should I expect much slower disk I/O transfers? Should I think about hosting my static assets (CSS, Javascript, downloads, etc) on something like Amazon S3 because of this?

I've also heard of people getting one Linode to run the database server, and another to host the rest (Apache, Django apps, etc). Is this a common strategy? Won't this increase the latency between the application and database?

I haven't had too many problems with my current provider, except that their customer service is very lacking. I've had a disk go bad twice now, and I've had to learn how to put a fresh disk back into a RAID in Linux. I'd like to avoid that in the future. However, now that I know how to do this, I could stay with "the devil I know" and get another server from them, or I could try the Linode route. I'm just concerned about latency and having a smaller amount of resources available to me.

Thanks for entertaining my questions.

So how much of the 2G Ram did you use?

Without any idea of the pages, or the traffic, there's no way to guess what you'll need.

Start small, start simple, and ramp up as needed (which is the main advantage VPS has over dedicated)

Splitting the Database and Web engine is the last resort (doubles the point of failure, doubles the admin overhead, doubles the backup management, adds some latency, etc etc).

Fire up a single 512M, tune it wisely, test it, go from there.

_________________
Either provide enough details for people to help, or sit back and listen to the crickets chirp.
Security thru obscurity is a myth - and really really annoying.

Here's a Munin snapshot of my memory usage.



Traffic is very light.

The Atlanta datacenter filters the ports typically used for IRC. If you want to run your IRC server, you'd be best served by selecting a different datacenter.

Disk is local to the host machine, so serving static files should not be slow, as long as your webserver is configured properly (e.g., using daemon mode for mod_wsgi).

Each workload is different, but in general you're probably much better off running on a single 1GB Linode than on two 512s.

Your graph seems to indicate you'll need at least a 1GB Linode, although judicious tuning may bring memory usage down.

Linode's disks are in a RAID configuration, but this is no substitute for backups. You can use the (extra cost) service offered by Linode, or run backups yourself to your favorite storage provider or wherever you want (you probably won't have to worry about bumping up against the monthly network transfer cap given the recent 10x increase).

Edited to add:

Note that RAID configuration is completely handled by the host system, so when a disk fails, Linode staff replace it and take care of rebuilding the array. This all happens without any involvement needed from you.

Also, moving to a larger or smaller Linode plan is just a matter of putting in a support ticket. So if you guess wrong during initial set-up, it's an easy mistake to fix.

You can resize the Linode yourself from the Linode manager.

_________________
Paid support
How to ask for help
1. Give details of your problem
2. Post any errors
3. Post relevant logs.
4. Don't hide details i.e. your domain, it just makes things harder
5. Be polite or you'll be eaten by a grue

Thanks for the advice so far.



This is what I was confused about. I thought that because of the virtualization, applications have an extra layer between them and the file system, and are also competing with other users for the disk.

Although I am using daemon mode for mod_wsgi, I am not certain what this has to do with the serving of static files. I am using Apache to serve the static files directly using an Alias, and not going through mod_wsgi, if that is what you mean.

Thanks!

I'm not familiar enough with Xen to know if there's really a full extra layer, or if it's simply a shared device exposed to all guests (much as the disk is a shared device to all processes on a dedicated host), but I think a mental model of all guests going through a common layer to access the disk is valid.

It's absolutely true that the bandwidth/latency of the I/O subsystem is shared among all guests on a host (as is the CPU), so if comparing apples to apples to a similarly equipped (hardware wise) dedicated server, worst case I/O performance will absolutely be worse, and average performance may or may not be as well.

With that said, there are a lot of variables involved that affect how any of the shared resources actually perform. Statistically speaking, sharing works very well due to few guests really fully utilizing all resources simultaneously.

Plus, comparisons are not always apples to apples. For example, you mention your current dedicated server is RAID 1 so is limited to the throughput of a single disk. Linode hosts are (to my understanding) RAID 10 which in a minimum 4-disk configuration should have twice the performance over RAID 1. You don't mention drive types but the Linode hosts are 15K enterprise drives, so there may be a gain there as well. Of course, a 2 or 3x increase is balanced by up to maybe 40 guests sharing it (on a 512), so average rate may still be lower, but when others are idle you may actually find yourself bursting higher than your current dedicated host.

I do think it's important to assume that you will not want to depend on the disk for your working set (e.g., active swapping) as the hit you take on a VPS for that compared to a dedicated box can be less predictable. So using a Linode provisioned with enough memory for your typical working set is important.

But that's really a long way to say it's hard to say, and you sort of just have to try. My bet is that you will find the performance more than sufficient for your needs, but that'll never be more than an educated guess. Personally, if I were in your shoes, I'd take advantage of the fact that I had a known quantity in my existing server, and just go ahead and set up a Linode in parallel to replicate my current configuration. You have an easy opportunity to do an A/B comparison while leaving your current server in place.

Given your current stats, I do think that at least without tuning it's probably not worth all that much to try to shoe-horn it into a Linode 512, unless as you say you split functionality. But I'd probably just start with an adequately sized Linode as the simplest way to replicate your current environment - you can always choose to split functionality (and then potentially downsize the original Linode) if desired at some point in the future.

-- David

Oops. I rarely ever go into the manager, so I didn't know that. Thanks!



The point I was making is that disks are local to the host machine your Linode runs on; disk I/O doesn't have to travel over a network to reach a SAN or something. db3l did a better job explaining the issues involved.



I am not very familiar with mod_wsgi, but my impression is that embedded mode is like mod_php in that a python interpreter is invoked even for static file requests. This eats up memory and CPU. Daemon mode appears to be more like FastCGI and connects to an already-running python instance only if the request is for a script needing interpretation.

You should be aware of such details as they will affect your resource footprint.

My linode has better disk performance than any dedicated server I've ever used, although all those dedicated servers had 7200RPM disks as opposed to a Linode host's RAIDed 15k RPM SAS disks.

Edit: Well yes, if you don't know what you are doing, you can let requests for static files go to mod_wsgi (in either daemon or embedded modes). However you generally try to avoid that for performance reasons.

Well it sounds like my impressions about disk I/O under virtualization are unfounded, or at least it works good enough for most use cases. In any event, it sounds like I should take db3l's advice and build up a Linode and see how it performs myself. Only if it seems slower will I consider hosting static files on a CDN like S3.

Thanks a lot!

spammer

I suppose I should post a follow up. I moved my modest sites, one by one, to Linode several months ago. Everything has been really awesome. I love the control panels and the knowledge base of DIY help articles.

My fears about slow disk I/O don't seem to be founded. Mind you I haven't done any extensive benchmarking. It is certainly fast enough for my needs.

The only interesting story I have is that on my old host's dedicated server, a lowly dual-core box running Ubuntu 8.04, it took 90 seconds to do a MySQL dump and compress of one of my databases in a nightly cron job. When I first moved it to Linode running Ubuntu 12.04, it took about 120 seconds. I didn't know why it was slower. But after a week it mysteriously started taking 90 seconds. I don't know if MySQL had to "warm up" or maybe someone moved off of my box. I was migrated after the Fremont RAM upgrade and now it is taking 60 seconds.

Again, probably not a significant data point, just an observation. In any event, the Linode experience has been much better for me compared to my old dedicated server host.