Then you've misconfigured your server... 4GB is overkill to handle 5k uniques per day. I know little about drupal, but even if it causes your PHP processes to consume 120MB of RAM each, you shouldn't need more than what, a dozen PHP processes to handle that load? Less? How many drupal pageloads are you getting simultaneously?
It's a Drupal problem not a server config problem. That particular Drupal installation is a real beast, with massive memory consumption, and huge (>3 seconds) page load times.
It seems typical for the server to be handling 10 to 30 page page loads simultaneously (I've limited MaxClients to 30 for now due to the memory they consume).
Note that the site has spikes in traffic because much of its traffic comes from a weekly newsletter.
Another disclaimer: this is not my site, and the site is hosted with Linode's competitor. It's possible that their "slices" are just inferior to Linode's