Beyond the web server performance it might also be worth considering the actual page performance and browser caching.
The website 'http://www.greenandgoldrugby.com/' takes 8.2s to load in my web browser or 6.6s after its cached its content. Some of that time could be saved by setting a decent noexpires header on some of the other areas of wordpress (it appears some is set up correctly).
YSlow also complains that your pulling down about 50 different js files and 19 stylesheets. Checking yslow, and firebug's net page will give you a hint at what is slowing down page loads.
Load impact reports a rather sluggish load too, although it handles the 50 client test fine.
http://loadimpact.com/result/www.greena ... c4945960d4