Linode Forum
Linode Community Forums
 FAQFAQ    SearchSearch    MembersMembers      Register Register 
 LoginLogin [ Anonymous ] 
Post new topic  Reply to topic
Author Message
PostPosted: Sun Nov 14, 2010 9:54 pm 
Offline
Junior Member

Joined: Sun Nov 14, 2010 9:46 pm
Posts: 28
Hi all, I have a site on Media Template GS that crashes on busy nights.
I am in the process of moving it to a linode but the performance is so much worse & dies at 40 users when I run a basic LoadImpact test.
I am using the optimisation guide here but it's not helping, any tips?


Top
   
 Post subject:
PostPosted: Sun Nov 14, 2010 10:19 pm 
Offline
Senior Member
User avatar

Joined: Sun Dec 27, 2009 11:12 pm
Posts: 1038
Location: Colorado, USA
I've ordered one of these...

Image

Hopefully that will help us discover any useful detail that might be buried in your question.


Top
   
 Post subject:
PostPosted: Sun Nov 14, 2010 10:29 pm 
Offline
Junior Member

Joined: Sun Nov 14, 2010 9:46 pm
Posts: 28
no harm in a bit of sarcasm, I enjoy a bit myself...

linode 512, Ubuntu 10.0.4 LTS with LAMP installed.
Running WP & BuddyPress.
The site gets 20k page views on a busy night, 5k average on week days.

I do run a lot of plugins but with all disabled the load test still craps out.

Currently hosting on: http://xfactor-updates.com
Linode IP for testing is http://109.74.198.57/


let me know if any other details would be helpful.
any tips appreciated before I switch over.


php info: http://109.74.198.57/php.php
apc info: http://109.74.198.57/apc.php


Last edited by xfactor-updates on Sun Nov 14, 2010 11:07 pm, edited 1 time in total.

Top
   
 Post subject:
PostPosted: Sun Nov 14, 2010 10:59 pm 
Offline
Senior Member

Joined: Wed May 13, 2009 1:18 am
Posts: 681
That referenced page is for a machine with a GB of RAM, and the suggested MaxClients of 250 is going to cause real problems for probably all but the largest Linodes. To be honest, I think the settings are probably too high even for that machine. For example, if I'm reading it right they limit PHP to 64MB of memory, which means that even without the Apache process overhead, at 250 simultaneous clients they risk needing like 16GB of memory worst case. Of course, in practice most PHP requests likely use a fraction of that memory which is how they get away with it at all.

Once you overcommit memory heavily and start thrashing on a VPS, performance is going to tank. The odds are good that during your stress test, you're forcing your Linode to start so many Apache processes in parallel that you're swapping heavily and everything essentially comes to a stop while waiting for all the I/O to complete.

There are a large number of Apache tuning threads here in the forum (searching for MaxClients might be a good start) that you might try reviewing.

Certainly if you're on a Linode 512, just for the heck of it, drop MaxClients down to 15-20 or even lower to start with and see how it does. Watch peak memory usage during the stress tests.

Peak transaction throughput may suffer at that low value, but you shouldn't keel over completely at any point and if you find yourself with spare working memory under load, you can start raising the value, and start playing with requests per child and so on. PHP caching is definitely also worth pursuing. But MaxClients is the single big knob that should at least let you get into the ballpark in terms of resources.

Last time I tried this myself (Linode 512 with Apache fronting a Silverstripe CMS site over mysql), I settled on about 25 for MaxClients for testing with absolutely no other PHP or mysqld tuning. My apache processes were averaging about 16-18MB in resident size, and I was getting 20 requests/s. The system floated with around 50-75MB free during the test. The processing was entirely CPU bound, so Apache's requests per child configuration made little difference, but I could keep churning 20 requests/s as long as needed.

Now there's a variety of things I could do to try to increase the processing rate, but increasing MaxClients would not be one of them. Even just going above 30 pretty much guaranteed I'd thrash and performance would absolutely tank. But, for example, I tried with 50, and ended up going from almost 100% cpu bound, to almost 100% I/O wait bound and no free memory. My transaction rate during tests dropped to 3/s (an 85% drop) and 72% of 200 test transactions failed to complete. I even generated a kernel panic when the machine OOM'd without finding a killable process (I had only configured 256MB of swap). All from changing MaxClients from 25 to 50.

It's useful to note that MaxClients keeps memory usage in check, but even low settings can deliver high request rates as long as the page processing is fast and memory could in theory permit a higher value. For example, static content through the same system (no PHP used by the page, resident apache process size around 4-6MB) was about 6000 requests/sec. I had a lot more free memory during the test, but even without leveraging all of it got decent rates.

Hmm, this got pretty long - I was originally just going to say "drop MaxClients" :-)

-- David


Last edited by db3l on Sun Nov 14, 2010 11:13 pm, edited 3 times in total.

Top
   
 Post subject:
PostPosted: Sun Nov 14, 2010 11:02 pm 
Offline
Senior Member
User avatar

Joined: Sun Dec 27, 2009 11:12 pm
Posts: 1038
Location: Colorado, USA
There's a handful of discussions on the forum about optimization, plus several Linode Library articles that discuss configuring a LAMP staff for optimal traffic.

The link to the page you listed for optimization may be good advice for some people - but it's counter productive to a limited memory VPS system (especially the max clients and maxrrequestsperchild is VERY high for a 512M VPS system).


Top
   
 Post subject:
PostPosted: Sun Nov 14, 2010 11:12 pm 
Offline
Junior Member

Joined: Sun Nov 14, 2010 9:46 pm
Posts: 28
db3l, vonskippy,
perfect thanks, that's the kind of advise I need.
Will reduce the MaxClients and report back.
I have no issues upgrading to a higher memory Linode, just want to get optimal settings working first before I switch over.


Top
   
 Post subject:
PostPosted: Mon Nov 15, 2010 12:34 am 
Online
Senior Member

Joined: Fri Jan 09, 2009 5:32 pm
Posts: 634
xfactor-updates wrote:
db3l, vonskippy,
perfect thanks, that's the kind of advise I need.
Will reduce the MaxClients and report back.
I have no issues upgrading to a higher memory Linode, just want to get optimal settings working first before I switch over.


Definitely do some tuning first. I run a phpBB on my 512. I know WP is heavier, but my peak last month was 50k pageviews in a day and my linode wasn't sweating in the least. I had some sweating back in January when I got about 40k in a 3-4 hour period, but that was also prior to the bump to 512 (it used to be 360).


Top
   
 Post subject:
PostPosted: Mon Nov 15, 2010 3:06 am 
Offline
Senior Member

Joined: Fri May 02, 2008 8:44 pm
Posts: 1121
Rule of thumb for LAMP servers:

MaxClients and/or ServerLimit should be no more than RAM (MB) / 20


Top
   
 Post subject:
PostPosted: Mon Nov 15, 2010 3:47 am 
Offline
Senior Member

Joined: Thu Jul 22, 2010 8:23 pm
Posts: 60
xfactor,

Are you using a cache plugin for WP?


Top
   
 Post subject:
PostPosted: Tue Nov 16, 2010 7:53 pm 
Offline
Junior Member

Joined: Sun Nov 14, 2010 9:46 pm
Posts: 28
I have tried lots of different Apache, PHP & MySQL configs now & the load test still craps out.

Can anyone point me in the direction of optimal config files for a WordPress set up on a 512 linode?

Does upgrading to a 1024 linode make much of a difference?
I have no problems upgrading but while I'm testing I'll try any OS on 512 to see what results in the best performance.


Top
   
 Post subject:
PostPosted: Tue Nov 16, 2010 8:09 pm 
Offline
Senior Member

Joined: Wed May 13, 2009 1:18 am
Posts: 681
xfactor-updates wrote:
Does upgrading to a 1024 linode make much of a difference?

It's only likely to make a difference if the reason for the failure is running out of memory, and the needed configuration would fit into 1GB but not 512MB. Otherwise you'll just have the same issue slightly later on the load curve with the Linode 1024. So to answer that question you need to know the memory working set your configuration needs.

Are you sure you aren't running out of memory with your testing? Can you provide some data on the configurations you've tried and their results? What does "crap out" mean with respect to your testing?

I have a difficult time believing that a low enough MaxClients can't prevent a lock-up on a Linode 512 no matter what the application or load, though perhaps at a horrible transaction rate. But maybe by "crap out" (or your original "dies" comment) you just mean it isn't meeting your expectations for transaction rate?

Knowing the actual configuration modifications you've made, and how the test is failing might help. Perhaps it's actually some bug in the application stack that is hanging things up (in which case the Linode should still be responsive but your app itself fails to work) which might be a completely different kettle of fish.

Although I suppose you could take the other tack of just creating a very large Linode as a test and see if it performs differently. You can always cancel and any remaining fee will be prorated back to your account as a credit.

-- David


Top
   
 Post subject:
PostPosted: Tue Nov 16, 2010 8:21 pm 
Offline
Junior Member

Joined: Sun Nov 14, 2010 9:46 pm
Posts: 28
db3l
thanks for the fast reply, by crap out, I mean I can't access the site via HTTP & SSH responses are extremely slow.

If I run a clean install of WP & import my posts by SQL, without activating any plugins I am having the same issue.

What OS and WAMP config files would you recommend for a WordPress blog with 5k page views a day & I will work from there.


Top
   
 Post subject:
PostPosted: Tue Nov 16, 2010 9:19 pm 
Offline
Senior Member

Joined: Fri May 02, 2008 8:44 pm
Posts: 1121
Open another SSH connection to your server, and run top. Leave it running, and do whatever it takes to make your server "crap out" again. While it's crapped out, take a screenshot of top and post it here. That'll be the easiest way for knowledgeable people to figure out what's happening.

If you don't have a caching plugin installed on your WordPress blog, try WP Super Cache or W3 Total Cache. Try them both, one at a time, but not both of them at the same time. Take some time to select all the recommended settings, and do your load test again. See if either plugin makes things better.

You shouldn't have to upgrade your Linode to handle 20K page views per day. As soon as you find the culprit and fix it, you'll be able to get by with a Linode 512 and save $$$.


Top
   
 Post subject:
PostPosted: Tue Nov 16, 2010 10:15 pm 
Offline
Senior Member

Joined: Wed May 13, 2009 1:18 am
Posts: 681
xfactor-updates wrote:
db3l
thanks for the fast reply, by crap out, I mean I can't access the site via HTTP & SSH responses are extremely slow.

If I run a clean install of WP & import my posts by SQL, without activating any plugins I am having the same issue.

With what exact configuration for Apache?

I also second hybinet's suggestion - using top is a quick way to watch your memory usage during the test. If you find yourself exhausting free memory, the working set of your current configuration is too high.

-- David


Top
   
 Post subject:
PostPosted: Wed Nov 17, 2010 9:30 am 
Offline
Senior Member

Joined: Mon Oct 27, 2008 10:24 am
Posts: 173
Website: http://www.worshiproot.com
Enable some sort of caching plugin, drop MaxClients down to 15, and turn off KeepAlives. (I realize these are all things people have already said; I'm just trying to distill the most important points out for you)

Assuming most of your traffic is from readers who aren't logged in, that will get you a _long_ way. Most of the caching plugins out there just serve a static file to unauthenticated users. Apache can serve a _lot_ of static files very quickly.

If you find you still have a lot of free RAM (I mean 'free' in the sense that it's only being used for disk i/o caching http://www.linuxatemyram.com/ ), then feel free to bump MaxClients up a bit. Start low and work your way up.

Remember though, Linux likes to cache disk reads, and is pretty good about improving i/o performance if you leave enough memory lying around for it to use, so don't get too aggressive in reducing your free memory.


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
RSS

Powered by phpBB® Forum Software © phpBB Group