Linode Forum
Linode Community Forums
 FAQFAQ    SearchSearch    MembersMembers      Register Register 
 LoginLogin [ Anonymous ] 
Post new topic  Reply to topic
Author Message
 Post subject: Really high disk IO
PostPosted: Mon Apr 04, 2011 9:59 am 
Offline
Senior Newbie

Joined: Sat Oct 02, 2010 5:55 am
Posts: 16
Hi,

I've got two different Linodes, and each one is hosting a website with similar traffic in pageviews and unique visitors: around 400K visitors/month and around 800k pageviews/month.

Those sites perform well, but there's a big difference in them: the disk IO usage. One of the sites has minimal disk IO, and the other one is using lots of disk IO, and I'm getting email alerts all the time (I've set the limit for emails high, but even with that I keep on receiving several emails per day).

The servers are running quite similar things: Both are Ubuntu 10.04 LTS machines, with Nginx web server, PHP-FPM, MySQL and a WordPress blog on each of them.

So I don't know what's the big difference that makes one so different from the other in terms of disk IO consumption.

Can I know which is the cause? How? What should I look for in logs, if I have to look those? What services can be really consuming? Could it be some wordpress plugin? I doubt this, because the config is quite similar between the two...

I attach a couple of images on which you can see the difference in IO Rate. Look at max, min and avg. Nonsense :(

Server 1: good http://i.imgur.com/QKGsl.jpg
Server 2: baaad http://i.imgur.com/oD8os.jpg


Top
   
 Post subject:
PostPosted: Mon Apr 04, 2011 10:30 am 
Offline
Senior Member

Joined: Sun Mar 07, 2010 7:47 pm
Posts: 1970
Website: http://www.rwky.net
Location: Earth
Yes that is quite a difference!

If I was to hazard a guess I'd say either

1) You're swapping a lot
or
2) Your mysql settings need tweaking

install iotop (apt-get install iotop) and run

iotop -ao for say 10 minutes then paste the output here.

_________________
Paid support
How to ask for help
1. Give details of your problem
2. Post any errors
3. Post relevant logs.
4. Don't hide details i.e. your domain, it just makes things harder
5. Be polite or you'll be eaten by a grue


Top
   
 Post subject:
PostPosted: Mon Apr 04, 2011 10:34 am 
Offline
Senior Member

Joined: Fri May 02, 2008 8:44 pm
Posts: 1121
I agree with obs that this is probably swap or mysql (or both).

Could you post a screenshot of vmstat 5 (wait for the latter to print a dozen lines or so) on the "baaad" box?


Top
   
 Post subject: Iotop - vmstat results
PostPosted: Mon Apr 04, 2011 12:29 pm 
Offline
Senior Newbie

Joined: Sat Oct 02, 2010 5:55 am
Posts: 16
Thanks for the suggestions, I'm posting screenshots for the info you requested:

iotop-ml (good): http://i.imgur.com/O4LOh.jpg
iotop-mc (baaad): http://i.imgur.com/kPfMD.jpg
vmstat (baaad): http://i.imgur.com/6oSfw.jpg

I hope that helps!


Top
   
 Post subject: Iotop - vmstat results
PostPosted: Mon Apr 04, 2011 12:30 pm 
Offline
Senior Newbie

Joined: Sat Oct 02, 2010 5:55 am
Posts: 16
Thanks for the suggestions, I'm posting screenshots for the info you requested:

iotop-ml (good): http://i.imgur.com/O4LOh.jpg
iotop-mc (baaad): http://i.imgur.com/kPfMD.jpg
vmstat (baaad): http://i.imgur.com/6oSfw.jpg

I hope that helps!


Top
   
 Post subject:
PostPosted: Mon Apr 04, 2011 12:43 pm 
Offline
Senior Member

Joined: Sun Mar 07, 2010 7:47 pm
Posts: 1970
Website: http://www.rwky.net
Location: Earth
You're not swapping which is good.

I assume the iotops are are from around the same length of time, the good box has done..well pretty much nothing. The second one you have two processes that have done data writing/reading in the MBs, one is an nginx worker and the other is kjournald which is part of the ext3 journaling.

To be honest it looks like the bad one is receiving traffic and the good one isn't, are you serving up large image files/video files on the bad one?

Also what's the content of your /etc/fstab on the bad one.

_________________
Paid support
How to ask for help
1. Give details of your problem
2. Post any errors
3. Post relevant logs.
4. Don't hide details i.e. your domain, it just makes things harder
5. Be polite or you'll be eaten by a grue


Top
   
 Post subject:
PostPosted: Mon Apr 04, 2011 1:00 pm 
Offline
Senior Newbie

Joined: Sat Oct 02, 2010 5:55 am
Posts: 16
Well, in fact as I've said the traffic in both sites is almost the same, and the design is not that heavy on none of them. (Not advertising myself, just if you want to see how is the site displaying content)

The good: www.muylinux.com
the baaad: www.muycomputer.com

There are more images on MuyComputer (the bad one), but we don't serve videos or heavy downloads, so I guess there's just a bad theming there or bad images managing on the WordPress' theme. But I'm not sure.

How can I know mysql is not doing something wrong? There's a cache on mysql, but maybe it's not well optimized...


Top
   
 Post subject:
PostPosted: Mon Apr 04, 2011 5:00 pm 
Offline
Senior Member

Joined: Fri May 02, 2008 8:44 pm
Posts: 1121
On the baaad box, nginx seems to be writing a lot to disk. Do you have any extra logging enabled?


Top
   
 Post subject:
PostPosted: Tue Apr 05, 2011 3:37 am 
Offline
Senior Newbie

Joined: Sat Oct 02, 2010 5:55 am
Posts: 16
extra logging? Sorry, I don't know what's that. I'm logging the typical access and error messages, but nothing else. You mean slow queries in mysql?


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic


Who is online

Users browsing this forum: No registered users and 0 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
RSS

Powered by phpBB® Forum Software © phpBB Group