So my Linode has been around a while now. For over a year now I haven't have any problems with disk IO. I worked hard to optimize everything the best I could and it was always really low.

Then one day it started spiking like this:



Previously, it was pretty much the same as the low peaks all the time.

I don't know when it started or why. I have another linode with nearly the same configuration and it doesn't have these problems. I think it started some time after I did some updates to the some of the software.

I'm running lighttpd and mysql. That's about it. Traffic to the site has not increased at all, so it's not just heavier load.

Everything is running fine in general, and I raised my threshold so Linode would stop sending me emails about it, but it still bothers me.

Anyone seen this before? Any ideas on things I could do hunt down the culprit?

Install iotop and see which process is using the most I/O during one of those spikes.

You seem to be having 3 spikes per hour (every 20 minutes), except between midnight and 6am when the interval is slightly longer. Do you have any cron jobs that run every 20 minutes during the day? Do you have a control panel that might perform periodic updates, etc?

Thanks for the tips.

Cron isn't even installed, so it's not a cron job. As far as I know, I don't have anything that would run periodically like that. But obviously, something is.

I installed iotop, but it won't run:

Could not run iotop as some of the requirements are not met:
- Python >= 2.5 for AF_NETLINK support: Found
- Linux >= 2.6.20 with I/O accounting support: Not found

Any way around that?

This is what I'm seeing with a ps aux:

/sbin/udevd --daemon
dhclient3 -e IF_METRIC=100 -pf /var/run/dhclient.eth0.pid -lf /var/lib/dhcp3/dhclient.eth0.leases eth0
/usr/sbin/ntpd -p /var/run/ntpd.pid -u 109:114 -g
/sbin/syslogd -u syslog
/bin/dd bs 1 if /proc/kmsg of /var/run/klogd/kmsg
/sbin/klogd -P /var/run/klogd/kmsg
/usr/sbin/sshd
/usr/bin/mysqld_safe
/usr/sbin/mysqld 
logger -p daemon.err -t mysqld_safe -i -t mysqld
/usr/sbin/lighttpd -f /etc/lighttpd/lighttpd.conf
/usr/bin/php5-cgi
/usr/bin/php5-cgi
/usr/bin/php5-cgi
/usr/bin/php5-cgi
/sbin/getty 38400 tty1
/usr/bin/php5-cgi
/usr/bin/php5-cgi
/usr/bin/php5-cgi
/usr/bin/php5-cgi

Do any of those look like they could be doing something every 20 minutes?

Run Latest Paravirt. It wants a Linux version higher than 2.6.20, and Latest Stable is 2.6.18.

_________________
Disclaimer: I am no longer employed by Linode; opinions are my own alone.

We have the same problem here. Actually, it's been getting gradually worse and worse. Being that we're inexperienced, we eventually thought it might be RAM so we upgraded. As soon as we doubled our RAM, the oscillation magnitude also doubled.

Our spikes are also 3 per hour. After getting iotop going, I found that syslogd writes a lot to disk. A bit of Google searching said that its default is to run every 20 minutes.....

Now what? Any suggestions?