Munin graphs show absolutely nothing. Meaning it stopped recording at the time of the spike.

The linode CPU graph shows 70% CPU usage no Disk I/O and no Network I/O.

No spike in traffic. Only thing running is 1 mongrel nothing fancy just a blog.

Logging into Lish shows shows nothing. No login prompt.

It's like the entire thing "blue screened" if that were possible.

Anyone else still having problems?

Well you need to look at what was going on up to that problem. Are you watching Apache and MySQL in Munin? Those aren't on by default.

I was getting similar issues. What I found to help was tuning Apache, removing unneeded features in Virtualmin.

Here's some tips. This is written with Virtualmin in mind, but could be applied elsewhere....
- Don't run BIND DNS for your domains. Use Linode's DNS Manager instead. Turn off this feature in Virtualmin.

- If you forward your emails to another provider (e.g. GMail, Yahoo), disable SpamAssassin and Virus Filtering in Virtualmin

- Disable DAV in Virtualmin if you aren't using it. Also, edit your httpd.conf via Webmin and comment out the following lines:

#LoadModule dav_module modules/mod_dav.so
#LoadModule dav_fs_module modules/mod_dav_fs.so

Then open the subversion.conf file and comment out:

#LoadModule dav_svn_module     modules/mod_dav_svn.so
#LoadModule authz_svn_module   modules/mod_authz_svn.so

This keeps Apache from loading an unneeded module. Restart Apache.

- Tune Apache. There's two sections you need to tune up prefork and mpm. These are in the httpd.conf. Here's my setup:

<IfModule prefork.c>
StartServers       8
MinSpareServers    5
MaxSpareServers   10
ServerLimit      128
MaxClients       128
MaxRequestsPerChild  500
</IfModule>

<IfModule worker.c>
StartServers         2
MaxClients         150
MinSpareThreads     25
MaxSpareThreads     75 
ThreadsPerChild     25
MaxRequestsPerChild  0
</IfModule>

Restart Apache to take effect.

-Take a close look at the LoadModule section in your httpd.conf. You may not need every module that's loading. With Virtualmin, there's quite a few.

- Look at the Features and Plugins page. If you aren't using it, uncheck it. Dovecot, Postgres, Mailman, subversion, etc. After you disable the feature, visit the System->Bootup section in Webmin and verify that those services aren't set to start at boot. For some reason Postgres was set to startup on boot for me and I wasn't using it.

- Use SFTP instead of FTP. Then you can stop ProFTPd as well. You can use a client such as Filezilla do your transfers this way. Its also much more secure.

That ought to get you started.

_________________
Digital Deviation - http://digitaldeviation.com

Yup Apache and MySQL are in there. There is no spike or indication of any run-up to the problem.

Everything is showing almost zero activity and then all of a sudden the graph just disappears.

Would be great if I could see what was actually using CPU. Anyone know of any other way to track what programs are using what? Kind of like a recording of top?

It really looks like an all out crash.

There's no BIND running, no virtualmin, Apache is stripped down. No memory issues at all on the box.

If it wasn't for the Linode CPU graph I'd have no idea that the box was doing anything at all. Would just appear to the outside world to have gone down.

The only thing that spikes is CPU? What's your IO rate at when the CPU spikes?

_________________
Digital Deviation - http://digitaldeviation.com

Ah shoot I didn't save the graphs from linode. Next time I will.

CPU was right at 70% Disk I/O and network activity were 0. Not even registering on the graph.

New linode user, and happy with it.

Started noticing the same CPU and I/O problems, CPU maxing for hours at 400%, I/O going up and down in the graph.

To maybe help someone else looking for solutions, here is what I did. I started with a Ubuntu 8.10.

apache: changed settings for maxclients and threads to around a 1/4 of the default, as mentioned earlier.
mysql: skip-innodb, set it to log slow-queries

these helped lower the CPU max to around 280%. In my case skipping InnoDB helped, even though I had to switch to another engine for some databases.

timezone: set to my timezone (Europe)

As the peaks started around the hour, I asked myself if cronjobs were partle to blame. They weren't.

ufw: can't remember what I set as default


About 6 hours later I was back at 400%. So I decided to switch to Ubuntu 8.04, as it is LTS and 8.10 isn't.

repeated the above steps, and changed UFW to default deny. Graphs are now looking nicely, bubbling at the bottom at the X axis.

It could be a bug in your code (php ... etc.) where a tight loop keeps exhausting the total allowable memory for an apache thread. When that thread runs out of memory, it freezes so the next request spawns another thread, the next thread ends up sharing the same faith. That happens till your system runs out of memory. So, mysql starves to death ... and so on ... The path of execution can be such that the end result occurs once in a few days. If you have access to a shell. At the prompt type ps -ef | grep httpd (or appache) and see what is going on, then you can use strace to see what and where the dead thread doing last.