Apache consuming all CPU

Hey all,

Until recently my Linode 4096 (running LAMP on Ubuntu 16) has been happily hosting websites and a TeamSpeak server.

Apache seems to have got into the habit of consuming all of the CPU. This has only been happening for the last few days.

I've used htop and other tools to take a look and see if there's anything obvious such as a runaway script, but to no avail.

I personally have made no changes, however accept that the server is set to auto-update both itself and any of the Wordpress installations that reside upon it.

With regards to tuning, I've previous used apachebuddy to get recommendations.

Here's the output of apachebuddy right now:

########################################################################################
##       ___                     __        ___      ____            __    __          ##
##      /   |  ____  ____ ______/ /_  ___ |__ \    / __ )__  ______/ /___/ /_  __     ##
##     / /| | / __ \/ __ `/ ___/ __ \/ _ \__/ /   / __  / / / / __  / __  / / / /     ##
##    / ___ |/ /_/ / /_/ / /__/ / / /  __/ __/   / /_/ / /_/ / /_/ / /_/ / /_/ /      ##
##   /_/  |_/ .___/\__,_/\___/_/ /_/\___/____/  /_____/\__,_/\__,_/\__,_/\__, /       ##
##         /_/ apache2buddy.pl                                          /____/        ##
##                                                                                    ##
########################################################################################

About...
[ INFO     ] apache2buddy.pl is a fork of apachebuddy.pl.
[ INFO     ] MD5SUMs now availiable at https://raw.githubusercontent.com/richardforth/apache2buddy/master/md5sums.txt
[ INFO     ] SHA256SUMs now availiable at https://raw.githubusercontent.com/richardforth/apache2buddy/master/sha256sums.txt
[ INFO     ] apache2buddy.pl is now released under the Apache 2.0 License. See https://raw.githubusercontent.com/richardforth/apache2buddy/master/LICENSE
[ INFO     ] apache2buddy.pl is now hosted from github. See https://github.com/richardforth/apache2buddy
[ INFO     ] Changelogs and updates in github. See https://raw.githubusercontent.com/richardforth/apache2buddy/master/changelog

Performing Auto-Discovery, and Pre-Flight Checks...
[ OK       ] This script is being run as root.
[ OK       ] The utility 'pmap' exists and is available for use: /usr/bin/pmap
[ OK       ] 'php' exists and is available for use: /usr/bin/php

[ OK       ] The port (port 80) is a valid port.
[ INFO     ] We are attempting to discover the operating system type and version number ...
[ INFO     ] OS Name: Ubuntu
[ INFO     ] OS Release: 16.04
[ OK       ] The operating system is supported.
[ INFO     ] Hostname: arthur.abouthere.uk
[ INFO     ] Primary IP: xxxxxxxxxxx
[ INFO     ] We are checking the service running on port 80...
[ INFO     ] The process listening on port 80 is /usr/sbin/apache2
[ INFO     ] The process running on port 80 is Apache/2.4.18 (Ubuntu).
[ INFO     ] Apache has been running 0d 0h 05m 55s.
[ INFO     ] The full path to the Apache config file is: /etc/apache2/apache2.conf
[ INFO     ] Apache is using prefork model.
[ INFO     ] pidfile setting is /var/run/apache2/apache2$SUFFIX.pid.
[ INFO     ] Actual pidfile is /var/run/apache2/apache2.pid.
[ INFO     ] Parent PID: 5427.
[ OK       ] Memory usage of parent PID is less than 50MB: 8428 Kilobytes.
[ INFO     ] Your server has 3948 MB of PHYSICAL memory.
[ INFO     ] Your MaxRequestWorkers setting is 40.
[ OK       ] Current Apache Process Count is 14, including the parent PID.
[ WARNING  ] ServerLimit directive not found, assuming default values.
[ INFO     ] Your ServerLimit setting is 256.
[ WARNING  ] MaxRequestsPerChild directive not found.

Detecting PHP Memory Limits...
[ INFO     ] Your PHP Memory Limit (Per-Process) is -1 MB.

Detecting additional services for consideration...
[ INFO     ] MySQL Detected => Using 105.07 MB of memory.

Detecting If Maxclients or MaxRequestWorkers has been hit recently....

PRO TIP: If this process appears to hang, press CTRL + c to exit the program, and then
go check for a large error log file in /var/log/httpd or /var/log/apache2.

[ WARNING  ] MaxClients has been hit recently (maximum of 5 results shown), consider the dates and times below:
[Sat Jun 18 22:20:30.342626 2016] [mpm_prefork:error] [pid 4140] AH00161: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting
[Mon Sep 26 18:51:29.649918 2016] [mpm_prefork:error] [pid 3860] AH00161: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting
[Tue Oct 25 01:50:16.538411 2016] [mpm_prefork:error] [pid 19249] AH00161: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting
[Sat Nov 05 14:53:37.463401 2016] [mpm_prefork:error] [pid 3802] AH00161: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting
[Sun Nov 06 23:44:54.207091 2016] [mpm_prefork:error] [pid 13269] AH00161: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting

Analyzing apache memory use...
[ INFO     ] apache2 is currently using 1302.90 MB of memory.
[ INFO     ] The smallest apache process is using 38.39 MB of memory
[ INFO     ] The average apache process is using 54.44 MB of memory
[ INFO     ] The largest apache process is using 69.81 MB of memory

Results...
[ OK       ] Going by the average Apache process, Apache can potentially use 2177.61 MB RAM:
        Without considering services: 55.16 % of total installed RAM
        Considering extra services: 56.67 % of remaining RAM
[ OK       ] Going by the largest Apache process, Apache can potentially use 2792.41 MB RAM:
        Without considering services: 70.73 % of total installed RAM
        Considering extra services: 72.67 % of remaining RAM

Generating reports...
### GENERAL FINDINGS & RECOMMENDATIONS ###
--------------------------------------------------------------------------------
Apache2buddy.pl report for server: xxxxxxxxxxxxxxxxx:

Settings considered for this report:

    Your server's physical RAM:                                   3948 MB
    Remaining Memory after other services considered:             3842 MB
    Apache's MaxRequestWorkers directive:                         40       <--------- Current Setting    
    Apache MPM Model:                                             prefork
    Largest Apache process (by memory):                           69 MB
[ !! ]     Your MaxRequestWorkers setting is too low.
    Your recommended MaxRequestWorkers setting is between 49 and 55.        <------- Acceptable Range (10% of MAX)
    Max potential memory usage:                                   2792 MB
    Percentage of TOTAL RAM allocated to Apache:                  70.73  %
    Percentage of REMAINING RAM allocated to Apache:              72.67  %
--------------------------------------------------------------------------------
A log file entry has been made in: /var/log/apache2buddy.log for future reference.

Last 5 entries:

2016/11/07 09:08:01 Model: "Prefork" Memory: "3948 MB" MaxRequestWorkers: "42" Recommended: "57" Smallest: "23.78 MB" Avg: "49.75 MB" Largest: "67.52 MB" Highest Pct Remaining RAM: "73.04%" (71.83% TOTAL RAM)
2016/11/07 09:11:07 Model: "Prefork" Memory: "3948 MB" MaxRequestWorkers: "42" Recommended: "469" Smallest: "8.27 MB" Avg: "8.27 MB" Largest: "8.27 MB" Highest Pct Remaining RAM: "8.96%" (8.80% TOTAL RAM)
2016/11/07 09:17:46 Model: "Prefork" Memory: "3948 MB" MaxRequestWorkers: "42" Recommended: "54" Smallest: "8.27 MB" Avg: "37.46 MB" Largest: "71.76 MB" Highest Pct Remaining RAM: "77.67%" (76.35% TOTAL RAM)
2016/11/07 09:54:31 Model: "Prefork" Memory: "3948 MB" MaxRequestWorkers: "112" Recommended: "51" Smallest: "8.27 MB" Avg: "47.87 MB" Largest: "73.92 MB" Highest Pct Remaining RAM: "215.54%" (209.71% TOTAL RAM)
2016/11/07 10:01:09 Model: "Prefork" Memory: "3948 MB" MaxRequestWorkers: "40" Recommended: "55" Smallest: "38.39 MB" Avg: "54.44 MB" Largest: "69.81 MB" Highest Pct Remaining RAM: "72.67%" (70.73% TOTAL RAM)

I know for the best results I need to have had Apache running for a period of time.

Here's my a screenshot of htop:

http://pasteboard.co/2qL0spyrm.png

And finally, this is what my current mpm-prefork.conf looks like:

 <ifmodule mpm_prefork_module="">StartServers            5
        MinSpareServers         5
        MaxSpareServers         10
        MaxRequestWorkers       40
        MaxConnectionsPerChild  0</ifmodule> 

 <ifmodule mod_status.c="">ExtendedStatus On
    <location server-status="">SetHandler server-status
        Order deny,allow
        Deny from all
        Allow from 127.0.0.1</location></ifmodule> 

Until a few days ago, I had got it set to the recommend Linode posted settings (found here)

 <ifmodule mpm_prefork_module="">StartServers 4
    MinSpareServers 20
    MaxSpareServers 40
    MaxClients 200
    MaxRequestsPerChild 4500</ifmodule> 

Understandably I'm a little confused about what could have caused the sudden issue with CPU. Wondered if this was perhaps a core update or something others were also experiencing.