Linode Forum
Linode Community Forums
 FAQFAQ    SearchSearch    MembersMembers      Register Register 
 LoginLogin [ Anonymous ] 
Post new topic  Reply to topic
Author Message
PostPosted: Tue Nov 27, 2012 9:49 am 
Offline
Senior Newbie
User avatar

Joined: Sun Apr 12, 2009 4:11 pm
Posts: 12
Hi -

Logwatch sent me this. I know MySQL crashed yesterday. Not sure why it or Apache would OOM. Also not sure what role kernel errors play here. Thanks very much for any assistance.

(Ubuntu 12.04.1 LTS)

Quote:
--------------------- Kernel Begin ------------------------


WARNING: Kernel Errors Present
[<c0122720>] ? mm_fault_error+0x160/0x160 ...: 16 Time(s)
[<c072e582>] ? error_code+0x5a/0x60 ...: 8 Time(s)

2 Time(s): cache: size-32768, object size: 32768, order: 3
1 Time(s): node 0: slabs: 1/1, objs: 1/1, free: 0
1 Time(s): node 0: slabs: 2/2, objs: 2/2, free: 0
2 Time(s): /usr/share/webm cpuset=/ mems_allowed=0
2 Time(s): /usr/share/webm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
11 Time(s): 0 pages HighMem
1 Time(s): 10020 pages shared
1 Time(s): 104736 pages non-shared
1 Time(s): 11631 pages in swap cache
1 Time(s): 117506 pages non-shared
1 Time(s): 123031 pages non-shared
1 Time(s): 123498 pages non-shared
1 Time(s): 123704 pages non-shared
1 Time(s): 12402 total pagecache pages
1 Time(s): 124207 pages non-shared
1 Time(s): 124326 pages non-shared
1 Time(s): 124363 pages non-shared
1 Time(s): 124461 pages non-shared
1 Time(s): 124515 pages non-shared
1 Time(s): 124911 pages non-shared
11 Time(s): 133104 pages RAM
1 Time(s): 1762 pages in swap cache
1 Time(s): 18484 total pagecache pages
1 Time(s): 2296 pages in swap cache
1 Time(s): 2447 total pagecache pages
1 Time(s): 2501 pages in swap cache
1 Time(s): 28879 pages shared
1 Time(s): 2936 total pagecache pages
1 Time(s): 3071 pages in swap cache
1 Time(s): 3325 total pagecache pages
1 Time(s): 3517 total pagecache pages
1 Time(s): 37423 pages shared
1 Time(s): 44325 total pagecache pages
1 Time(s): 4583 pages in swap cache
1 Time(s): 4758 pages in swap cache
1 Time(s): 5200 pages shared
1 Time(s): 5249 pages shared
1 Time(s): 5412 total pagecache pages
1 Time(s): 5606 pages in swap cache
11 Time(s): 5960 pages reserved
1 Time(s): 6113 pages shared
1 Time(s): 6283 pages shared
1 Time(s): 6493 pages in swap cache
1 Time(s): 6496 pages shared
1 Time(s): 6650 pages shared
1 Time(s): 6670 pages in swap cache
1 Time(s): 6886 pages shared
1 Time(s): 7297 pages in swap cache
1 Time(s): 7629 total pagecache pages
1 Time(s): 7676 total pagecache pages
1 Time(s): 7898 pages shared
1 Time(s): 8216 total pagecache pages
11 Time(s): CPU 0: hi: 0, btch: 1 usd: 0
4 Time(s): CPU 0: hi: 186, btch: 31 usd: 0
1 Time(s): CPU 0: hi: 186, btch: 31 usd: 4
1 Time(s): CPU 0: hi: 186, btch: 31 usd: 24
2 Time(s): CPU 0: hi: 186, btch: 31 usd: 30
1 Time(s): CPU 0: hi: 186, btch: 31 usd: 42
1 Time(s): CPU 0: hi: 186, btch: 31 usd: 58
1 Time(s): CPU 0: hi: 186, btch: 31 usd: 113
11 Time(s): CPU 1: hi: 0, btch: 1 usd: 0
6 Time(s): CPU 1: hi: 186, btch: 31 usd: 0
1 Time(s): CPU 1: hi: 186, btch: 31 usd: 27
1 Time(s): CPU 1: hi: 186, btch: 31 usd: 30
1 Time(s): CPU 1: hi: 186, btch: 31 usd: 32
1 Time(s): CPU 1: hi: 186, btch: 31 usd: 59
1 Time(s): CPU 1: hi: 186, btch: 31 usd: 125
11 Time(s): CPU 2: hi: 0, btch: 1 usd: 0
3 Time(s): CPU 2: hi: 186, btch: 31 usd: 0
1 Time(s): CPU 2: hi: 186, btch: 31 usd: 1
1 Time(s): CPU 2: hi: 186, btch: 31 usd: 5
1 Time(s): CPU 2: hi: 186, btch: 31 usd: 27
1 Time(s): CPU 2: hi: 186, btch: 31 usd: 30
1 Time(s): CPU 2: hi: 186, btch: 31 usd: 35
1 Time(s): CPU 2: hi: 186, btch: 31 usd: 80
1 Time(s): CPU 2: hi: 186, btch: 31 usd: 133
1 Time(s): CPU 2: hi: 186, btch: 31 usd: 156
11 Time(s): CPU 3: hi: 0, btch: 1 usd: 0
5 Time(s): CPU 3: hi: 186, btch: 31 usd: 0
1 Time(s): CPU 3: hi: 186, btch: 31 usd: 3
1 Time(s): CPU 3: hi: 186, btch: 31 usd: 7
1 Time(s): CPU 3: hi: 186, btch: 31 usd: 25
1 Time(s): CPU 3: hi: 186, btch: 31 usd: 30
1 Time(s): CPU 3: hi: 186, btch: 31 usd: 115
1 Time(s): CPU 3: hi: 186, btch: 31 usd: 158
11 Time(s): Call Trace:
1 Time(s): DMA free:2044kB min:84kB low:104kB high:124kB active_anon:1276kB inactive_anon:1588kB active_file:1240kB inactive_file:592kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15808kB mlocked:0kB dirty:0kB writeback:0kB mapped:716kB shmem:1460kB slab_reclaimable:48kB slab_unreclaimable:24kB kernel_stack:8kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
1 Time(s): DMA free:2056kB min:84kB low:104kB high:124kB active_anon:1464kB inactive_anon:1588kB active_file:1300kB inactive_file:344kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15808kB mlocked:0kB dirty:0kB writeback:0kB mapped:716kB shmem:1460kB slab_reclaimable:48kB slab_unreclaimable:20kB kernel_stack:8kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:12 all_unreclaimable? no
1 Time(s): DMA free:2080kB min:84kB low:104kB high:124kB active_anon:2040kB inactive_anon:2456kB active_file:0kB inactive_file:60kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15808kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:812kB slab_reclaimable:40kB slab_unreclaimable:72kB kernel_stack:24kB pagetables:20kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:165 all_unreclaimable? yes
1 Time(s): DMA free:2080kB min:84kB low:104kB high:124kB active_anon:2128kB inactive_anon:2304kB active_file:8kB inactive_file:52kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15808kB mlocked:0kB dirty:0kB writeback:4kB mapped:12kB shmem:640kB slab_reclaimable:40kB slab_unreclaimable:60kB kernel_stack:88kB pagetables:24kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:141 all_unreclaimable? yes
1 Time(s): DMA free:2080kB min:84kB low:104kB high:124kB active_anon:2148kB inactive_anon:2220kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15808kB mlocked:0kB dirty:0kB writeback:0kB mapped:8kB shmem:560kB slab_reclaimable:40kB slab_unreclaimable:172kB kernel_stack:96kB pagetables:24kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
1 Time(s): DMA free:2080kB min:84kB low:104kB high:124kB active_anon:2192kB inactive_anon:2304kB active_file:0kB inactive_file:92kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15808kB mlocked:0kB dirty:0kB writeback:0kB mapped:32kB shmem:1480kB slab_reclaimable:60kB slab_unreclaimable:24kB kernel_stack:40kB pagetables:12kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
1 Time(s): DMA free:2080kB min:84kB low:104kB high:124kB active_anon:2204kB inactive_anon:2348kB active_file:0kB inactive_file:16kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15808kB mlocked:0kB dirty:0kB writeback:0kB mapped:32kB shmem:1480kB slab_reclaimable:52kB slab_unreclaimable:24kB kernel_stack:32kB pagetables:12kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
1 Time(s): DMA free:2084kB min:84kB low:104kB high:124kB active_anon:2104kB inactive_anon:2140kB active_file:52kB inactive_file:120kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15808kB mlocked:0kB dirty:0kB writeback:4kB mapped:32kB shmem:696kB slab_reclaimable:40kB slab_unreclaimable:156kB kernel_stack:88kB pagetables:16kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:455 all_unreclaimable? yes
1 Time(s): DMA free:2084kB min:84kB low:104kB high:124kB active_anon:2144kB inactive_anon:2304kB active_file:4kB inactive_file:0kB unevictable:0kB isolated(anon):112kB isolated(file):0kB present:15808kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:664kB slab_reclaimable:40kB slab_unreclaimable:28kB kernel_stack:80kB pagetables:20kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:103 all_unreclaimable? yes
1 Time(s): DMA free:2084kB min:84kB low:104kB high:124kB active_anon:2244kB inactive_anon:2284kB active_file:0kB inactive_file:20kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15808kB mlocked:0kB dirty:0kB writeback:0kB mapped:12kB shmem:812kB slab_reclaimable:52kB slab_unreclaimable:24kB kernel_stack:24kB pagetables:44kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:60 all_unreclaimable? yes
1 Time(s): DMA free:2108kB min:84kB low:104kB high:124kB active_anon:2088kB inactive_anon:2140kB active_file:32kB inactive_file:264kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15808kB mlocked:0kB dirty:0kB writeback:4kB mapped:36kB shmem:1444kB slab_reclaimable:44kB slab_unreclaimable:52kB kernel_stack:8kB pagetables:12kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
11 Time(s): DMA per-cpu:
1 Time(s): DMA: 1*4kB 17*8kB 40*16kB 35*32kB 3*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2092kB
1 Time(s): DMA: 1*4kB 3*8kB 33*16kB 34*32kB 7*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2092kB
2 Time(s): DMA: 1*4kB 9*8kB 49*16kB 33*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2044kB
1 Time(s): DMA: 11*4kB 24*8kB 42*16kB 37*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2092kB
1 Time(s): DMA: 19*4kB 39*8kB 32*16kB 25*32kB 6*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2084kB
1 Time(s): DMA: 25*4kB 26*8kB 43*16kB 34*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2084kB
1 Time(s): DMA: 3*4kB 16*8kB 22*16kB 32*32kB 9*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2092kB
1 Time(s): DMA: 5*4kB 28*8kB 31*16kB 26*32kB 8*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2084kB
1 Time(s): DMA: 7*4kB 15*8kB 11*16kB 34*32kB 11*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2116kB
1 Time(s): DMA: 7*4kB 19*8kB 41*16kB 37*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2084kB
9 Time(s): Free swap = 0kB
1 Time(s): Free swap = 118344kB
1 Time(s): Free swap = 137272kB
1 Time(s): Killed process 29433 (apache2) total-vm:102384kB, anon-rss:52932kB, file-rss:0kB
1 Time(s): Killed process 29528 (apache2) total-vm:191540kB, anon-rss:133988kB, file-rss:0kB
1 Time(s): Killed process 29902 (apache2) total-vm:102336kB, anon-rss:22324kB, file-rss:0kB
1 Time(s): Killed process 29977 (mysqld) total-vm:333508kB, anon-rss:9664kB, file-rss:0kB
1 Time(s): Killed process 30143 (apache2) total-vm:102336kB, anon-rss:41896kB, file-rss:0kB
1 Time(s): Killed process 30212 (apache2) total-vm:103060kB, anon-rss:38388kB, file-rss:0kB
1 Time(s): Killed process 30219 (apache2) total-vm:110128kB, anon-rss:54168kB, file-rss:0kB
1 Time(s): Killed process 30221 (apache2) total-vm:104692kB, anon-rss:41108kB, file-rss:24kB
1 Time(s): Killed process 4524 (mysqld) total-vm:348068kB, anon-rss:26300kB, file-rss:0kB
11 Time(s): Mem-Info:
1 Time(s): Normal free:2704kB min:2816kB low:3520kB high:4224kB active_anon:219192kB inactive_anon:219476kB active_file:744kB inactive_file:1108kB unevictable:0kB isolated(anon):52kB isolated(file):68kB present:512064kB mlocked:0kB dirty:0kB writeback:0kB mapped:684kB shmem:108kB slab_reclaimable:28080kB slab_unreclaimable:11688kB kernel_stack:1808kB pagetables:5324kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2945 all_unreclaimable? yes
1 Time(s): Normal free:2720kB min:2816kB low:3520kB high:4224kB active_anon:219676kB inactive_anon:219832kB active_file:584kB inactive_file:996kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:512064kB mlocked:0kB dirty:0kB writeback:0kB mapped:724kB shmem:784kB slab_reclaimable:27616kB slab_unreclaimable:11516kB kernel_stack:1800kB pagetables:5420kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2545 all_unreclaimable? yes
1 Time(s): Normal free:2732kB min:2816kB low:3520kB high:4224kB active_anon:219792kB inactive_anon:219900kB active_file:372kB inactive_file:572kB unevictable:0kB isolated(anon):128kB isolated(file):0kB present:512064kB mlocked:0kB dirty:0kB writeback:168kB mapped:744kB shmem:488kB slab_reclaimable:27748kB slab_unreclaimable:11376kB kernel_stack:1792kB pagetables:5368kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1391 all_unreclaimable? yes
1 Time(s): Normal free:2772kB min:2816kB low:3520kB high:4224kB active_anon:219004kB inactive_anon:219144kB active_file:1012kB inactive_file:1312kB unevictable:0kB isolated(anon):128kB isolated(file):0kB present:512064kB mlocked:0kB dirty:0kB writeback:108kB mapped:1156kB shmem:976kB slab_reclaimable:27464kB slab_unreclaimable:11496kB kernel_stack:1776kB pagetables:5792kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:3685 all_unreclaimable? yes
1 Time(s): Normal free:2772kB min:2816kB low:3520kB high:4224kB active_anon:219232kB inactive_anon:219240kB active_file:548kB inactive_file:812kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:512064kB mlocked:0kB dirty:0kB writeback:0kB mapped:540kB shmem:972kB slab_reclaimable:27556kB slab_unreclaimable:11492kB kernel_stack:1760kB pagetables:5680kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2217 all_unreclaimable? yes
1 Time(s): Normal free:2780kB min:2816kB low:3520kB high:4224kB active_anon:220276kB inactive_anon:220284kB active_file:36kB inactive_file:152kB unevictable:0kB isolated(anon):128kB isolated(file):0kB present:512064kB mlocked:0kB dirty:4kB writeback:0kB mapped:276kB shmem:764kB slab_reclaimable:27468kB slab_unreclaimable:11608kB kernel_stack:1744kB pagetables:5612kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:529 all_unreclaimable? yes
1 Time(s): Normal free:2788kB min:2816kB low:3520kB high:4224kB active_anon:219700kB inactive_anon:219920kB active_file:524kB inactive_file:648kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:512064kB mlocked:0kB dirty:0kB writeback:12kB mapped:604kB shmem:976kB slab_reclaimable:27540kB slab_unreclaimable:11544kB kernel_stack:1744kB pagetables:5560kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1851 all_unreclaimable? yes
1 Time(s): Normal free:2816kB min:2816kB low:3520kB high:4224kB active_anon:220008kB inactive_anon:221364kB active_file:8kB inactive_file:108kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:512064kB mlocked:0kB dirty:0kB writeback:8kB mapped:208kB shmem:680kB slab_reclaimable:28500kB slab_unreclaimable:11112kB kernel_stack:1712kB pagetables:5080kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:201 all_unreclaimable? yes
1 Time(s): Normal free:3888kB min:2816kB low:3520kB high:4224kB active_anon:220412kB inactive_anon:220436kB active_file:68kB inactive_file:184kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:512064kB mlocked:0kB dirty:0kB writeback:88kB mapped:316kB shmem:688kB slab_reclaimable:28348kB slab_unreclaimable:11184kB kernel_stack:1464kB pagetables:4976kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
1 Time(s): Normal free:4216kB min:2816kB low:3520kB high:4224kB active_anon:195832kB inactive_anon:196092kB active_file:25712kB inactive_file:21644kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:512064kB mlocked:0kB dirty:68kB writeback:24kB mapped:15772kB shmem:1132kB slab_reclaimable:29448kB slab_unreclaimable:11096kB kernel_stack:1656kB pagetables:4412kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:82 all_unreclaimable? no
1 Time(s): Normal free:4312kB min:2816kB low:3520kB high:4224kB active_anon:149632kB inactive_anon:149808kB active_file:77108kB inactive_file:62460kB unevictable:0kB isolated(anon):0kB isolated(file):108kB present:512064kB mlocked:0kB dirty:108kB writeback:0kB mapped:13568kB shmem:16068kB slab_reclaimable:29880kB slab_unreclaimable:10660kB kernel_stack:1608kB pagetables:3964kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:49 all_unreclaimable? no
11 Time(s): Normal per-cpu:
1 Time(s): Normal: 566*4kB 49*8kB 3*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2704kB
1 Time(s): Normal: 601*4kB 82*8kB 86*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 4436kB
1 Time(s): Normal: 628*4kB 28*8kB 8*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2864kB
1 Time(s): Normal: 647*4kB 30*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2844kB
1 Time(s): Normal: 648*4kB 14*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2720kB
1 Time(s): Normal: 651*4kB 21*8kB 2*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2804kB
1 Time(s): Normal: 667*4kB 15*8kB 3*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2836kB
1 Time(s): Normal: 689*4kB 3*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2780kB
1 Time(s): Normal: 691*4kB 1*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2772kB
1 Time(s): Normal: 794*4kB 130*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 4216kB
1 Time(s): Normal: 878*4kB 41*8kB 3*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 3888kB
1 Time(s): Out of memory: Kill process 29433 (apache2) score 73 or sacrifice child
1 Time(s): Out of memory: Kill process 29528 (apache2) score 176 or sacrifice child
1 Time(s): Out of memory: Kill process 29902 (apache2) score 73 or sacrifice child
1 Time(s): Out of memory: Kill process 29977 (mysqld) score 71 or sacrifice child
1 Time(s): Out of memory: Kill process 30143 (apache2) score 73 or sacrifice child
1 Time(s): Out of memory: Kill process 30212 (apache2) score 71 or sacrifice child
1 Time(s): Out of memory: Kill process 30219 (apache2) score 79 or sacrifice child
1 Time(s): Out of memory: Kill process 30221 (apache2) score 75 or sacrifice child
1 Time(s): Out of memory: Kill process 4524 (mysqld) score 110 or sacrifice child
1 Time(s): Pid: 18922, comm: apache2 Not tainted 3.5.2-linode45 #1
1 Time(s): Pid: 23616, comm: apache2 Not tainted 3.5.2-linode45 #1
1 Time(s): Pid: 29528, comm: apache2 Not tainted 3.5.2-linode45 #1
1 Time(s): Pid: 29547, comm: miniserv.pl Not tainted 3.5.2-linode45 #1
1 Time(s): Pid: 30009, comm: mysqld Not tainted 3.5.2-linode45 #1
1 Time(s): Pid: 30081, comm: /usr/share/webm Not tainted 3.5.2-linode45 #1
1 Time(s): Pid: 30136, comm: apache2 Not tainted 3.5.2-linode45 #1
1 Time(s): Pid: 30214, comm: apache2 Not tainted 3.5.2-linode45 #1
1 Time(s): Pid: 30221, comm: apache2 Not tainted 3.5.2-linode45 #1
1 Time(s): Pid: 30227, comm: apache2 Not tainted 3.5.2-linode45 #1
1 Time(s): Pid: 30326, comm: /usr/share/webm Not tainted 3.5.2-linode45 #1
2 Time(s): SLAB: Unable to allocate memory on node 0 (gfp=0x20)
1 Time(s): Swap cache stats: add 1010198, delete 1005440, find 194980520/195074597
1 Time(s): Swap cache stats: add 1121277, delete 1114607, find 195123126/195225106
1 Time(s): Swap cache stats: add 1161697, delete 1159935, find 195141635/195247703
1 Time(s): Swap cache stats: add 1179851, delete 1172554, find 195144902/195252043
1 Time(s): Swap cache stats: add 1221611, delete 1215118, find 195167240/195278999
1 Time(s): Swap cache stats: add 1239860, delete 1237359, find 195185647/195299303
1 Time(s): Swap cache stats: add 1260371, delete 1248740, find 195190747/195306322
1 Time(s): Swap cache stats: add 1280798, delete 1277727, find 195198043/195315915
1 Time(s): Swap cache stats: add 877305, delete 872722, find 192807923/192893729
1 Time(s): Swap cache stats: add 887937, delete 882331, find 193564369/193650979
1 Time(s): Swap cache stats: add 991791, delete 989495, find 194977489/195071092
11 Time(s): Total swap = 262140kB



Breaking in here to ask: what specifically do the below refer to? What am I reading? Errors that happened to a process? And what's different from those above?

Quote:
1 Time(s): [ 1987] 0 1987 712 0 3 0 0 upstart-socket-
1 Time(s): [ 1987] 0 1987 712 12 0 0 0 upstart-socket-
5 Time(s): [ 1987] 0 1987 712 13 0 0 0 upstart-socket-
1 Time(s): [ 1987] 0 1987 712 13 2 0 0 upstart-socket-
1 Time(s): [ 1987] 0 1987 712 24 0 0 0 upstart-socket-
1 Time(s): [ 2075] 0 2075 1672 0 3 -17 -1000 sshd
1 Time(s): [ 2075] 0 2075 1672 1 3 -17 -1000 sshd
1 Time(s): [ 2075] 0 2075 1672 3 1 -17 -1000 sshd
3 Time(s): [ 2075] 0 2075 1672 4 1 -17 -1000 sshd
2 Time(s): [ 2075] 0 2075 1672 11 1 -17 -1000 sshd
1 Time(s): [ 2075] 0 2075 1672 12 1 -17 -1000 sshd
1 Time(s): [ 2085] 101 2085 8071 50 0 0 0 rsyslogd
1 Time(s): [ 2085] 101 2085 8071 55 0 0 0 rsyslogd
2 Time(s): [ 2085] 101 2085 8071 68 0 0 0 rsyslogd
3 Time(s): [ 2085] 101 2085 8071 80 0 0 0 rsyslogd
2 Time(s): [ 2085] 101 2085 8071 81 0 0 0 rsyslogd
9 Time(s): [ 2098] 0 2098 733 0 0 0 0 dhclient3
1 Time(s): [ 2104] 102 2104 816 0 0 0 0 dbus-daemon
1 Time(s): [ 2104] 102 2104 816 6 2 0 0 dbus-daemon
5 Time(s): [ 2104] 102 2104 816 7 2 0 0 dbus-daemon
1 Time(s): [ 2104] 102 2104 816 15 3 0 0 dbus-daemon
1 Time(s): [ 2104] 102 2104 816 23 2 0 0 dbus-daemon
9 Time(s): [ 2190] 0 2190 619 0 0 0 0 atd
2 Time(s): [ 2191] 0 2191 656 15 1 0 0 cron
7 Time(s): [ 2191] 0 2191 656 21 0 0 0 cron
1 Time(s): [ 2244] 103 2244 6117 31 0 0 0 whoopsie
1 Time(s): [ 2244] 103 2244 6117 33 0 0 0 whoopsie
2 Time(s): [ 2244] 103 2244 6117 33 1 0 0 whoopsie
1 Time(s): [ 2244] 103 2244 6117 33 3 0 0 whoopsie
2 Time(s): [ 2244] 103 2244 6117 34 0 0 0 whoopsie
1 Time(s): [ 2244] 103 2244 6117 34 1 0 0 whoopsie
1 Time(s): [ 2244] 103 2244 6117 34 3 0 0 whoopsie
2 Time(s): [ 2681] 0 2681 602 0 0 0 0 getty
7 Time(s): [ 2681] 0 2681 602 1 0 0 0 getty
3 Time(s): [ 2731] 106 2731 1434 29 0 0 0 ntpd
2 Time(s): [ 2731] 106 2731 1434 29 2 0 0 ntpd
1 Time(s): [ 2731] 106 2731 1434 35 0 0 0 ntpd
1 Time(s): [ 2731] 106 2731 1434 36 0 0 0 ntpd
1 Time(s): [ 2731] 106 2731 1434 36 1 0 0 ntpd
1 Time(s): [ 2731] 106 2731 1434 38 1 0 0 ntpd
1 Time(s): [ 4524] 108 4524 87017 6575 1 0 0 mysqld
1 Time(s): [ 5952] 0 5952 1146 11 1 0 0 master
2 Time(s): [ 5952] 0 5952 1146 19 1 0 0 master
2 Time(s): [ 5952] 0 5952 1146 21 0 0 0 master
1 Time(s): [ 5952] 0 5952 1146 22 1 0 0 master
1 Time(s): [ 5952] 0 5952 1146 22 2 0 0 master
1 Time(s): [ 5952] 0 5952 1146 23 2 0 0 master
1 Time(s): [ 5952] 0 5952 1146 23 3 0 0 master
1 Time(s): [ 5954] 107 5954 1163 19 0 0 0 qmgr
5 Time(s): [ 5954] 107 5954 1163 20 0 0 0 qmgr
1 Time(s): [ 5954] 107 5954 1163 23 1 0 0 qmgr
2 Time(s): [ 5954] 107 5954 1163 24 0 0 0 qmgr
5 Time(s): [ 6135] 107 6135 1799 3 0 0 0 tlsmgr
1 Time(s): [ 6135] 107 6135 1799 4 0 0 0 tlsmgr
1 Time(s): [ 6135] 107 6135 1799 25 1 0 0 tlsmgr
1 Time(s): [ 6135] 107 6135 1799 26 1 0 0 tlsmgr
1 Time(s): [ 6135] 107 6135 1799 27 0 0 0 tlsmgr
9 Time(s): [ 7178] 1002 7178 1458 0 0 0 0 screen
1 Time(s): [ 7179] 1002 7179 1862 0 1 0 0 bash
8 Time(s): [ 7179] 1002 7179 1862 1 1 0 0 bash
1 Time(s): [ 7347] 0 7347 4838 178 0 0 0 miniserv.pl
1 Time(s): [ 7347] 0 7347 4838 195 0 0 0 miniserv.pl
1 Time(s): [ 7347] 0 7347 4838 196 3 0 0 miniserv.pl
1 Time(s): [ 7347] 0 7347 4838 241 0 0 0 miniserv.pl
1 Time(s): [ 7347] 0 7347 4838 250 2 0 0 miniserv.pl
1 Time(s): [ 7347] 0 7347 4838 317 0 0 0 miniserv.pl
1 Time(s): [ 7347] 0 7347 4838 332 3 0 0 miniserv.pl
1 Time(s): [ 7347] 0 7347 4838 339 1 0 0 miniserv.pl
1 Time(s): [ 7347] 0 7347 4838 343 0 0 0 miniserv.pl
2 Time(s): [ 7387] 1002 7387 1860 0 3 0 0 bash
7 Time(s): [ 7387] 1002 7387 1860 1 3 0 0 bash
9 Time(s): [ 7486] 1002 7486 3188 0 1 0 0 vim
9 Time(s): [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
4 Time(s): [11370] 1000 11370 12041 0 0 0 0 memcached
2 Time(s): [11370] 1000 11370 12041 0 1 0 0 memcached
2 Time(s): [11370] 1000 11370 12041 0 2 0 0 memcached
1 Time(s): [11370] 1000 11370 12041 3 1 0 0 memcached
1 Time(s): [11740] 0 11740 710 0 2 0 0 upstart-udev-br
1 Time(s): [11740] 0 11740 710 4 2 0 0 upstart-udev-br
5 Time(s): [11740] 0 11740 710 5 2 0 0 upstart-udev-br
1 Time(s): [11740] 0 11740 710 16 2 0 0 upstart-udev-br
1 Time(s): [11740] 0 11740 710 17 2 0 0 upstart-udev-br
1 Time(s): [11742] 0 11742 711 0 0 -17 -1000 udevd
8 Time(s): [11742] 0 11742 711 1 0 -17 -1000 udevd
2 Time(s): [19800] 1002 19800 1705 0 0 0 0 screen
7 Time(s): [19800] 1002 19800 1705 1 0 0 0 screen
2 Time(s): [19801] 1002 19801 1859 0 3 0 0 bash
7 Time(s): [19801] 1002 19801 1859 1 3 0 0 bash
2 Time(s): [19857] 1002 19857 1860 0 0 0 0 bash
7 Time(s): [19857] 1002 19857 1860 1 0 0 0 bash
2 Time(s): [19963] 1002 19963 3199 0 0 0 0 vim
7 Time(s): [19963] 1002 19963 3199 1 0 0 0 vim
2 Time(s): [21674] 1002 21674 3187 0 2 0 0 vim
7 Time(s): [21674] 1002 21674 3187 1 2 0 0 vim
2 Time(s): [21676] 1002 21676 1859 0 1 0 0 bash
7 Time(s): [21676] 1002 21676 1859 1 1 0 0 bash
2 Time(s): [21782] 1002 21782 1860 0 0 0 0 bash
7 Time(s): [21782] 1002 21782 1860 1 0 0 0 bash
9 Time(s): [22370] 1002 22370 1458 0 2 0 0 screen
9 Time(s): [22371] 1002 22371 1860 0 1 0 0 bash
1 Time(s): [24402] 109 24402 12555 28 0 -13 -900 postgres
1 Time(s): [24402] 109 24402 12555 28 2 -13 -900 postgres
3 Time(s): [24402] 109 24402 12555 29 0 -13 -900 postgres
1 Time(s): [24402] 109 24402 12555 29 2 -13 -900 postgres
1 Time(s): [24402] 109 24402 12555 29 3 -13 -900 postgres
1 Time(s): [24402] 109 24402 12555 30 0 -13 -900 postgres
1 Time(s): [24402] 109 24402 12555 31 0 -13 -900 postgres
2 Time(s): [24404] 109 24404 12586 20 0 0 0 postgres
2 Time(s): [24404] 109 24404 12586 20 3 0 0 postgres
1 Time(s): [24404] 109 24404 12586 21 0 0 0 postgres
2 Time(s): [24404] 109 24404 12586 21 1 0 0 postgres
1 Time(s): [24404] 109 24404 12586 21 2 0 0 postgres
1 Time(s): [24404] 109 24404 12586 33 0 0 0 postgres
1 Time(s): [24405] 109 24405 12555 13 2 0 0 postgres
2 Time(s): [24405] 109 24405 12555 14 0 0 0 postgres
1 Time(s): [24405] 109 24405 12555 14 1 0 0 postgres
4 Time(s): [24405] 109 24405 12555 14 2 0 0 postgres
1 Time(s): [24405] 109 24405 12555 14 3 0 0 postgres
2 Time(s): [24406] 109 24406 12727 100 1 0 0 postgres
2 Time(s): [24406] 109 24406 12727 101 0 0 0 postgres
2 Time(s): [24406] 109 24406 12727 101 3 0 0 postgres
1 Time(s): [24406] 109 24406 12727 114 2 0 0 postgres
1 Time(s): [24406] 109 24406 12727 114 3 0 0 postgres
1 Time(s): [24406] 109 24406 12727 124 0 0 0 postgres
1 Time(s): [24407] 109 24407 5180 64 2 0 0 postgres
1 Time(s): [24407] 109 24407 5180 65 0 0 0 postgres
2 Time(s): [24407] 109 24407 5180 66 0 0 0 postgres
3 Time(s): [24407] 109 24407 5180 66 1 0 0 postgres
1 Time(s): [24407] 109 24407 5180 66 2 0 0 postgres
1 Time(s): [24407] 109 24407 5180 66 3 0 0 postgres
2 Time(s): [24636] 0 24636 1110 0 1 0 0 newrelic-daemon
7 Time(s): [24636] 0 24636 1110 1 1 0 0 newrelic-daemon
8 Time(s): [24638] 0 24638 25485 0 0 0 0 newrelic-daemon
1 Time(s): [24638] 0 24638 25485 0 1 0 0 newrelic-daemon
2 Time(s): [25141] 1002 25141 3184 0 1 0 0 vim
7 Time(s): [25141] 1002 25141 3184 1 1 0 0 vim
2 Time(s): [26701] 1002 26701 1860 0 2 0 0 bash
7 Time(s): [26701] 1002 26701 1860 1 2 0 0 bash
2 Time(s): [27283] 0 27283 2415 0 0 0 0 sshd
7 Time(s): [27283] 0 27283 2415 1 0 0 0 sshd
9 Time(s): [27427] 1000 27427 2415 0 1 0 0 sshd
2 Time(s): [27428] 1000 27428 2592 0 2 0 0 bash
7 Time(s): [27428] 1000 27428 2592 1 2 0 0 bash
1 Time(s): [29086] 107 29086 1151 12 0 0 0 pickup
1 Time(s): [29086] 107 29086 1151 13 0 0 0 pickup
1 Time(s): [29086] 107 29086 1151 13 2 0 0 pickup
2 Time(s): [29086] 107 29086 1151 15 0 0 0 pickup
1 Time(s): [29086] 107 29086 1151 15 1 0 0 pickup
1 Time(s): [29086] 107 29086 1151 17 0 0 0 pickup
1 Time(s): [29086] 107 29086 1151 17 1 0 0 pickup
1 Time(s): [29086] 107 29086 1151 17 2 0 0 pickup
2 Time(s): [29388] 33 29388 26114 12612 0 0 0 apache2
2 Time(s): [29433] 33 29433 25596 3533 1 0 0 apache2
1 Time(s): [29433] 33 29433 25596 13233 1 0 0 apache2
2 Time(s): [29460] 33 29460 21541 256 1 0 0 apache2
1 Time(s): [29501] 33 29501 25589 5479 2 0 0 apache2
1 Time(s): [29501] 33 29501 25589 10279 2 0 0 apache2
1 Time(s): [29510] 33 29510 25584 9398 0 0 0 apache2
1 Time(s): [29510] 33 29510 25584 13230 0 0 0 apache2
1 Time(s): [29516] 33 29516 20390 2247 3 0 0 apache2
1 Time(s): [29516] 33 29516 20390 8497 3 0 0 apache2
1 Time(s): [29524] 33 29524 25530 13420 1 0 0 apache2
1 Time(s): [29524] 33 29524 25723 13577 2 0 0 apache2
2 Time(s): [29526] 33 29526 17055 4902 2 0 0 apache2
1 Time(s): [29528] 33 29528 27243 14579 1 0 0 apache2
1 Time(s): [29528] 33 29528 47885 33497 0 0 0 apache2
2 Time(s): [29529] 33 29529 17055 4902 1 0 0 apache2
1 Time(s): [29532] 33 29532 12942 658 2 0 0 apache2
1 Time(s): [29532] 33 29532 13042 773 1 0 0 apache2
1 Time(s): [29533] 33 29533 12942 658 3 0 0 apache2
1 Time(s): [29533] 33 29533 13042 775 3 0 0 apache2
1 Time(s): [29534] 33 29534 20388 8442 0 0 0 apache2
1 Time(s): [29534] 33 29534 22713 10545 3 0 0 apache2
1 Time(s): [29535] 33 29535 12942 658 1 0 0 apache2
2 Time(s): [29539] 33 29539 12942 658 0 0 0 apache2
2 Time(s): [29542] 33 29542 12942 658 1 0 0 apache2
2 Time(s): [29543] 33 29543 12942 658 2 0 0 apache2
1 Time(s): [29547] 0 29547 4838 266 0 0 0 miniserv.pl
1 Time(s): [29547] 0 29547 5597 2933 2 0 0 /usr/share/webm
1 Time(s): [29614] 0 29614 4838 368 3 0 0 /usr/share/webm
1 Time(s): [29615] 33 29615 12677 354 3 0 0 apache2
1 Time(s): [29633] 33 29633 22191 7998 0 0 0 apache2
1 Time(s): [29633] 33 29633 22191 8279 0 0 0 apache2
1 Time(s): [29633] 33 29633 22191 8873 1 0 0 apache2
1 Time(s): [29633] 33 29633 22191 9028 0 0 0 apache2
1 Time(s): [29633] 33 29633 22255 9827 0 0 0 apache2
1 Time(s): [29633] 33 29633 25270 11775 0 0 0 apache2
1 Time(s): [29633] 33 29633 25399 3383 0 0 0 apache2
1 Time(s): [29636] 33 29636 22228 4138 2 0 0 apache2
1 Time(s): [29636] 33 29636 22228 5774 1 0 0 apache2
1 Time(s): [29636] 33 29636 22228 6570 1 0 0 apache2
1 Time(s): [29636] 33 29636 22228 6592 1 0 0 apache2
1 Time(s): [29636] 33 29636 22228 9031 1 0 0 apache2
1 Time(s): [29636] 33 29636 22228 9062 1 0 0 apache2
1 Time(s): [29636] 33 29636 22228 9671 0 0 0 apache2
1 Time(s): [29640] 0 29640 2415 6 0 0 0 sshd
6 Time(s): [29640] 0 29640 2415 7 0 0 0 sshd
1 Time(s): [29791] 1000 29791 2415 28 0 0 0 sshd
3 Time(s): [29791] 1000 29791 2415 29 0 0 0 sshd
2 Time(s): [29791] 1000 29791 2415 29 1 0 0 sshd
1 Time(s): [29791] 1000 29791 2415 47 1 0 0 sshd
2 Time(s): [29792] 1000 29792 2592 103 3 0 0 bash
1 Time(s): [29792] 1000 29792 2592 104 0 0 0 bash
1 Time(s): [29792] 1000 29792 2592 104 1 0 0 bash
1 Time(s): [29792] 1000 29792 2592 123 3 0 0 bash
1 Time(s): [29792] 1000 29792 2592 132 3 0 0 bash
1 Time(s): [29792] 1000 29792 2592 141 0 0 0 bash
1 Time(s): [29902] 33 29902 25584 5581 2 0 0 apache2
1 Time(s): [29902] 33 29902 25584 6985 2 0 0 apache2
1 Time(s): [29977] 108 29977 83161 2443 0 0 0 mysqld
1 Time(s): [29977] 108 29977 83161 2570 0 0 0 mysqld
1 Time(s): [29977] 108 29977 83377 2416 0 0 0 mysqld
1 Time(s): [29977] 108 29977 83417 2700 0 0 0 mysqld
1 Time(s): [29977] 108 29977 83417 2702 1 0 0 mysqld
1 Time(s): [29977] 108 29977 83417 2720 0 0 0 mysqld
1 Time(s): [29977] 108 29977 83417 2751 0 0 0 mysqld
1 Time(s): [30081] 0 30081 4838 321 1 0 0 miniserv.pl
1 Time(s): [30081] 0 30081 6263 2851 0 0 0 /usr/share/webm
1 Time(s): [30081] 0 30081 6526 3572 0 0 0 /usr/share/webm
1 Time(s): [30081] 0 30081 8546 5251 2 0 0 /usr/share/webm
1 Time(s): [30082] 0 30082 4838 411 1 0 0 miniserv.pl
1 Time(s): [30136] 0 30136 12599 241 0 0 0 apache2
1 Time(s): [30136] 0 30136 12599 241 1 0 0 apache2
1 Time(s): [30136] 0 30136 12599 242 0 0 0 apache2
2 Time(s): [30136] 0 30136 12599 242 1 0 0 apache2
1 Time(s): [30136] 0 30136 12599 250 0 0 0 apache2
2 Time(s): [30136] 0 30136 12599 269 0 0 0 apache2
1 Time(s): [30136] 0 30136 12599 270 0 0 0 apache2
1 Time(s): [30143] 33 30143 25584 10474 0 0 0 apache2
1 Time(s): [30143] 33 30143 25584 10478 1 0 0 apache2
1 Time(s): [30143] 33 30143 25584 11215 1 0 0 apache2
1 Time(s): [30168] 0 30168 4838 86 2 0 0 miniserv.pl
1 Time(s): [30168] 0 30168 4838 92 2 0 0 miniserv.pl
1 Time(s): [30168] 0 30168 4838 200 2 0 0 miniserv.pl
1 Time(s): [30168] 0 30168 5337 2690 3 0 0 /usr/share/webm



I deleted a segment of output to meet Linode forum posting character limit. Full text of logwatch output at http://pastebin.com/weRgNDaJ


Quote:
3 Time(s): [<c0102eb4>] ? xen_clts+0x44/0x50
2 Time(s): [<c0103106>] ? load_TLS_descriptor+0x36/0x70
4 Time(s): [<c0103828>] ? xen_mc_flush+0x78/0x110
8 Time(s): [<c0103ed5>] ? pte_pfn_to_mfn+0xb5/0xd0
7 Time(s): [<c0104e6e>] ? xen_batched_set_pte+0x1e/0xf0
3 Time(s): [<c0109d90>] ? math_state_restore+0x20/0xf0
3 Time(s): [<c0109e60>] ? math_state_restore+0xf0/0xf0
8 Time(s): [<c0122821>] ? do_page_fault+0x101/0x3a0
1 Time(s): [<c0122889>] ? do_page_fault+0x169/0x3a0
9 Time(s): [<c0138a4b>] ? has_ns_capability_noaudit+0xb/0x20
1 Time(s): [<c013a9da>] ? mod_timer_pending+0xba/0x110
1 Time(s): [<c0159d42>] ? update_curr+0x202/0x220
1 Time(s): [<c015be37>] ? find_busiest_group+0x107/0xa00
1 Time(s): [<c016c5c6>] ? sys_futex+0x76/0x140
3 Time(s): [<c01898ab>] ? handle_edge_irq+0x6b/0xf0
1 Time(s): [<c018bee4>] ? __call_rcu+0x94/0x170
1 Time(s): [<c018c014>] ? call_rcu_sched+0x14/0x20
3 Time(s): [<c018d2f2>] ? rcu_irq_exit+0x52/0xa0
7 Time(s): [<c018d644>] ? __delayacct_freepages_end+0x24/0x30
1 Time(s): [<c0199034>] ? generic_perform_write+0xc4/0x1e0
1 Time(s): [<c01991af>] ? generic_file_buffered_write+0x5f/0xa0
1 Time(s): [<c019a5d0>] ? __generic_file_aio_write+0x210/0x500
1 Time(s): [<c019a934>] ? generic_file_aio_write+0x74/0xf0
1 Time(s): [<c019ade1>] ? grab_cache_page_write_begin+0x71/0xe0
8 Time(s): [<c019b232>] ? filemap_fault+0x1f2/0x3c0
9 Time(s): [<c019c3ed>] ? T.681+0x7d/0x1b0
9 Time(s): [<c019c778>] ? T.680+0x258/0x290
9 Time(s): [<c019ca21>] ? out_of_memory+0x271/0x320
2 Time(s): [<c019f8d8>] ? warn_alloc_failed+0x98/0x100
1 Time(s): [<c019ffaf>] ? __alloc_pages_nodemask+0x13f/0x6d0
2 Time(s): [<c01a0348>] ? __alloc_pages_nodemask+0x4d8/0x6d0
9 Time(s): [<c01a0523>] ? __alloc_pages_nodemask+0x6b3/0x6d0
1 Time(s): [<c01ac183>] ? vm_mmap_pgoff+0x73/0x90
8 Time(s): [<c01b1b55>] ? __do_fault+0x75/0x550
1 Time(s): [<c01b1f09>] ? __do_fault+0x429/0x550
8 Time(s): [<c01b5140>] ? handle_pte_fault+0xa0/0x2f0
8 Time(s): [<c01b5485>] ? handle_mm_fault+0xf5/0x1b0
2 Time(s): [<c01c8541>] ? T.933+0x31/0xe0
2 Time(s): [<c01c8831>] ? T.930+0x91/0x250
2 Time(s): [<c01c8c3e>] ? cache_alloc_refill+0x24e/0x290
2 Time(s): [<c01c8d3e>] ? __kmalloc+0xbe/0xd0
1 Time(s): [<c01cb55e>] ? do_sync_readv_writev+0xbe/0x100
1 Time(s): [<c01cb5a0>] ? do_sync_readv_writev+0x100/0x100
1 Time(s): [<c01cb66d>] ? do_sync_write+0xcd/0x110
2 Time(s): [<c01cb826>] ? rw_verify_area+0x66/0x120
1 Time(s): [<c01cba9e>] ? do_sendfile+0x1be/0x240
1 Time(s): [<c01cbc68>] ? sys_sendfile+0x98/0xb0
1 Time(s): [<c01cbd2a>] ? do_readv_writev+0xaa/0x1a0
1 Time(s): [<c01cbe5e>] ? vfs_writev+0x3e/0x60
1 Time(s): [<c01cbf81>] ? sys_writev+0x41/0xa0
1 Time(s): [<c01cc230>] ? vfs_write+0xa0/0x170
1 Time(s): [<c01cc3d1>] ? sys_write+0x41/0x80
1 Time(s): [<c01d024e>] ? sys_fstat64+0x1e/0x30
1 Time(s): [<c01dd7e0>] ? sys_select+0x40/0xb0
3 Time(s): [<c01ed710>] ? splice_from_pipe_begin+0x10/0x10
1 Time(s): [<c01ed77b>] ? pipe_to_sendpage+0x6b/0x90
1 Time(s): [<c01ed809>] ? splice_from_pipe_feed+0x69/0xf0
1 Time(s): [<c01ee09b>] ? __splice_from_pipe+0x4b/0x60
1 Time(s): [<c01ee2ad>] ? splice_from_pipe+0x5d/0x80
1 Time(s): [<c01ee310>] ? default_file_splice_write+0x40/0x40
1 Time(s): [<c01ee330>] ? generic_splice_sendpage+0x20/0x30
1 Time(s): [<c01ee4cf>] ? do_splice_from+0x5f/0x90
1 Time(s): [<c01ee500>] ? do_splice_from+0x90/0x90
1 Time(s): [<c01ee525>] ? direct_splice_actor+0x25/0x40
1 Time(s): [<c01eeadf>] ? splice_direct_to_actor+0xbf/0x1d0
1 Time(s): [<c01eec48>] ? do_splice_direct+0x58/0x80
1 Time(s): [<c0275382>] ? ext3_writeback_write_end+0xc2/0x140
1 Time(s): [<c0275601>] ? ext3_write_begin+0x71/0x220
9 Time(s): [<c0452d64>] ? security_capable_noaudit+0x14/0x20
1 Time(s): [<c0453cb7>] ? security_file_permission+0x17/0x90
9 Time(s): [<c04a6a77>] ? ___ratelimit+0x97/0x110
3 Time(s): [<c04c3d2e>] ? __xen_evtchn_do_upcall+0x1ce/0x210
1 Time(s): [<c04c50bd>] ? xen_evtchn_do_upcall+0x1d/0x30
2 Time(s): [<c04e77be>] ? add_interrupt_randomness+0x2e/0x160
1 Time(s): [<c058d3be>] ? kernel_sendpage+0x4e/0x90
1 Time(s): [<c058d400>] ? kernel_sendpage+0x90/0x90
2 Time(s): [<c058d9b0>] ? sock_aio_dtor+0x10/0x10
1 Time(s): [<c058dada>] ? sock_aio_write+0x12a/0x180
2 Time(s): [<c0596698>] ? pskb_expand_head+0x48/0x1e0
2 Time(s): [<c0596dbd>] ? __pskb_pull_tail+0x4d/0x2a0
2 Time(s): [<c059bccf>] ? netif_skb_features+0xaf/0xc0
2 Time(s): [<c05a00b5>] ? dev_hard_start_xmit+0x1b5/0x3e0
2 Time(s): [<c05a03df>] ? dev_queue_xmit+0xff/0x340
1 Time(s): [<c05b3d98>] ? eth_header+0x28/0xc0
2 Time(s): [<c05b507a>] ? sch_direct_xmit+0xba/0x180
1 Time(s): [<c05dfccc>] ? nf_iterate+0x6c/0x90
2 Time(s): [<c0632dd8>] ? ip_local_out+0x18/0x20
2 Time(s): [<c0633850>] ? ip_finish_output2+0x260/0x260
1 Time(s): [<c06339ba>] ? ip_finish_output+0x16a/0x2e0
1 Time(s): [<c0633bc4>] ? ip_output+0x94/0xb0
1 Time(s): [<c063c253>] ? tcp_sendmsg+0xa63/0xc20
1 Time(s): [<c063c8ef>] ? do_tcp_sendpages+0x4df/0x540
1 Time(s): [<c063c950>] ? do_tcp_sendpages+0x540/0x540
1 Time(s): [<c063c9cb>] ? tcp_sendpage+0x7b/0x90
2 Time(s): [<c0647315>] ? tcp_transmit_skb+0x395/0x660
2 Time(s): [<c064900d>] ? tcp_fragment+0x1ad/0x2e0
2 Time(s): [<c0649cdd>] ? tcp_write_xmit+0x1dd/0x4f0
1 Time(s): [<c064a01d>] ? tcp_push_one+0x2d/0x40
1 Time(s): [<c064a054>] ? __tcp_push_pending_frames+0x24/0x90
1 Time(s): [<c065b0a7>] ? inet_sendmsg+0x47/0xb0
1 Time(s): [<c065b190>] ? inet_dgram_connect+0x80/0x80
1 Time(s): [<c065b1f9>] ? inet_sendpage+0x69/0x110
10 Time(s): [<c0720000>] ? sctp_v6_get_saddr+0x40/0x40
1 Time(s): [<c072dcf5>] ? _raw_spin_lock+0x5/0x10
9 Time(s): [<c072dd71>] ? _raw_spin_unlock_irqrestore+0x11/0x20
3 Time(s): [<c072dff1>] ? syscall_call+0x7/0xb
1 Time(s): active_anon:37727 inactive_anon:37849 isolated_anon:0
1 Time(s): active_anon:49324 inactive_anon:49420 isolated_anon:0
1 Time(s): active_anon:55277 inactive_anon:55321 isolated_anon:32
1 Time(s): active_anon:55320 inactive_anon:55404 isolated_anon:13
1 Time(s): active_anon:55344 inactive_anon:55386 isolated_anon:28
1 Time(s): active_anon:55429 inactive_anon:55572 isolated_anon:0
1 Time(s): active_anon:55457 inactive_anon:55556 isolated_anon:0
1 Time(s): active_anon:55509 inactive_anon:55546 isolated_anon:32
1 Time(s): active_anon:55550 inactive_anon:55917 isolated_anon:0
1 Time(s): active_anon:55606 inactive_anon:55626 isolated_anon:32
1 Time(s): active_anon:55654 inactive_anon:55696 isolated_anon:0
1 Time(s): active_file:133 inactive_file:175 isolated_file:0
1 Time(s): active_file:138 inactive_file:203 isolated_file:0
1 Time(s): active_file:146 inactive_file:264 isolated_file:0
1 Time(s): active_file:17 inactive_file:50 isolated_file:0
1 Time(s): active_file:194 inactive_file:343 isolated_file:17
1 Time(s): active_file:19587 inactive_file:15763 isolated_file:0
1 Time(s): active_file:2 inactive_file:50 isolated_file:0
1 Time(s): active_file:266 inactive_file:358 isolated_file:0
1 Time(s): active_file:6753 inactive_file:5497 isolated_file:0
1 Time(s): active_file:9 inactive_file:37 isolated_file:0
1 Time(s): active_file:93 inactive_file:148 isolated_file:0
5 Time(s): apache2 cpuset=/ mems_allowed=0
1 Time(s): apache2 invoked oom-killer: gfp_mask=0x10200da, order=0, oom_adj=0, oom_score_adj=0
4 Time(s): apache2 invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
2 Time(s): apache2: page allocation failure: order:3, mode:0x20
1 Time(s): free:1200 slab_reclaimable:6914 slab_unreclaimable:2897
1 Time(s): free:1203 slab_reclaimable:7031 slab_unreclaimable:2935
1 Time(s): free:1204 slab_reclaimable:6950 slab_unreclaimable:2850
1 Time(s): free:1214 slab_reclaimable:6876 slab_unreclaimable:2913
1 Time(s): free:1214 slab_reclaimable:6899 slab_unreclaimable:2880
1 Time(s): free:1215 slab_reclaimable:6877 slab_unreclaimable:2945
1 Time(s): free:1217 slab_reclaimable:6895 slab_unreclaimable:2901
1 Time(s): free:1224 slab_reclaimable:7140 slab_unreclaimable:2784
1 Time(s): free:1492 slab_reclaimable:7100 slab_unreclaimable:2802
1 Time(s): free:1568 slab_reclaimable:7374 slab_unreclaimable:2779
1 Time(s): free:1589 slab_reclaimable:7482 slab_unreclaimable:2671
11 Time(s): lowmem_reserve[]: 0 0 0 0
11 Time(s): lowmem_reserve[]: 0 500 500 500
1 Time(s): mapped:136 shmem:409 pagetables:1425 bounce:0
1 Time(s): mapped:154 shmem:404 pagetables:1396 bounce:0
1 Time(s): mapped:180 shmem:388 pagetables:1334 bounce:0
1 Time(s): mapped:182 shmem:399 pagetables:1360 bounce:0
1 Time(s): mapped:189 shmem:325 pagetables:1353 bounce:0
1 Time(s): mapped:297 shmem:418 pagetables:1452 bounce:0
1 Time(s): mapped:3571 shmem:4382 pagetables:991 bounce:0
1 Time(s): mapped:4122 shmem:648 pagetables:1103 bounce:0
1 Time(s): mapped:60 shmem:540 pagetables:1273 bounce:0
1 Time(s): mapped:71 shmem:331 pagetables:1409 bounce:0
1 Time(s): mapped:87 shmem:542 pagetables:1247 bounce:0
1 Time(s): miniserv.pl cpuset=/ mems_allowed=0
1 Time(s): miniserv.pl invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
1 Time(s): mysqld cpuset=/ mems_allowed=0
1 Time(s): mysqld invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
3 Time(s): unevictable:0 dirty:0 writeback:0 unstable:0
1 Time(s): unevictable:0 dirty:0 writeback:2 unstable:0
1 Time(s): unevictable:0 dirty:0 writeback:22 unstable:0
1 Time(s): unevictable:0 dirty:0 writeback:28 unstable:0
1 Time(s): unevictable:0 dirty:0 writeback:4 unstable:0
1 Time(s): unevictable:0 dirty:0 writeback:42 unstable:0
1 Time(s): unevictable:0 dirty:1 writeback:0 unstable:0
1 Time(s): unevictable:0 dirty:17 writeback:6 unstable:0
1 Time(s): unevictable:0 dirty:27 writeback:0 unstable:0

---------------------- Kernel End -------------------------


Top
   
PostPosted: Wed Nov 28, 2012 1:57 am 
Offline
Senior Member
User avatar

Joined: Sun Jan 18, 2009 2:41 pm
Posts: 830
If you're using Apache with the prefork MPM (apache2ctl -M | grep mpm should tell you what you're using), you probably need to lower your MaxClients setting. The link will lead you to older posts in the forum on this topic which should help you choose a suitable value.

The whoopsie daemon is nothing more than a waste of memory and CPU unless you have configured it to send crash reports to Canonical. You can remove it with apt-get purge whoopsie.

You are running both PostgreSQL and MySQL. Do you have a specific need to run two different database servers? If not, I would eliminate one of them.

Doing these things will reduce the memory demand on your server and the chances that it will run out of memory in the future.


Top
   
PostPosted: Thu Nov 29, 2012 3:03 pm 
Offline
Senior Newbie
User avatar

Joined: Sun Apr 12, 2009 4:11 pm
Posts: 12
Thanks Vance. Reducing MaxClients really seems to have done the trick.

-r


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
RSS

Powered by phpBB® Forum Software © phpBB Group