Since I upgraded to VirtualBox 2.1.4 I have some kind of memory
related problem that invokes the oom-killer. I hope that some kernel or
memory management guru have some ideas of what could be the cause and
what I can do to solve the problem!
First some background:
About four months ago I built a new box. The box has always been
running Debian/Lenny 32 bit, which now also is Debian/Stable. The CPU
is "CPU0: Intel(R) Core(TM)2 Quad CPU Q9550 @ 2.83GHz stepping 07" and
the box has 8 GB of memory. The kernel is "Linux big 2.6.26-1-686-bigmem
#1 SMP Sat Jan 10 19:13:22 UTC 2009 i686 GNU/Linux"
Until 2009-03-07 (March 7) I have been running virtualbox-2.0
(2.0.6-39765_Debian_lenny) and until that day everything had been rock
stable. My mostly used VM is a Windows 2000 installation with 1 GB of
memory. Both the host and the guest was working perfectly and I must
say that I'm really impressed of VirtualBox and grateful to the
VirtualBox developers!
However, (isn't it always something...) on March 7 I updated
VirtualBox to version 2.1 (2.1.4-42893_Debian_lenny) and since then I
have more or less daily problems with memory management, resulting in
that the oom-killer kicks in and kill some processes.
The box is very light loaded and the only memory deserving application
is VirtualBox. As I understand it there is always huge amounts of free
memory and I can't understand why the kernel doesn't agree with me...
When the oom-killer does it's job I get really extensive information
in /var/log/messages. I hope that someone more capable than me can
help me in analyzing the log and maybe help me understand what's going
on and what I can do to resolve the problem.
So far I have tried some memory management tuning by adding the
following rows to /etc/sysctl.conf
vm.min_free_kbytes = 16384
vm.lowmem_reserve_ratio = "128 32 32"
I can't really say if that have made any improvement and since I'm
really not knowing what I'm doing, I'm probably on the wrong track...
Attaching the relevant part of /var/log/messages from one time when
the oom-killer was invoked.
Please let me know if you need any more information and I'm very
grateful to anyone that could help me resolve this problem!
Code: Select all
=========================================================
Mar 18 22:58:56 big -- MARK --
Mar 18 23:38:56 big -- MARK --
Mar 19 00:38:56 big -- MARK --
Mar 19 00:58:56 big -- MARK --
Mar 19 01:18:01 big kernel: [1447935.367814] __ratelimit: 337 messages suppressed
Mar 19 01:18:01 big kernel: [1447935.367814] __ratelimit: 337 messages suppressed
Mar 19 01:18:01 big kernel: [1447935.367821] Xorg invoked oom-killer: gfp_mask=0xd0, order=0, oomkilladj=0
Mar 19 01:18:01 big kernel: [1447935.367821] Xorg invoked oom-killer: gfp_mask=0xd0, order=0, oomkilladj=0
Mar 19 01:18:01 big kernel: [1447935.367824] Pid: 3402, comm: Xorg Tainted: P 2.6.26-1-686-bigmem #1
Mar 19 01:18:01 big kernel: [1447935.367824] Pid: 3402, comm: Xorg Tainted: P 2.6.26-1-686-bigmem #1
Mar 19 01:18:01 big kernel: [1447935.367842] [<c015fe1e>] oom_kill_process+0x4f/0x195
Mar 19 01:18:01 big kernel: [1447935.367842] [<c015fe1e>] oom_kill_process+0x4f/0x195
Mar 19 01:18:01 big kernel: [1447935.367856] [<c0160248>] out_of_memory+0x14e/0x17f
Mar 19 01:18:01 big kernel: [1447935.367856] [<c0160248>] out_of_memory+0x14e/0x17f
Mar 19 01:18:01 big kernel: [1447935.367868] [<c01621aa>] __alloc_pages_internal+0x2b8/0x34e
Mar 19 01:18:01 big kernel: [1447935.367868] [<c01621aa>] __alloc_pages_internal+0x2b8/0x34e
Mar 19 01:18:01 big kernel: [1447935.367881] [<c016224c>] __alloc_pages+0x7/0x9
Mar 19 01:18:01 big kernel: [1447935.367881] [<c016224c>] __alloc_pages+0x7/0x9
Mar 19 01:18:01 big kernel: [1447935.367885] [<c016225d>] __get_free_pages+0xf/0x1b
Mar 19 01:18:01 big kernel: [1447935.367885] [<c016225d>] __get_free_pages+0xf/0x1b
Mar 19 01:18:01 big kernel: [1447935.367889] [<c018970a>] __pollwait+0x44/0xac
Mar 19 01:18:01 big kernel: [1447935.367889] [<c018970a>] __pollwait+0x44/0xac
Mar 19 01:18:01 big kernel: [1447935.367896] [<c02b7377>] unix_poll+0x17/0x8c
Mar 19 01:18:01 big kernel: [1447935.367896] [<c02b7377>] unix_poll+0x17/0x8c
Mar 19 01:18:01 big kernel: [1447935.367903] [<c025e5eb>] sock_poll+0xc/0xe
Mar 19 01:18:01 big kernel: [1447935.367903] [<c025e5eb>] sock_poll+0xc/0xe
Mar 19 01:18:01 big kernel: [1447935.367907] [<c018907a>] do_select+0x21c/0x3bd
Mar 19 01:18:01 big kernel: [1447935.367907] [<c018907a>] do_select+0x21c/0x3bd
Mar 19 01:18:01 big kernel: [1447935.367920] [<c01896c6>] __pollwait+0x0/0xac
Mar 19 01:18:01 big kernel: [1447935.367920] [<c01896c6>] __pollwait+0x0/0xac
Mar 19 01:18:01 big kernel: [1447935.367927] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:01 big kernel: [1447935.367927] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:01 big kernel: [1447935.367935] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:01 big kernel: [1447935.367935] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:01 big kernel: [1447935.367941] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:01 big kernel: [1447935.367941] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:01 big kernel: [1447935.367948] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:01 big kernel: [1447935.367948] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:01 big kernel: [1447935.367954] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:01 big kernel: [1447935.367954] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.367961] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.367961] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.367968] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.367968] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.367974] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.367974] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.367981] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.367981] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.367987] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.367987] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.367994] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.367994] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.368000] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.368000] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.368007] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.368007] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.368014] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.368014] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.368021] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.368021] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.368027] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.368027] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.368034] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.368034] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.368041] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.368041] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.368048] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.368048] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.368054] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.368054] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:02 big kernel: [1447935.368062] [<c018949d>] core_sys_select+0x282/0x29f
Mar 19 01:18:02 big kernel: [1447935.368062] [<c018949d>] core_sys_select+0x282/0x29f
Mar 19 01:18:02 big kernel: [1447935.368099] [<c013aa4b>] __remove_hrtimer+0x5d/0x64
Mar 19 01:18:02 big kernel: [1447935.368099] [<c013aa4b>] __remove_hrtimer+0x5d/0x64
Mar 19 01:18:02 big kernel: [1447935.368109] [<c013b06c>] hrtimer_try_to_cancel+0x5c/0x63
Mar 19 01:18:02 big kernel: [1447935.368109] [<c013b06c>] hrtimer_try_to_cancel+0x5c/0x63
Mar 19 01:18:02 big kernel: [1447935.368121] [<c0189840>] sys_select+0xce/0x180
Mar 19 01:18:02 big kernel: [1447935.368121] [<c0189840>] sys_select+0xce/0x180
Mar 19 01:18:02 big kernel: [1447935.368129] [<c012c6ee>] sys_setitimer+0x74/0x7f
Mar 19 01:18:02 big kernel: [1447935.368129] [<c012c6ee>] sys_setitimer+0x74/0x7f
Mar 19 01:18:02 big kernel: [1447935.368143] [<c0108853>] sysenter_past_esp+0x78/0xb1
Mar 19 01:18:02 big kernel: [1447935.368143] [<c0108853>] sysenter_past_esp+0x78/0xb1
Mar 19 01:18:02 big kernel: [1447935.368160] =======================
Mar 19 01:18:02 big kernel: [1447935.368160] =======================
Mar 19 01:18:02 big kernel: [1447935.368163] Mem-info:
Mar 19 01:18:02 big kernel: [1447935.368163] Mem-info:
Mar 19 01:18:02 big kernel: [1447935.368166] DMA per-cpu:
Mar 19 01:18:02 big kernel: [1447935.368166] DMA per-cpu:
Mar 19 01:18:02 big kernel: [1447935.368170] CPU 0: hi: 0, btch: 1 usd: 0
Mar 19 01:18:02 big kernel: [1447935.368170] CPU 0: hi: 0, btch: 1 usd: 0
Mar 19 01:18:02 big kernel: [1447935.368174] CPU 1: hi: 0, btch: 1 usd: 0
Mar 19 01:18:02 big kernel: [1447935.368174] CPU 1: hi: 0, btch: 1 usd: 0
Mar 19 01:18:02 big kernel: [1447935.368178] CPU 2: hi: 0, btch: 1 usd: 0
Mar 19 01:18:02 big kernel: [1447935.368178] CPU 2: hi: 0, btch: 1 usd: 0
Mar 19 01:18:02 big kernel: [1447935.368181] CPU 3: hi: 0, btch: 1 usd: 0
Mar 19 01:18:02 big kernel: [1447935.368181] CPU 3: hi: 0, btch: 1 usd: 0
Mar 19 01:18:02 big kernel: [1447935.368185] Normal per-cpu:
Mar 19 01:18:02 big kernel: [1447935.368185] Normal per-cpu:
Mar 19 01:18:02 big kernel: [1447935.368188] CPU 0: hi: 186, btch: 31 usd: 161
Mar 19 01:18:02 big kernel: [1447935.368188] CPU 0: hi: 186, btch: 31 usd: 161
Mar 19 01:18:02 big kernel: [1447935.368192] CPU 1: hi: 186, btch: 31 usd: 165
Mar 19 01:18:02 big kernel: [1447935.368192] CPU 1: hi: 186, btch: 31 usd: 165
Mar 19 01:18:02 big kernel: [1447935.368197] CPU 2: hi: 186, btch: 31 usd: 173
Mar 19 01:18:02 big kernel: [1447935.368197] CPU 2: hi: 186, btch: 31 usd: 173
Mar 19 01:18:02 big kernel: [1447935.368200] CPU 3: hi: 186, btch: 31 usd: 146
Mar 19 01:18:02 big kernel: [1447935.368200] CPU 3: hi: 186, btch: 31 usd: 146
Mar 19 01:18:02 big kernel: [1447935.368204] HighMem per-cpu:
Mar 19 01:18:02 big kernel: [1447935.368204] HighMem per-cpu:
Mar 19 01:18:02 big kernel: [1447935.368208] CPU 0: hi: 186, btch: 31 usd: 153
Mar 19 01:18:02 big kernel: [1447935.368208] CPU 0: hi: 186, btch: 31 usd: 153
Mar 19 01:18:02 big kernel: [1447935.368211] CPU 1: hi: 186, btch: 31 usd: 158
Mar 19 01:18:02 big kernel: [1447935.368211] CPU 1: hi: 186, btch: 31 usd: 158
Mar 19 01:18:02 big kernel: [1447935.368215] CPU 2: hi: 186, btch: 31 usd: 155
Mar 19 01:18:02 big kernel: [1447935.368215] CPU 2: hi: 186, btch: 31 usd: 155
Mar 19 01:18:02 big kernel: [1447935.368219] CPU 3: hi: 186, btch: 31 usd: 174
Mar 19 01:18:02 big kernel: [1447935.368219] CPU 3: hi: 186, btch: 31 usd: 174
Mar 19 01:18:02 big kernel: [1447935.368224] Active:537043 inactive:522645 dirty:2 writeback:1 unstable:0
Mar 19 01:18:02 big kernel: [1447935.368224] Active:537043 inactive:522645 dirty:2 writeback:1 unstable:0
Mar 19 01:18:02 big kernel: [1447935.368226] free:801910 slab:191425 mapped:29385 pagetables:2175 bounce:0
Mar 19 01:18:02 big kernel: [1447935.368226] free:801910 slab:191425 mapped:29385 pagetables:2175 bounce:0
Mar 19 01:18:02 big kernel: [1447935.368231] DMA free:7248kB min:292kB low:364kB high:436kB active:0kB inactive:32kB present:16256kB pages_scanned:15 all_unreclaimable? no
Mar 19 01:18:02 big kernel: [1447935.368231] DMA free:7248kB min:292kB low:364kB high:436kB active:0kB inactive:32kB present:16256kB pages_scanned:15 all_unreclaimable? no
Mar 19 01:18:02 big kernel: [1447935.368235] lowmem_reserve[]: 0 1746 17748 17748
Mar 19 01:18:02 big kernel: [1447935.368235] lowmem_reserve[]: 0 1746 17748 17748
Mar 19 01:18:02 big kernel: [1447935.368252] Normal free:14864kB min:16088kB low:20108kB high:24132kB active:11400kB inactive:10884kB present:894080kB pages_scanned:39243 all_unreclaimable? yes
Mar 19 01:18:02 big kernel: [1447935.368252] Normal free:14864kB min:16088kB low:20108kB high:24132kB active:11400kB inactive:10884kB present:894080kB pages_scanned:39243 all_unreclaimable? yes
Mar 19 01:18:02 big kernel: [1447935.368256] lowmem_reserve[]: 0 0 64008 64008
Mar 19 01:18:02 big kernel: [1447935.368256] lowmem_reserve[]: 0 0 64008 64008
Mar 19 01:18:02 big kernel: [1447935.368273] HighMem free:3185528kB min:512kB low:37376kB high:74240kB active:2136776kB inactive:2079664kB present:8193024kB pages_scanned:0 all_unreclaimable? no
Mar 19 01:18:02 big kernel: [1447935.368273] HighMem free:3185528kB min:512kB low:37376kB high:74240kB active:2136776kB inactive:2079664kB present:8193024kB pages_scanned:0 all_unreclaimable? no
Mar 19 01:18:02 big kernel: [1447935.368277] lowmem_reserve[]: 0 0 0 0
Mar 19 01:18:02 big kernel: [1447935.368277] lowmem_reserve[]: 0 0 0 0
Mar 19 01:18:02 big kernel: [1447935.368292] DMA: 127*4kB 92*8kB 76*16kB 40*32kB 15*64kB 4*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7260kB
Mar 19 01:18:02 big kernel: [1447935.368292] DMA: 127*4kB 92*8kB 76*16kB 40*32kB 15*64kB 4*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7260kB
Mar 19 01:18:02 big kernel: [1447935.368326] Normal: 1205*4kB 3*8kB 3*16kB 2*32kB 0*64kB 1*128kB 0*256kB 3*512kB 2*1024kB 1*2048kB 1*4096kB = 14812kB
Mar 19 01:18:02 big kernel: [1447935.368326] Normal: 1205*4kB 3*8kB 3*16kB 2*32kB 0*64kB 1*128kB 0*256kB 3*512kB 2*1024kB 1*2048kB 1*4096kB = 14812kB
Mar 19 01:18:02 big kernel: [1447935.368357] HighMem: 53523*4kB 55646*8kB 27466*16kB 18855*32kB 6585*64kB 3608*128kB 708*256kB 744*512kB 3*1024kB 1*2048kB 8*4096kB = 3185404kB
Mar 19 01:18:02 big kernel: [1447935.368357] HighMem: 53523*4kB 55646*8kB 27466*16kB 18855*32kB 6585*64kB 3608*128kB 708*256kB 744*512kB 3*1024kB 1*2048kB 8*4096kB = 3185404kB
Mar 19 01:18:02 big kernel: [1447935.368391] 640546 total pagecache pages
Mar 19 01:18:02 big kernel: [1447935.368391] 640546 total pagecache pages
Mar 19 01:18:02 big kernel: [1447935.368394] Swap cache: add 35, delete 32, find 0/0
Mar 19 01:18:02 big kernel: [1447935.368394] Swap cache: add 35, delete 32, find 0/0
Mar 19 01:18:02 big kernel: [1447935.368398] Free swap = 8008256kB
Mar 19 01:18:02 big kernel: [1447935.368398] Free swap = 8008256kB
Mar 19 01:18:02 big kernel: [1447935.368402] Total swap = 8008392kB
Mar 19 01:18:02 big kernel: [1447935.368402] Total swap = 8008392kB
Mar 19 01:18:02 big kernel: [1447935.391525] 2293760 pages of RAM
Mar 19 01:18:02 big kernel: [1447935.391525] 2293760 pages of RAM
Mar 19 01:18:02 big kernel: [1447935.391527] 2064384 pages of HIGHMEM
Mar 19 01:18:02 big kernel: [1447935.391527] 2064384 pages of HIGHMEM
Mar 19 01:18:02 big kernel: [1447935.391529] 228611 reserved pages
Mar 19 01:18:02 big kernel: [1447935.391529] 228611 reserved pages
Mar 19 01:18:02 big kernel: [1447935.391530] 504715 pages shared
Mar 19 01:18:02 big kernel: [1447935.391530] 504715 pages shared
Mar 19 01:18:02 big kernel: [1447935.391532] 3 pages swap cached
Mar 19 01:18:02 big kernel: [1447935.391532] 3 pages swap cached
Mar 19 01:18:02 big kernel: [1447935.391533] 2 pages dirty
Mar 19 01:18:02 big kernel: [1447935.391533] 2 pages dirty
Mar 19 01:18:02 big kernel: [1447935.391534] 1 pages writeback
Mar 19 01:18:02 big kernel: [1447935.391534] 1 pages writeback
Mar 19 01:18:03 big kernel: [1447935.391536] 29385 pages mapped
Mar 19 01:18:03 big kernel: [1447935.391536] 29385 pages mapped
Mar 19 01:18:03 big kernel: [1447935.391537] 191425 pages slab
Mar 19 01:18:03 big kernel: [1447935.391537] 191425 pages slab
Mar 19 01:18:03 big kernel: [1447935.391539] 2175 pages pagetables
Mar 19 01:18:03 big kernel: [1447935.391539] 2175 pages pagetables
Mar 19 01:18:03 big kernel: [1447935.391541] Out of memory: kill process 12570 (VBoxSVC) score 565205 or a child
Mar 19 01:18:03 big kernel: [1447935.391541] Out of memory: kill process 12570 (VBoxSVC) score 565205 or a child
Mar 19 01:18:03 big kernel: [1447935.391583] Killed process 9992 (VirtualBox)
Mar 19 01:18:03 big kernel: [1447935.391583] Killed process 9992 (VirtualBox)
Mar 19 01:18:03 big kernel: [1447935.395130] Xorg invoked oom-killer: gfp_mask=0xd0, order=0, oomkilladj=0
Mar 19 01:18:03 big kernel: [1447935.395130] Xorg invoked oom-killer: gfp_mask=0xd0, order=0, oomkilladj=0
Mar 19 01:18:03 big kernel: [1447935.395134] Pid: 3402, comm: Xorg Tainted: P 2.6.26-1-686-bigmem #1
Mar 19 01:18:03 big kernel: [1447935.395134] Pid: 3402, comm: Xorg Tainted: P 2.6.26-1-686-bigmem #1
Mar 19 01:18:03 big kernel: [1447935.395152] [<c015fe1e>] oom_kill_process+0x4f/0x195
Mar 19 01:18:03 big kernel: [1447935.395152] [<c015fe1e>] oom_kill_process+0x4f/0x195
Mar 19 01:18:03 big kernel: [1447935.395165] [<c0160248>] out_of_memory+0x14e/0x17f
Mar 19 01:18:03 big kernel: [1447935.395165] [<c0160248>] out_of_memory+0x14e/0x17f
Mar 19 01:18:03 big kernel: [1447935.395176] [<c01621aa>] __alloc_pages_internal+0x2b8/0x34e
Mar 19 01:18:03 big kernel: [1447935.395176] [<c01621aa>] __alloc_pages_internal+0x2b8/0x34e
Mar 19 01:18:03 big kernel: [1447935.395188] [<c016224c>] __alloc_pages+0x7/0x9
Mar 19 01:18:03 big kernel: [1447935.395188] [<c016224c>] __alloc_pages+0x7/0x9
Mar 19 01:18:03 big kernel: [1447935.395193] [<c016225d>] __get_free_pages+0xf/0x1b
Mar 19 01:18:03 big kernel: [1447935.395193] [<c016225d>] __get_free_pages+0xf/0x1b
Mar 19 01:18:03 big kernel: [1447935.395196] [<c018970a>] __pollwait+0x44/0xac
Mar 19 01:18:03 big kernel: [1447935.395196] [<c018970a>] __pollwait+0x44/0xac
Mar 19 01:18:03 big kernel: [1447935.395203] [<c02b7377>] unix_poll+0x17/0x8c
Mar 19 01:18:03 big kernel: [1447935.395203] [<c02b7377>] unix_poll+0x17/0x8c
Mar 19 01:18:03 big kernel: [1447935.395209] [<c025e5eb>] sock_poll+0xc/0xe
Mar 19 01:18:03 big kernel: [1447935.395209] [<c025e5eb>] sock_poll+0xc/0xe
Mar 19 01:18:03 big kernel: [1447935.395214] [<c018907a>] do_select+0x21c/0x3bd
Mar 19 01:18:03 big kernel: [1447935.395214] [<c018907a>] do_select+0x21c/0x3bd
Mar 19 01:18:03 big kernel: [1447935.395228] [<c01896c6>] __pollwait+0x0/0xac
Mar 19 01:18:03 big kernel: [1447935.395228] [<c01896c6>] __pollwait+0x0/0xac
Mar 19 01:18:03 big kernel: [1447935.395234] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395234] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395242] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395242] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395249] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395249] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395257] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395257] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395264] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395264] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395271] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395271] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395278] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395278] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395285] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395285] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395292] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395292] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395299] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395299] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395306] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395306] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395313] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395313] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395321] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395321] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395328] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395328] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395335] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395335] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395343] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395343] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395351] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395351] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395358] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395358] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395365] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395365] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395373] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395373] [<c0122206>] default_wake_function+0x0/0x8
Mar 19 01:18:03 big kernel: [1447935.395381] [<c018949d>] core_sys_select+0x282/0x29f
Mar 19 01:18:03 big kernel: [1447935.395381] [<c018949d>] core_sys_select+0x282/0x29f
Mar 19 01:18:03 big kernel: [1447935.395421] [<c013aa4b>] __remove_hrtimer+0x5d/0x64
Mar 19 01:18:03 big kernel: [1447935.395421] [<c013aa4b>] __remove_hrtimer+0x5d/0x64
Mar 19 01:18:03 big kernel: [1447935.395427] [<c013b06c>] hrtimer_try_to_cancel+0x5c/0x63
Mar 19 01:18:03 big kernel: [1447935.395427] [<c013b06c>] hrtimer_try_to_cancel+0x5c/0x63
Mar 19 01:18:03 big kernel: [1447935.395437] [<c0189840>] sys_select+0xce/0x180
Mar 19 01:18:03 big kernel: [1447935.395437] [<c0189840>] sys_select+0xce/0x180
Mar 19 01:18:03 big kernel: [1447935.395442] [<c012c6ee>] sys_setitimer+0x74/0x7f
Mar 19 01:18:03 big kernel: [1447935.395442] [<c012c6ee>] sys_setitimer+0x74/0x7f
Mar 19 01:18:03 big kernel: [1447935.395452] [<c0108853>] sysenter_past_esp+0x78/0xb1
Mar 19 01:18:03 big kernel: [1447935.395452] [<c0108853>] sysenter_past_esp+0x78/0xb1
Mar 19 01:18:03 big kernel: [1447935.395465] =======================
Mar 19 01:18:03 big kernel: [1447935.395465] =======================
Mar 19 01:18:03 big kernel: [1447935.395466] Mem-info:
Mar 19 01:18:03 big kernel: [1447935.395466] Mem-info:
Mar 19 01:18:03 big kernel: [1447935.395468] DMA per-cpu:
Mar 19 01:18:03 big kernel: [1447935.395468] DMA per-cpu:
Mar 19 01:18:03 big kernel: [1447935.395470] CPU 0: hi: 0, btch: 1 usd: 0
Mar 19 01:18:03 big kernel: [1447935.395470] CPU 0: hi: 0, btch: 1 usd: 0
Mar 19 01:18:03 big kernel: [1447935.395472] CPU 1: hi: 0, btch: 1 usd: 0
Mar 19 01:18:03 big kernel: [1447935.395472] CPU 1: hi: 0, btch: 1 usd: 0
Mar 19 01:18:03 big kernel: [1447935.395473] CPU 2: hi: 0, btch: 1 usd: 0
Mar 19 01:18:03 big kernel: [1447935.395473] CPU 2: hi: 0, btch: 1 usd: 0
Mar 19 01:18:03 big kernel: [1447935.395475] CPU 3: hi: 0, btch: 1 usd: 0
Mar 19 01:18:03 big kernel: [1447935.395475] CPU 3: hi: 0, btch: 1 usd: 0
Mar 19 01:18:03 big kernel: [1447935.395477] Normal per-cpu:
Mar 19 01:18:03 big kernel: [1447935.395477] Normal per-cpu:
Mar 19 01:18:03 big kernel: [1447935.395479] CPU 0: hi: 186, btch: 31 usd: 161
Mar 19 01:18:03 big kernel: [1447935.395479] CPU 0: hi: 186, btch: 31 usd: 161
Mar 19 01:18:03 big kernel: [1447935.395481] CPU 1: hi: 186, btch: 31 usd: 165
Mar 19 01:18:03 big kernel: [1447935.395481] CPU 1: hi: 186, btch: 31 usd: 165
Mar 19 01:18:03 big kernel: [1447935.395482] CPU 2: hi: 186, btch: 31 usd: 173
Mar 19 01:18:03 big kernel: [1447935.395482] CPU 2: hi: 186, btch: 31 usd: 173
Mar 19 01:18:03 big kernel: [1447935.395484] CPU 3: hi: 186, btch: 31 usd: 146
Mar 19 01:18:03 big kernel: [1447935.395484] CPU 3: hi: 186, btch: 31 usd: 146
Mar 19 01:18:03 big kernel: [1447935.395486] HighMem per-cpu:
Mar 19 01:18:03 big kernel: [1447935.395486] HighMem per-cpu:
Mar 19 01:18:04 big kernel: [1447935.395488] CPU 0: hi: 186, btch: 31 usd: 153
Mar 19 01:18:04 big kernel: [1447935.395488] CPU 0: hi: 186, btch: 31 usd: 153
Mar 19 01:18:04 big kernel: [1447935.395489] CPU 1: hi: 186, btch: 31 usd: 158
Mar 19 01:18:04 big kernel: [1447935.395489] CPU 1: hi: 186, btch: 31 usd: 158
Mar 19 01:18:04 big kernel: [1447935.395491] CPU 2: hi: 186, btch: 31 usd: 168
Mar 19 01:18:04 big kernel: [1447935.395491] CPU 2: hi: 186, btch: 31 usd: 168
Mar 19 01:18:04 big kernel: [1447935.395493] CPU 3: hi: 186, btch: 31 usd: 174
Mar 19 01:18:04 big kernel: [1447935.395493] CPU 3: hi: 186, btch: 31 usd: 174
Mar 19 01:18:04 big kernel: [1447935.395496] Active:533006 inactive:522651 dirty:2 writeback:1 unstable:0
Mar 19 01:18:04 big kernel: [1447935.395496] Active:533006 inactive:522651 dirty:2 writeback:1 unstable:0
Mar 19 01:18:04 big kernel: [1447935.395498] free:805909 slab:191425 mapped:29385 pagetables:2175 bounce:0
Mar 19 01:18:04 big kernel: [1447935.395498] free:805909 slab:191425 mapped:29385 pagetables:2175 bounce:0
Mar 19 01:18:04 big kernel: [1447935.395501] DMA free:7248kB min:292kB low:364kB high:436kB active:0kB inactive:56kB present:16256kB pages_scanned:45 all_unreclaimable? no
Mar 19 01:18:04 big kernel: [1447935.395501] DMA free:7248kB min:292kB low:364kB high:436kB active:0kB inactive:56kB present:16256kB pages_scanned:45 all_unreclaimable? no
Mar 19 01:18:04 big kernel: [1447935.395504] lowmem_reserve[]: 0 1746 17748 17748
Mar 19 01:18:04 big kernel: [1447935.395504] lowmem_reserve[]: 0 1746 17748 17748
Mar 19 01:18:04 big kernel: [1447935.395509] Normal free:14864kB min:16088kB low:20108kB high:24132kB active:11400kB inactive:10884kB present:894080kB pages_scanned:39243 all_unreclaimable? yes
Mar 19 01:18:04 big kernel: [1447935.395509] Normal free:14864kB min:16088kB low:20108kB high:24132kB active:11400kB inactive:10884kB present:894080kB pages_scanned:39243 all_unreclaimable? yes
Mar 19 01:18:04 big kernel: [1447935.395512] lowmem_reserve[]: 0 0 64008 64008
Mar 19 01:18:04 big kernel: [1447935.395512] lowmem_reserve[]: 0 0 64008 64008
Mar 19 01:18:04 big kernel: [1447935.395516] HighMem free:3201524kB min:512kB low:37376kB high:74240kB active:2120648kB inactive:2079664kB present:8193024kB pages_scanned:0 all_unreclaimable? no
Mar 19 01:18:04 big kernel: [1447935.395516] HighMem free:3201524kB min:512kB low:37376kB high:74240kB active:2120648kB inactive:2079664kB present:8193024kB pages_scanned:0 all_unreclaimable? no
Mar 19 01:18:04 big kernel: [1447935.395519] lowmem_reserve[]: 0 0 0 0
Mar 19 01:18:04 big kernel: [1447935.395519] lowmem_reserve[]: 0 0 0 0
Mar 19 01:18:04 big kernel: [1447935.395523] DMA: 127*4kB 92*8kB 76*16kB 40*32kB 15*64kB 4*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7260kB
Mar 19 01:18:04 big kernel: [1447935.395523] DMA: 127*4kB 92*8kB 76*16kB 40*32kB 15*64kB 4*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7260kB
Mar 19 01:18:04 big kernel: [1447935.395531] Normal: 1205*4kB 3*8kB 3*16kB 2*32kB 0*64kB 1*128kB 0*256kB 3*512kB 2*1024kB 1*2048kB 1*4096kB = 14812kB
Mar 19 01:18:04 big kernel: [1447935.395531] Normal: 1205*4kB 3*8kB 3*16kB 2*32kB 0*64kB 1*128kB 0*256kB 3*512kB 2*1024kB 1*2048kB 1*4096kB = 14812kB
Mar 19 01:18:04 big kernel: [1447935.395539] HighMem: 54590*4kB 55783*8kB 27554*16kB 18961*32kB 6606*64kB 3617*128kB 710*256kB 748*512kB 4*1024kB 1*2048kB 8*4096kB = 3201648kB
Mar 19 01:18:04 big kernel: [1447935.395539] HighMem: 54590*4kB 55783*8kB 27554*16kB 18961*32kB 6606*64kB 3617*128kB 710*256kB 748*512kB 4*1024kB 1*2048kB 8*4096kB = 3201648kB
Mar 19 01:18:04 big kernel: [1447935.395548] 640546 total pagecache pages
Mar 19 01:18:04 big kernel: [1447935.395548] 640546 total pagecache pages
Mar 19 01:18:04 big kernel: [1447935.395550] Swap cache: add 35, delete 32, find 0/0
Mar 19 01:18:04 big kernel: [1447935.395550] Swap cache: add 35, delete 32, find 0/0
Mar 19 01:18:04 big kernel: [1447935.395552] Free swap = 8008256kB
Mar 19 01:18:04 big kernel: [1447935.395552] Free swap = 8008256kB
Mar 19 01:18:04 big kernel: [1447935.395553] Total swap = 8008392kB
Mar 19 01:18:04 big kernel: [1447935.395553] Total swap = 8008392kB
Mar 19 01:18:04 big kernel: [1447935.431522] 2293760 pages of RAM
Mar 19 01:18:04 big kernel: [1447935.431522] 2293760 pages of RAM
Mar 19 01:18:04 big kernel: [1447935.431522] 2064384 pages of HIGHMEM
Mar 19 01:18:04 big kernel: [1447935.431522] 2064384 pages of HIGHMEM
Mar 19 01:18:04 big kernel: [1447935.431522] 228611 reserved pages
Mar 19 01:18:04 big kernel: [1447935.431522] 228611 reserved pages
Mar 19 01:18:04 big kernel: [1447935.431522] 351731 pages shared
Mar 19 01:18:04 big kernel: [1447935.431522] 351731 pages shared
Mar 19 01:18:04 big kernel: [1447935.431522] 3 pages swap cached
Mar 19 01:18:04 big kernel: [1447935.431522] 3 pages swap cached
Mar 19 01:18:04 big kernel: [1447935.431522] 2 pages dirty
Mar 19 01:18:04 big kernel: [1447935.431522] 2 pages dirty
Mar 19 01:18:04 big kernel: [1447935.431522] 1 pages writeback
Mar 19 01:18:04 big kernel: [1447935.431522] 1 pages writeback
Mar 19 01:18:04 big kernel: [1447935.431522] 28937 pages mapped
Mar 19 01:18:04 big kernel: [1447935.431522] 28937 pages mapped
Mar 19 01:18:04 big kernel: [1447935.431522] 191425 pages slab
Mar 19 01:18:04 big kernel: [1447935.431522] 191425 pages slab
Mar 19 01:18:04 big kernel: [1447935.431522] 2101 pages pagetables
Mar 19 01:18:04 big kernel: [1447935.431522] 2101 pages pagetables
Mar 19 01:18:04 big kernel: [1447935.431522] Out of memory: kill process 3484 (apache2) score 21304 or a child
Mar 19 01:18:04 big kernel: [1447935.431522] Out of memory: kill process 3484 (apache2) score 21304 or a child
Mar 19 01:18:04 big kernel: [1447935.431522] Killed process 13205 (apache2)
Mar 19 01:18:04 big kernel: [1447935.431522] Killed process 13205 (apache2)
Mar 19 01:38:56 big -- MARK --
Mar 19 01:58:56 big -- MARK --
Mar 19 02:18:56 big -- MARK --
Mar 19 02:58:56 big -- MARK --