Virtualbox-4.3.0 - multiple hard crashes

Discussions related to using VirtualBox on Linux hosts.
gordboy
Posts: 3
Joined: 14. Mar 2011, 15:02
Primary OS: Ubuntu other
VBox Version: PUEL
Guest OSses: Windows 7, Windows XP
Location: Scotland

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by gordboy »

Hi guys.

I noticed this in my system logs the other day. Seems to explain why the 4.3 series is having problems particularly with NAT in the guests.

NAT[1743]: segfault at 9cd43826 ip a547d956 sp 9b41cfc8 error 4 in VBoxDD.so[a535f000+270000]

Doesn't happen all the time, but NAT doesn't work either, the guest is not getting an IP address from the virtual DHCP server.

I wound back to 4.2.18 for the time being, as neither XP nor Mint are working properly under both 4.3.0 and 4.3.2.
skoehler
Posts: 117
Joined: 1. Dec 2008, 12:12

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by skoehler »

gordboy wrote:NAT[1743]: segfault at 9cd43826 ip a547d956 sp 9b41cfc8 error 4 in VBoxDD.so[a535f000+270000]
Sorry, but this isn't even remotely related to anything we discuss here.
saintdev
Posts: 6
Joined: 5. Nov 2013, 07:28

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by saintdev »

saintdev wrote:I have recompiled with frame pointers and will attempt to get a backtrace when I'm prepared to lockup my system again.

Code: Select all

$ uname -a
Linux khopesh 3.10.17-ck.0 #2 SMP PREEMPT Tue Oct 22 16:26:22 MDT 2013 x86_64 Intel(R) Core(TM)2 Quad CPU Q9300 @ 2.50GHz GenuineIntel GNU/Linux
New kernel log with frame pointers. Not sure if it is of much help, the first trace is empty. I have also included the VM log and settings from VBoxManage showvminfo --details.

Edit: I am also willing to do more testing and/or supply the virtual machine. This VM can consistently lock up my host in a minute or two by just doing an update.
Attachments
khopesh-dmesg.log
Kernel log
(86.99 KiB) Downloaded 17 times
vminfo.txt
VM settings
(3.44 KiB) Downloaded 13 times
Arch Router Clone-2013-11-11-13-04-19.log
VM log
(57.7 KiB) Downloaded 14 times
skoehler
Posts: 117
Joined: 1. Dec 2008, 12:12

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by skoehler »

saintdev wrote:This VM can consistently lock up my host in a minute or two by just doing an update.
An update of what?
saintdev
Posts: 6
Joined: 5. Nov 2013, 07:28

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by saintdev »

skoehler wrote:
saintdev wrote:This VM can consistently lock up my host in a minute or two by just doing an update.
An update of what?
When I update the Arch guest.

All I have to do is run pacman -Syu and the guest is not even able to finish updating before causing a kernel panic on the host.
skoehler
Posts: 117
Joined: 1. Dec 2008, 12:12

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by skoehler »

saintdev wrote:When I update the Arch guest.

All I have to do is run pacman -Syu and the guest is not even able to finish updating before causing a kernel panic on the host.
So it's a linux guest? There is a tool called "stress". It allows you to generate plenty of CPU and I/O load. Can you reproduce the crash with that?
Ramshankar
Oracle Corporation
Posts: 793
Joined: 7. Jan 2008, 16:17

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by Ramshankar »

Code: Select all

[131856.147729] huh, entered softirq 1 TIMER ffffffff8104a350 preempt_count 00000200, exited with 00000101?
[131856.147735] huh, entered softirq 2 NET_TX ffffffff81522420 preempt_count 00000200, exited with 00000201?
[131856.147773] huh, entered softirq 3 NET_RX ffffffff81524460 preempt_count 00000200, exited with 00000bfe?
[131856.147781] huh, entered softirq 8 HRTIMER ffffffff8105ff60 preempt_count 00000200, exited with 000001ff?
[131856.147787] huh, entered softirq 9 RCU ffffffff810a6a70 preempt_count 00000200, exited with 000001fc?
[131856.147792] huh, entered softirq 3 NET_RX ffffffff81524460 preempt_count 00000201, exited with 00000201?
That's definitely sounds like some kernel preemption got borked. Will see if I can put out a VirtualBox test build later to try narrow down the problem more. Could you post "cat /boot/config-`uname -r`" or whatever the equivalent is on Arch linux.
Oracle Corp.
saintdev
Posts: 6
Joined: 5. Nov 2013, 07:28

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by saintdev »

skoehler wrote:
saintdev wrote:When I update the Arch guest.

All I have to do is run pacman -Syu and the guest is not even able to finish updating before causing a kernel panic on the host.
So it's a linux guest? There is a tool called "stress". It allows you to generate plenty of CPU and I/O load. Can you reproduce the crash with that?
I'll give it a try and report back, most likely it will.
saintdev
Posts: 6
Joined: 5. Nov 2013, 07:28

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by saintdev »

Ramshankar wrote:Could you post "cat /boot/config-`uname -r`" or whatever the equivalent is on Arch linux.
Do you want the guest (Arch) or host (Gentoo) config? I already posted the host's config in my previous post (except I enabled frame pointers for the second kernel log).

Edit: Also, would testing a kernel with preemption disabled help you in any way?
Ramshankar
Oracle Corporation
Posts: 793
Joined: 7. Jan 2008, 16:17

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by Ramshankar »

saintdev wrote:
Ramshankar wrote:Could you post "cat /boot/config-`uname -r`" or whatever the equivalent is on Arch linux.
Do you want the guest (Arch) or host (Gentoo) config? I already posted the host's config in my previous post (except I enabled frame pointers for the second kernel log).

Edit: Also, would testing a kernel with preemption disabled help you in any way?
No, the host kernel config. is enough. Thanks. I can't yet say what exactly goes wrong with it, whether CONFIG_PREEMPT_RCU or CONFIG_PREEMPT itself is causing problems. Here's what a stock Ubuntu kernel has which doesn't, afaict, have such lockups:

Code: Select all

# CONFIG_PREEMPT_RCU is not set
CONFIG_PREEMPT_NOTIFIERS=y
# CONFIG_PREEMPT_NONE is not set
CONFIG_PREEMPT_VOLUNTARY=y
# CONFIG_PREEMPT is not set
Again, it's just something to narrow down the problem a little bit than what is known now.
Oracle Corp.
MarkKnecht
Posts: 54
Joined: 14. Feb 2010, 22:34
Primary OS: Linux other
VBox Version: OSE self-compiled
Guest OSses: Win 7, Win XP, Gentoo 32-bit

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by MarkKnecht »

Ramshankar wrote:
saintdev wrote:
Ramshankar wrote:Could you post "cat /boot/config-`uname -r`" or whatever the equivalent is on Arch linux.
Do you want the guest (Arch) or host (Gentoo) config? I already posted the host's config in my previous post (except I enabled frame pointers for the second kernel log).

Edit: Also, would testing a kernel with preemption disabled help you in any way?
No, the host kernel config. is enough. Thanks. I can't yet say what exactly goes wrong with it, whether CONFIG_PREEMPT_RCU or CONFIG_PREEMPT itself is causing problems. Here's what a stock Ubuntu kernel has which doesn't, afaict, have such lockups:

Code: Select all

# CONFIG_PREEMPT_RCU is not set
CONFIG_PREEMPT_NOTIFIERS=y
# CONFIG_PREEMPT_NONE is not set
CONFIG_PREEMPT_VOLUNTARY=y
# CONFIG_PREEMPT is not set
Again, it's just something to narrow down the problem a little bit than what is known now.
For the sake of conversation, this is what I have in my 3.10.18 gentoo-sources kernel:

mark@c2RAID6 ~ $ zcat /proc/config.gz | grep PREEMPT
CONFIG_TREE_PREEMPT_RCU=y
CONFIG_PREEMPT_RCU=y
CONFIG_PREEMPT_NOTIFIERS=y
# CONFIG_PREEMPT_NONE is not set
# CONFIG_PREEMPT_VOLUNTARY is not set
CONFIG_PREEMPT=y
CONFIG_PREEMPT_COUNT=y
CONFIG_DEBUG_PREEMPT=y
mark@c2RAID6 ~ $

As with everything Gentoo we're sort of on our own. I have no reason to trust these settings, assuming they are involved at all. However this crashes with 4.3.x and works fine with 4.2.18. I am willing to try some changes if there's a reasonable willingness to evaluate going forward.

I do currently have more kernel debugging turned on than I used to.

- Mark
skoehler
Posts: 117
Joined: 1. Dec 2008, 12:12

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by skoehler »

the CONFIG_PREEMPT_* settings are controlled by the option "Preemption Model" under "Processor Type and Features" in the kernel's menuconfig. I switched from "Preemptible Kernel (Low-latency Desktop)" to "Voluntary Kernel Preemption (Desktop)". The CONFIG_PREEMPT_* settings now match what Ramshankar posted above about the Ubuntu kernel.
I will let you know, whether I experience the host crash with virtualbox 4.3 again.
nsousa
Posts: 1
Joined: 13. Nov 2013, 17:06

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by nsousa »

skoehler wrote:the CONFIG_PREEMPT_* settings are controlled by the option "Preemption Model" under "Processor Type and Features" in the kernel's menuconfig. I switched from "Preemptible Kernel (Low-latency Desktop)" to "Voluntary Kernel Preemption (Desktop)". The CONFIG_PREEMPT_* settings now match what Ramshankar posted above about the Ubuntu kernel.
I will let you know, whether I experience the host crash with virtualbox 4.3 again.
I also made these changes and I can report that I'm not having any problems.

My host is 32 bit Gentoo/linux. Guest is Windows 7 Professional 64-bits.
Before this change the host would crash during the boot or login procedure. It has been working for over 30 minutes without any problem.
web-devel
Posts: 2
Joined: 7. Nov 2013, 01:25

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by web-devel »

Perryg wrote:
web-devel wrote:Last week I upgraded to Suse 12.3/Virtualbox 4.3.0 and feel I now have a very similar problem. Hard freeze on the base Suse OS. The console is sadly unresponsive. Turning VM to use only one processor seems to help. I'd love to downgrade but a package < 4.3.0 does not appear to be available for Suse 12.3.
I would upgrade to 4.3.2 myself.
Update for my case:
I Downgraded to version 4.2.14. from this location.
http://forums.opensuse.org/english/get- ... 2-a-2.html and have no problems since.
MarkKnecht
Posts: 54
Joined: 14. Feb 2010, 22:34
Primary OS: Linux other
VBox Version: OSE self-compiled
Guest OSses: Win 7, Win XP, Gentoo 32-bit

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by MarkKnecht »

skoehler wrote:the CONFIG_PREEMPT_* settings are controlled by the option "Preemption Model" under "Processor Type and Features" in the kernel's menuconfig. I switched from "Preemptible Kernel (Low-latency Desktop)" to "Voluntary Kernel Preemption (Desktop)". The CONFIG_PREEMPT_* settings now match what Ramshankar posted above about the Ubuntu kernel.
I will let you know, whether I experience the host crash with virtualbox 4.3 again.
As of yesterday I recompiled with the more modest preempt settings:

mark@c2RAID6 ~ $ zcat /proc/config.gz | grep PREEMPT
# CONFIG_PREEMPT_RCU is not set
CONFIG_PREEMPT_NOTIFIERS=y
# CONFIG_PREEMPT_NONE is not set
CONFIG_PREEMPT_VOLUNTARY=y
# CONFIG_PREEMPT is not set
mark@c2RAID6 ~ $

and then updated to virtualbox-4.3.2 under Gentoo. I did not update the guest additions immediately but rather ran 2 Win 7 VMs along with the VMWare Player VM with no problems. In that state I've run about 12 hours without problems. I don't think I ever went more than 1-2 hours with the other preempt settings & 4.3.x so these setting changes appear to be a reasonable workaround. (But only a work around - NOT a solution...)

I've just this morning updated the guest additions on both VMs and am now working with that setup. I'll continue to track this over the next few days and report back later.

I will say that these settings do seem to have slowed things down. The machine doesn't feel quite as fast but that may be imaginary as I haven't run anything other than the low latency desktop setting for years.

I do hope the Virtualbox developers will continue to work on this issue.
Locked