Virtualbox-4.3.0 - multiple hard crashes

Discussions related to using VirtualBox on Linux hosts.
Locked
MarkKnecht
Posts: 54
Joined: 14. Feb 2010, 22:34
Primary OS: Linux other
VBox Version: OSE self-compiled
Guest OSses: Win 7, Win XP, Gentoo 32-bit

Virtualbox-4.3.0 - multiple hard crashes

Post by MarkKnecht »

Just a heads-up post looking for any others experiencing issues.

My host machine is a mostly stable Gentoo 64-bit box which runs a couple of Virtualbox Windows guests all day long. I've been running the 4.2.X series for months with no problems. I updated to 4.3 today and immediately had 2 hard crashes while running my regular VMs. The box became completely unresponsive. Couldn't even ping it from another box much less log in. Host keys in VB didn't work. Linux magic keys didn't work. Only way out seems to have been a hard reset.

I've downgraded back to 4.2.18 but haven't done any testing as I'm waiting on an fsck to finish. Will report more info later if I have anything to say.
Ramshankar
Oracle Corporation
Posts: 793
Joined: 7. Jan 2008, 16:17

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by Ramshankar »

Could you attach the VBox.log for a VM that when run caused the crash?

Anything in the system log regarding virtualbox/crashes would also be useful.
Oracle Corp.
MarkKnecht
Posts: 54
Joined: 14. Feb 2010, 22:34
Primary OS: Linux other
VBox Version: OSE self-compiled
Guest OSses: Win 7, Win XP, Gentoo 32-bit

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by MarkKnecht »

OK, I hope I'm giving you the right stuff. I've downgraded back to 4.2.18 and I'm not seeing problems so far. I'll keep you up to date on that if things change and for some reason it appears to not be related to 4.3.0.

Two VM logs attached:

Builder VM
Win7-DualMonitor

I believe they were both running at the time of the last crash. THey are both from around noon my time although they don't have the same time stamps so take this with a grain of salt.

Good luck & please let me know if you find anything obvious.
Attachments
Win7-DualMonitor-VBox.log
Log from Win7-DualMonitor VM crash
(58.29 KiB) Downloaded 49 times
Builder-VM-VBox.log
Log from Builder VM crash
(65.16 KiB) Downloaded 35 times
Ramshankar
Oracle Corporation
Posts: 793
Joined: 7. Jan 2008, 16:17

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by Ramshankar »

I can't tell from just the VBox.log what the exact problem is but it is so I have an idea of the general configuration of things (both in terms of host and guest features)

One of the VM has 3D enabled and the other one is a 4 CPU Windows 7 VM. Both should work but obviously something goes wrong and crashes. In an effort to narrow down the problem, can you try with just a single VM with 1 guest-CPU and without 3D and then if things are stable slowly ramp it up? If not, we'll have to try and reproduce the problem ourselves on a similar host environment you have.

Could you post "uname -a" output from your Gentoo host?
Oracle Corp.
MarkKnecht
Posts: 54
Joined: 14. Feb 2010, 22:34
Primary OS: Linux other
VBox Version: OSE self-compiled
Guest OSses: Win 7, Win XP, Gentoo 32-bit

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by MarkKnecht »

The following is stuff from /var/log/messages pre & post crash. None of it looks very specific to this issue to me anyway. The rebuild events on /dev/md3 were something I started by hand after the crash to ensure the RAID still was good.

As for more if you want or need it.

HTH,
Mark


Oct 15 11:19:57 c2RAID6 udisksd[2657]: Acquired the name org.freedesktop.UDisks2 on the system message bus
Oct 15 11:19:57 c2RAID6 kernel: [ 40.797528] ata13.00: configured for UDMA/133
Oct 15 11:19:57 c2RAID6 kernel: [ 40.797531] ata13: EH complete
Oct 15 11:19:57 c2RAID6 kernel: [ 40.912579] ata14.00: configured for UDMA/133
Oct 15 11:19:57 c2RAID6 kernel: [ 40.912581] ata14: EH complete
Oct 15 11:19:57 c2RAID6 dbus[2095]: [system] Activating service name='org.kde.powerdevil.backlighthelper' (using servicehelper)
Oct 15 11:19:57 c2RAID6 dbus[2095]: [system] Successfully activated service 'org.kde.powerdevil.backlighthelper'
Oct 15 11:19:58 c2RAID6 kernel: [ 42.007050] EXT4-fs (md3): re-mounted. Opts: commit=60,commit=0
Oct 15 11:20:01 c2RAID6 cron[2704]: (root) CMD (test -x /usr/sbin/run-crons && /usr/sbin/run-crons)
Oct 15 11:20:12 c2RAID6 polkitd[2390]: Registered Authentication Agent for unix-session:/org/freedesktop/ConsoleKit/Session1 (system bus name :1.20 [/usr/lib64/kde4/libexec/polkit-kde-authentication-agent-1], object path /org/kde/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8)
Oct 15 11:20:19 c2RAID6 su[2809]: Successful su for root by mark
Oct 15 11:20:19 c2RAID6 su[2809]: + /dev/pts/1 mark:root
Oct 15 11:20:19 c2RAID6 su[2809]: pam_unix(su:session): session opened for user root by mark(uid=1000)
Oct 15 11:21:38 c2RAID6 mdadm[1829]: RebuildStarted event detected on md device /dev/md/3
Oct 15 11:21:38 c2RAID6 kernel: [ 141.986001] md: data-check of RAID array md3
Oct 15 11:21:38 c2RAID6 kernel: [ 141.986004] md: minimum _guaranteed_ speed: 1000 KB/sec/disk.
Oct 15 11:21:38 c2RAID6 kernel: [ 141.986005] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for data-check.
Oct 15 11:21:38 c2RAID6 kernel: [ 141.986010] md: using 128k window, over a total of 484088160k.
Oct 15 11:30:01 c2RAID6 cron[3498]: (root) CMD (test -x /usr/sbin/run-crons && /usr/sbin/run-crons)
Oct 15 11:40:01 c2RAID6 cron[5102]: (root) CMD (test -x /usr/sbin/run-crons && /usr/sbin/run-crons)
Oct 15 11:44:12 c2RAID6 kernel: [ 1496.536643] warning: `VirtualBox' uses 32-bit capabilities (legacy support in use)
Oct 15 11:44:13 c2RAID6 vmnetBridge: RTM_NEWLINK: name:vboxnet0 index:6 flags:0x00001002
Oct 15 11:44:13 c2RAID6 vmnet-natd: RTM_NEWLINK: name:vboxnet0 index:6 flags:0x00001002
Oct 15 11:44:16 c2RAID6 kernel: [ 1500.972844] device eth0 entered promiscuous mode
Oct 15 11:49:54 c2RAID6 kernel: [ 1838.775379] device eth0 left promiscuous mode
Oct 15 11:49:54 c2RAID6 kernel: [ 1839.287143] vboxnetflt: 0 out of 1722 packets were not sent (directed to host)
Oct 15 11:50:01 c2RAID6 cron[6112]: (root) CMD (test -x /usr/sbin/run-crons && /usr/sbin/run-crons)
Oct 15 11:54:12 c2RAID6 kernel: [ 2097.063197] device eth0 entered promiscuous mode
Oct 15 11:54:58 c2RAID6 mdadm[1829]: Rebuild32 event detected on md device /dev/md/3
Oct 15 11:59:01 c2RAID6 cron[12834]: (root) CMD (rm -f /var/spool/cron/lastrun/cron.hourly)
Oct 15 12:00:01 c2RAID6 cron[12850]: (root) CMD (test -x /usr/sbin/run-crons && /usr/sbin/run-crons)
Oct 15 12:10:01 c2RAID6 cron[15926]: (root) CMD (test -x /usr/sbin/run-crons && /usr/sbin/run-crons)
Oct 15 12:20:01 c2RAID6 cron[17190]: (root) CMD (test -x /usr/sbin/run-crons && /usr/sbin/run-crons)
Oct 15 12:28:19 c2RAID6 mdadm[1829]: Rebuild46 event detected on md device /dev/md/3
Oct 15 12:30:01 c2RAID6 cron[20865]: (root) CMD (test -x /usr/sbin/run-crons && /usr/sbin/run-crons)
Oct 15 12:30:01 c2RAID6 run-crons[20880]: (root) CMD (/etc/cron.daily/logrotate)
Oct 15 12:30:02 c2RAID6 run-crons[21038]: (root) CMD (/etc/cron.daily/makewhatis)
Oct 15 12:30:05 c2RAID6 run-crons[21627]: (root) CMD (/etc/cron.daily/mlocate)
Oct 15 12:46:03 c2RAID6 syslog-ng[2051]: syslog-ng starting up; version='3.4.2'
Oct 15 12:46:03 ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ kernel: [ 0.000000] Linux version 3.10.16-gentoo (root@c2RAID6) (gcc version 4.6.3 (Gentoo 4.6.3 p1.13, pie-0.5.2) ) #1 SMP PREEMPT Mon Oct 14 11:05:04 PDT 2013
Oct 15 12:46:03 c2RAID6 kernel: [ 0.000000] Command line: root=LABEL=RAID6root video=vesafb vga=0x307
Oct 15 12:46:03 c2RAID6 kernel: [ 0.000000] e820: BIOS-provided physical RAM map:
Oct 15 12:46:03 c2RAID6 kernel: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009e3ff] usable
Oct 15 12:46:03 c2RAID6 kernel: [ 0.000000] BIOS-e820: [mem 0x000000000009e400-0x000000000009ffff] reserved
Oct 15 12:46:03 c2RAID6 kernel: [ 0.000000] BIOS-e820: [mem 0x00000000000e4000-0x00000000000fffff] reserved
Ramshankar
Oracle Corporation
Posts: 793
Joined: 7. Jan 2008, 16:17

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by Ramshankar »

Hmm, I wonder if your gentoo is somehow configured with some odd combination of kernel parameters that trips up VirtualBox..

The syslog doesn't give me much info either. Like I said, what might be useful is if you try with a single VCPU VM without 3D and see if even that works.
Oracle Corp.
MarkKnecht
Posts: 54
Joined: 14. Feb 2010, 22:34
Primary OS: Linux other
VBox Version: OSE self-compiled
Guest OSses: Win 7, Win XP, Gentoo 32-bit

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by MarkKnecht »

According to the settings tab I don't have 3D enabled. Never have to the best of my knowledge. Don't need it.

As for a single VCPU VM, are you asking me to simply reset my current VM to 1 CPU? That's easy to do and I'll try it after I finish doing some backups. However that would require that I reinstall 4.3.0. No big deal. I just want to be clear I'm doing the experiment you're asking me to do.

Your 'single VCPU' comment brings up something that seems to have changed between 4.2.18 and 4.3. In 4.2.18 I set my CPU count for this VM to 8. The host is an Intel i7 980x which is a 6 physical CPU processor with hyperthreading. 4.2.18 never complained and in Win 7 I saw 8 processors.

In 4.3 I saw a message about an incorrect setting WRT CPUs, telling me I should never set it higher than the number of physical CPUs. I lowered the setting to 4 but still got a second crash. (This all happened yesterday)

Anyway, in 4.2.18 I was able to get 8 CPUs in the Win7 Task Manager. Now it seems I'm not going to be able to get as many. Is this by design or is there a way around it? One of my apps is computational and multi-threaded so it uses as many CPUs as I choose to make available.

Thanks
MarkKnecht
Posts: 54
Joined: 14. Feb 2010, 22:34
Primary OS: Linux other
VBox Version: OSE self-compiled
Guest OSses: Win 7, Win XP, Gentoo 32-bit

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by MarkKnecht »

Attaching 3.10.16 kernel config taken from zcat /proc/config.gz. (Boy this system is picky about file names and extensions!)
Attachments
Gentoo_3_10_16_config.txt
3.10.16 kernel config
(79.28 KiB) Downloaded 38 times
Ramshankar
Oracle Corporation
Posts: 793
Joined: 7. Jan 2008, 16:17

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by Ramshankar »

4.3 warns you about the CPU based on the physical cores on your system rather than hyperthreads. In general, you shouldn't exceed the physical core count for performance reasons (scheduling).

As for the crash, did you get a chance to try with a single VCPU VM (yes, changing the number of guest-CPUs to 1)?
Oracle Corp.
MarkKnecht
Posts: 54
Joined: 14. Feb 2010, 22:34
Primary OS: Linux other
VBox Version: OSE self-compiled
Guest OSses: Win 7, Win XP, Gentoo 32-bit

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by MarkKnecht »

I am running 4.3 with the VM set to 1 processor as of 30 minutes ago. No problems so far, but the previous settings didn't fail immediately. It was only after doing some work for awhile so I'll likely have to run this way for a day or two to feel confident, and then push the processor count up a bit and see if I can trigger a failure I guess. That could take, what days I guess to figure anything out.

It the old thing about 'No evidence of disease' vs 'Evidence of no disease'. They are truly different things.

I'll work this way for 1-2 days and report back what happens.

As for the 4.3 # of VCPU warning, I can only say I've always run these VMs for the last 3-4 years with CPU counts based on hyperthread desires and not physical CPUs. If there was a problem before I guess I just got lucky.

Thanks,
Mark
jde
Posts: 2
Joined: 17. Oct 2013, 08:15

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by jde »

Hi,

I experienced hard crashed after upgrading from Version 4.2.18 too. My box is a 64 bit OpenSUSE 12.3 Notebook with NVidia graphics. The last lines of the VM log show the following:

00:02:00.305880 Display::handleDisplayResize(): uScreenId = 0, pvVRAM=00007f5ba4000000 w=1280 h=969 bpp=32 cbLine=0x1400, flags=0x1
00:02:00.314481 Guest Log: VBoxDisp[0]: VBVA enabled
00:02:00.314520 VBVA_INFO_SCREEN: [0] @0,0 1280x969, line 0x1400, BPP 32, flags 0x1
00:02:00.314531 Display::handleDisplayResize(): uScreenId = 0, pvVRAM=00007f5ba4000000 w=1280 h=969 bpp=32 cbLine=0x1400, flags=0x1
00:02:00.314538 Display::handleDisplayResize(): Warning: resize postponed.
00:02:00.325755 Display::handleDisplayResize(): uScreenId = 0, pvVRAM=00007f5ba4000000 w=1280 h=969 bpp=32 cbLine=0x1400, flags=0x1
00:02:00.377804 gl version string: 04.3.0 NVIDIA 319.32
00:02:00.377824 gl version: 0x40300
00:02:00.377830 gl extensions: GL_AMD_multi_draw_indirect GL_ARB_arrays_of_arrays GL_ARB_base_instance GL_ARB_blend_func_extended GL_ARB_clear_buffer_object GL_ARB_color_buffer_float GL_ARB_compatibility GL_ARB_compressed_texture_pixel_storage GL_ARB_conservative_depth GL_ARB_compute_shader GL_ARB_copy_buffer GL_ARB_copy_image GL_ARB_debug_output GL_ARB_depth_buffer_float

Looks like having problems with 3D or maybe 2D video acceleration. Both are enabled in my WinXP VM, which crashed while starting before the desktop is shown. It's configured with 4 VCPUs, the number of physical CPUs. So this shouldn't be the problem. After making some backups I'll try the VM with 3D an 2D video switched off...

Any other ideas?

Thanks,
Jens
jde
Posts: 2
Joined: 17. Oct 2013, 08:15

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by jde »

Hi again,

starting my WinXP VM with VirtualBox 4.3 worked without crashing the host after disabling 3D and 2D-video acceleration. After installing the new guest additions (actually upgrade from 4.2.18 to 4.3) I could re-enable 3D and 2D-video acceleration and everything works fine :-)

Jens
MarkKnecht
Posts: 54
Joined: 14. Feb 2010, 22:34
Primary OS: Linux other
VBox Version: OSE self-compiled
Guest OSses: Win 7, Win XP, Gentoo 32-bit

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by MarkKnecht »

Ramshankar wrote:I can't tell from just the VBox.log what the exact problem is but it is so I have an idea of the general configuration of things (both in terms of host and guest features)

One of the VM has 3D enabled and the other one is a 4 CPU Windows 7 VM. Both should work but obviously something goes wrong and crashes. In an effort to narrow down the problem, can you try with just a single VM with 1 guest-CPU and without 3D and then if things are stable slowly ramp it up? If not, we'll have to try and reproduce the problem ourselves on a similar host environment you have.

Could you post "uname -a" output from your Gentoo host?
Sorry. I'm sick with a cold and was not reading as carefully as I should.

1) According to the Virtualbox GUI neither VM has 3D enabled. One has 2D enabled. Maybe you mistyped?

2) Kernel is gentoo-sources-3.10.16:

mark@c2RAID6 ~ $ uname -a
Linux c2RAID6 3.10.16-gentoo #1 SMP PREEMPT Mon Oct 14 11:05:04 PDT 2013 x86_64 Intel(R) Core(TM) i7 CPU X 980 @ 3.33GHz GenuineIntel GNU/Linux
mark@c2RAID6 ~ $


c2RAID6 ~ # uname -a
Linux c2RAID6 3.10.16-gentoo #1 SMP PREEMPT Mon Oct 14 11:05:04 PDT 2013 x86_64 Intel(R) Core(TM) i7 CPU X 980 @ 3.33GHz GenuineIntel GNU/Linux
c2RAID6 ~ #


3) The machine called "Builder VM" was the VM with 2D enabled. I ran it yesterday afternoon and all morning with 2D enabled and 1 processor. No problems. I then disabled 2D and set the processor count to 2. The VM crashed the the whole Gentoo box while booting Win 7 in the VM. No way out (as best I could tell) except for a hard reset. Log file attached.

Thanks for your help.
Attachments
VBox.log
2 VCPU log file after crash
(56.83 KiB) Downloaded 25 times
MarkKnecht
Posts: 54
Joined: 14. Feb 2010, 22:34
Primary OS: Linux other
VBox Version: OSE self-compiled
Guest OSses: Win 7, Win XP, Gentoo 32-bit

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by MarkKnecht »

Hi,
OK, in reviewing some of the virtualization settings (PAE/NX, Vt-x, Nested Paging, IO APIC, etc) I ran across this part of the Virtualbox manual:
Warning
Do not run other hypervisors (open-source or commercial virtualization products) together with VirtualBox! While several hypervisors can normally be installed in parallel, do not attempt to run several virtual machines from competing hypervisors at the same time. VirtualBox cannot track what another hypervisor is currently attempting to do on the same host, and especially if several products attempt to use hardware virtualization features such as VT-x, this can crash the entire host. Also, within VirtualBox, you can mix software and hardware virtualization when running multiple VMs. In certain cases a small performance penalty will be unavoidable when mixing VT-x and software virtualization VMs. We recommend not mixing virtualization modes if maximum performance and low overhead are essential. This does not apply to AMD-V.
I do not know yet whether it's involved but I have (for many YEARS now) been running a copy of VMWare Player in parallel with Virtualbox. It was running during (I think) all of the crashes that have occurred over the last couple of days with 4.3. If I understand the text above the VB team is telling me I cannot do that? It's _never_ been a problem before but maybe it's a problem now?

I'd LOVE to get rid of VMWare Player completely but it has one feature for that VM that I cannot live without. I work at home on my Linux box but use that VMWare VM to watch online media. (Netflix, Hulu, etc.) WIth Player I can tag the VM window to display on all 6 KDE desktops and when I go full screen with Player it does display on all desktops. This means I can scroll through all my desktops looking at different apps I'm running and the media appears to not change. Unfortunately Virtualbox has never done that. If I go full screen with a VB VM then Virtualbox only displays on one desktop which doesn't work for me.

Anyway, I'll do some more experiments to see if the is somehow related to Player running at the same time, but it appears to be a 4.3 regression because this has never been a problem with earlier releases.
socratis
Site Moderator
Posts: 27330
Joined: 22. Oct 2010, 11:03
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Win(*>98), Linux*, OSX>10.5
Location: Greece

Re: Virtualbox-4.3.0 - multiple hard crashes

Post by socratis »

MarkKnecht wrote:It's _never_ been a problem before but maybe it's a problem now?
It is my understanding that the VT-x code was heavily reworked in the 4.3 release, so that may be part of the problem.
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Locked