4.3.14 Multiple BSODs Win 7 x64 Pro Guest

[surforacle]

VirtualBox Version: 4.3.14r95030

Host:

Win7 x64 Pro SP1 (all WU patches current)
Motherboard: Intel BOXDP55WG
CPU: 4-core I5-750 LGA1156 (Family 6, CPU Model 1E, Stepping 5, CPU Revision 4) Supports VT-x w/ Ext Page Tables
16 GB RAM
Video Adapter: Nvidia Geoforce 9500 GT
Host machine running in Dual-Display mode (running off single display adapter)

Guest VM:

Win7 x64 Pro SP1 (all WU patches current)
4096 MB RAM allocated to VM
2 CPUs Enabled, PAE/NX *not* enabled
PIIX3 Chipset
VT-x/AMD-V Enabled
Nested Paging Enabled
I/O APIC Enabled
4.3.14r95030 VM Guest Additions Installed

Note: No Anti-Virus products installed in either the Host or Guest. (Yes you read that correctly)

Problem Description:

Mutiple BSODs of Win7 x64 Pro guest during normal operation with a variety of different kernel trap addresses and stack frames. This is not a boot-time problem; it occurs somewhat randomly (no evident correlation to specific user or application activity), with a frequency of one to several times per day.

After observing a few crashes, using the kernel debugger, and turning on driver verification (for dxgmms1 and VBoxVideoWddm) I thought the fault code nexus might be video related because the first few crashes (after enabling driver verification) trapped in dxgmms1 or VBoxVideoWddm, e.g.

dxgmms1!VIDMM_DMA_POOL::RemoveDmaBufferFromPool+ba fffff880`04429892 488b4b78 mov rcx,qword ptr [rbx+78h]
or
VBoxVideoWddm+27066 fffff880`04230066 4c8911 mov qword ptr [rcx],r10 (called from dxgkrnl)


Subsequently I got crashes at a variety of locations however (no increased frequency.) here's a few fault IPs from separate BSOD crashes:

dxgmms1!VIDMM_GLOBAL::ReferenceAllocationForPreparation+0x19
nt!KiProcessExpiredTimerList+110
nt!ViIrpDatabaseFindPointer+46

I can post minidumps but I'm skeptical of the outcome as I am seeing BSOD crashes in a variety of unrelated stack frames.

I have tried disabling/enabling 2D acceleration without any differences. I have not (yet) tried disabling driver verification to see if that has any effect on fault location(s).

Background and Possibly Germane Info::

I have used VirtualBox on the same Host machine since about 3.2.12 without any guest BSOD/kernel panics, using primarily Win Xp x32, DOS, and both 32- and 64-bit Ubuntu guests, but have not used a Win 7 x64 Pro as a guest prior to 4.3.14. I have a good degree of confidence in the hardware, having previously run x32 and x64 guests in both SMP and single core configurations, largely without incident. But I had not previously run Win 7 x64 as a guest.

These BSOD crashes occur with another guest running (Win Xp x32); there is no other apparent guest or Host involvement during the Win7 BSOD.

I actually never observe the BSOD screen, even after following the recommendation

VBoxManage setextradata GuestName "VBoxInternal/PDM/HaltOnReset" 1

sometimes I will see a red bar momentarily appear across the top of the guest's window, but that's it.

I'm pretty sure II have operated a few Ubuntu guests (both x32 and x64) on 4.3.14 (since upgrading from 4.2.6) without similar incidents.

I wish there was a simple fix, but I think I am facing an uninstall of VM Guest Additions in 16 VM guests in order to roll back VirtualBox to a prior release.

Advice from anyone having good prior experiences with Win 7 x64 SP1 guests that want to recommend and earlier VirtualBox release would be quite welcome; I need to get some work done that requires W7. (I don't use my Host W7 machine OS for anything but hosting.)

[mpack]

VM log please. Minimum information needed for assistance.

[surforacle]

Here's a log from the guest. Thank you for your reply.

I don't see anything in the logs immediately prior to the crash, just

02:30:29.634689 AHCI#0: Reset the HBA

at the moment of the BSOD. The 7 minute interval after the crash is a reboot, me copying the kernel & minidump to a guest share, scheduling a disk check, and then shutting down (to do the disk check on next reboot).

Nothing in particular catches my eye in the log other than it describing the host processor as a Core-i7. Same Nehalem family, but not even on the same socket as the processor in my motherboard (Core i5, Lynnfield). Probably not significant though. (I've been using VB for 4 years on the same hardware; hard to imagine that it got anything wrong there).

I am willing to help out capturing further extended information, etc. VB has been very useful to me the past 4 years and I would like to give back.

That said if anyone wants to recommend a specific prior 4.x release that worked well for them with a combo of Win7 x64 host / Win7 x64 guest, that would be useful to me if I am forced to roll back (e.g. to a 4.2.x) in VB releases.

[socratis]

I took a look at your logs and your problem description, and I could see anything that strikes me as out of the ordinary. The message that you refer to (Reset the HBA) is perfectly normal.

Except maybe one little thing. You seem to be having four snapshots in your "Snapshots" directory [1], but you only seem to be actively using one [2]. Is there a reason for that? Could you try a different snapshot and see if the problem persists?

[1]

00:00:02.973149 ... \W7x64-CIMECH\Snapshots/{8110e9c4-74f6-4af1-9090-82d4981299f8}.vdi
00:00:02.973151 ... \W7x64-CIMECH\Snapshots/{d134f531-5286-4877-adde-77ab7e1e0d35}.vdi
00:00:02.973154 ... \W7x64-CIMECH\Snapshots/{c4944aaf-cc70-4c49-8403-a3a7002098b4}.vdi
00:00:02.973156 ... \W7x64-CIMECH\Snapshots/{77ef11fe-99f1-4f15-bcd7-d038c2740223}.vdi
00:00:02.973158 ... \W7x64-CIMECH\W7x64-CIMECH.vdi

[2]

00:00:01.861322 ... \W7x64-CIMECH\Snapshots/{8110e9c4-74f6-4af1-9090-82d4981299f8}.vdi

[surforacle]

I took a look at your logs and your problem description, and I could see anything that strikes me as out of the ordinary. The message that you refer to (Reset the HBA) is perfectly normal.

Well, normal after a BSOD/crash.

Except maybe one little thing. You seem to be having four snapshots in your "Snapshots" directory [1], but you only seem to be actively using one [2]. Is there a reason for that? Could you try a different snapshot and see if the problem persists?

I had to do a bunch of application installs in sequence, starting with some older 3.x .NET frameworks, and after that some (.NET dependent) time-limited trial software. Not only that but my W7 Pro x64 distro DVD is pre-SP1, so getting everything up to SP1 patchlevel is a huge boring time sink before I can even begin customizing. So those snapshots are there so I could clone from a given snapshot in case I needed to back up quite a long ways. Disk space is less expensive than my time hopefully

There's certainly a lot of things I could try. Right now I dropped the guest to single-core to see if no SMP changes anything ... not that I would want to run that way in the long term.

As I mentioned, the fact that I don't see a common locus in the stack frame of the kernel debugger log from crash to crash suggests something hard to debug - e.g. kernel heap corruption or similar. The bugcheck codes don't seem to be very uniform either, e.g.:

BUGCHECK_STR: 0xc4_13a PROCESS_NAME: csrss.exe [ DRIVER_VERIFIER_DETECTED_VIOLATION (c4) ]
BUGCHECK_STR: 0xD1 PROCESS_NAME: svchost.exe [ DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1) ]
BUGCHECK_STR: 0x50 PROCESS_NAME: csrss.exe [ PAGE_FAULT_IN_NONPAGED_AREA (50) ]
BUGCHECK_STR: 0x7E PROCESS_NAME: System [ SYSTEM_THREAD_EXCEPTION_NOT_HANDLED (7e) ]
BUGCHECK_STR: 0xA PROCESS_NAME: System [ IRQL_NOT_LESS_OR_EQUAL (a) ]
BUGCHECK_STR: 0xA PROCESS_NAME: csrss.exe [ IRQL_NOT_LESS_OR_EQUAL (a) ]
BUGCHECK_STR: 0xA PROCESS_NAME: System [ IRQL_NOT_LESS_OR_EQUAL (a) ]


I don't know what to make of it. But it *is* looking like those many snapshots will get some use. Ugh.

[Perryg]

I really think you should either install the 4.3.15 test build or revert back to 4.3.12. There are a lot of issues with 4.3.14 and it is not only virus software that triggers it. Some debuggers and even video drivers ( host side ) have been a cause since the hardening of VirtualBox was mandated.

[surforacle]

I really think you should either install the 4.3.15 test build or revert back to 4.3.12. There are a lot of issues with 4.3.14 and it is not only virus software that triggers it. Some debuggers and even video drivers ( host side ) have been a cause since the hardening of VirtualBox was mandated.

I'm taking your advice and uninstalling .14 & installing 4.3.12r93733 right now.

Dropping to one core didn't make a difference. I also turned off Driver Verification (in the W7 guest) completely. That didn't prevent a BSOD, but interestingly, I loaded up a bunch of apps and was able to force a BSOD (lots of foreground window changes). Did the kernel debugging afterwords and the trap location was back in the DirectX driver

dxgmms1!VIDMM_GLOBAL::UnreferenceDmaBuffer+0x7d

That's a sample of only one, so I don't know if that is meaningful, but it makes me wonder if turning on Driver Verification (for dxgmms1 and VBoxVideoWddm) was actually making the faulting locations more ambiguous.


The other thing that is immediately obvious (now that I am on 4.3.12 and have the Guest Extensions installed in the W7 x64 guest: I notice straight away that I do not hear any two-tone audio alert noises when I resize the guest window. I had thought that was odd when I first installed 4.3.14, but I couldn't find anything in any of the Windows event logs (in the guest) that indicated a driver or other system hiccup. iirc I also looked through the driver tree to see if there were any "not working" drivers and found nothing there either. So, I don't know what those audio tones were. (They are a little like the hot-plug notice sounds associated with USB device connection/disconnection).

Well, anyway; I will report back whether I continue to observe BSODs or can operate cleanly. I fired up a bunch of apps in 4.3.12 and so far so good. I will give it some more time though before I report back that this conclusively changes things.

[mpack]

It sounds like you should also look for updates to your hosts graphics drivers.

[surforacle]

Might be a little soon to say conclusively, but after a little over a day's use of the W7 x64 guest following the (Host) rollback to 4.3.12, I have had no further BSODs. I did not roll back the guest to a prior snapshot (nor change the host-side video driver); in the guest the only thing I did was to uninstall the 4.3.14 guest additions and then install the 4.3.12 guest additions (after uninstalling 4.3.14 from the host and installing 4.3.12)

I will follow up with a confirmation in a week or so (or sooner if I experience any further BSODs in the W7 guest).

Thanks to everyone who responded.

[surforacle]

FWIW

After 1 full week, no further BSODs on Win7 x64 Pro guest using a rollback to 4.3.12 (on the W7 Pro x64 host).

Curiously - while I was trying to resolve a problem with a CentOS7 x64 guest using 4.3.12** - I discovered that I still had the 4.3.14 extensions installed on the host (I did roll back all the guest additions in all my guests as well). Despite that, no BSODs in the Win 7x64 Pro guest during that time. Does that mean that the Host-side VirtualBox extensions in 4.3.14 have nothing to do with the guest W7 x64 problem? Don't know but that seems to be evidence that that is the case.

cheers


** way Off-topic but here. (Basically, getting VirtualBox 4.3.12 guest extensions to build correctly on RHEL 7 or CentOS 7) This bugfix is not in the 4.3.12 Linux guest extensions, and the 4.3.14 Linux guest extensions kernel interface code is wildly different in many many source files. I was able to build the 4.3.12 guest additions by dropping a single file (memobj-r0drv-linux.c) from the 4.3.14 guest extension ".run" file. into the 4.3.12 tree and re-packaging that back into a modded 4.3.12 ".run" file using "makeself-2.1.5"***. Probably other ways of skinning the cat, but I saw strange recommendations in these forums such as "just install the 4.3.14 guest additions" ... & that didn't seem like the right approach. Although the way I did it was rather tedious (you have to carve the tar file off the back of the .run file before unpacking).

*** e.g. $ makeself.sh --bzip2 --copy install VBoxLinuxAdditions-4.3.12-mod.run v4.3.12-mod '/bin/sh' './install.sh $0 1> /dev/null'