3D acceleration causes crash after login (ubuntu)

[Poison]

This problem has been pestering me for weeks. I have tried everything I could find online. I'll try to list everything relevant, but let me know if you need any more information.

- First and foremost, prior to this problem occurring I had a linux mint VM that ran on this machine without any problems (3D acceleration working fine) for quite some time. I believe this problem began with a recent iteration of VBox, but I am not certain and can not pinpoint exactly when things got screwy. I do not remember the version number I was using prior.
- I have tried several different ISO's. All linux based. The same problem occurs on all of them. I am currently trying to get Ubuntu 16.04 64bit to work.
- Virtualization is enabled for my computer
- i7 5700HQ, GTX 980m, 32GB ram, Windows 10 (all drivers for everything are up to date afaik)
- I have tried every permutation of installing the guest additions that I could think of. Countless clean installs. I have tried installing the packages recommended at Ubuntu install time.
- Starts fine as long as 3D acceleration is NOT enabled
- Crash occurs after typing in password and clicking login. Basically I never get logged in, and if I try to exit VBox, it begins not responding.
- I have tried a variety of different settings for ram allocation, processors, and all kinds of other setting recommendations that I found whilst trying to solve this problem.
- THE STRANGEST PART: sometimes it miraculously works. I can not reproduce it intentionally. It will start up, ill run unity_support_test, everything will look good. Reboot, and out of nowhere it stops working again. This is without changing any settings.


Any recommendations?

[socratis]

From a quick look the VM is underpowered:

• 1 GB of RAM. You have 28 GB available on the host at the start, you could afford 2 GB for the guest.
• 16 MB of vRAM. You most certainly can afford at least 64 MB.
• 1 CPU. Your host has 4 CPUs. I'd give the VM 2 CPUs.
• But the biggest problem I believe is the following:

  Code: Select all

  00:00:23.403089 supR3HardenedErrorV: supR3HardenedScreenImage/LdrLoadDll: rc=VERR_CR_PKCS7_KEY_USAGE_MISMATCH fImage=1 fProtect=0x0 fAccess=0x0 \Device\HarddiskVolume3\Windows\System32\DriverStore\FileRepository\nvmii.inf_amd64_a62dff8086e215aa\nvd3dumx.dll: WinVerifyTrust failed with hrc=CERT_E_CHAINING on '\Device\HarddiskVolume3\Windows\System32\DriverStore\FileRepository\nvmii.inf_amd64_a62dff8086e215aa\nvd3dumx.dll'
  
  00:00:23.403246 supR3HardenedErrorV: supR3HardenedMonitor_LdrLoadDll: rejecting 'C:\WINDOWS\System32\DriverStore\FileRepository\nvmii.inf_amd64_a62dff8086e215aa\nvd3dumx.dll' (C:\WINDOWS\System32\DriverStore\FileRepository\nvmii.inf_amd64_a62dff8086e215aa\nvd3dumx.dll): rcNt=0xc0000190
  
  00:00:23.403603 supR3HardenedErrorV: supR3HardenedScreenImage/LdrLoadDll: cached rc=VERR_CR_PKCS7_KEY_USAGE_MISMATCH fImage=1 fProtect=0x0 fAccess=0x0 cHits=1 \Device\HarddiskVolume3\Windows\System32\DriverStore\FileRepository\nvmii.inf_amd64_a62dff8086e215aa\nvd3dumx.dll
  
  00:00:23.403640 supR3HardenedErrorV: supR3HardenedMonitor_LdrLoadDll: rejecting 'C:\WINDOWS\System32\DriverStore\FileRepository\nvmii.inf_amd64_a62dff8086e215aa\nvd3dumx.dll' (C:\WINDOWS\System32\DriverStore\FileRepository\nvmii.inf_amd64_a62dff8086e215aa\nvd3dumx.dll): rcNt=0xc0000190

  That's a bunch of NVidia related files that are rejected due to hardened security. Try to update your host's graphics drivers.

[Poison]

I made the changes you suggested. I also updated the OpenGL Version from 4.5.0 NVIDIA 376.19 -> 4.5.0 NVIDIA 376.20.

I believe that this is the latest version there is. Unfortunately, the error remains. I attached log files of the machine running correctly (before guest additions is installed) and crashing (after) with the latest version drivers and setting changes.

[socratis]

• The guest additions is a red herring I believe. It's the 3D that is causing this. When the 3D functionality is invoked, the NVidia driver is trying to load some files in the VirtualBox process. The loading of these files fails, because they're not properly signed.
• Is it only this guest? You mentioned a Mint guest that run (at least at some point).
• What happens if you downgrade to an older version of VirtualBox? It's not as difficult as you might think.
• I'm not that great in reading the tea leaves in the VBoxHardening.log, but could you upload a zipped version of that as well?

[mpack]

That's weird: because I also have a Win10-64bit host, and I'm using an NVidia GeForce GTX 970, and I have no problem enabling 3D acceleration in guests. Do you have any theme hacks installed on the host?

I was confused by the word "crash", as I struggled to find details of the crash report. You have described a hang (software is still running, but not responding), not a crash (software terminates abruptly, usually with a low level error message).

p.s. The guest OS is probably irrelevant to this question. The problem merely comes when the guest tries to initialize a 3D accelerated graphics mode.

p.p.s. Save a virtual tree (i.e. server space). Zip your logs!

[Poison]

@socratis
It is every guest I have tried. Mint previously worked, and then abruptly stopped working. I have not tried downgrading. I can upload a copy of the VBoxHardening log.

@mpack
You are correct, it hangs and stops responding. Process has to be killed. No theme hacks.


Here's the kicker. I just woke up after a night of having made no more changes. No computer restarts, no log offs, no nothing. It is once again working correctly for absolutely no reason. I'll attach these logs.

I am going to have to wait until it stops working again to get the VBoxHardening logs. Shouldn't be long.

[tsubery]

I have similar issue. Sometimes it crashes immediately and sometimes it crashes as soon as I open chrome browser in the guest. My host is Windows 10, i7 6700K, Nvidia GTX970M.
Any help would be appreciated.
I have tried about 4 different nvidia drivers from the recent months. Uninstalling and reinstalling guest additions. Couldn't find any indication of error interesting in the log files (but probably don't know what to look for).

[socratis]

@Poison
If it works with 3D disabled, then it's a problem with the drivers of the graphics card, unless something funky comes up in the VBoxHardening.log. But it definitely happens when entering in 3D area...

@tsubery
Your VBoxHardening.log seems fine. But, you face a known problem. If you had just Googled for it (not from within your VM, it would crash ), you would have found the answer in half the time it took you to register. You need to disable 3D acceleration in Chrome.

[tsubery]

I tried disabling hardware acceleration in chrome. The problem is, It crashes even if I don't start chrome at all. Either immediately after login or when I resize virtual box or enter/exit the vm. Basically, I could not use the vm with 3d acceleration for more than a couple of minutes of normal usage.

[Poison]

@socratis
The working.zip logs were functioning with 3D enabled, and unity_support_test looks good across the board. I just started it up again, and it seems to still be working 3D enabled and all. This inconsistency is driving me crazy. I'm going to restart the host and see what happens.

Well, after a reboot of the host it is still working. Unfortunately, this is what happened last time I got it working before it decided to stop. If and when I run into problems again, I'll post all the logs here. My gut suspicion is it has something to do with windows updates... but I wouldn't know since windows likes to update without my permission and when I'm not looking.

[Poison]

Once again, without doing anything at all (no restarts, no log off, no change in internet connection, no nothing), it has stopped working. Same error. Logs attached.

I restarted my computer several times earlier and it worked correctly every time. It's almost like after a certain time of day this error starts to occur....


Snippet from VBoxHardening.log:

Code: Select all

31fc.3bb8: supHardenedWinVerifyImageByHandle: -> -23303 (\Device\HarddiskVolume3\Windows\System32\DriverStore\FileRepository\nvmi.inf_amd64_ae396a6228d16bb2\nvd3dumx.dll) WinVerifyTrust
31fc.3bb8: Error (rc=0):
31fc.3bb8: supR3HardenedScreenImage/LdrLoadDll: rc=Unknown Status -23303 (0xffffa4f9) fImage=1 fProtect=0x0 fAccess=0x0 \Device\HarddiskVolume3\Windows\System32\DriverStore\FileRepository\nvmi.inf_amd64_ae396a6228d16bb2\nvd3dumx.dll: fKeyUsage=0x0, missing 0x1: \Device\HarddiskVolume3\Windows\System32\DriverStore\FileRepository\nvmi.inf_amd64_ae396a6228d16bb2\nvd3dumx.dll
31fc.3bb8: supR3HardenedWinVerifyCacheInsert: \Device\HarddiskVolume3\Windows\System32\DriverStore\FileRepository\nvmi.inf_amd64_ae396a6228d16bb2\nvd3dumx.dll
31fc.3bb8: Error (rc=0):
31fc.3bb8: supR3HardenedMonitor_LdrLoadDll: rejecting 'C:\WINDOWS\System32\DriverStore\FileRepository\nvmi.inf_amd64_ae396a6228d16bb2\nvd3dumx.dll' (C:\WINDOWS\System32\DriverStore\FileRepository\nvmi.inf_amd64_ae396a6228d16bb2\nvd3dumx.dll): rcNt=0xc0000190
31fc.3bb8: supR3HardenedMonitor_LdrLoadDll: returns rcNt=0xc0000190 'C:\WINDOWS\System32\DriverStore\FileRepository\nvmi.inf_amd64_ae396a6228d16bb2\nvd3dumx.dll'
31fc.3bb8: supR3HardenedScreenImage/LdrLoadDll: cache hit (Unknown Status -23303 (0xffffa4f9)) on \Device\HarddiskVolume3\Windows\System32\DriverStore\FileRepository\nvmi.inf_amd64_ae396a6228d16bb2\nvd3dumx.dll
31fc.3bb8: Error (rc=0):
31fc.3bb8: supR3HardenedScreenImage/LdrLoadDll: cached rc=Unknown Status -23303 (0xffffa4f9) fImage=1 fProtect=0x0 fAccess=0x0 cHits=1 \Device\HarddiskVolume3\Windows\System32\DriverStore\FileRepository\nvmi.inf_amd64_ae396a6228d16bb2\nvd3dumx.dll
31fc.3bb8: Error (rc=0):
31fc.3bb8: supR3HardenedMonitor_LdrLoadDll: rejecting 'C:\WINDOWS\System32\DriverStore\FileRepository\nvmi.inf_amd64_ae396a6228d16bb2\nvd3dumx.dll' (C:\WINDOWS\System32\DriverStore\FileRepository\nvmi.inf_amd64_ae396a6228d16bb2\nvd3dumx.dll): rcNt=0xc0000190
31fc.3bb8: supR3HardenedMonitor_LdrLoadDll: returns rcNt=0xc0000190 'C:\WINDOWS\System32\DriverStore\FileRepository\nvmi.inf_amd64_ae396a6228d16bb2\nvd3dumx.dll'
31fc.3bb8: supR3HardenedMonitor_LdrLoadDll: returns rcNt=0x0 hMod=00007ffa67510000 'C:\WINDOWS\System32\gdi32.dll'
3198.31f0: supR3HardNtChildWaitFor[2]: Quitting: ExitCode=0x1 (rcNtWait=0x0, rcNt1=0x0, rcNt2=0x103, rcNt3=0x103, 122947 ms, the end);
3280.3284: supR3HardNtChildWaitFor[1]: Quitting: ExitCode=0x1 (rcNtWait=0x0, rcNt1=0x0, rcNt2=0x103, rcNt3=0x103, 123594 ms, the end);

[socratis]

The snippet is showing clearly that the Nvidia drivers are being rejected. Why sometimes they are and sometimes they're not, I have no clue. Especially since mpack has the same setup and he has no problems. Maybe some weird graphics settings, advanced I-don't-know, or weird je-ne-sais-quoi.

The thing is if you can't get it to reproduce reliably, we can't seem to be able to find something consistent (except that they're rejected), I'd disable 3D acceleration and live with it for the moment, unless you want to escalate it to the bugtracker and either search for an appropriate bug, or open a new bug report where you'll attach your logs and other vital information.

Mind you that the 3D acceleration isn't getting too much love these days, mainly because they're overwhelmed. And since this is not reproducible, or traceable, and even worst since it is clearly a hardening issue, I personally wouldn't even bother with the bugtracker.

Finally, one last thought that I have (and I'm not 100% sure about if you are wiling to do it; I wouldn't), is to remove completely the Nvidia graphics driver. Replace your graphics driver with the basic Microsoft Graphics Driver (or something like that), search filesystem and registry for any traces of Nvidia and after that install the latest non-beta Nvidia driver. Maybe something cached is interfering. Long shot, but you never know...

Final final thought. Maybe you have a corrupt certificates database? I'm not on Windows and I haven't searched for something like that, but I vaguely remember that I've seen Microsoft's own tools to fix such issues.

[mpack]

Especially since mpack has the same setup and he has no problems. Maybe some weird graphics settings, advanced I-don't-know, or weird je-ne-sais-quoi.

Almost the same setup. I have the GeForce 970, his is the 980 - I'm assuming the difference will be small. Possibly a driver issue? Mine is v358.87.

[socratis]

tsubery had the 970M. Is the "M" for mobile? Are you tsubery(970M) and Poison(980M) both on laptops? And what about you mpack?

I highly doubt that Nvidia would have signed parts of their drivers, it's gotta be something else going on here. mpack, if you're not on a laptop, do you think that it could be a feature that kicks in only in the mobile models? Like high/low performance mode?

[mpack]

Are you tsubery(970M) and Poison(980M) both on laptops? And what about you mpack?

I wouldn't be caught dead with a laptop. (otoh I do also have two tablets and a phablet when I need mobile computing. I don't run VMs on any of the latter).

My PC in question is an HP Envy Phoenix 860-010ng. I don't know the HP model naming system, but I believe "ng" means intended for German market, as that's what language it spoke when it turned up. The Envy right of the box includes the graphics, an SSD and 2TB hdd second drive, so it's very good for VM. 4-core 16GB. Pricey, but I like the HP brand, and I only buy a new desktop every 5 years or so.