3D support of the host is not working properly

[duzenko]

I haven't used my VirtualBox VM's for a while and now when I try to start one I'm getting these errors.
VBox.log attached.
What can be the problem here?

Failed to open a session for the virtual machine xp.

This VM was configured to use 3D acceleration. However, the 3D support of the host is not working properly and the VM cannot be started. To fix this problem, either fix the host 3D support (update the host graphics driver?) or disable 3D acceleration in the VM settings (VERR_NOT_AVAILABLE).

Result Code: E_FAIL (0x80004005)
Component: ConsoleWrap
Interface: IConsole {872da645-4a9b-1727-bee2-5585105b9eed}

[andyp73]

There is a big hint in the error message. You have two options:

either fix the host 3D support (update the host graphics driver?)

or

disable 3D acceleration in the VM settings

Only you know whether running without 3D acceleration is going to cause you a problem and therefore which of the proposed solutions is the one to focus on.

-Andy.

[duzenko]

Thank you very much for your advice.
My host supports 3D just fine.
My question is what exactly VirtualBox complains about.

[andyp73]

My question is what exactly VirtualBox complains about.

Looking at the source code for VirtualBox it checks whether 3D support is available by spawning the VBoxTestOGL.exe file. My guess is the VERR_NOT_AVAILABLE return code means that the executable can't be found as a result of an improper or corrupt VirtualBox installation.

As you are on a Windows host, open up a command prompt and run the following:

Code: Select all

C:\> "C:\Program Files\Oracle\VirtualBox\VBoxTestOGL.exe" --log test3d.out --test 3D

Post the contents of the log file here if it isn't very long otherwise zip it and attach it to the ticket.

-Andy.

[duzenko]

Here it is

[socratis]

VERR_NOT_AVAILABLE return code means that the executable can't be found as a result of an improper or corrupt VirtualBox installation.

Actually it seems that it's crashing!

@duzenko
Can you update your drivers? Can you post the VBoxHardening.log (ZIPPED) from a failed session? Quit VirtualBox, wait for 1', start VirtualBox, start your VM, get a crash, fetch the VBoxHardening.log. ZIPPED.

[duzenko]

Attached.
My driver is 12/2017 and I'd rather not mess with it because it's perfectly stable in every other program using 3D, OpenGL included.

[andyp73]

Looking at the assertion in the log file from manually running VBoxTestOGL.exe then it appears that we are executing some code that was needed for dealing with issues in ATI drivers. Specifically, the comments say:

Code: Select all

    /*
     * ATI WDDM-based graphics have an issue with rendering fullscreen.
     * See public tickets #9775 & #9267 .
     * Namely ATI drivers check whether ogl window is foreground and fullscreen
     * and if so - do D3DKMTSetDisplayMode for ogl surface,
     * which prevented any other data from being displayed, no matter what.
     *
     * Here we check whether we're using an ATI card and if so, patch the ogl ICD driver's IAT
     * to replace GetForegroundWindow reference with our renderspuAtiQuirk_GetForegroundWindow,
     * which always returns NULL.
     */

Without talking to the devs (who probably wouldn't talk to me anyway - not knowing me from Adam) I don't know whether this is still needed for all ATI graphics drivers on Windows hosts. I still think that your fastest option is going to be to disable 3D acceleration for this guest.

The only adversary that is listed in the hardening log file is Malwarebytes Web Protection. I don't know what it does but can't immediately see how it could be causing the problem with the OpenGL test crashing.

-Andy.

[andyp73]

Can you open the registry editor (regedit.exe) on the host and navigate down the tree to:

Code: Select all

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class\{4D36E968-E325-11CE-BFC1-08002BE10318}

When you get there, right click on "0000" and then select "Export". In the dialog that opens change the "Save as type" option to "Win9x/NT4 Registration Files" and post the .reg file here?

-Andy.

[duzenko]

Here it is.

Not sure if it does fail to patch the driver why abort the whole 3d support thing. At least warn user and allow to proceed on his own risk.

[andyp73]

I have had a quick look at the entries from your registry file and they don't appear to be correct!

Code: Select all

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class\{4d36e968-e325-11ce-bfc1-08002be10318}\0000]

This is the point in the registry that the VBoxTestOGL.exe application reads and specifically the "OpenGLVendorName" and "OpenGLDriverName" key values. Your registry has no sensible entries in it here.

The registry is moderately clever, in that the previous settings get moved to "0001" probably as a result of the last host graphics driver update but even in there the two values don't appear to be right.

Admittedly, VBoxTestOGL.exe shouldn't crash, and there is a fair amount of error checking in there already. Re-installing either the same or the latest version of the ATI drivers might fix the registry entries (or not).
When I have some time available I'll butcher the registry on a test system and see if I can find a way to get it to report an error and die nicely.

-Andy.

[duzenko]

Why on Earth would anyone be looking in the registry for vendor name?!
It's

Code: Select all

glGetString(GL_VENDOR);

Anyway, if it can't find the vendor in the registry it should just continue or warn user, not abort the whole thing.