Gnome, apps buggy with 3D enabled (Debian)

[garbage_collected]

Some apps are misbehaving under Gnome shell when I enable 3D acceleration and I'm wondering if someone wouldn't mind checking my VBox and glxinfo logs, please. I don't know if the problem is with my configuration, with VirtualBox, or with Gnome/Debian itself.

Gnome works fine, but VLC and SimpleScreenRecorder have issues. VLC causes Gnome to freeze when I click on the 'X' to close the window while a video is playing. A still image of VLC remains on the screen as an artifact, even though the process has terminated. Nothing else on the desktop is responsive at that point except for the "Activities" button in the upper left corner of Gnome. Clicking on "Activities" clears the VLC artifact and restores responsiveness to the desktop.

SimpleScreenRecorder (compiled from git) has a more serious issue: everything is black. The recording preview window shows a black screen, and the output video is all black.

With 3D acceleration disabled both of these apps work correctly with Mesa/Gallium.
I have an Arch guest with 3D enabled which does not have these issues, however it's running XFCE instead of Gnome. Therefore I know that Chromium/Humper works in at least some cases.

There's no debug output from the apps themselves that indicate any errors. I've also tried limiting the virtual CPUs to one and the RAM to 1GiB, as suggested in another thread, but did not help.

Configuration:
VBox 5.0.12 r104815
Guest Additions compiled from CD image in Debian
Windows 7 x64 host, SP1
Radeon HD 7950 graphics

Sorry about the .zip archive but there's a 128kB file size limit. I see some messages about missing NV OpenGL extensions in the mesa table. Not sure if related.

EDIT: forgot to mention, the Totem/Videos app in Gnome has the same issue as VLC. It only exits properly when 3D acceleration is disabled.

Many thanks!

[Perryg]

Code: Select all

00:00:01.386399 supR3HardenedErrorV: supR3HardenedScreenImage/NtCreateSection: rc=VERR_BAD_EXE_FORMAT fImage=1 fProtect=0x7 fAccess=0x2 \Device\HarddiskVolume2\Windows\System32\drivers\atikmdag.sys: 
00:00:01.386637 supR3HardenedErrorV: supR3HardenedScreenImage/NtCreateSection: cached rc=VERR_BAD_EXE_FORMAT fImage=1 fProtect=0x2 fAccess=0x7 cHits=1 \Device\HarddiskVolume2\Windows\System32\drivers\atikmdag.sys

Seems to be an issue with the hosts GPU. I would try to update the drivers.

Code: Select all

00:00:01.427263   NumCPUs           <integer> = 0x0000000000000004 (4)
00:00:01.799316 CPUM: Physical host cores: 4

You have maxed out your processor count. I would set it to two.

[garbage_collected]

Thanks, Perry. I set the CPUs to 2 and I purged and reinstalled the latest AMD drivers for my Radeon (16.1 'Crimson'), but the issue persists.

I had never noticed before but the error related to 'atikmdag.sys' is there in my VBox log for the Arch linux guest as well. Strangely, 3D acceleration for VLC and SimpleScreenRecorder works fine in that VM.

I didn't find anything in a bug search, but there is one thread I came across on the AMD forum where someone suggested downgrading to VirtualBox 4.3.28.
https://community.amd.com/thread/184189

Is it possible to run an older version of VBox in parallel with 5.x, so that I can try it?

[Perryg]

Is it possible to run an older version of VBox in parallel with 5.x, so that I can try it?

No.

The host driver is your issue for what ever reason. Downgrading may be your only hope until the driver gets fixed.