vmwgfx failure with Ubuntu kernel upgrade (6.5.0-14)

[bradford.wetmore]

Hi. During an upgrade I ran into an apparent kernel bug with an unsupported driver that rendered the GUI unusable until I downgraded the kernel. I have:

• VirtualBox 7.0.12, with matching Extensions/Guest Additions installed

• Windows 11 Host

• Linux Ubuntu Guest (details below)

• VMSVGA displays in VirtualBox, no 3D acceleration. Full Screen. 1-2 monitors, 64 MB Video Memory, Scale Factor 100%

Before the upgrade:

% uname -a
Linux congas 6.2.0-39-generic #40~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Nov 16 10:53:04 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

% lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.3 LTS
Release: 22.04
Codename: jammy

Everything was working fine. When I attempted an upgrade today to the offered 6.5.0-14 kernel...

The following NEW packages will be installed:
linux-headers-6.5.0-14-generic linux-hwe-6.5-headers-6.5.0-14
linux-image-6.5.0-14-generic linux-modules-6.5.0-14-generic
linux-modules-extra-6.5.0-14-generic
The following packages have been kept back:
transmission-common transmission-gtk
The following packages will be upgraded:
dnsmasq-base libc-bin libc-dev-bin libc6 libc6:i386 libc6-dbg libc6-dev
linux-generic-hwe-22.04 linux-headers-generic-hwe-22.04
linux-image-generic-hwe-22.04 locales python3-update-manager update-manager
update-manager-core
14 upgraded, 5 newly installed, 0 to remove and 2 not upgraded.
10 standard LTS security updates
Need to get 161 MB of archives.
After this operation, 721 MB of additional disk space will be used.
Do you want to continue? [Y/n] yes

...the display took a holiday. Some boot attempts wouldn't allow me to use the cursor (just a purple or black box) or log in, others had the screen flashing/redrawing itself all over the place.

There was an error during boot, which is likely of interest. (Sorry, hand typing this in. Any typos are mine. )

[ 3.663400] vmwgfx 0000:00:0Z.0 [drm] *ERROR* vmwgfx seems to be running on an unsupported hypervisor.
[ 3.663412] vmwgfx 0000:00:0Z.0 [drm] *ERROR* This configuration is likely broken.
[ 3.663412] vmwgfx 0000:00:0Z.0 [drm] *ERROR* Please switch to a supported graphics device to avoid problems.
...deleted...
[ 3.663412] UBSAN: array-index-out-of-bounds in /tmp/vbox.0/common/log/log.c41
[ 3.663412] index 1 is out of range for type 'uint32-t [1]'
...deleted...
[ 3.663412] UBSAN: array-index-out-of-bounds in /tmp/vbox.0/common/log/log.c:551:41
[ 3.663412] index 32 is out of range for type 'uint32-t [1]'
...deleted...
[ 3.663412] UBSAN: array-index-out-of-bounds in /tmp/vbox.0/common/log/log.c:4161:34
[ 3.663412] index 32 is out of range for type 'uint32-t [1]'

I have a picture, but can't get it under 128kb.

When tried to switch to one of the other display options, the interface complained about invalid settings detected, and the GUI output was much worse.

When I turned on 3D acceleration, the GUI worked normally(?), but still had the boot errors above.

Using `grub`, I was able to downgrade the kernel to 6.2.0-39, and all was right again.

It sounds similar to https://www.reddit.com/r/virtualbox/com ... el_update/ , but there was nothing conclusive in that thread.

The first three lines are apparently from the Linux vmwgfx GPU drivers. https://github.com/torvalds/linux/blob/ ... wgfx_drv.c

[fth0]

I think that the VirtualBox developers heard about the changed behavior in the vmwgfx driver, but I don't know if your issue is already solved in VirtualBox 7.0.14, which is expected in the coming days (around 2024-01-16).

Could you try if installing VirtualBox 7.0.13r161031 (or newer) on the host and the VirtualBox Guest Additions 7.0.13r160984 (or newer) in the guest solves your issue? Both can be found on the VirtualBox test builds page and should be more or less the same as the expected release updates.

[bertje]

We encountered the same issue with latest VirtualBox versions 6.1.48 and 7.0.12 on Windows 10 and Linux hosts and any guest running Ubuntu Wayland kernel version 6.5. I could not find a solution on internet.

VirtualBox is currently a nightmare with kernels 6.4 and 6.5. Support for these kernels is recently added to VirtualBox (see changelog) and contains major bugs: Screen flickering, white screens during Ubuntu 23.10 installation, very slow GUI guest response and your reported issue which makes our production machines useless after installing the latest Ubuntu updates.

Downgrading to kernel 6.3 solved the issue. Additionally, we figured out that downgrading from VirtualBox 7.0.12 to VirtualBox 6.1.48 also increases performance dramatically in Ubuntu guests. Alternatively, VMWare Workstation 16/17 works out of the box.

[appyface]

I just ran into a similar error today, I get these three lines when booting:

[ 4.413521] vmwgfx 0000:00:02.0: [drm] *ERROR* vmwgfx seems to be running on an unsupported hypervisor.
[ 4.413526] vmwgfx 0000:00:02.0: [drm] *ERROR* This configuration is likely broken.
[ 4.413529] vmwgfx 0000:00:02.0: [drm] *ERROR* Please switch to a supported graphics device to avoid problems.

Host is Win7 Pro SP1 x64 + post hotfixes.
Guest is Lubuntu 22.04
VBox 7.0.8, installed GA is 7.0.8

What is odd, I have three VM's originally created with the same Lubuntu 22.04 ISO and same VM settings. I upgraded all three VMs' Lubuntu today via Menu -> Preferences -> Apply Full Upgrade. The other two VMs rebooted fine after upgrade. Only this one has the above three error messages and stuck there. I restored to snapshot taken before the upgrade and it boots up fine again. Tried the upgrade again and same, three errors and won't come up.

I can restore the snapshot back and just use it without the updates, but curious how to find the error with this one VM. As far as I can recall, they are pretty much clones of each other VM config wise and OS wise.

EDIT: Well, I didn't intend to but left this VM running with the error messages on display, and it eventually booted all the way up. I'll have to test how well it functions. The other two VMs do not have these three error messages, they just show the one ubiquitous "can't send to host log" error message, and they boot up right away.

[fth0]

I get these three lines when booting: [...]

FWIW, older versions of the vmwgfx driver instead threw two error messages that could be ignored, since they had no consequences in VirtualBox VMs whatsoever. I don't know if and when the changes have what effect.

The other two VMs rebooted fine after upgrade. Only this one has the above three error messages and stuck there.

Do you have 3D acceleration enabled or disabled in the VMs?

[bertje]

Issue is fully reproducible and related to this reported graphics issue.

Do you have 3D acceleration enabled or disabled in the VMs?

In the guest Display settings, set 128MB graphics memory and never enable 3D graphics acceleration as this is not supported or functional for many years (and I don't see this on the near feature supported like VMWare).

[appyface]

@fth0 and @bertje

Thank you for replies. I can confirm all three VMs have 128MB graphics memory and 3D is not selected.

That link to the graphics issue report got me thinking. From within the erroring VM, I can see the 7.0.8 GA are installed. However, I compared the erroring VMs VBox log file, with the log from one of the other VMs. In the "good" VMs log, I have this line:

00:00:22.391614 VMMDev: Guest Additions information report: Version 7.0.8 r156879 '7.0.8'

But in the erroring VMs log I have this line:

00:00:27.413855 VMMDev: Guest Additions information report: Version 6.0.0 r127566 '6.0.0'

I am not proficient with Linux but it appears this means a built-in/default GA is running instead of the newer GA that I installed. I followed same commands to install on this VM that I used on the others - that procedure does not include removing any built-in GA.

I will look for instructions on how to sort out the GA on this VM, seems likely that will solve the issue.

Thanks again for help.

[bertje]

Good point. It is recommended to re-install the VirtualBox guest additions after changing VirtualBox host installation and after installing Ubuntu updates in the guest as this changes kernel versions now to 6.5. Make sure that gcc-12 is now required before installing guest additions.

[appyface]

I restored the erroring VM again to snapshot before the OS update, and 7.0.8 GA is listed in the log. Since this VM does eventually boot up I should be able to resolve this.

If, however, the erroring VM wouldn't ever boot up (as I initially thought was the case), how would I solve this? I guess I'd have to pre-install some upgraded dependencies then reinstall the GA, prior to running the full update? (sorry, as I mentioned, I'm not proficient with Linux)

[bertje]

Are you able to access the Grub menu (press ESC or SHIFT key quickly at boottime) to select a previously installed kernel via the advanced menu like this? Then you can also try booting the recovery mode and start the root shell.

[appyface]

No, ESC turns on a long boot log (but doesn't stop it) and SHIFT doesn't do anything AFAICT. I started pressing as soon as the VM window launched and kept repeating until it got through the boot sequence. Do I need to press F12 first?

So, trying to get 7.0.8 installed on the updated erroring VM (since at least it eventually boots up) but no luck so far.

Ran this command run on updated erroring VM: sudo apt-get install build-essential gcc make perl dkms
I get a message already up to date, plus version numbers. The same command run on an updated "good" VM brings up the same:

Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
build-essential is already the newest version (12.9ubuntu3).
gcc is already the newest version (4:11.2.0-1ubuntu1).
make is already the newest version (4.3-4.1build1).
dkms is already the newest version (2.8.7-2ubuntu2.2).
perl is already the newest version (5.34.0-3ubuntu1.3).
0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.

So I uninstalled the 7.0.8 GA and now my GA functions like shared clipboard and shared drives are no longer working... That seems odd? Since 6.0.0 is reported in the logs as the current GA it should not have mattered?

Anyway, I next tried rebuilding linux-headers-generic: sudo apt-get intall linux-headers-generic
And there were some updates:
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
linux-headers-5.15.0-91 linux-headers-5.15.0-91-generic
The following NEW packages will be installed:
linux-headers-5.15.0-91 linux-headers-5.15.0-91-generic linux-headers-generic
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 15.2 MB of archives.
After this operation, 103 MB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://us.archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-91 all 5.15.0-91.101 [12.3 MB]
Get:2 http://us.archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-91-generic amd64 5.15.0-91.101 [2,860 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.91.88 [2,342 B]
Fetched 15.2 MB in 2s (8,436 kB/s)
Selecting previously unselected package linux-headers-5.15.0-91.
(Reading database ... 297428 files and directories currently installed.)
Preparing to unpack .../linux-headers-5.15.0-91_5.15.0-91.101_all.deb ...
Unpacking linux-headers-5.15.0-91 (5.15.0-91.101) ...
Selecting previously unselected package linux-headers-5.15.0-91-generic.
Preparing to unpack .../linux-headers-5.15.0-91-generic_5.15.0-91.101_amd64.deb ...
Unpacking linux-headers-5.15.0-91-generic (5.15.0-91.101) ...
Selecting previously unselected package linux-headers-generic.
Preparing to unpack .../linux-headers-generic_5.15.0.91.88_amd64.deb ...
Unpacking linux-headers-generic (5.15.0.91.88) ...
Setting up linux-headers-5.15.0-91 (5.15.0-91.101) ...
Setting up linux-headers-5.15.0-91-generic (5.15.0-91.101) ...
/etc/kernel/header_postinst.d/dkms:
* dkms: running auto installation service for kernel 5.15.0-91-generic
...done.
Setting up linux-headers-generic (5.15.0.91.88) ...


I shutdown then restarted the VM. Tried installing 7.0.8 GA again, and got error message this time:

currently loaded module vboxguest version () does not match to VirtualBox Guest Additions installation version (7.0.8 r156879)
The log file /var/log/vboxadd-setup.log may contain further information.

I don't know how to get the log off the VM without guest additions to give me a share on the host so I an post it here? But the error message listed right off the bat in the log is:

Error building the module (...) the complier differs from the one used to build the kernel
The kernel was built by: x86_64-linux-gnu-gcc-12 (Ubuntu 12.3.0-lubuntu1~22.04) 12.3.0
You are using: gcc-12 (Ubuntu 12.3.0-lubuntu1~22.04) 12.3.0

I'm lost now, I don't know enough about linux to troubleshoot this further. TIA for any ideas.

[bertje]

So I uninstalled the 7.0.8 GA and now my GA functions like shared clipboard and shared drives are no longer working... That seems odd?

Clipboard, shared folders and larger screens only works after (re)-installing GA (Guest Additions) from the ISO via guest menu "Devices | Insert Guest Additions CD image". Then VirtualBox host version is identical to the Guest Additions. Unexpected behavior occurs when VirtualBox on the host is for example v6.1.48 and installed guest additions is for example v6.0.0. VirtualBox GA contains major issues since kernel release 6.4 and this is not the first time where VirtualBox GA clipboard/shared folder/graphics/audio was broken for a long time, so I'm not surprised.

After installing guest updates, all our trouble started and the boottime increased over 5 minutes (related to outdated Guest Additions):

Code: Select all

A start job is running for vboxadd.service (1min 15s / 5min 3s)

Did you see the same?

The F12 key enters the BIOS or UEFI settings of the guest. It is better to start a root shell via the Grub menu to reinstall the guest additions. But it looks like you're able to boot the Ubuntu guest to open a shell?

The log file /var/log/vboxadd-setup.log may contain further information.

Can you post the contents of this file, for example via https://pastebin.com/?

Code: Select all

$ cat /var/log/vboxadd-setup.log

Guessing, but maybe gcc-12 is not installed which is a new recent requirement:

Code: Select all

# Minimum packages to build/install Guest Additions
$ sudo apt install build-essential gcc-12
$ cd /media/$USER/VBox_GAs_6.1.48
$ sudo ./VBoxLinuxAdditions.run

# check if there is any error reported

$ sudo reboot now

Does this work for you?

Note: There is also a hidden uninstall option available via ./VBoxLinuxAdditions.run uninstall (not visible with --help) you can try to remove previous versions first.

[appyface]

Yes that is the command I used to uninstall the GA I had installed via the ISO mount originally.

As I mentioned, I don't know how to get the log file off the guest?

The error message shows gcc-12 is what I have - if the name is verbatim then the gcc-12 that built the kernel has additional characters in the name. But I don't know, since I've never looked at anything like this before.

[bertje]

As I mentioned, I don't know how to get the log file off the guest?

I'm not sure how you booted the guest. Can you open a webbrowser in the guest? Then you can use pastebin.com for example or login to this forum. Alternatively, you can boot the guest from a Ubuntu 22.04 ISO to open a webbrowser and open the file from the mounted disk.

[appyface]

I booted the guest like I do any other - just clicked start in Virtualbox on the guest and it fires up. There is just a long pause on the erroring VM with those three messages on display, then eventually the desktop comes up.

I was able to launch Firefox and access pastebin.com, here's the full log: http://pastebin.com/RVL3tdhE