Kernel 6.3.5 page_fault panic when starting VMs

Discussions related to using VirtualBox on Linux hosts.
Post Reply
ariapaige
Posts: 3
Joined: 5. Jun 2023, 10:09
Primary OS: Fedora other
VBox Version: OSE Fedora
Guest OSses: Windows and Linux

Kernel 6.3.5 page_fault panic when starting VMs

Post by ariapaige »

Recently updated to the 6.3.5 kernel on my Fedora 38 box, and despite building the new kernel modules, I can no longer get any of my VMs to start. They hang with the message "Starting Virtual Machine..."

Looking at the logs
VBox.log
Windows 10 vm log
(1.69 KiB) Downloaded 20 times
for the VM itself, the logs just end very early with the line:

00:00:00.786443 GUI: UIDesktopWidgetWatchdog::sltHandleHostScreenAvailableGeometryCalculated: Screen 0 work area is actually resized to: 0x32 x 1920x1168

This kinda makes me think gfx drivers? Maybe? but not sure why or how to TS based on what I'm seeing tbh. Checked dmesg
dmesg.txt
dmesg vbox
(6.52 KiB) Downloaded 14 times
right after starting the machine and see the following in the trace:

[ 891.564156] ? VBoxDrvLinuxIOCtl_7_0_8+0x169/0x260 [vboxdrv]
[ 891.564177] ? __x64_sys_ioctl+0x91/0xd0
[ 891.564181] ? do_syscall_64+0x5d/0x90
[ 891.564182] ? syscall_exit_to_user_mode+0x1b/0x40
[ 891.564185] ? do_syscall_64+0x6c/0x90
[ 891.564187] ? ksys_write+0xd8/0xf0
[ 891.564189] ? syscall_exit_to_user_mode+0x1b/0x40
[ 891.564191] ? do_syscall_64+0x6c/0x90
[ 891.564193] ? exc_page_fault+0x7c/0x180
[ 891.564195] ? entry_SYSCALL_64_after_hwframe+0x72/0xdc
[ 891.564199] </TASK>
[ 891.564199] Modules linked in: vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE)

I'm attaching the full vm log and dmesg log to the post here. Wanted to get everyone's thoughts, I'm a little perplexed.

Steps I've tried:

- Rebuilding the modules
- Running the sbin config script
- booting into old kernel to test, which, is of course borked now too, bleh.
- purge vbox, then fully rebuild and install, rebuilt kernel modules again, still no luck
birdie
Posts: 431
Joined: 2. May 2010, 14:19
Primary OS: Fedora other
VBox Version: VirtualBox+Oracle ExtPack
Guest OSses: Windows, Linux, other Unixes
Location: Artem S. Tashkinov
Contact:

Re: Kernel 6.3.5 page_fault panic when starting VMs

Post by birdie »

That's weird.

I'm also on Fedora 38, only running a custom kernel and VBox 7.0.8 works just fine here.
scottgus1
Site Moderator
Posts: 20945
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: VirtualBox+Oracle ExtPack
Guest OSses: Windows, Linux

Re: Kernel 6.3.5 page_fault panic when starting VMs

Post by scottgus1 »

I've read on the forum that new kernels may require new Virtualbox versions. Maybe the test builds or development snapshots might help?
marmel
Posts: 75
Joined: 1. Nov 2013, 03:44

Re: Kernel 6.3.5 page_fault panic when starting VMs

Post by marmel »

ariapaige wrote: 5. Jun 2023, 10:28 Recently updated to the 6.3.5 kernel on my Fedora 38 box, and despite building the new kernel modules, I can no longer get any of my VMs to start. They hang with the message "Starting Virtual Machine..."

Looking at the logs VBox.log for the VM itself, the logs just end very early with the line:

00:00:00.786443 GUI: UIDesktopWidgetWatchdog::sltHandleHostScreenAvailableGeometryCalculated: Screen 0 work area is actually resized to: 0x32 x 1920x1168
Hi @ariapaige,

Just to cover some basic troubleshooting... are you ensuring IBT is off via kernel parameter?

Your log looks similar to one with IBT enabled in the kernel, which VBox does not yet build modules with support for.

See Forum Thread:
viewtopic.php?f=7&t=108948

Bug Tracker:
https://www.virtualbox.org/ticket/21435


EDIT: Here's what my kernel log looks like when VBox is launched with IBT enabled:

Code: Select all

Jun 05 09:46:11 kernel: SUPR0GipMap: fGetGipCpu=0x1b
Jun 05 09:46:11 kernel: traps: Missing ENDBR: 0xffffa1fb4749a430
Jun 05 09:46:11 kernel: ------------[ cut here ]------------
Jun 05 09:46:11 kernel: kernel BUG at arch/x86/kernel/traps.c:255!
Jun 05 09:46:11 kernel: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
Jun 05 09:46:11 kernel: CPU: 17 PID: 1919 Comm: EMT-0 Tainted: P           OE      6.3.5-arch1-1 #1 649d963afc0261175aabf0511660febbb7b06177
Jun 05 09:46:11 kernel: Hardware name: Micro-Star International Co., Ltd. MS-7D31/MPG Z690 EDGE WIFI DDR4 (MS-7D31), BIOS 1.30 03/22/2022
Jun 05 09:46:11 kernel: RIP: 0010:exc_control_protection+0xb8/0xc0
Jun 05 09:46:11 kernel: Code: 48 8b 93 80 00 00 00 be fc 00 00 00 48 c7 c7 2b db 7a a7 e8 0a 62 36 ff e9 7b ff ff ff 48 c7 43 50 00 00 00 00 e9 6e ff ff ff <0f> 0b 66 0f 1f 44 00 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90
Jun 05 09:46:11 kernel: RSP: 0018:ffffa1fb4722fd28 EFLAGS: 00010002
Jun 05 09:46:11 kernel: RAX: 0000000000000028 RBX: ffffa1fb4722fd48 RCX: 0000000000000000
Jun 05 09:46:11 kernel: RDX: 0000000000000000 RSI: ffff9581f0461680 RDI: ffff9581f0461680
Jun 05 09:46:11 kernel: RBP: 0000000000000003 R08: 0000000000000000 R09: ffffa1fb4722fbd0
Jun 05 09:46:11 kernel: R10: 0000000000000003 R11: ffff9582307a78a8 R12: 0000000000000000
Jun 05 09:46:11 kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
Jun 05 09:46:11 kernel: FS:  00007fe96020c6c0(0000) GS:ffff9581f0440000(0000) knlGS:0000000000000000
Jun 05 09:46:11 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 05 09:46:11 kernel: CR2: 00007fe93053c000 CR3: 00000001b2136000 CR4: 0000000000f50ee0
Jun 05 09:46:11 kernel: PKRU: 55555554
Jun 05 09:46:11 kernel: Call Trace:
Jun 05 09:46:11 kernel:  <TASK>
Jun 05 09:46:11 kernel:  ? die+0x36/0x90
Jun 05 09:46:11 kernel:  ? do_trap+0xda/0x100
Jun 05 09:46:11 kernel:  ? exc_control_protection+0xb8/0xc0
Jun 05 09:46:11 kernel:  ? do_error_trap+0x6a/0x90
Jun 05 09:46:11 kernel:  ? exc_control_protection+0xb8/0xc0
Jun 05 09:46:11 kernel:  ? exc_invalid_op+0x50/0x70
Jun 05 09:46:11 kernel:  ? exc_control_protection+0xb8/0xc0
Jun 05 09:46:11 kernel:  ? asm_exc_invalid_op+0x1a/0x20
Jun 05 09:46:11 kernel:  ? exc_control_protection+0xb8/0xc0
Jun 05 09:46:11 kernel:  ? exc_control_protection+0x6e/0xc0
Jun 05 09:46:11 kernel:  asm_exc_control_protection+0x26/0x30
Jun 05 09:46:11 kernel: RIP: 0010:0xffffa1fb4749a430
Jun 05 09:46:11 kernel: Code: 89 da 5b 41 5c 41 5d 5d e9 5d 05 fb ff 48 83 c4 28 b8 8e f8 ff ff 5b 41 5c 41 5d 5d c3 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 <55> 48 8d 35 c8 7b 1d 00 48 89 e5 41 54 49 89 fc 53 e8 aa 11 fb ff
Jun 05 09:46:11 kernel: RSP: 0018:ffffa1fb4722fdf8 EFLAGS: 00010286
Jun 05 09:46:11 kernel: RAX: ffffa1fb4749a430 RBX: ffffa1fb47689010 RCX: ffff957332136a20
Jun 05 09:46:11 kernel: RDX: 0000000000000000 RSI: ffff957280000f70 RDI: ffff95728cde8b10
Jun 05 09:46:11 kernel: RBP: ffffa1fb4722fe48 R08: 0000000000000024 R09: ffffa1fb47688000
Jun 05 09:46:11 kernel: R10: ffff9582307fbd80 R11: 0000000000000000 R12: 0000000000000000
Jun 05 09:46:11 kernel: R13: ffffffffc48e33a0 R14: ffff95728cde8b10 R15: 0000000000000004
Jun 05 09:46:11 kernel:  ? supdrvIOCtl+0x2d20/0x31b0 [vboxdrv 0525ec1c9382960c8c86a70e049c1555c2493315]
Jun 05 09:46:11 kernel:  ? _copy_from_user+0x4a/0x60
Jun 05 09:46:11 kernel:  ? VBoxDrvLinuxIOCtl_7_0_8+0x169/0x260 [vboxdrv 0525ec1c9382960c8c86a70e049c1555c2493315]
Jun 05 09:46:11 kernel:  ? __x64_sys_ioctl+0x91/0xd0
Jun 05 09:46:11 kernel:  ? do_syscall_64+0x5d/0x90
Jun 05 09:46:11 kernel:  ? syscall_exit_to_user_mode+0x1b/0x40
Jun 05 09:46:11 kernel:  ? exc_page_fault+0x7c/0x180
Jun 05 09:46:11 kernel:  ? entry_SYSCALL_64_after_hwframe+0x72/0xdc
Jun 05 09:46:11 kernel:  </TASK>
Jun 05 09:46:11 kernel: Modules linked in: rfkill xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c iptable_filter dm_crypt cbc encrypted_keys trusted asn1_encoder tee hid_logitech_hidpp mousedev hid_logitech_dj snd_usb_audio snd_usbmidi_lib snd_rawmidi snd_seq_device joydev mc snd_sof_pci_intel_tgl snd_sof_intel_hda_common soundwire_intel soundwire_generic_allocation soundwire_cadence snd_sof_intel_hda snd_sof_pci snd_sof_xtensa_dsp snd_sof snd_sof_utils snd_soc_hdac_hda intel_rapl_msr snd_hda_ext_core intel_rapl_common snd_soc_acpi_intel_match snd_soc_acpi x86_pkg_temp_thermal soundwire_bus intel_powerclamp coretemp snd_soc_core kvm_intel snd_compress ac97_bus snd_pcm_dmaengine kvm nls_iso8859_1 snd_intel_dspcfg snd_intel_sdw_acpi vfat fat snd_hda_codec irqbypass rapl snd_hda_core intel_cstate iTCO_wdt snd_hwdep spi_nor intel_pmc_bxt snd_pcm pmt_telemetry mei_hdcp mei_pxp ee1004 iTCO_vendor_support snd_timer pmt_class intel_uncore mtd mxm_wmi wmi_bmof pcspkr mei_me igc i2c_i801 snd i2c_smbus mei soundcore
Jun 05 09:46:11 kernel:  intel_vsec serial_multi_instantiate acpi_tad acpi_pad mac_hid vboxnetflt(OE) vboxnetadp(OE) vboxdrv(OE) sg crypto_user fuse loop dm_mod ip_tables x_tables usbhid nvidia_uvm(POE) nvidia_drm(POE) nvidia_modeset(POE) raid10 ext4 crc32c_generic crc16 mbcache md_mod jbd2 nvidia(POE) crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyval_generic gf128mul ghash_clmulni_intel sha512_ssse3 xhci_pci aesni_intel nvme crypto_simd spi_intel_pci nvme_core cryptd spi_intel xhci_pci_renesas nvme_common video wmi
Jun 05 09:46:11 kernel: ---[ end trace 0000000000000000 ]---
Jun 05 09:46:11 kernel: RIP: 0010:exc_control_protection+0xb8/0xc0
Jun 05 09:46:11 kernel: Code: 48 8b 93 80 00 00 00 be fc 00 00 00 48 c7 c7 2b db 7a a7 e8 0a 62 36 ff e9 7b ff ff ff 48 c7 43 50 00 00 00 00 e9 6e ff ff ff <0f> 0b 66 0f 1f 44 00 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90
Jun 05 09:46:11 kernel: RSP: 0018:ffffa1fb4722fd28 EFLAGS: 00010002
Jun 05 09:46:11 kernel: RAX: 0000000000000028 RBX: ffffa1fb4722fd48 RCX: 0000000000000000
Jun 05 09:46:11 kernel: RDX: 0000000000000000 RSI: ffff9581f0461680 RDI: ffff9581f0461680
Jun 05 09:46:11 kernel: RBP: 0000000000000003 R08: 0000000000000000 R09: ffffa1fb4722fbd0
Jun 05 09:46:11 kernel: R10: 0000000000000003 R11: ffff9582307a78a8 R12: 0000000000000000
Jun 05 09:46:11 kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
Jun 05 09:46:11 kernel: FS:  00007fe96020c6c0(0000) GS:ffff9581f0440000(0000) knlGS:0000000000000000
Jun 05 09:46:11 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 05 09:46:11 kernel: CR2: 00007fe93053c000 CR3: 00000001b2136000 CR4: 0000000000f50ee0
Jun 05 09:46:11 kernel: PKRU: 55555554
Jun 05 09:46:11 kernel: note: EMT-0[1919] exited with irqs disabled
And this single line is the telltale of any kind of module compiled without IBT support:

Code: Select all

kernel: traps: Missing ENDBR: 0xffffa1fb4749a430
ariapaige
Posts: 3
Joined: 5. Jun 2023, 10:09
Primary OS: Fedora other
VBox Version: OSE Fedora
Guest OSses: Windows and Linux

Re: Kernel 6.3.5 page_fault panic when starting VMs

Post by ariapaige »

SOLVED:

So it's totally IBT. I added the line GRUB_CMDLINE_LINUX_DEFAULT="ibt=off" to '/etc/default/grub' to disable IBT, then did a mkconfig with:

sudo grub2-mkconfig -o /boot/grub2/grub.cfg

After rebooting I ran ye 'ol "/sbin/vboxconfig" to rebuild the kernel modules and BAM! working ^_^

So yes, in conclusion, IBT seems to make VB very very sad haha

Thanks @marmel
Attachments
Screenshot from 2023-06-05 09-41-50.png
Screenshot from 2023-06-05 09-41-50.png (92.18 KiB) Viewed 1603 times
fth0
Volunteer
Posts: 5690
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: VirtualBox+Oracle ExtPack
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: Kernel 6.3.5 page_fault panic when starting VMs

Post by fth0 »

FWIW, I've checked the Fedora Linux kernel change log, and Independent Branch Tracking (IBT) was enabled in Fedora Linux kernel 6.3.0 rc6.
ariapaige
Posts: 3
Joined: 5. Jun 2023, 10:09
Primary OS: Fedora other
VBox Version: OSE Fedora
Guest OSses: Windows and Linux

Re: Kernel 6.3.5 page_fault panic when starting VMs

Post by ariapaige »

Good to know. It's always something I swear :lol:
Post Reply