[Solved] All of a sudden, all my Arch-based VM's fail to launch (boot)
-
- Posts: 21
- Joined: 28. Dec 2018, 13:39
- Primary OS: Linux other
- VBox Version: OSE Debian
- Guest OSses: Win 7, Win 11, Arch Linux, Debian, Slackware, Arco
- Location: Tarragona. Spain
[Solved] All of a sudden, all my Arch-based VM's fail to launch (boot)
This is maddening. A few days ago after updating my various VM's (some based on Arch, some on Debian), upon opening VB later to launch a given VM, it gets to GRUB, boots, and then stops about 1-2 seconds into the process and freezes. At that point I am obligated to go to File > Power > Force to Close the close out the failed VM.
I'm at a loss to understand here. This is not happening with my Debian or Debian-based VM's, only Arch based. On my desktop computer I have several different Linux distros installed to various dedicated SSD's. Two of these distros have VB installed: Fedora 36 (cinnamon) and Linux Mint 20.3 (cinnamon). At first I noticed this failure of Arch-based VM's to open on my Fedora system. I chalked it up to a Fedora issue, then uninstalled VB 6.1.34 and installed 6.1.36 thinking that would solve it, but no joy. A day or so later I noticed the same thing happening on my VB installation on Linux Mint. So I uninstalled VB 6.1.34 and again, installed version 6.1.36 but again no joy.
There seems to be one or two things going on here: A problem between VB and the Linux kernel on the HOST machine, but also a problem with the Linux kernel on the GUEST machine as it relates the host. Tbh I simply don't know what to do here.
I'm attaching two log files to this post. One is the zipped log of the failed Arch VM, and the other is the zipped log of a successful boot of a Debian VM. Can someone look at these and maybe clue me in as to what may be happening? As a side note, the fellows over at Arch forums gave me the usual snotty RTFM response, which of course was no help at all because I have read the Arch wiki and scoured the Internet in an attempt to solve this problem.
Computer specs: Home built, Asus VII Crosshair x470, AMD Ryzen 7 3700x, AMD Radeon 580 GPU, 16MB DDR4
Last edited by Jack21 on 5. Aug 2022, 11:44, edited 1 time in total.
-
- Volunteer
- Posts: 5677
- Joined: 14. Feb 2019, 03:06
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Linux, Windows 10, ...
- Location: Germany
Re: All of a sudden, all my Arch-based VM's fail to launch (boot)
The biggest difference that I noticed is the 3D Acceleration setting. Please try the other combinations (Debian without and Arch Linux with 3D acceleration).
-
- Posts: 21
- Joined: 28. Dec 2018, 13:39
- Primary OS: Linux other
- VBox Version: OSE Debian
- Guest OSses: Win 7, Win 11, Arch Linux, Debian, Slackware, Arco
- Location: Tarragona. Spain
Re: All of a sudden, all my Arch-based VM's fail to launch (boot)
Yeah that's not it. Disabling 3D Accel on the Debian VM just produces Cinnamon DE complaining about "software rendering mode" (expected behavior). The Arch (this is plain vanilla btw, a basic installation with NO DE) VM still won't boot.fth0 wrote:The biggest difference that I noticed is the 3D Acceleration setting. Please try the other combinations (Debian without and Arch Linux with 3D acceleration).
-
- Site Moderator
- Posts: 20945
- Joined: 30. Dec 2009, 20:14
- Primary OS: MS Windows 10
- VBox Version: PUEL
- Guest OSses: Windows, Linux
Re: All of a sudden, all my Arch-based VM's fail to launch (boot)
What happens in the VM stays in the VM. OS updates inside the VM can hose the OS or move the OS out of compliance with Virtualbox, but the OS's help features and channels must be used to fix the OS, same as if the problem happened on a real PC.Jack21 wrote:after updating my various VM's (some based on Arch, some on Debian), upon opening VB later to launch a given VM, it gets to GRUB, boots, and then stops about 1-2 seconds into the process and freezes.
The "failing" log does not look to me to show a failure. The VM 'hardware' seems to boot fine.
Please try making a new Arch VM, using the Arch ISO you originally used. If it boots, then the updates inside the existing VM are giving trouble.
-
- Posts: 21
- Joined: 28. Dec 2018, 13:39
- Primary OS: Linux other
- VBox Version: OSE Debian
- Guest OSses: Win 7, Win 11, Arch Linux, Debian, Slackware, Arco
- Location: Tarragona. Spain
Re: All of a sudden, all my Arch-based VM's fail to launch (boot)
Exactly. Something in the last kernel update has changed, or something in a recent Arch package update has changed. Yesterday I took the latest Arch iso, and installed entirely by hand a completely vanilla installation sans DE. During the install I added both the linux and linux-lts kernels. After all was said and done with installation finished I shut down the VM with the "shutdown now" command at the Arch root prompt. Upon entering VB again and choosing to start my just built Arch VM, it once again halts after passing the Grub boot loader (same behavior choosing either kernel at Grub).scottgus1 wrote:What happens in the VM stays in the VM. OS updates inside the VM can hose the OS or move the OS out of compliance with Virtualbox, but the OS's help features and channels must be used to fix the OS, same as if the problem happened on a real PC.Jack21 wrote:after updating my various VM's (some based on Arch, some on Debian), upon opening VB later to launch a given VM, it gets to GRUB, boots, and then stops about 1-2 seconds into the process and freezes.
The "failing" log does not look to me to show a failure. The VM 'hardware' seems to boot fine.
Please try making a new Arch VM, using the Arch ISO you originally used. If it boots, then the updates inside the existing VM are giving trouble.
For "fun" I installed a VB daily build on my Fedora 36 install. Made no difference, same problem. I'm simply at a loss for what to do other than file a bug report with Virtualbox, or should I file one with the Arch devs? Don't know which way to proceed from here.
-
- Volunteer
- Posts: 5677
- Joined: 14. Feb 2019, 03:06
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Linux, Windows 10, ...
- Location: Germany
Re: All of a sudden, all my Arch-based VM's fail to launch (boot)
FWIW, Debian read ~600 MB from the virtual hard disk, whereas Arch Linux read ~20 MB from the virtual hard disk.
I may have found something:
I may have found something:
Although VirtualBox did not provide IBPB to the VM, Arch Linux tried to use it, while Debian didn't. I've found an Arch Linux bug report which seems to fit the bill (Modifications in 5.18.15 addressing Retbleed may prevent booting on AMD CPUs). For a test, please try if adding spectre_v2=off to the kernel parameters helps, and report back.Arch_fail_to_boot.log wrote:00:00:00.806585 SVM Feature Identification (leaf A): 00:00:00.806552 Mnemonic - Description = guest (host) 00:00:00.806615 IBPB - Supports the IBPB command in IA32_PRED_CMD = 0 (1) [...[ 00:00:04.761738 IEM: wrmsr(0x49,0x0`00000001) -> #GP(0)
-
- Posts: 21
- Joined: 28. Dec 2018, 13:39
- Primary OS: Linux other
- VBox Version: OSE Debian
- Guest OSses: Win 7, Win 11, Arch Linux, Debian, Slackware, Arco
- Location: Tarragona. Spain
Re: All of a sudden, all my Arch-based VM's fail to launch (boot)
BINGO! That did it. Have added the spectre_v2=off kernel parameter to all my Arch VM's (even the one using rEFInd) and all functions as before. Thank you SO MUCH for that lead. When this all started a few days ago I went through the log files with a fine tooth comb, but clearly my experience level (after nearly six freakin years of using both Arch and Debian linux) just wasn't high enough to catch this. Just wow. Thank you again!fth0 wrote:FWIW, Debian read ~600 MB from the virtual hard disk, whereas Arch Linux read ~20 MB from the virtual hard disk.
I may have found something:
Although VirtualBox did not provide IBPB to the VM, Arch Linux tried to use it, while Debian didn't. I've found an Arch Linux bug report which seems to fit the bill (Modifications in 5.18.15 addressing Retbleed may prevent booting on AMD CPUs). For a test, please try if adding spectre_v2=off to the kernel parameters helps, and report back.Arch_fail_to_boot.log wrote:00:00:00.806585 SVM Feature Identification (leaf A): 00:00:00.806552 Mnemonic - Description = guest (host) 00:00:00.806615 IBPB - Supports the IBPB command in IA32_PRED_CMD = 0 (1) [...[ 00:00:04.761738 IEM: wrmsr(0x49,0x0`00000001) -> #GP(0)
-
- Posts: 21
- Joined: 28. Dec 2018, 13:39
- Primary OS: Linux other
- VBox Version: OSE Debian
- Guest OSses: Win 7, Win 11, Arch Linux, Debian, Slackware, Arco
- Location: Tarragona. Spain
Re: SOLVED All of a sudden, all my Arch-based VM's fail to launch (boot)
As a side thought, why am I able to boot my Fedora and Arch bare metal installations (not VM's, but actual installs to dedicated SSD's) without adding this kernel parameter to my Grub? Just slightly confused here.
-
- Volunteer
- Posts: 5677
- Joined: 14. Feb 2019, 03:06
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Linux, Windows 10, ...
- Location: Germany
Re: [Solved] All of a sudden, all my Arch-based VM's fail to launch (boot)
In the log messages quoted, you can see that the host's physical CPU provides IBPB, it's just that VirtualBox's virtual CPU does not provide it to the VM. And Arch Linux kernel 5.18.15 has a bug not verifying the availability of IBPB before enabling it.
Can you confirm that for Arch Linux kernel 5.18.16, the spectre_v2=off kernel parameter is no longer necessary?
Can you confirm that for Arch Linux kernel 5.18.16, the spectre_v2=off kernel parameter is no longer necessary?
-
- Posts: 21
- Joined: 28. Dec 2018, 13:39
- Primary OS: Linux other
- VBox Version: OSE Debian
- Guest OSses: Win 7, Win 11, Arch Linux, Debian, Slackware, Arco
- Location: Tarragona. Spain
Re: [Solved] All of a sudden, all my Arch-based VM's fail to launch (boot)
Well ok, get this. I removed the spectre kernel parameter from my vanilla Arch VM, and it boots up under kernel 5.18.16 ok. But when choosing kernel 5.15.58-2-lts to boot from, it fails as before. Apparently the kernel devs haven't gotten around to fixing the regression with ltx?fth0 wrote:In the log messages quoted, you can see that the host's physical CPU provides IBPB, it's just that VirtualBox's virtual CPU does not provide it to the VM. And Arch Linux kernel 5.18.15 has a bug not verifying the availability of IBPB before enabling it.
Can you confirm that for Arch Linux kernel 5.18.16, the spectre_v2=off kernel parameter is no longer necessary?
-
- Volunteer
- Posts: 5677
- Joined: 14. Feb 2019, 03:06
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Linux, Windows 10, ...
- Location: Germany
Re: [Solved] All of a sudden, all my Arch-based VM's fail to launch (boot)
Should be answered in the Arch Linux ticket.
-
- Posts: 21
- Joined: 28. Dec 2018, 13:39
- Primary OS: Linux other
- VBox Version: OSE Debian
- Guest OSses: Win 7, Win 11, Arch Linux, Debian, Slackware, Arco
- Location: Tarragona. Spain
Re: [Solved] All of a sudden, all my Arch-based VM's fail to launch (boot)
Update: It appears as of lts kernel 5.15.59-2-lts that this issue is solved as well.