VirtualBox 6.0.24 cannot start Ubuntu 14.04 guest on CentOS 7 host

[Prof Eliot]

I have two 64-bit Ubuntu VMs I tried to set up on a CentOS 7 host, and neither will start. One is a VM that is working fine under Windows 10; the other is a brand new VM, created to see if the defaults work (they don't). I am going to attach:

- /proc/cpuinfo-like information from the Windows host (where the VM works)
- a log from Windows where things work
- the .vbox file from the working Windows setup
- /proc/cpuinfo from the Centos 7 Linux host (where things don't work)
- a log from the Linux host
- the .vbox file from the Centos 7 host

(The latter 3 I will add to a comment since I am allowed only 3 attachments!)

Note that I have tried a variety of settings (host I/O cache or not, acceleration or not, number of CPUs, amount of video memory, which video drive (e.g., VBoxVGA), etc., as people have suggested in a variety of other posts, but without success.

The machine was copied by exporting the VM as an appliance, copying it over the network and then importing the appliance.

The CentOS 7 host's uname -a reports:

Linux compute-0-1 3.10.0-957.21.3.el7.x86_64 #1 SMP Tue Jun 18 16:35:19 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

yum reports that this package is installed: VirtualBox-6.0-6.0.24_139119_el7-1.x86_64

/sbin/vboxconfig happily built the kernel modules.


Thanks for any suggestions / advice!

[Prof Eliot]

Here are my other attachments ... Eliot Moss

[scottgus1]

Thanks for the diagnostic data! The working host's log was from a restored session, not a start from cold boot. Cold Boot logs how more diagnostic data.

Logs fit perfectly when zipped, so there would be no need to strip out data to make them fit.

A guest moved from one Virtualbox host to another Virtualbox host can be directly copied, if the guest is in the optimum format * . It does not need to be exported/imported. Check to see that your guest is in optimum format then copy the guest folder to the new guest. Remove the imported guest then double-click the copied guest.

( * optimum format is where the guest disk file(s) (and Snapshots folder if used) are in the guest folder next to the guest's .vbox file. A guest in this format can be copied to any capable Virtualbox host and run without import/export.)

If Virtualbox cannot start a guest it quite often relates an error message. If you got a Virtualbox error message we need to know what the exact error message is.

Your working .vbox file shows the guest still apparently has a saved state. If you exported the guest while it was save-stated the imported guest may at least have bad data not comparing to the original host's CPU.

[Prof Eliot]

Thank you, scottgus1!

I exported to an appliance while the VM was shutdown, so there was no saved state at that time. Not sure how that changes things. I am attaching a Windows (working) host cold boot log for you now, and thanks for the tip about zipping the log file. And nice to know that I can copy a (powered down) VM from one system to another without doing the appliance export thing.

On the (failing) Centos 7 host I am not getting a message. The boot process simply does not proceed any farther than what you see in the log. I have to do a VBoxManage emergencystop to kill the stuck VM (which will give an error message, but not a useful one). This is true of a fresh VM as well as the copied one I am reporting. The fresh VM has a blank hard disk and should be booting the installation of Ubuntu from ubuntu-14.04.4-desktop-amd64.iso, the same OS I am trying to boot in the copied machine, but seems to get stuck at the same place (I'm attaching that log for you as well, and that machine's .vbox file). Note: I had an earlier version of the VM running on CentOS 7, but its disk was too small. I tried to grow it by converting vmdk to vdi then applying gparted, but managed to damage it and decided to just copy my working laptop machine over. I've gone through the whole path of installing a more recent VirtualBox version, getting the kernel modules to build without complaint, getting the appliaance copied over, etc., and now no machine seems to start. I'm guessing there's some small thing I am doing wrong, but I really don't know what it might be at this point.

Regards - Eliot Moss

[fth0]

I don't have much experience with VirtualBox running on large compute nodes, but noticed the following:

First of all, the Ubuntu Test VBox.log file starts with a time offset of 24 seconds. Typical values are below 5 seconds, so there is already something unknown happening while the VirtualBox VM process is starting. Do you know what's going on then?

The majority of the Ubuntu Test VBox.log file shows expected content, with the exception of the last 5 lines:

00:00:27.365113 X Server details: vendor: The Cygwin/X Project, release: 12005000, protocol version: 11.0, display string: localhost:10.0
00:00:27.365140 Using XKB for keycode to scan code conversion
00:00:28.121970 GUI: UIInvisibleWindow::sltFallback: Move event haven't came. Screen: 0, work area: 0x0 x 1620x1080
00:00:28.122002 GUI: UIDesktopWidgetWatchdog::sltHandleHostScreenAvailableGeometryCalculated: Screen 0 work area is actually resized to: 0x0 x 1620x1080

In a typical VBox.log file, the 1st, 2nd and 4th lines are present, but within the first 30 lines of the VBox.log file, so there is another unknown delay happening. Besides that, I'd expect a Cygwin/X Server rather running on a Windows OS than on a Linux OS. Is it correct, that the CentOS 7 host uses a Cygwin/X Server?

I've never seen the 3rd line before, I'll have to further investigate ...

00:02:14.697846 ERROR [COM]: aRC=VBOX_E_INVALID_VM_STATE (0x80bb0002) aIID={872da645-4a9b-1727-bee2-5585105b9eed} aComponent={ConsoleWrap} aText={Invalid machine state: Starting (must be Running, Paused or Stuck)}, preserve=false aResultDetail=0

The 5th line is probably a consequential failure. I've seen it in the past, so maybe it's worth googling for it ...

If nothing else helps, for a test, you could take this fresh VM and remove all optional components from its configuration (USB, Network, Audio, ...).

[scottgus1]

The Ububtu test vm is using the VboxSVGA video card for Windows guests, not the default VMSVGA for Linux guests that the New Guest Wizard chose. I don't know if this is the reason for the failure, but the guest log shuts down during setup with:

00:02:14.697846 ERROR [COM]: aRC=VBOX_E_INVALID_VM_STATE (0x80bb0002) aIID={872da645-4a9b-1727-bee2-5585105b9eed} aComponent={ConsoleWrap} aText={Invalid machine state: Starting (must be Running, Paused or Stuck)}, preserve=false aResultDetail=0

Based on fth0's posted info maybe the wrong video card is pertinent.

The working vbox.log shows VboxVGA, the old video card for all guests. It also shows mismatched Extension Pack:

00:00:01.742155 VirtualBox VM 6.0.22 r137980 win.amd64 (May 14 2020 19:17:53) release log
00:00:01.803615 Installed Extension Packs:
00:00:01.803629 Oracle VM VirtualBox Extension Pack (Version: 6.1.8 r137981; VRDE Module: VBoxVRDP unusable because of 'VBoxExtPackVMRegister returned VERR_VERSION_MISMATCH, pReg=0000000000000000 ErrInfo='VirtualBox version mismatch - expected 6.1 got 6.0'')

which doesn't always but one day will cause a problem.

[Prof Eliot]

To fth0: Thank you! Part of the explanation for what you're seeing is that the CentOS host is a server in a machine and I was running X from my laptop (Cygwin/X, Windows OS). Also, running things that way is _very slow_ (for X actions).

That 5th with "Invalid machine state" suggests that I should apply emergencystop to that VM before trying to start it.

Both: I will ty out using VboxVGA and see if I can find and install the Extension Pack. There did not seem to be an rpm for it, so I wasn't sure exactly how to install it on the server, but I can look harder on that ...

[scottgus1]

CentOS host is a server in a machine and I was running X from my laptop

Is this some kind of remote desktop? Or is it nested virtualization?

That 5th with "Invalid machine state" suggests that I should apply emergencystop to that VM before trying to start it.

I don't understand how you arrive at this. VBOX_E_INVALID_VM_STATE is a Virtualbox error indicating something isn't set right somewhere and Virtualbox doesn't know how to fix it. Is "emergency stop" supposed to be powering off the guest? Powering off is just 'pulling the plug', which could cause more damage.

If you don't need 3D acceleration in the guest, VboxVGA will do. If 3D is needed, use VMSVGA.

[Prof Eliot]

Yes, I see that the extension packs are just one per release (not host or guest specific). I was able to fetch the matching one.

And I changed the graphics controller to VBoxVGA. I've never used 3D since my use for these VMs is not really graphics related.
Once I get the machine working, headless and a network connection for ssh login will be fine. (And I have had that working
before.)

I also deleted the obsolete Shared Folder information (referred to windows paths).

I was able to speed up the X Windows displaying by using ssh compression over the slower link. This is not virtualization or RDP - just regular X windows but over a slow network link back to the X server on my laptop.

Still, the behavior is the same as before.

[fth0]

Part of the explanation for what you're seeing is that the CentOS host is a server in a machine and I was running X from my laptop (Cygwin/X, Windows OS). Also, running things that way is _very slow_ (for X actions).

I was able to speed up the X Windows displaying by using ssh compression over the slower link. This is not virtualization or RDP - just regular X windows but over a slow network link back to the X server on my laptop.

Instead of using Cygwin/X or SSH with X11 forwarding, you could use either X2Go or NoMachine. Those two solutions are considerably faster, because they reduce the X Window round trip times by a significant amount by proxying the X11 protocol. I'm using them myself in the current home office times and can recommend them.

Is this some kind of remote desktop? Or is it nested virtualization?

You've probably heard about the X Window System (X11, X), which is the underlying technology for most graphical desktops on Linux (and other Unix) systems. The windows managers (e.g. Cinnamon, MATE, KDE, Gnome, XFCE, ...) all are running on top of the X Window System. X11 is designed for a client-server relationship, so that any program can be running on PC S, and the graphical display, mouse and keyboard are on PC C. To confuse the situation a little bit, the program on PC S is called an X client, while on PC C the X server is running. If you are running a Linux OS, the X clients and the X server are usually both running on the same PC. But they can also be separated by any kind of network.

Prof Eliot uses a laptop with a Windows OS and Cygwin/X as X server, the X clients are running on the large compute node, and the VirtualBox Manager and/or the VirtualBox VM are X clients. I haven't tried this setup (yet).

[Prof Eliot]

Yes, I understand X. Thanks for the hints. I was able to boot a previous machine with this same X setup, when I ran gparted to try to adjust the partitions on a disk that I expanded.

I should add that the boot process stops at the same place when I do a headless start. So it would appear to be something else.

Lastly, I had a 5.0.x version of this machine running fine on this same host (i.e., same host, same guest), so the problem seems to be something about the 6.0.y series of VirtualBox (or my installation of it) and/or the .vbox file.

I appreciate both of you continuing to engage and suggest!

[fth0]

Yes, I understand X.

I supposed that, too. The detailed explanation wasn't targeted at you.

I should add that the boot process stops at the same place when I do a headless start. So it would appear to be something else.

Good to know. You can generally divide the startup of a VM in two parts: First, the VirtualBox VM process creates and initializes the virtual hardware (drivers in the host OS, devices for the guest OS). Second, the guest is started, beginning with the virtual BIOS, similar to a real PC starting with the BIOS after power on.

Your problem seems to happen in the middle of the first part. It could be something CPU related (like the APIC id ordering), and we do not know many users of VirtualBox with large CPUs.

I appreciate both of you continuing to engage and suggest!

Well, it's a welcome challenge in the midst of the daily 'Hyper-V is enabled' problems.

[Prof Eliot]

What do you think I should try next?

Three directions came to mind:

- Update the host in question to 6.1.x. My only objection to that is that I seem to recall trying it on Windows and the working VM no longer worked, though it was long enough ago I do not recall the issue.

- The disk in question is shared with another server running a slightly later version of Centos 7. I could see if installing VB there and running under that different version makes a difference.

- I could downgrade to VB 5.0.x, which I know worked with a similar VM recently on the same host.

Appreciative of your advice / insights. Indeed, the booting is not getting past the first stage, which is troubling, but also suggests it is not my disk or DVD contents, but something else. I observe that I have _really_ long-running jobs on the host in question and am not in a position to do a host restart (the jobs are five months in, and I don't know when they'll finish!).

[Prof Eliot]

Today I went in to my office for a while, to have the advantage of higher bandwidth to the server.

I played around with various settings in 6.0.24 without success.

I then decided to:

- Install 6.1.12, on the off-chance that the VB version would make a difference, and

- With 6.1.12, create a new VM, at the least to see what setting VB 6.1.12 thinks might be good.

That did not work right away, but for some reason I thought to entirely disable the audio device. (Previously I had tried just turning off audio input and output - not quite the same.) And then the VM booted!

So the answer was that the audio device was somehow interfering. I don't need audio in my work, so that's fine. It could be that using a different one of VB's audio cards might work - or not.

My "old" VM also booted up just fine as soon as I disabled the audio device. I do need the VBoxVGA display setup for it to auto=detect and use the whole available window. However, that server VM I will usually be running headless since X over the network link is not fun and ssh from an xterm is fine.

Thought you all would want the report.

I suspect that 6.0 would have worked as well if I fully disabled the audio device, but it's not worth my while to uninstall, re-install, etc.

Thanks for your suggestions along the way. "Turn off / disable devices" was the key thing this time!

Eliot Moss

[Prof Eliot]

A little more info for the VB team on this ...

- The PulseAudio and ALSA audio device definitely prevent booting

- They do seem to start a /usr/bin/pulseaudio process on the host. This host is server, which I am guessing does not really have audio devices, only some component(s) is not smart enough to figure that out and tell the user anything and/or just disable the device with a warning message and drive on.

- The Null audio device does boot

Regards - Eliot