All VMs fail with guru meditation error

Discussions related to using VirtualBox on Linux hosts.
Post Reply
bdmayes
Posts: 3
Joined: 26. Mar 2012, 17:57

All VMs fail with guru meditation error

Post by bdmayes »

I suddenly cannot get any VM to start on my Ubuntu machine. The (I think) relevant sections in the logs show:
00:00:00.792984 Changing the VM state from 'POWERING_ON' to 'RUNNING'
00:00:00.792993 Console: Machine state changed to 'Running'
00:00:00.794930 Changing the VM state from 'RUNNING' to 'GURU_MEDITATION'
00:00:00.794952 Console: Machine state changed to 'GuruMeditation'
00:00:00.796260 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
00:00:00.796260 !!
00:00:00.796260 !!         VCPU0: Guru Meditation -79 (VERR_INVALID_STATE)
00:00:00.796263 !!
00:00:00.796266 !!
00:00:00.796267 !! {mappings, <NULL>}
00:00:00.796267 !!
00:00:00.796272 
00:00:00.796272 The mappings are DISABLED.
00:00:00.796274 00000000fd800000 - 00000000febfffff  Hypervisor Memory Area
00:00:00.796276 00000000fec00000 - 00000000ffbfffff  Above 4GB Base RAM
00:00:00.796277 !!
00:00:00.796277 !! {hma, <NULL>}
00:00:00.796278 !!
and
00:00:02.793906 ERROR [COM]: aRC=E_ACCESSDENIED (0x80070005) aIID={ab4164db-c13e-4dab-842d-61ee3f0c1e87} aComponent={DisplayWrap} aText={The console is not powered up}, preserve=false aResultDetail=0
00:00:02.794673 GUI: Aborting startup due to invalid machine state detected: 1
I have no idea when this started happening but it must have been in the past day or so. As far as I know, the only thing that might have changed in that time frame is that I installed the qemu-kvm package in Ubuntu (the host), then added my user to the kvm group and rebooted. I was just spinning up some react-native development on my machine and android-studio was complaining about not having access to /dev/kvm so I installed qemu-kvm and added my user to the kvm group to fix that.

Not sure what is causing the E_ACCESSDENIED error. Would be nice if it logged the exact resource/file handle/whatever that it was trying to access when it encountered that error...
Attachments
VBox.log.gz
(52.18 KiB) Downloaded 108 times
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: All VMs fail with guru meditation error

Post by scottgus1 »

I'm no Linux guy, just trying to branch out...

A very easy way to get help is to do a search in a web browser, using the CAPITALIZED_UNDERSCORE_SPACED_WORDS, with the text "site:forums.virtualbox.org" added to restrict searching to this forum, like this:

VERR_INVALID_STATE site:forums.virtualbox.org

One link I found showed the possibility that having qemu on your host at the same time as Virtualbox might be the problem.
socratis
Site Moderator
Posts: 27330
Joined: 22. Oct 2010, 11:03
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Win(*>98), Linux*, OSX>10.5
Location: Greece

Re: All VMs fail with guru meditation error

Post by socratis »

bdmayes wrote:the only thing that might have changed in that time frame is that I installed the qemu-kvm package in Ubuntu (the host)
scottgus1 wrote:One link I found showed the possibility that having qemu on your host at the same time as Virtualbox might be the problem.
Scott, it's not the QEMU part, it's the KVM part that 'bdmayes' installed, QEMU is just the front-end for KVM. KVM has been pulling a "Hyper-V" on Linux hosts, for a couple of years now, i.e. they grab VT-x in a really aggressive way, sometimes even permanently. I haven't seen an obvious pattern that I can recognize from "far and away". And I think that this is what the Guru refers to...
bdmayes wrote:Not sure what is causing the E_ACCESSDENIED error
00:00:02.793906 ERROR [COM]: aRC=E_ACCESSDENIED (0x80070005) aIID={ab4164db-c13e-4dab-842d-61ee3f0c1e87}
                aComponent={DisplayWrap} aText={The console is not powered up}, preserve=false aResultDetail=0
This error happens when there cannot be an update to the Preview in VirtualBox Manager. I used to see them as a red-herring and I would discard them, wouldn't even bother. But, I was told that they are actual errors to be investigated.

Well, the "investigation" that I see in this case is that you couldn't get the Preview window updated because "The console is not powered up", i.e. you've crashed and there's nothing to get the preview from. See? I was right to not get bothered by them. ;)



PS. Hey Scott, I'm so going to "steal" your CAPITALIZED_UNDERSCORE_SPACED_WORDS sentence, I like it. I'm going to be quoting it mucho plenty... :)
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: All VMs fail with guru meditation error

Post by scottgus1 »

socratis wrote: I'm so going to "steal" your CAPITALIZED_UNDERSCORE_SPACED_WORDS sentence, I like it.
I'm happy to contribute!

So KVM acts like Hyper-V, never got taught to share its toys, huh? Will have to remember that.
socratis
Site Moderator
Posts: 27330
Joined: 22. Oct 2010, 11:03
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Win(*>98), Linux*, OSX>10.5
Location: Greece

Re: All VMs fail with guru meditation error

Post by socratis »

scottgus1 wrote:So KVM acts like Hyper-V, never got taught to share its toys, huh?
The sad part in this whole story is that KVM did not do that in the past. I guess they decided at some point that the "VT-x lock"/"execute privileged code"/"VT-x unlock" that's required for toy-sharing, was taking too many CPU cycles... :?
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
bdmayes
Posts: 3
Joined: 26. Mar 2012, 17:57

Re: All VMs fail with guru meditation error

Post by bdmayes »

Thank you both. I can confirm that

Code: Select all

sudo apt remove qemu-kvm
and a reboot has resolved the issue. I wonder if there a way vbox could detect such a state and then output a more useful error message? Additionally, I wonder if filing a bug with the kvm package to let them know they are too aggressively holding VT-x would be worth my time. If this is an ongoing issue for a few years I suspect someone has already reported it however, and/or they don't care. :(

Thanks for the info. This is really irritating to have to install this package (and reboot!) to do development in Android Studio, but then have to uninstall it (and reboot again!) just to get another VM functional. At least I have a solution, even if it does involve muttering a few curse words. :D
socratis
Site Moderator
Posts: 27330
Joined: 22. Oct 2010, 11:03
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Win(*>98), Linux*, OSX>10.5
Location: Greece

Re: All VMs fail with guru meditation error

Post by socratis »

bdmayes wrote:if filing a bug with the kvm package to let them know they are too aggressively holding VT-x would be worth my time
If they don't hear from the user-base that they're creating a coexistence problem, they're not going to address it. Just saying...
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
buckyball7
Posts: 3
Joined: 23. Apr 2019, 17:14

Re: All VMs fail with guru meditation error (another user)

Post by buckyball7 »

I am having the exact same problem, but do not have qemu-kvm installed.

I have tried VirtualBox 6.0.14 and 6.1.0, the latter with several different Linux kernels (4.15.x, 5.0.0, 5.3.0), all to no avail.

Any ideas?
Andrew
Attachments
VBox.log.gz
(52.12 KiB) Downloaded 16 times
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: All VMs fail with guru meditation error

Post by scottgus1 »

bdmayes wrote:This is really irritating to have to install this package (and reboot!) to do development in Android Studio, but then have to uninstall it (and reboot again!) just to get another VM functional.
My no-doubt-too-surface research into 'kvm' shows that it might be a type-1 hypervisor. If so, it sits between the host OS and the hardware, rather like Hyper-V on Windows hosts. This post: https://www.tenforums.com/tutorials/139 ... puter.html explains how type-1 hypervisor Hyper-V interferes with Virtualbox et al. on Windows hosts, and some boot switches to turn Hyper-V on & off. Maybe there's something similar that can be done with qemu/kvm? At least uninstalling might not be needed.
buckyball7
Posts: 3
Joined: 23. Apr 2019, 17:14

Re: All VMs fail with guru meditation error (another user)

Post by buckyball7 »

In my previous post, I indicated that the same error was occurring for me, but that qemu-kvm was not installed.

However, the libvirt0 package and dependencies WAS installed and there were a number of virtual network interfaces that had been created. Here's what I found:

Code: Select all

Display network interfaces/addresses:
$ ip addr

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 28:d2:44:04:a8:6a brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.80/24 brd 192.168.2.255 scope global noprefixroute eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::a05e:7f01:e40d:1fa4/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: wlan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 60:36:dd:fe:01:73 brd ff:ff:ff:ff:ff:ff
4: mpqemubr0-dummy: <BROADCAST,NOARP> mtu 1500 qdisc noop master mpqemubr0 state DOWN group default qlen 1000
    link/ether 52:54:00:9d:8a:ab brd ff:ff:ff:ff:ff:ff
5: mpqemubr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether 52:54:00:9d:8a:ab brd ff:ff:ff:ff:ff:ff
    inet 10.77.6.1/24 brd 10.77.6.255 scope global mpqemubr0
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fe9d:8aab/64 scope link 
       valid_lft forever preferred_lft forever
6: tap-1e376645a40: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel master mpqemubr0 state DOWN group default qlen 1000
    link/ether c6:db:51:ac:8e:d8 brd ff:ff:ff:ff:ff:ff
Interfaces 4,5,and 6 appear to be associated with the networking for qemu.

So I manually shut them down and removed them and... voila! VirtualBox works again!

Here are the commands I used to remove the interfaces.

(Note: I am new to the 'ip' command which is replacing the traditional 'ifconfig' and 'route' commands, so there may be better ways to do this.)

Code: Select all

Display routing table:

$ ip route
default via 192.168.2.1 dev eth0 proto static metric 100 
10.77.6.0/24 dev mpqemubr0 proto kernel scope link src 10.77.6.1 linkdown 
169.254.0.0/16 dev eth0 scope link metric 1000 
192.168.2.0/24 dev eth0 proto kernel scope link src 192.168.2.80 metric 100 

Delete route for qemu bridge:

$ sudo ip route del 10.77.6.0/24

Take note that interface 5 below has an IP address assigned - need to remove it:

$ ip addr

1: lo: (snip)
2: eth0: (snip)
3: wlan0: (snip)
4: mpqemubr0-dummy: (snip)
5: mpqemubr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether 52:54:00:9d:8a:ab brd ff:ff:ff:ff:ff:ff
    inet 10.77.6.1/24 brd 10.77.6.255 scope global mpqemubr0
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fe9d:8aab/64 scope link 
       valid_lft forever preferred_lft forever
6: (snip)

Remove IP address on qemu bridge device 'mpqemubr0' :

$ sudo ip addr del  10.77.6.1/24 dev mpqemubr0

Make sure bridge interface is down:

$ sudo ip link set dev mpqemubr0 down

Remove bridge interfaces:

$ sudo ip link del dev mpqemubr0
$ sudo ip link del dev mpqemubr0-dummy
$ sudo ip link del dev tap-1e376645a40

Finally check the interfaces again:

$ ip addr

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 28:d2:44:04:a8:6a brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.80/24 brd 192.168.2.255 scope global noprefixroute eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::a05e:7f01:e40d:1fa4/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: wlan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 60:36:dd:fe:01:73 brd ff:ff:ff:ff:ff:ff

Nice and clean!
At this point, VirtualBox guests launch as per normal, across a range of guest OS.

Now I just need to figure out where the addresses are getting set up and rip that out.

I'll update this post when that's sorted.
(***Moderators - should I move these two posts to their own thread? Same symptoms, but different fix?)
shiskey
Posts: 1
Joined: 10. Aug 2020, 14:34

Re: All VMs fail with guru meditation error

Post by shiskey »

Great thread!

In my case the 4, 5 and 6 ones in ip addr as mentioned by the previous helpful individual, was due to multipass.

I need not have to manually remove the networks.. Just had to sudo snap remove multipass and everything got back to normal again!

Finally got that hint after hours of browsing thanks to this thread!
khudnut
Posts: 1
Joined: 26. Feb 2020, 00:48

Re: All VMs fail with guru meditation error

Post by khudnut »

From what I can see, this discussion is not terribly old (Aug 2020) and not fully answered as to the original issue:
"All VMs failed with guru meditation error". I had this problem on CentOS 8, latest version, latest 4.18 kernel. After
reading the first few posts on this thread I had the idea: restart the services and processes. I got a little way with
that using the usual systemctl start/restart on the VBox system daemons. (That is outside the scope of this thread,
but you can use the fine manuals to see how to do systemctl commands.) That did not help. Then, I did what
what many linux administrators will do.
0) Have a backup. If you don't have a backup, you don't have and excuse.
1) I was running the virtualbox GUI. Exit from that.
2) ps -ef | grep -i <virtual machine name>
In my case the v. machine was e2web-vm2-a and I actually did
ps -ef | grep -i vm2
This results in a process and as expected I found the PID and the PPID was 1, meaning the virtual machine was
running as a nohup'd background process.
2a) Again, I had a backup so I was not concerned. Therefore:
kill -9 PID
3) ps -ef again to check that the kill worked. It did. Open virtualbox GUI again and I see that the machine is now in
a rather normal state and not running. Exit virtualbox GUI
4) As usual: nohup vboxheadless -s <virtual machine name> &
5) Check: everything was back to normal.

So, to summarize: You need a backup. You need that before you even get here. Assuming that: kill the process and
restart the virtual machine.

Hope this helps. I'm not a VirtualBox guru.
--
Karl Hudnut
Post Reply