Guru meditations with AOS 503 (eCS) as guest

Discussions about using non Windows and Linux guests such as FreeBSD, DOS, OS/2, OpenBSD, etc.
massimo-ecs
Posts: 29
Joined: 25. Feb 2020, 22:48

Re: Guru meditations with AOS 503 (eCS) as guest

Post by massimo-ecs »

scottgus1 wrote:This log does show a guru meditation, good.

The log has some 26 different reboots of the VM OS over almost 108 hours, The guru meditates at 107:47:38.153674, some 50 minutes after the last reboot. The error that the guru meditates about, VERR_VMX_UNABLE_TO_START_VM, may be related to an unexpected failure to continue to access the host's hardware virtualization, VT-x.

Does your Windows host OS have 3rd-party AV or security software on it?

hi,
another guru meditation log, but still with version VirtualBox-6.1.34a-150636-Win
Attachments
server2_web-2022-07-23-18-20-32.zip
(174.01 KiB) Downloaded 54 times
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: Guru meditations with AOS 503 (eCS) as guest

Post by scottgus1 »

This log is from July 22, 2022, grid. It is still 6.1.34.

Here is an interesting pattern:
00:00:24.895871 PIT: mode=2 count=0x10000 (65536) - 18.20 Hz (ch=0)
00:00:25.498616 Display::i_handleDisplayResize: uScreenId=0 pvVRAM=0000000005fd0000 w=800 h=600 bpp=8 cbLine=0x320 flags=0x0 origin=0,0
10:20:04.770070 Reset initiated by keyboard controller
10:20:04.770123 Changing the VM state from 'RUNNING' to 'RESETTING'
......
10:20:24.381106 PIT: mode=2 count=0x10000 (65536) - 18.20 Hz (ch=0)
10:20:24.992108 Display::i_handleDisplayResize: uScreenId=0 pvVRAM=0000000005fd0000 w=800 h=600 bpp=8 cbLine=0x320 flags=0x0 origin=0,0
10:20:49.018193 Reset initiated by keyboard controller
10:20:49.018288 Changing the VM state from 'RUNNING' to 'RESETTING'
......
10:21:08.491738 PIT: mode=2 count=0x10000 (65536) - 18.20 Hz (ch=0)
10:21:09.105877 Display::i_handleDisplayResize: uScreenId=0 pvVRAM=0000000005fd0000 w=800 h=600 bpp=8 cbLine=0x320 flags=0x0 origin=0,0
17:20:13.106847 Reset initiated by keyboard controller
17:20:13.106947 Changing the VM state from 'RUNNING' to 'RESETTING'
......
17:20:32.810893 PIT: mode=2 count=0x10000 (65536) - 18.20 Hz (ch=0)
17:20:33.441600 Display::i_handleDisplayResize: uScreenId=0 pvVRAM=0000000005fd0000 w=800 h=600 bpp=8 cbLine=0x320 flags=0x0 origin=0,0
17:20:57.729338 Reset initiated by keyboard controller
17:20:57.729434 Changing the VM state from 'RUNNING' to 'RESETTING'
......
17:21:17.323160 PIT: mode=2 count=0x10000 (65536) - 18.20 Hz (ch=0)
17:21:17.938941 Display::i_handleDisplayResize: uScreenId=0 pvVRAM=0000000005fd0000 w=800 h=600 bpp=8 cbLine=0x320 flags=0x0 origin=0,0
26:54:44.210758 Changing the VM state from 'RUNNING' to 'GURU_MEDITATION'
26:54:44.210784 Console: Machine state changed to 'GuruMeditation'
26:54:44.211000 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
26:54:44.211001 !!
26:54:44.211001 !! VCPU0: Guru Meditation 1155 (VINF_EM_TRIPLE_FAULT)
The pattern I see is these series of lines, with sometimes hours' worth of no log activity between the "Display" 800x600 line and the Reset request. (Some reset requests have very little delay: I surmise that these are resets right after reboot.)

The guru meditates right after a long no-log-activity delay. Were you by chance resetting the VM when the guru meditated?
massimo-ecs
Posts: 29
Joined: 25. Feb 2020, 22:48

Re: Guru meditations with AOS 503 (eCS) as guest

Post by massimo-ecs »

i didn't anything
i just found the VM's services down, i RDP to the host and i found guru meditation
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: Guru meditations with AOS 503 (eCS) as guest

Post by scottgus1 »

OK, thanks. I'll have to declare this problem beyond me, as far as this particular VM is concerned.

A Triple Fault error is sort of like Virtualbox's last effort to describe the error: it means something tried to do something for which Virtualbox had no way to handle the command. Like a "Beats the living daylights out of me what to do now..."

Since the OS is a non-usual OS, I'd wonder f it is every so often trying something Virtualbox doesn't know how to do. Whatever it is, it's not happening often: hours or days can go by.

A possible experiment, to see if it's the host Virtualbox or the VM OS at fault. Please make a new VM, install Ubuntu or Mint OS, and leave it running in the background all the time. Check every so often if it guru-meditates as well. If it does not, then the problem probably lies in the AOS VM's OS.
massimo-ecs
Posts: 29
Joined: 25. Feb 2020, 22:48

Re: Guru meditations with AOS 503 (eCS) as guest

Post by massimo-ecs »

i've 2 other VMs on the same host these are eCS2.2b never had a gugu meditation that i recall
but they have a very low load

while this one is a web server and has much "overhead" expecially with WP websites

is there a way i can from commandline see if there is a guru meditation and stop and restart the VM with a script?
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: Guru meditations with AOS 503 (eCS) as guest

Post by scottgus1 »

You can monitor output of vboxmanage showvminfo "VM name". look for the "State" line:

Code: Select all

State:                       powered off
"powered off" should say "running" when the VM is running. I'm not sure what that line says when the VM's guru has meditated, but it should be something else than "running".

Put the monitor in a loop with a slight delay, perhaps 10 seconds, so it doesn't swamp a whole CPU core, then do the power-off & restart or reset as needed.
massimo-ecs
Posts: 29
Joined: 25. Feb 2020, 22:48

Re: Guru meditations with AOS 503 (eCS) as guest

Post by massimo-ecs »

please give a look at this guru meditation of today
is that something about vga?

thanks

about the script i did something, yes in the showvminfo output it write "guru meditation"
Attachments
server2_web-2022-07-26-16-02-09.zip
(175.61 KiB) Downloaded 50 times
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: Guru meditations with AOS 503 (eCS) as guest

Post by scottgus1 »

massimo-ecs wrote:in the showvminfo output it write "guru meditation"
Thanks for letting me know! :D

This was the same Triple Fault, about 53 minutes after the latest reset.

This VM resets and reboots a lot. Is that expected behavior?
massimo-ecs
Posts: 29
Joined: 25. Feb 2020, 22:48

Re: Guru meditations with AOS 503 (eCS) as guest

Post by massimo-ecs »

no, this is not an expected behaviour
and i hope that the guest VM will not compromise
this is a production server
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: Guru meditations with AOS 503 (eCS) as guest

Post by scottgus1 »

massimo-ecs wrote:no, this is not an expected behaviour
OK, so the VM's OS is doing something funny.

You can see it in the logs: multiple reboots started by:
Changing the VM state from 'RUNNING' to 'RESETTING'
You should look in whatever the VM's OS has for an Event Viewer and try to pin down what's causing those resets.
massimo-ecs wrote: i hope that the guest VM will not compromise
this is a production server
If you're not doing backups of this VM, it's really time to start. Get the data that's being served and the server scripts backed up onto other media: you may be looking at an OS reinstall.

So far I don't see anything in the logs indicating that this is being caused by Virtualbox.
massimo-ecs
Posts: 29
Joined: 25. Feb 2020, 22:48

Re: Guru meditations with AOS 503 (eCS) as guest

Post by massimo-ecs »

the VM is of course under bkup using VSS technology (daily and weekly *full* bkups)

this guest machine is a web server and sometimes it has "overhead" i mean "a lot of load"

so, if you say that it's not an host related problem (e.g. antivirus or other software) i will try to understand what happens in the guest VM

i've seen right now that the VM has "eComStation" as OS, instead of "ArcaOS"
i've now fixed it to ArcaOS, let's hope it help

thanks you
massimo-ecs
Posts: 29
Joined: 25. Feb 2020, 22:48

Re: Guru meditations with AOS 503 (eCS) as guest

Post by massimo-ecs »

that's surely not the wrong VM type/OS since i just had another guru medidation
this happens on a production web server and it's very very frustrating
fth0
Volunteer
Posts: 5668
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: Guru meditations with AOS 503 (eCS) as guest

Post by fth0 »

server2_web-2022-07-26-16-02-09.log file wrote:
00:00:00.588506 VirtualBox VM 6.1.36 r152435 win.amd64 (Jul 19 2022 21:51:27) release log
00:00:00.588511 Log opened 2022-07-24T22:24:55.864569900Z
06:24:23.546745 Reset initiated by keyboard controller
06:25:06.645441 Reset initiated by keyboard controller
23:24:26.333376 Reset initiated by keyboard controller
23:25:09.020182 Reset initiated by keyboard controller
30:24:22.472590 Reset initiated by keyboard controller
30:25:04.266683 Reset initiated by keyboard controller
38:36:25.673205 Reset initiated by keyboard controller
39:29:56.136912 Changing the VM state from 'RUNNING' to 'GURU_MEDITATION'
In your previous log files, you generally had a lot more guest OS reboots than Guru Meditations, so I'd suggest to investigate the former. The latter may be just a variant of the former.

Note the similar minute values of the timestamps, which indicate that the cause of the reboots could be an hourly recurring event. The first reboot was around 2022-07-24T22:24:55.864569900Z + 06:24:23.546745 = 2022-07-25T04:49:19Z.
massimo-ecs
Posts: 29
Joined: 25. Feb 2020, 22:48

Re: Guru meditations with AOS 503 (eCS) as guest

Post by massimo-ecs »

i don't understand at 6,24 the VM do a standard scheduled reboot (a standard, reboot not a power off)

the others are reboots done manually by me, due to maintenance works (updates or such)

e.g. today i've changed the os type from eCS to ArcaOS, some hours later i turned off the COM1 since i don't need it
(all of these, anyway standard reboots, not power-off, in the hope to avoid guru meditations)
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: Guru meditations with AOS 503 (eCS) as guest

Post by scottgus1 »

massimo-ecs wrote:the VM is of course under bkup using VSS technology (daily and weekly *full* bkups)
If these backups are being done in the host OS while the VM is running, then they're corrupted and not reliable. Such a host backup of a running VM does not let the VM OS know it is being backed up, so the VM OS cannot prepare its databases, file writes, etc. Data will be corrupted and dirty.

The only safe methods to backup a VM are:

A. with the VM completely shut down, not save-stated. Then host backups will be OK, if all the VM files are backed up.

B. if the VM must be running, then the OS inside the VM must handle the backup using native or 3rd-party software installed inside the VM capable of telling the VM OS to prepare itself for live backup. Then save the backups out through a Bridged or Host-Only network onto other media besides the disk holding the VM's files. The VM's files still need to be backed up via method A every so often.

Please start a new topic if you want to discuss this more.
Post Reply