Cant restore some saved machines

Discussions related to using VirtualBox on Windows hosts.
Post Reply
lazna
Posts: 16
Joined: 20. Dec 2013, 02:19

Cant restore some saved machines

Post by lazna »

Have backup script which every night:

1. check machine state
2. if machine is in state 'running', perform

Code: Select all

vboxmanage.exe controlvm "machine" savestate
3. perform

Code: Select all

vboxmanage.exe export "machine" -o file_name
4. Perform

Code: Select all

vboxmanage.exe startvm "machine" --type headless
this happens durring night. Morning, time by time, found one or more virtual machines in 'saved' state. This cant be manually started untill "remaining processes" named vboxheadless.exe are not killed in taskmanager. Expect this is same reason why machines was not started by

Code: Select all

vboxmanage startvm
after its export.

What am I doing wrong? Why there are remaining processes in memory after 'savestate' command?

6.1.26r145957
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: Cant restore some saved machines

Post by scottgus1 »

We'd need to see some kind of feedback on the error condition that happens, perhaps with text output of the commands and the 'std err' stream redirected to a text file. Web-search how to redirect std-out and std-err to text files.

FWIW I too have had times where a VM refused to turn off, though it was not while save-stating or running headless. I had to run a timer in the batch file that tested for the VM's state to be powered off, and upon expiration of the time, would run a 'controlvm poweroff', which would kill the stuck VM.

However, I really feel a Stack-Exchange-style 'frame challenge' is required. Have you tested restoring your backups? Because I strongly suspect that you're not getting what you think you're backing up.
lazna wrote:backup script ...
vboxmanage.exe controlvm "machine" savestate
A saved state can only be reliably restarted on the original host with the original version of Virtualbox that was running the VM when it was save-stated. The saved state includes processor states, so only the original CPU can run it, and saved states cannot survive across Virtualbox version changes. The only way to get the saved-state VM to restart without original hardware and Virtualbox is to discard the saved state, which loses all unsaved data in the VM and drops the VM to the same as if it were powered off unceremoniously, with dirty databases, at which point 'controlvm poweroff' would be faster.
lazna wrote:vboxmanage.exe controlvm "machine" savestate
...
vboxmanage.exe export "machine"
I just tried exporting and re-importing a saved-state VM. It did not import as a saved-state VM, but as a powered-off VM instead. The saved state was missing. I don't always use export/import, so I may have missed something, but I don't think saved state is compatible with export/import.

Export is not recommended for backups, because the exported OVA is not a bit-for-bit copy of the original and cannot be file-compare checked to see if the backup is a good backup. Also, some UUIDs of the VM might change, which might cause software activation issues.

The best backup is a folder copy of a fully and peacefully shut-down not save-stated VM's folder, along with any disk files stored outside the VM folder. If the disk file is inside the VM folder next to the VM's .vbox file, the backup copy of such a folder can be copied and started on any capable Virtualbox host. This backup is not locked to a particular host or version of Virtualbox like a saved state is, and it can be file-compared to confirm backup integrity.
lazna
Posts: 16
Joined: 20. Dec 2013, 02:19

Re: Cant restore some saved machines

Post by lazna »

Thanks for info, will try copying folders.
lazna
Posts: 16
Joined: 20. Dec 2013, 02:19

Re: Cant restore some saved machines

Post by lazna »

One more question regarding of EXPORT:

You say the export cant be vefified if it is good export. There are no any built-in mechanisms which guarantee you the backup was seccessully done? I mean you cant rely on export was OK if exitcode is 0?

backup by copying folder is problem for me, because various VM running from various drives and paths. Scripting this could be a bit problematic..
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: Cant restore some saved machines

Post by scottgus1 »

I have not seen the source code for 'Export', but an exit code from a program such as 'export' only shows that the program did not see the errors it was programmed to look for. Other errors can still have occurred. If a host glitch happens during reading from or writing to the drive, or another program goes crazy and overwrites a portion of the backup file, the program may not know about it and may therefore not raise an error. Thus a final comparison of the output backup file and the original is necessary to confirm the backup. A hash of the original and copy may also be useful for further checks at a later date when the original has been used more and has changed.

I have overseen an office server of Virtualbox VMs and did folder backups with file-compare and hash confirmations to several on-site and off-site locations. I was quite confident due to the low-level file compares and equal hashes that a total destruction of the office would result in only a half hour maximum of data loss and a complete restore of the office server within a couple hours was easily achieved. The confirmation of successful backups gave me that confidence (along with making sure capable hardware was also available). Also, trying to restore the backups on another host and seeing successful restores and working VMs with no lost data gave me assurance that my confidence was not misplaced.
lazna wrote:various VM running from various drives and paths.
If you keep a list of folders for your VMs in an ini file and read through the file in your backup batch file, you can back up by copying folders easily.

Parse the <MachineRegistry> section of Virtualbox.xml to find the path to each VM's .vbox file. If you have each VM's disk file inside the folder containing the VM's .vbox file, then copying and confirming the folder containing the .vbox file will be the backup of the VM. If the disk file must be outside the folder (necessary in a couple rare cases) the .vbox file also contains the path to disk files that are stored outside the VM folder, so parsing the .vbox file can get you these paths for backup too.
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: Cant restore some saved machines

Post by scottgus1 »

I just found another poster viewtopic.php?f=3&t=103980 who has headless processes not closing .

Perhaps a bug has crept in. If you downgrade Virtualbox a few versions does the same headless-remaining problem happen?

Update: the other poster found that rolling back to 6.1.22 got the commands working again. Please try the same and confirm if rolling back makes the commands work again. Then roll up to 6.1.26. If they fail, roll back to 6.1.22 again. If they start working again, this will show the devs that a bug crept in.
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: Cant restore some saved machines

Post by scottgus1 »

A Bugtracker ticket has been opened on this problem: https://www.virtualbox.org/ticket/20574
lazna
Posts: 16
Joined: 20. Dec 2013, 02:19

Re: Cant restore some saved machines

Post by lazna »

scottgus1 wrote:I just found another poster viewtopic.php?f=3&t=103980 who has headless processes not closing .

Perhaps a bug has crept in. If you downgrade Virtualbox a few versions does the same headless-remaining problem happen?

Update: the other poster found that rolling back to 6.1.22 got the commands working again. Please try the same and confirm if rolling back makes the commands work again. Then roll up to 6.1.26. If they fail, roll back to 6.1.22 again. If they start working again, this will show the devs that a bug crept in.
it does not look like my case, because my VM refuse start in both normal / headlesss mode

L.
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: Cant restore some saved machines

Post by scottgus1 »

If headless is still stuck running then normal won't start either.

Did you try rolling back to 6.1.22?
fth0
Volunteer
Posts: 5668
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: Cant restore some saved machines

Post by fth0 »

This issue should be fixed in the VirtualBox test builds 6.1.27r147422 and newer.
lazna
Posts: 16
Joined: 20. Dec 2013, 02:19

Re: Cant restore some saved machines

Post by lazna »

just noticed, this behaviour could be somehow related to low disk space on Host machine. Not sure, only observation.
lazna
Posts: 16
Joined: 20. Dec 2013, 02:19

Re: Cant restore some saved machines

Post by lazna »

Low disk space was probably red herring, it happens even with a tens of free gigabytes on disk.

But today I found, that why I kill process VBoxSVC.exe, than remaining processes of all 'frozen' virtuals gone and those could be freely started. It look like VBoxSVC somehow 'frozinng' single virtuals.
fth0
Volunteer
Posts: 5668
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: Cant restore some saved machines

Post by fth0 »

FWIW, the VirtualBox 6.1.26 regression with hanging VBoxHeadless processes has been solved in the current VirtualBox 6.1.28.
lazna
Posts: 16
Joined: 20. Dec 2013, 02:19

Re: Cant restore some saved machines

Post by lazna »

Just upgraded to 6.1.28 and backup script seems working fine.
Post Reply