Re-installing crashed VirtualBox guest through GUI

[JohnShmo]

Hello;
I appreciate your patience in advance. I have been thoroughly enjoying the benefits of VirtualBox when it functions as it allows me to use MS programs which are superior to what I can get native on Linux.
Recently the whole guest system crashed and I had to play around by re-installing/upgrading the whole program.
viewtopic.php?f=7&t=83800
That worked, but then even the thread gave a SQL error (I'm battin a 1000 here).

So I'm opening a new thread and I apologize in advance if this topic is posted elsewhere. I would assume that it is, but I cannot find it.
I have the .vbox file, the .log files, and several .vdi screenshots. However the VirtualBox GUI does not allow me to recreate the guest setup which was there previously.

As I understand it, a .vbox file is the actual virtual hard disk and .vdi is a snapshot in time which only retains the current state of a virtual drive.

The GUI for creating a new virtual hard drive does not allow .vbox files, if I instead choose a .vdi file (which IS an option) then this message appears
the medium '....VirtualBox VMs/New group/BilltheGates Clone/Snapshots/{1b4d801a-fb3b-431d-9828-73eea720bc74}.vdi' is not found in the media registry ('...../.config/VirtualBox/VirtualBox.xml').
It is very clear (both from the warning within the file and other posts on this forum) that one should NOT mess with the VirtualBox.xml file and so I have not. - If you would like to see the contents of this file, I can upload them.
There is no option within VirtualBox to access VBox Manager until a guest OS is created
I've also looked into importing an appliance (.ovf) but I'm assuming that this is a plan of action I should have prepared for before the SNAFU

Thank you in advance for your advice

[mpack]

As I understand it, a .vbox file is the actual virtual hard disk and .vdi is a snapshot in time which only retains the current state of a virtual drive.

I'm afraid that is wholly mistaken.

A .vbox file is an xml which describes the recipe for the VM, records which drives are attached, what snapshots have been taken (if any). This is an extremely important file. It does not represent a disk drive.

A .vdi is a disk image container. It may contain either a snapshot or a complete image. Usually if it came from the snapshots subfolder and has a weird name such as {1234abcd.xxxx}.vdi then it's a snapshot. Otherwise its the base VDI for the VM. These too are vital files.

There are other files in the VM folder or subfolders, but they are not vital.

A VM which uses snapshots consists of a <vmname>.vbox file in the VM folder, a base <vmname>.vdi in the same folder, and a bunch of snapshot VDI files in the VM/Snapshots subfolder. You need ALL of these files to reconstitute the VM. E.g. if you don't have the base VDI then you don't have a VM. The VM folder structure needs to be correct, the .vbox has to be consistent with the available VDI files, and you have to use Machine|Add... to re-register the VM - note that you are restoring a VM, not installing a disk drive, so it isn't appropriate to be looking at the media manager.

[JohnShmo]

Okay mpack, thank you for that, it clears things up more for me.

Unfortunately what I have for all of the clones that I've created are-
1- .vbox file (sometimes this has data and sometimes it's 0 bytes)
2. .vbox-prev file
3. log folder with multiple log files
4. snapshots folder with multiple snapshots files {1234abcd.xxxx}.vdi



Is that the correct data needed to reconstruct the clone?

[mpack]

The .vbox file can't be zero bytes, because that means it has no data, which means you don't have a VM. And I don't see how the size can fluctuate: VirtualBox can't use a 0 byte file, unless of course the VM was already running.

If the VM was a linked clone then it's possible that the base VDI is in the original VMs folder. The .vbox file would tell you that, if it existed!

One thing puzzles me... why are you talking about creating clones? You don't (in fact probably can't) create clones of non working VMs.

Let's pick a test case: what do you have left from a VM that was previously working before the crash? I.e. no clones.

[JohnShmo]

Regarding the 0-byte file, that's exactly what I thought as well. I don't know how a functional guest can lose all of it's data like this, but that seems to be what happened.
I do have two earlier VirtualBox guest machines with the same file structure but with 50kb .vbox files. Theoretically either of those should be capable of being restarted. I simply have not had success doing so.

BTW, I've been careful enough to store all working files (.psd, .dwg, etc) on the host machine through shared folders. So the primary goal with restarting an existing guest is simply to break the cycle of constantly building new guest machines every time a crash like this occurs and to retain customized MS program settings.

[mpack]

VirtualBox rewrites the .vbox on every run (settings changes get written back there), so if the VM was running then its no surprise that the open file shows 0 bytes. However you have NOT so far indicated that the VM was running: please clarify. Note also that "running" doesn't mean that you successfully reached the guest desktop: if you start the VM then it is running, and if it then shows an error then the VM is still running until you close the error dialog.

50K would be a pretty huge .vbox file. 5K would be more like it. The only way you could reach 50K would be if you went way over the top with the number of snapshots, and in that case it was always a disaster waiting to happen (search the site: snapshots are not safe).

Starting from a blank manager (no registered VMs), the way to recover a VM is to register the .vbox file using the Machine|Add... menu item. I suggest that you pick a VM which seems to have the full complement of files that I outlined earlier, try a Machine|Add... and report what happens.

[JohnShmo]

"VirtualBox rewrites the .vbox on every run"
I'm not surprised to hear that. That's the most logical explanation for a 0kb file.

Sorry to not have provided full info. Before the crash, I had two guest machines. One of those was running with the Windows OS, but the other was turned off. The guest which was running experienced a fatal corruption of some kind. I was not able to (or perhaps simply ignorant of a way to) resolve the issue without rebooting. I can imagine the guest which was on could lose the data, but I'm unsure why the other one experienced this. Unless it was due to the VirtualBox software issue in the first place ( viewtopic.php?f=7&t=83800 ).
Two legacy .vbox filesizes from 2yrs ago were - 51.4kB and 46.1kB. There were six snapshots, each of which ranged from 2-10GB which seems exceptional to me
I would love to learn how to save and properly backup a guest machine so that I can delete superfluous data. Perhaps it would be best for me to remove all .vdi/.vbox files and start fresh with what you wrote here- viewtopic.php?f=35&t=55003
I did not realize that snapshots were unsafe. I'm grateful for that clarification.

Using machine| Add was successful at getting Virtualbox to set up one of the legacy guests. However there was no matching .vdi for the guest (i.e. 'BilltheGates.vbox' did not have a corresponding 'BilltheGates.vdi'), just the snapshots (is that what you mean when you say they are not safe? Do they overwrite the original .vdi which is necessary for backup?)

[mpack]

Using machine| Add was successful at getting Virtualbox to set up one of the legacy guests. However there was no matching .vdi for the guest (i.e. 'BilltheGates.vbox' did not have a corresponding 'BilltheGates.vdi'), just the snapshots (is that what you mean when you say they are not safe? Do they overwrite the original .vdi which is necessary for backup?)

No, that is not why snapshots are unsafe. They do NOT overwrite older files. What they are is a house of cards: each snapshot is a difference list with respect to the immediately older image. The snapshot is not stand alone. Eventually you build a long chain of dependant snapshots: now introduce a glitch which kills the base image, and poof: all your data is gone.

If you want your data to be safe, just copy the VM folder to secondary storage. Snapshots are not backups in any sense.

Two legacy .vbox filesizes from 2yrs ago were - 51.4kB and 46.1kB. There were six snapshots, each of which ranged from 2-10GB which seems exceptional to me

There's nothing exceptional about those sizes. A snapshot VDI contains all the disk sectors which have changed since you told VirtualBox to preserve the current disk state (i.e. since you created a snapshot). Each snapshot in the chain therefore has the potential - for as long as it remains the current state - to grow to the maximum size of the original disk. So, if you created an 80GB logical drive and 5 snapshots, then it is possible for the data (including the base) to occupy 6 * 80GB = 480GB. This would be highly unlikely, but you can see that 10GB is nothing.



Returning to your problem, I'd be interested in seeing that 50kb .vbox file. If you like, you can zip it up and attach here. At least I can see what media it wants to see.

Perhaps it would be best for me to remove all .vdi/.vbox files and start fresh with what you wrote here- viewtopic.php?f=35&t=55003

I'm not sure what you're saying here. If you have a proper backup that you could apply that FAQ to then certainly I'd recommend doing that. In fact that's pretty much what I suggested in my last post (last sentence).

[JohnShmo]

This clarifies things a bit more regarding snapshots. I'll have to be more careful about that.

I'm happy to share the .vbox file and I look forward to hearing your thoughts on this. http://www.towardzeroimpact.net/BilltheGates.zip

If you have a proper backup that you could apply that FAQ to then certainly I'd recommend doing that.

Unless I'm terribly mistaken, the backup is not complete. I assume this is the cause for the error msg 'disk image files are not accessible.' Am I correct in assuming that the 'disk image' that it's looking for is the .vdi which you mentioned earlier?
"A .vdi is a disk image container. It may contain either a snapshot or a complete image. Usually if it came from the snapshots subfolder and has a weird name such as {1234abcd.xxxx}.vdi then it's a snapshot. Otherwise its the base VDI for the VM. These too are vital files."

Thank you for the continued time.

[mpack]

Ok, below is the hard disk registry section of your .vbox file. In this section the xml lists all of the hard disks that the VM requires, with their numeric IDs and their current location (path and filename).

First however I'll deal with the contents of the zip, which are not what I expected.

<root folder>
...+--- "Bill-The-Gates.vbox"
...<Bill-The-Gates>
......+--- empty
...<Win7>
......+--- empty

If those folders <Bill-The-Gates> and <Win7> are supposed to be VM folders then they should not be empty. The "Bill-The-Gates.vbox" file should be inside the <Bill-The-Gates> folder along with all the other files belonging to the "Bill-The-Gates" VM. Don't attempt to fix this yet, but when you do the method is to unregister the VM, then move the file(s), then re-register using Machine|Add..., i.e. the method from the "Move a VM" FAQ mentioned earlier.

Turning now to the registered media for the "Bill-The-Gates.vbox" VM.

<HardDisks>
...<HardDisk uuid="{6c1cc830-e556-4af2-8cec-a0c42a8fe6fb}" location="/home/bikejunkie/VirtualBox VMs/win7x64.vdi" format="VDI" type="Normal">
......<HardDisk uuid="{80bde535-06cc-45ee-8c72-44317656d0e6}" location="Snapshots/{80bde535-06cc-45ee-8c72-44317656d0e6}.vdi" format="VDI">
.........<HardDisk uuid="{194b9090-8d67-487d-8e6c-734d265b8e29}" location="Snapshots/{194b9090-8d67-487d-8e6c-734d265b8e29}.vdi" format="VDI">
............<HardDisk uuid="{26889bbb-e343-4679-b058-03fb9ac05bdd}" location="Snapshots/{26889bbb-e343-4679-b058-03fb9ac05bdd}.vdi" format="VDI">
...............<HardDisk uuid="{7b943706-02b4-4acc-a853-4a77d36d132a}" location="Snapshots/{7b943706-02b4-4acc-a853-4a77d36d132a}.vdi" format="VDI">
..................<HardDisk uuid="{afcc7746-b03b-478d-85d4-1be33ce8ead6}" location="Snapshots/{afcc7746-b03b-478d-85d4-1be33ce8ead6}.vdi" format="VDI"/>
...............</HardDisk>
............</HardDisk>
.........</HardDisk>
......</HardDisk>
...</HardDisk>
</HardDisks>

I have highlighted the base VDI filename in red. The VM references a base VDI with an unexpected name in an external folder (a folder arrangement that btw is just setting you up for accidents and confusion: it is far better to keep all VM files together). Clearly this VM was not created from scratch: it appears that you had a "Win7x64.vdi" file obtained from somewhere, and you built this VM around the external VDI. A more robust solution would have been to either move that base VDI into the VM folder, or (if you wanted to keep that VDI aside as a template) then clone it and then move the clone into the VM folder.

The VM folder should contain a Snapshots subfolder which in turn should contain all of the snapshot VDIs listed in the above media registry (highlighted in blue). Note that all these paths are relative to the folder containing the .vbox file.

Does that /home/bikejunkie/VirtualBox VMs/win7x64.vdi file still exist?

[JohnShmo]

You've solved the riddle!!
Thank you again. I had put the win7x64 file onto an external hard drive when space was getting tight. I hadn't realized that this guest was referencing it.

Would it be a fair bet now to delete (through Virtualbox not through terminal) the older snapshots and only keep the win7x64 file, log files, and the Bill-the-Gates.vbox? The multiple virtual machines have been consuming a lot of space to due many generations of errors and incorrect attempts to restart an old copy
I also wonder if it would be wise to edit the Bill-the-Gates.vbox file in order to correct the path and keep the win7x64 file in the same folder as you suggested.

[mpack]

Provided you now have a backup of all the files then yes, I would suggest that you now use the GUI to delete all the snapshots from the VM, leaving only the current state, and avoid all use of snapshots in the future. Apart from being much safer, this will also ensure that your VM uses the minimum possible space.

And yes, I would move the base VDI into the VM folder (you MUST leave that until all snapshots are deleted). Unless you are very confident then I don't recommend editing the .vbox file. Instead use the GUI. Once the media registry in the .vbox shows that the base VDI is the only one referenced then you can use File|Virtual Media Manager to release the VDI from the VM, then you can move the VDI into the VM folder, then use the Storage panel in the VM settings to register the media again with that VM.

If you are tight for space on a particular drive then the FAQ referenced earlier tells you how to move a VM. Always treat the VM as a unit, don't mess with individual files if you can possibly avoid it. However I don't normally recommend moving a VM to a removable drive for several reasons: they are usually very slow, not designed for active use (they are really just data transport), and because of the danger of absent mindedly unplugging the drive while the VM is running.

[JohnShmo]

Well the plan didn't go so well. There ended up being an error that caused all Vbox files to fail. But thanks to this thread I do understand more and after enough head-banging on the keyboard I got a guest to work and created an .ova on the host machine and a backup drive