Reconstruct vbox file?

[HuntStockwell]

I reformatted my hard drive and did a clean install of Win10 during the free upgrade period. I already had my vdi & snapshot files stored on another drive and thought I was safe - whoops, I was unaware of the vbox files. I want to restore the state of my guest (XP on Win10 host). I have created a new vbox file for the vdi. I figure I can just create as many snapshots against the base vdi as I have in the saved snapshot folder to get the vbox file structure correct and replace the UUIDs with the ones from my saved snapshot folder. Unfortunately, when I attempt this process replacing the UUIDs in the vbox file with the snapshot UUIDs based on snapshot creation date & time, I get an error:

Code: Select all

Parent UUID {915e7bdd-fd2c-4c03-be09-10aeca98dfc3} of the medium 'E:\Machines\Testbed\Snapshots\{f21683f7-09a9-4c3b-8e0c-a4d7bd8b00b7}.vdi' does not match UUID {f43b345e-4ff4-4aa6-a652-bf5033788f37} of its parent medium stored in the media registry

I thought I understood from another post I read that the snapshot files stored a reference to parent and/or child files near the beginning of the file but that it was only readable using a hex editor. I downloaded HxD Hex Editor and opened a snapshot, but cannot find anything that looks like a UUID unless I need a certain setting?

Can anyone offer any advice to help me get my guest running again? Thanks in advance.

Currently using VirtualBox 5.1.4
Snapshots created from 2010 on using roughly whatever version was current at the time.

[mpack]

I hope your backup includes the base VDI and all the snapshots, else nothing can be done.

Put all the VDIs together in one folder. Identify the latest snapshot, and clone it using CloneVDI. This will give you a single clone VDI representing the entire snapshot chain. Use the clone in a new VM - not the one you've been messing with.

Tip: when creating a new VM, when you get to the disk creation step you of course choose "Use existing..", however you should move the clone VDI into the (new) VM folder before selecting it. This creates a self contained VM folder.

The .vbox identifies the unique PC, and the file is not recoverable unless you made a backup. The recreated VM will be a completely new PC as far as any guest software is concerned. E.g. if it's a Windows guest then it will require reactivation. Ditto any protected guest apps.

[socratis]

replacing the UUIDs in the vbox file with the snapshot UUIDs based on snapshot creation date & time

You can't go solely based on the time and modification date. Just for completion and since you showed an interest in keeping the snapshots...

It is doable, but you got to be really careful about it. It's not for the faint of heart. You can't just replace the VDI's UUIDs. You have to replace the snapshot UUIDs as well. Open a Command Prompt window. Navigate to the VirtualBox installation directory (typically "C:\Program Files\Oracle\VirtualBox\"). Run:

• VBoxManage showmediuminfo <VDIs>

for every VDI that you have in your snapshots folder, as well as the Base.vdi (in fact you start from it). Each VDI will output the information showing its parent's UUID, its children's UUIDs and the snapshot UUID where it belongs . Example of a Minix VM with two snapshots (Basic, Snapshot 1, Current state):

• VBoxManage showmediuminfo /Users/Shared/VirtualBox/Machines/Minix/Minix.vdi
  UUID:           3cf06c72-af30-48b4-8236-68a022c6bb5f
  Parent UUID:    base
  ...
  In use by VMs:  Minix (UUID: 75dd144a-2452-438a-bccf-69e28d920eaf) [Basic (UUID: 37f370ee-1a0f-4f55-b34a-130c8f98a48a)]
  Child UUIDs:    9a20e11e-b908-413d-9e85-14340955ff54
  
  VBoxManage showmediuminfo /Users/Shared/VirtualBox/Machines/Minix/Snapshots/\{9a20e11e-b908-413d-9e85-14340955ff54\}.vdi
  UUID:           9a20e11e-b908-413d-9e85-14340955ff54
  Parent UUID:    3cf06c72-af30-48b4-8236-68a022c6bb5f
  ...
  In use by VMs:  Minix (UUID: 75dd144a-2452-438a-bccf-69e28d920eaf) [Snapshot 1 (UUID: fc8103d8-5ddc-4a5b-8b68-79948225087a)]
  Child UUIDs:    d568e6bb-5a8b-464d-a02b-586fad7ca9eb
  
  VBoxManage showmediuminfo /Users/Shared/VirtualBox/Machines/Minix/Snapshots/\{d568e6bb-5a8b-464d-a02b-586fad7ca9eb\}.vdi
  UUID:           d568e6bb-5a8b-464d-a02b-586fad7ca9eb
  Parent UUID:    9a20e11e-b908-413d-9e85-14340955ff54
  ...
  In use by VMs:  Minix (UUID: 75dd144a-2452-438a-bccf-69e28d920eaf)

I just color-coded the VDI UUIDs, not the snapshot UUIDs which have to match as well. If you really, really want to do it, as I said it's doable, but you have to be really, really careful. And have a backup.

Personally, I would go with mpack's advice and use CloneVDI.

[HuntStockwell]

Thank you both for the input. I thought I would try socratis' method first so I would get my ability to roll back changes again. I got a strange result when running vboxmanage-

vboxmanage showmediuminfo "d:\old drive\data\virtualboxdrives\testbed.vdi"
UUID: f43b345e-4ff4-4aa6-a652-bf5033788f37
Parent UUID: base
Location: D:\Old drive\Data\VirtualBoxDrives\Testbed.vdi
...
In use by VMs: WinXP Test (UUID: f032fc7a-52b0-4efb-ad65-d206d387af7e) [Snapshot 1 (UUID: 2a31a19b-bc4f-4e56-8d89-44b2ea929334)]
Child UUIDs: 2a31a19b-bc4f-4e56-8d89-44b2ea929334

vboxmanage showmediuminfo "e:\machines\testbed\snapshots\{2a31a19b-bc4f-4e56-8d89-44b2ea929334}.vdi"
UUID: 2a31a19b-bc4f-4e56-8d89-44b2ea929334
Parent UUID: f43b345e-4ff4-4aa6-a652-bf5033788f37
State: inaccessible
Access Error: Parent UUID {915e7bdd-fd2c-4c03-be09-10aeca98dfc3} of the medium 'E:\Machines\Testbed\Snapshots\{2a31a19b-bc4f-4e56-8d89-44b2ea929334}.vdi' does not match UUID {f43b345e-4ff4-4aa6-a652-bf5033788f37} of its parent medium stored in the media registry
...
Location: E:\Machines\Testbed\Snapshots\{2a31a19b-bc4f-4e56-8d89-44b2ea929334}.vdi
...
In use by VMs: WinXP Test (UUID: f032fc7a-52b0-4efb-ad65-d206d387af7e) [Snapshot 2 (UUID: 7f0d933e-4c4d-4f04-a74c-db5924a5f859)]
Child UUIDs: 7f0d933e-4c4d-4f04-a74c-db5924a5f859

For the rest of my snapshots, the parent-child references all match. This VM was working before I reformatted my hard drive and didn't save the vbox files.

An "dfc3" snapshot does not currently exist anywhere on my system. Any suggestions? Can I use HxD Hex Editor to change that one reference within the snapshot? Thanks.

[socratis]

Something's rotten in Denmark. What happens if you use CloneVDI? Does it work or does it error on you?

An "dfc3" snapshot does not currently exist anywhere on my system. Any suggestions? Can I use HxD Hex Editor to change that one reference within the snapshot? Thanks.

I wouldn't ever contemplate this without a full backup of everything. After that, you're on your own. Literally. I've never even attempted it.

One thing that might help you (and I know it's an old post) is the following: All about VDIs.

Good luck. You're going to need it...

[HuntStockwell]

I moved the source(base) VDI into the snapshot folder and ran CloneVDI.

If I listed the Source as the base VDI, a clone was successfully created, but it appears to be just the contents of the base VDI when the new VM successfully starts.
If I list the Source as any of the snapshots, I receive an error that the Source is a differencing image, which cannot be cloned currently. I also noted that while the Validation passed for each Source selection, the filesystem cannot be determined ("None") when any of the snapshots are used as the source except for 1 ("NTFS").

I'm feeling like this VM is pretty hosed at this point. Unless you have other ideas, it would seem that I'm left to try editing the hex to force that one snapshot to match the parent UUID (after making a copy). Unfortunately, I still can't figure out where the UUIDs are stored in the snapshots. It looks like it should be near the beginning if I'm understanding the content on the All About VDIs thread, which I had already read but looked at again- thanks.

CloneVDI can read the UUIDs when I click the Header button, so mpack must know how to find them.

[mpack]

It would help perhaps if you followed the instructions I gave in my first post. You don't clone the base VDI.

[HuntStockwell]

mpack,

Sorry, I guess I was not clear in my explanation. I did try what you suggested. I will make another attempt:

Code: Select all

CloneVDI Attempts
Attempt    Selected source    Validation Result    File System    Result after clicking Proceed
1          Snapshot 6         Ok                   None           Error "Source is a differencing image..."
2          Base VDI           Ok                   NTFS           Success, but VM is only a clone of the base VDI
3          Snapshot 1         Ok                   None           Error "Source is a differencing image..."
4          Snapshot 2         Ok                   NTFS           Error "Source is a differencing image..."
5          Snapshot 3         Ok                   None           Error "Source is a differencing image..."
6          Snapshot 4         Ok                   None           Error "Source is a differencing image..."
7          Snapshot 5         Ok                   None           Error "Source is a differencing image..."

[socratis]

@HuntStockwell
I have a (really) bad feeling about this. I've actually had it since you posted the output of the "VBoxManage showmediuminfo" and it showed that something was wrong from the very first snapshot. What I didn't like at that point is that your base VDI was stored outside the default folder structure, in:

        "D:\old drive\data\virtualboxdrives\testbed.vdi"

while the snapshot(s) are stored in another location:

        "e:\machines\testbed\snapshots\{2a31a19b-bc4f-4e56-8d89-44b2ea929334}.vdi"

That tells me that there has been a human intervention that manually bypassed all the defaults suggested by VirtualBox.

Now, I think that you don't quite understand what a snapshot is, so let me quote a passage which I find quite comprehensive and does a nice job of explaining differencing disks and snapshots (which are based on the concept of differencing disks) and you'll pretty easily figure out why they can be really bad.

When a virtual disk is first created for a new virtual machine, it is considered as the base disk for the guest - data for the guest is read from and written to that disk image.

The differencing disk records changes sector-by-sector to the whole disk image, not changes to any file in the disk. VirtualBox does not know what file system is employed on the disk image and therefore can not access any individual file of/on the disk image; only the guest OS is aware of that information.

First SnapShot creates a differencing disk for read/write access while the base disk becomes read-only - as the guest modifies its data, the data is written to the differencing disk and the base disk is untouched.

Second SnapShot creates another, new, differencing disk for read/write access while the first differencing disk becomes read-only along with the base disk.

Subsequent SnapShots create additional differencing disks, with the preceding differencing disk joining the hierarchy (pecking order/chain) of read-only disks.

Keep in mind that access to/from the virtual disks is sector-by-sector, not file-by-file.

When the guest requests that a sector be read, the latest SnapShot is read first. If the sector is not found there (Sector-Not-Found is returned), the next SnapShot in the chain (youngest to oldest), until the base virtual disk is reached. Then the sector on/in the base virtual disk is either read or Sector-Not-Found is returned.

So, even if you manually change the UUIDs of the snapshots, even if you trick VBoxManage and CloneVDI, if there is no consistency in the order of the snapshots, at best what you'll end up doing is mixing sectors. Parts of file A with file B, wiping parts of a file or "inventing" parts of a file. At its best scenario you might think of it as having a bunch of bad sectors all at the same time (when in fact they won't even be bad sectors).

Think of it as taking random sectors of my hard drive and mix them with random sectors of your hard drive. I believe "scottgus1" put it at its best in the thread viewtopic.php?f=6&t=79069 (you should read it BTW):

Expect some Frankenstein moments.

[mpack]

The "source is a difference image" error from CloneVDI means that you do not have the complete snapshot chain. Since none of the snapshots work I'd guess that you are missing an element from very early on, or perhaps the base VDI itself is a clone and not the original file: clones have the wrong UUIDs and will not be suitable for use in the snapshot chain.

> The "source is a difference image" error from CloneVDI means that you do not have the complete snapshot chain. (Take 2).
In fact you can check this yourself using CloneVDI. Open up a snapshot, click the "Show Header" button. The snapshot reports its own UUID and the UUID of the parent VDI (links connect backwards in time, not forwards). You should be able to find another VDI whose own UUID matches the parent UUID of the child, and which perhaps has a new parent UUID link to another VDI. If the entire chain is present then you should be able to construct a chain of parent links back to the base VDI. If you don't have this chain then you don't have a disk image. If you don't have this chain then CloneVDI says "source is a difference image" (since generally this means that user is attempting to clone a standalone snapshot - and of course there is no such thing).

[HuntStockwell]

@socratis
Yes, I chose from the very beginning to store the base VDI files in another location from the default, a "data" drive. Snapshots were stored in the default location up until I reformatted to install Win10. I knew the basics of what snapshots were before, but it wasn't until I began reading this forum that I understood how they worked. The quote from ChipMcK explains it very well.

@mpack
I understand what you are telling me that it seems my chain is incomplete. If that is the case, then the data I had within the VM is unable to be accessed. I would still like to try to access the VM, not for permanent use, but to get some of the data that was contained inside back, even if there is potential frankensteining. Can you tell me where I can find the parent UUID within a snapshot so that I can change it? Maybe the resulting chain doesn't boot, maybe it does. Maybe my data is all corrupt. I'd like to see what I can get out, if anything.

[mpack]

Have you verified that a member of the chain is missing? Anyway the command is "VBoxManage internalcommands sethduuid <vdi-filename> [<uuid>]".

As you say, forcing two VDIs to link together may give you a disk that is garbage. Make a backup first.

[HuntStockwell]

mpack, looking at the parent UUID references in each snapshot, it does seem like there is a break in the chain. I cannot explain it, since the VM worked before I reformatted the HD. I suppose it is possible that somehow, a snapshot at some point was stored elsewhere and I missed backing it up.

I have now attempted twice to change parent UUID references to make a complete chain. As it turns out, I have 2 snapshots that reference the same parent UUID - again I do not know how that could have happened. In attempt 1, I changed the UUID in snapshot E:\Machines\Testbed\Snapshots\{2a31a19b-bc4f-4e56-8d89-44b2ea929334}.vdi from the aforementioned {915e7bdd-fd2c-4c03-be09-10aeca98dfc3} to the base VDI UUID {f43b345e-4ff4-4aa6-a652-bf5033788f37}. Then I ran your CloneVDI program to make a single cloned VDI. The resulting VM would begin to boot, load WinXP, go to the screen with blue background and terminate just before it should have shown the login dialog.

I thought that maybe it failed because of the other potential snapshot that I had left out of the chain, so I restored my snapshot folder from the backup I made so that none had any changes. For attempt 22, I changed the UUID in snapshot E:\Machines\Testbed\Snapshots\{2a31a19b-bc4f-4e56-8d89-44b2ea929334}.vdi from the aforementioned {915e7bdd-fd2c-4c03-be09-10aeca98dfc3} to what may be the Snapshot 1, {f21683f7-09a9-4c3b-8e0c-a4d7bd8b00b7}.vdi. This snapshot also lists "dfc3" as its parent. I changed it to the base VDI UUID {f43b345e-4ff4-4aa6-a652-bf5033788f37}. Then I ran your CloneVDI program to make a single cloned VDI. The resulting VM would begin to boot, load WinXP, go to the screen with blue background and terminate just before it should have shown the login dialog.

After attempt 2, I thought to check some log files. The VBoxSVC.log file in the .VirtualBox folder contains the following error lines:

Code: Select all

00:00:07.313789          ERROR [COM]: aRC=VBOX_E_OBJECT_NOT_FOUND (0x80bb0001) aIID={edba9d10-45d8-b440-1712-46ac0c9bc4c5} aComponent={ExtPackManagerWrap} aText={No extension pack by the name 'Oracle VM VirtualBox Extension Pack' was found}, preserve=false aResultDetail=0
00:00:07.535139          ERROR [COM]: aRC=VBOX_E_IPRT_ERROR (0x80bb0005) aIID={b2547866-a0a1-4391-8b86-6952d82efaa0} aComponent={SessionMachine} aText={Saved screenshot data is not available (VERR_NOT_SUPPORTED)}, preserve=false aResultDetail=0

The VBox.log file in the \VirtualBox VMs\WinXP Clone\Logs folder seems to indicate that the guest requested to power off just past the 20 second mark. Here's an excerpt:

Code: Select all

00:00:18.018019 DSound: Guest source 'Line In' is using host recording device with GUID '{Default device}'
00:00:18.021589 OHCI: Software reset
00:00:18.021811 OHCI: USB Reset
00:00:18.079417 OHCI: USB Operational
00:00:18.383763 Display::handleDisplayResize: uScreenId=0 pvVRAM=00000000098b0000 w=1152 h=864 bpp=32 cbLine=0x1200 flags=0x1
00:00:18.383809 GUI: UIFrameBufferPrivate::NotifyChange: Screen=0, Origin=0x0, Size=1152x864, Sending to async-handler
00:00:18.383866 GUI: UIMachineView::sltHandleNotifyChange: Screen=0, Size=1152x864
00:00:18.383881 GUI: UIFrameBufferPrivate::handleNotifyChange: Size=1152x864
00:00:18.384001 GUI: UIFrameBufferPrivate::performResize: Size=1152x864, Directly using source bitmap content
00:00:18.384276 VMMDev: Guest Log: VBoxDisp[0]: VBVA enabled
00:00:18.384291 VBVA: InfoScreen: [0] @0,0 1152x864, line 0x1200, BPP 32, flags 0x1
00:00:18.384297 Display::handleDisplayResize: uScreenId=0 pvVRAM=00000000098b0000 w=1152 h=864 bpp=32 cbLine=0x1200 flags=0x1
00:00:18.384313 GUI: UIFrameBufferPrivate::NotifyChange: Screen=0, Origin=0x0, Size=1152x864, Sending to async-handler
00:00:18.396387 GUI: UIMachineView::sltHandleNotifyChange: Screen=0, Size=1152x864
00:00:18.396405 GUI: UIFrameBufferPrivate::handleNotifyChange: Size=1152x864
00:00:18.396423 GUI: UIFrameBufferPrivate::performResize: Size=1152x864, Directly using source bitmap content
00:00:19.266461 PIIX3 ATA: LUN#0: IDLE IMMEDIATE, CmdIf=0xe7 (-1 usec ago)
00:00:19.266492 PIIX3 ATA: LUN#0: aborting current command
00:00:19.562199 OHCI: USB Suspended
00:00:20.564172 VMMDev: Guest requests the VM to be turned off
00:00:20.564215 Changing the VM state from 'RUNNING' to 'POWERING_OFF'
00:00:20.564230 ****************** Guest state at power off for VCpu 0 ******************

Any ideas what is going on or is this my Frankenstein?

[HuntStockwell]

I should probably mention, in case you didn't already figure out, that I used the command VBoxManage internalcommands sethdparentuuid <file> <UUID> on my snapshots. I don't think it makes sense that it would, but, since I do not know the inner workings, maybe it makes a difference that I used it instead of the command you suggested.

[socratis]

I won't comment on your endeavors, as I believe it's an exercise in futility . A good exercise nonetheless. I do them all the time. For example, you made me look up the "sethdparentuuid". That's a good thing™.

Now, about one thing you said, I believe I might know the answer:

I have 2 snapshots that reference the same parent UUID - again I do not know how that could have happened.

I'll refer you to a picture from the User Manual, Chapter 1.10 Snapshots:



You can see that "SP2" and "SP3" have the same parent (XP Professional), as well as "Internet Explorer 6" and "Internet Explorer 7" (parent "SP2").

I'm not sure if this is what you had in your setup, or even if you had it at all in your setup, I'm simply pointing out how can two snapshots have the same parent.

Good luck...