[Solved] VERR_VD_VDI_INVALID_HEADER... :-/

[Gimni]

Last week, I got a memory error with a VirtualBox 5.1.22 Windows host and a running Debian Jessie VM.

I got no time to make a screenshot of that memory error but when I tried to restart my VM, I got a VERR_VD_VDI_INVALID_HEADER error. Following advices on this forum, I made a copy of the VDI file, tried to restart the VM but with no success.

I do have backups but unfortunately, I increased the VDI size since that last backup and made important changes to the VM that I'd like to recover. Yep... I know. :-/

Following mpack's recommandations, I've enclosed the first 64k of the VDI. The actual current size of the VDI is 18.663.604.224 bytes. Unfortunately, I can't remember the exact intended logical/maximum size of the disk; I think it was 25GB. :-/

Thus, if anybody (mpack ? has some advice, I'd be eternally grateful

TIA, G.

[mpack]

Thanks for the other info, but 64K from the VDI is far from enough I'm afraid. I need the first 2MB plus 1K, i.e. the first 2098176 bytes. This is mostly zeroes so it will compress quite well.

It's unusual that someone has made at least one backup of their "super important" data, so congrats on that. It may be useful to include the first 4K from that in the zip. It may allow me to recover things I often can't, such as the UUIDs (at the moment I don't know how much will need recovering).

[Gimni]

Here is the 2MB+ extract.

G.

[mpack]

Ok, I have a recovery procedure for you. Please perform the following steps very carefully, because I am not there to look over your shoulder and correct you.

As always I will assume that any hex editor you use has the same capability as frHed, the one I use.

1. Back up your existing VDI, if you have not already done so. It is important that nothing we do here can make matters worse.
2. Find your old backup VDI (the one from before the resize, not the one you just made), copy the first 512 bytes from this old VDI to your damaged VDI. DO NOT STOP THERE - WE'RE NOT DONE BY A LONG SHOT. You can now close the old backup VDI, we're done with it.
3. Go to offset 384 (0x180) in the repair vdi. Change the four bytes at that location to C0 78 00 00.
4. Go to offset 388 (0x184) in the repair vdi. Change the four bytes at that location to 85 45 00 00.
5. Save changes and exit the hex editor. The VDI header should now be repaired. Hopefully the image itself is fine.

The repaired VDI should be a plug-in replacement.

[Gimni]

OK, this is strange...

1. I followed your procedure: made a copy (jessie-edited.vdi) of the _broken_ vdi (jessie.vdi), copied the first 512 bytes from the _backup_ vdi (jessie-201705.vdi) and changed the 8 bytes you indicated.

2. in VirtualBox, I removed the jessie.vdi disk from the vm then tried to attach the jessie-edited.vdi file instead.

I got this error message: "Could not get the storage format of the medium 'D:\...\VirtualBox VMs\jessie\jessie-edited.vdi' (VERR_NOT_SUPPORTED)."

3. I tried the same with an unedited copy (jessie-copy.vdi) of the broken vdi and got the same VERR_NOT_SUPPORTED error message.

4. tried again steps 1 and 2 and got the same VERR_NOT_SUPPORTED message.

This is where it's weird:

5. I renamed the broken vdi to jessie-orig.vdi, renamed jessie-edited.vdi to jessie.vdi

6. I succesfully attached that jessie.vdi to the VM, started it and got the VERR_VD_VDI_INVALID_HEADER error.

7. I tried to attach the jessie-orig.vdi and got a VERR_NOT_SUPPORTED... :-/

So it seems:

- I could succesfully attach a vdi only when it's named "jessie.vdi", whether it was the broken, copied or edited vdi;

- if it was not named jessie.vdi, I get a VERR_NOT_SUPPORTED error;

- and whatever the vdi attached as jessie.vdi, I got the VERR_VD_VDI_INVALID_HEADER error when starting the VM.

I also tried all those steps by making sure to release the jessie.vdi in the Virtual Media Manager before re-attaching it. No more success.

Then I released and _removed_ the jessie.vdi from the Virtual Media Manager. Bad idea: now, I _always_ get the VERR_NOT_SUPPORTED error whatever the vdi or vdi filename I try to attach.

Grumpfff... :-q

[Gimni]

Btw, I can attach and start the VM with the backup VDI.

You'll find attached the first 4kB of that backup vdi, I forgot to send it earlier.

G.

[mpack]

There is no guarantee that corruption was restricted to the header. E.g. if the block map was also corrupted then repair may not be possible.

What error (if any) does CloneVDI give for the repaired VDI?

Also, is there any part of the story that you have failed to mention? E.g. I assume you are not using encryption? (I ask because there was extra stuff in the header that I took to be corruption, but it's worth asking).

[mpack]

Oh b****r. I just noticed that there was a step I forgot to consider: the old disk size was different hence you also need to patch the disk size field in the edited header.

And, there was a mistake in my estimation of the original logical disk size. A new procedure is outlined below. Changes are in red.

1. Back up your existing VDI, if you have not already done so. It is important that nothing we do here can make matters worse.
2. Find your old backup VDI (the one from before the resize, not the one you just made), copy the first 512 bytes from this old VDI to your damaged VDI. DO NOT STOP THERE - WE'RE NOT DONE BY A LONG SHOT. You can now close the old backup VDI, we're done with it.
3. Go to offset 368 (0x170). The 8 bytes from that address should be: 00 00 80 8C 07 00 00 00.
4. Go to offset 384 (0x180) in the repair vdi. Change the four bytes at that location to C8 78 00 00.
5. Go to offset 388 (0x184) in the repair vdi. Change the four bytes at that location to 85 45 00 00.
6. Save changes and exit the hex editor. The VDI header should now be repaired. Hopefully the image itself is fine.

The repaired VDI should be a plug-in replacement.

Weirdly, the edited 512 byte header you gave me already has the correct values for the eight disk size bytes (at offset 368).

[Gimni]

Hello, mpack.

Sorry for the late reply. I have just been able to apply the latest instructions you gave me.

And I recovered my broken VDI...

Thanks a lot for your help !!!

If you ever come to Belgium (for the FOSDEM, maybe ? , consider that you have an unlimited beer credit...

Thanks again, G.

[socratis]

I'm sure mpack is laughing as a happy camper would. And not about the beer (mostly)...
Marking this as [Solved].

[cableghost2]

I continue to receive this error message, VERR_VD_VDI_INVALID_HEADER on my VM's about every 3-4 days. Although I make backups, it's frustrating, constantly having to restore the image. I even went as far as creating a new image...same result.

What is the root cause of this? What settings/other can be made so the likelihood of this happening is minimized?

Scott

[mpack]

Nothing in VirtualBox causes it, I can tell you that much - because if it was otherwise then everyone would have the problem, and in my 10 years on here you are the first to report a recurrent VDI corruption problem. There must be some problem on your host, or you aren't restoring the actual VDI which has the actual problem.

In every other reported case there has been some underlying event that caused widespread problems, for example suffering a power loss while software (including the VM) was running. Or a memory error such as the OP reported.

[cableghost2]

In every other reported case there has been some underlying event that caused widespread problems, for example suffering a power loss while software (including the VM) was running. Or a memory error such as the OP reported.

After further testing, seems I have a bad disk. Will replace and try again. I'll report back if same behavior.