(Yet another) corrupt VMDK file

[fencekicker]

Hi,

So I had a VirtualBox VM and wanted to resize the drive, since it was almost full. I used File > Tools > Virtual Media Manager and tried to resize it from there, from 90 GB to 130. After the resize, I noticed a cryptic exclamation mark before the drive. I tried to start the VM and got this very helpful message:

Code: Select all

VM Name: ubuntu-64bit-dev-vm

Could not open the medium 'C:\chestii\VMs\VirtualBox VMs\ubuntu-64bit-dev-vm\ubuntu-dev-disk1.vmdk'.
VMDK: inconsistency between grain table and backup grain table in 'C:\chestii\VMs\VirtualBox VMs\ubuntu-64bit-dev-vm\ubuntu-dev-disk1.vmdk' (VERR_VD_VMDK_INVALID_HEADER).
VD: error VERR_VD_VMDK_INVALID_HEADER opening image file 'C:\chestii\VMs\VirtualBox VMs\ubuntu-64bit-dev-vm\ubuntu-dev-disk1.vmdk' (VERR_VD_VMDK_INVALID_HEADER).
Result Code:
E_FAIL (0X80004005)
Component:
MediumWrap
Interface:
IMedium {ad47ad09-787b-44ab-b343-a082a3f2dfb1}

Now, I searched for the error and found that most people suggest using vmware-vdiskmanager from either VMWare Workstation or the VDD toolkit. I tried both, but it seems VirtualBox really managed to mess up the VMDK:

Code: Select all

> <path\to>\vmware-vdd-8.0.1\bin\vmware-vdiskmanager.exe -R "c:\chestii\VMs\VirtualBox VMs\ubuntu-64bit-dev-vm\ubuntu-dev-disk1.vmdk"
The virtual disk, 'c:\chestii\VMs\VirtualBox VMs\ubuntu-64bit-dev-vm\ubuntu-dev-disk1.vmdk', is corrupted and cannot be repaired.

Is there anything else that can be tried? I know VMDK is a VMWare format (like people pointed out here: viewtopic.php?f=6&t=65621), but maybe there's something particular about how VirtualBox messes up the file .

[scottgus1]

What Virtualbox version are you using?

FWIW in a quick experiment in 7.0.8, I made a new dynamic vmdk, added it to an XP VM, did the required initialization in XP, made one partition on it, quick-formatted it and put files on it. Then I shut down the VM and expanded the VM.

When I started the VM again, the XP OS asked to re-initialize the disk, and it was back to a no-partition disk again with the data missing.

There might be a bug or something going on.

Meanwhile, unless you have a backup, you might try passing the vmdk through Mpack's CloneVDI and see what errors pop up.

[mpack]

I don't think CloneVDI bothers with the backup grain table, so it shouldn't produce this error. I would go ahead and convert the VMDK to VDI, since VMDK is not the recommended format in VirtualBox.

[fencekicker]

What Virtualbox version are you using?

I have 7.0.6, as far as I can tell.

Meanwhile, unless you have a backup, you might try passing the vmdk through Mpack's CloneVDI and see what errors pop up.

I did try to use VBoxManage to convert the VMDK, but that also failed. Let me take a look at CloneVDI, thanks. Alas, I don't have a backup.

[fencekicker]

Hi,

I've tried CloneVDI 4.01, but didn't get me very far. The "validation result" field says "Source file corrupt - block map contains errors".

[mpack]

In that case the VMDK is dead. It isn't just the backup grain table, it's the primary table too. Sounds like the corruption is extensive.

One thing I don't understand: your OP talks about resizing the drive, but AFAIK VirtualBox 7.x does not have a "resize VMDK" feature: it would have refused to carry out the resize function.

[scottgus1]

AFAIK VirtualBox 7.x does not have a "resize VMDK" feature: it would have refused to carry out the resize function.

The 7.0.6 changelog has this:

Storage: Added support for increasing the size of the following VMDK image variants: monolithicFlat, monolithicSparse, twoGbMaxExtentSparse, twoGbMaxExtentFlat

So it looks like we have a new feature.

Only thing is, my vmdk resize attempt above failed to preserve the data within the vmdk. So there's something wrong somewhere.

[fencekicker]

In that case the VMDK is dead. It isn't just the backup grain table, it's the primary table too. Sounds like the corruption is extensive.

OK, I guess that's life then. I'm still going to try to recover files from it. So far, I've had the most luck with photorec, which seems to look for some file patterns and then stores whatever it finds .

One thing I don't understand: your OP talks about resizing the drive, but AFAIK VirtualBox 7.x does not have a "resize VMDK" feature: it would have refused to carry out the resize function.

Maybe it's a bug then? I loaded another VMDK in the Virtual Media Manager and I'm able to use the "Size" slider below (see attachment). This seems to be the procedure recommended in a few places (for example, here: https://www.howtogeek.com/124622/how-to ... or-vmware/).

[scottgus1]

vmdk resizing is a new feature, see my post above your last.

[fencekicker]

vmdk resizing is a new feature, see my post above your last.

OK, now I saw your post, and re-read your first reply. It sucks when such an easy use case as yours fails right away.

So in your case, the VMDK was accepted by the VM, but the contents were scrambled, right?
In my case, the VMDK was not usable from a VM, but at least I can see some of the (raw) contents. I'll give testdisk a go, maybe it can see some partition boundaries and extract them somehow.

[mpack]

In my case, the VMDK was not usable from a VM, but at least I can see some of the (raw) contents.

Are you working with a monolithic-flat or monolithic-sparse image? Those would be the only variants where I'd consider it practical to recover data from a file with a corrupted block map (aka grain table). In all other variants, and in VDI, a dynamically allocated image effectively has a randomized block order and a working block map is needed to make sense of it. A non-specialist tool may recover file names from the directories inside the filesystem, but the file contents need to be considered with great suspicion.

[fencekicker]

Are you working with a monolithic-flat or monolithic-sparse image? Those would be the only variants where I'd consider it practical to recover data from a file with a corrupted block map (aka grain table).

Hmmm... Running 'less' on the VMDK I see some plain text after the leading data into the file saying:

# Disk DescriptorFile
version=1
CID=b31d4256
parentCID=ffffffff
createType="monolithicSparse"
[... some more lines ...]

So I guess it's monolithic-sparse?

[mpack]

Actually I withdraw my earlier comment, because monolithic-sparse is actually bad. Monolithic means it's a single file, that's ok. Sparse means it's dynamically allocated and you need a grain table to make sense of the block order. Not ok.

I rarely use VMDK so I have to think hard about the unfamiliar terms it uses.

Please see the attachment. This contains a special version of CloneVDI which should plow ahead and try to create the VDI regardless of block map and read errors it gets on the VMDK. The idea is that if it produces garbage then you are no worse off, but I'm hopeful that it might give you a working VDI... with an acceptable level of filesystem errors. (*) I offer absolutely no support or guarantees on this, but do let me know how you get on. I will withdraw the attachment after you have it, because I don't want this getting out.

(*) My reasoning is that VirtualBox will not have deliberately trashed the existing grain table entries, but it may have added entries which are garbage, e.g. address beyond end of file. Since you don't care about the contents of the extended part of the drive this may not matter at all, so we can ignore the errors. Obviously this involves a number of assumptions.

{mod edit: attachment deleted.}

[fencekicker]

[...]
Please see the attachment. This contains a special version of CloneVDI which should plow ahead and try to create the VDI regardless of block map and read errors it gets on the VMDK. The idea is that if it produces garbage then you are no worse off, but I'm hopeful that it might give you a working VDI... with an acceptable level of filesystem errors. (*) I offer absolutely no support or guarantees on this, but do let me know how you get on.[...]

Thanks, I downloaded the zip file. It seems to go ahead with the cloning, let's see what happens.

Thanks again for helping out!

[fencekicker]

So the clone operation finished, and it did create the VDI clone. The disk won't boot, so certainly the MBR (or GPT? I'm not sure what it had) is hosed. I'm going to try to run testdisk on the drive and see if it finds any partitions.

Using sed I was able to recover some text files from the VMDK. So you're saying 'monolithicSparse' means blocks might be scattered in the VMDK, and there's a table pointing to which block is where, right? So maybe if the file is small enough and it fits in a block, that would explain why I was able to recover it?