Weird Problem with a VMDK file

[HelterSkelter]

Hello everyone,

This is my first post here, usually I tend to resolve my own problems but this one is particularly painful and of utmost importance since it contains all my thesis work, so your feedback is greatly appreciated

Details:
- Running Virtualbox 4.3.26 r98988 on Windows 7 x64.
- I am working with a VMDK file, operating system is Ubuntu (32 bit). Total size is 21.5 GB (because I don't need more than that), and there is enough free storage in the system. I've included a screenshot of the VBoxManager that shows the specs in more detail.
- Kindly find uploaded a zip file containing the contents of my log folder (Vbox.log, log.1,2,3 and VboxStartup.log).

The Nature of the problem:
I was using this environment to run simulations for a thesis i'm working on using the Contiki platform, everything was working just perfectly until a couple of weeks ago. Normally one simulation would take about 8-9 hours to finish so I'd leave it open throughout the night and in the morning check the results.
HOWEVER, one day I woke up and I find the Ubuntu system frozen on the desktop screen and nothing works. I had to shut down the machine using the VM Manager and start again. No problem. This happened again a couple of times the past two weeks.

Yesterday, while attempting to start the machine again, it doesn't boot. For some reason it runs "fsck" and starts checking the disk, I've uploaded some screenshots - since the website won't let me post URLs just yet so bear with me - they are called "error1", "error2", "error3".
However, at one point it just stops (see the "CRASH" screenshot). And it crashes back to the desktop... Attempting to fix them manually or otherwise yields the same result.

I tried fixing the issue using Gparted, I booted from a CD image and ran it, used the "Check" option on the hard disk, about 55% of the way, without any warning or anything the VirtualBox Manager crashed, not just the virtual machine I was using but the entire program back to my desktop... Weird right ? Now I can't access this vmdk file in anyway and it's very important that I do because weeks of simulations are on there !

Any ideas ?
Thanks

[Smackey's dad]

Here is what I would suggest:
Create a backup of the raw file
Create a new VM and attach this hard disk as secondary drive and try to recover it

If you suspect the file itself is corrupted (not the file system inside) then
Attempt a clonehd
May have to hand massage the file - not easy and may not work

In future backup your VMs daily.

[HelterSkelter]

Hi,

I tried making a new VM, assigned an IDE primary master disk, and I attached the corrupted file in the SATA controller. The system boots and after a few seconds the Virtualbox crashes again back to the desktop....

[Smackey's dad]

Suggest reinstalling VirtualBox on the host. Make sure you uninstall completely first. Your VMs will not be affected by VB uninstall. Always good to have backup in any case.

[mpack]

Moving this topic to "Windows Hosts" since the issue clearly has little to do with the guest.

It smells like you have a heavily corrupted VMDK file on the host. Trying "VBoxManage clonehd" was probably pointless, since VBoxManage uses the same I/O functions as the VM, and hence will most likely crash as well.

Reinstalling the VirtualBox software is pointless too. That will do nothing at all for a corrupted VMDK.

Dare I ask what backups you've made of this important thesis work?

Hopefully you weren't using snapshots?

Running fsck on it may have been a mistake too. It's possible this would corrupt the disk further. When something like this happens it's best to make a another backup of the damaged files before attempting any repairs, so at least things can't get worse.

You might want to give CloneVDI a try. I'm not hopeful this will work, but the error message it gives might give us further insight. Possibly it will recover the disk enough that you can mount it in another VM and recover some files.

[HelterSkelter]

I tried running cloneVDI, around 47% it gave me this error: "Got OS error 0 when reading from the source file".

Dare I ask what backups you've made of this important thesis work?

Since I wrote most of the code I know how it works and I have a copy of it backed up on the cloud along with my important thesis documents, however, the programs and scripts I was running were specific to Contiki OS and some applications in it, and that's where most of my "data" is. I to access these raw files, they are just some numbers and images but to get each of them it required a simulation of about 10-11 hours.. I don't want to repeat weeks of work that's why I want to retrieve the data I already have !

[mpack]

Ah, I recognize that error, having encountered it myself just a few days ago when testing something for another user - I really should get around to providing something more readable.

It really did get OS error zero: this means that the read process reached "end of file" unexpectedly (with no hard error), meaning the file has been truncated somehow, perhaps by being moved via a FAT partition (FAT having a 4GB file size limit), or just copying the file to a full partition.

What size is the file? The size might give a clue as to the reason.

Regardless, VirtualBox can't have caused this - it must have been something you did when copying or manipulating host files in some way. Files can't become truncated in use - something modified the file outside of VirtualBox.

I'm afraid this is fatal - you need the original uncorrupted file, or a full backup.

[HelterSkelter]

It's about 21 GB.
22,479,310,848 bytes to be exact

[mpack]

Is that the exact current size? Not the "size on disk"?

If the former then it's a suspiciously round number, i.e. exactly divisible by 4K (the Windows cluster size). That argues that you tried to copy the file onto a nearly full partition and didn't notice when you ran out of space. If you still have the original file you copied from then that would help.

[mpack]

One thing does puzzle me. In your first post you said that you had limited the file to 21GB, whereas the read error at 47% implies that this host file was originally twice this size (physically), and who knows what the logical size was. Obviously, some wrong information was introduced somewhere.

[HelterSkelter]

I'm not sure I understand your questions. This might help

#Partition
Size: 22,743,613,440 bytes
Free Space: 60,686,282,752 bytes
File system: Linux Ext
Active partition: Active
Start Sector: 2048
Total Sectors: 44,421,120

#Physical Disk
Capacity : 22,479,310,848 Bytes
Total Sectors: 43,904,904
Bytes per sector: 512
Serial number: none

# Disk DescriptorFile
version=1
CID=42c7364a
parentCID=ffffffff
createType="monolithicSparse"

# Extent description
RW 45056000 SPARSE "COOJA.vmdk"

# The disk Data Base
#DDB

ddb.virtualHWVersion = "4"
ddb.adapterType="ide"
ddb.geometry.cylinders="16383"
ddb.geometry.heads="16"
ddb.geometry.sectors="63"
ddb.geometry.biosCylinders="1024"
ddb.geometry.biosHeads="255"
ddb.geometry.biosSectors="63"
ddb.uuid.image="a2b3d334-c443-422a-b482-8bb09ba643ae"
ddb.uuid.parent="00000000-0000-0000-0000-000000000000"
ddb.uuid.modification="6e4ee395-2a36-47aa-ad85-8ba130667c15"
ddb.uuid.parentmodification="00000000-0000-0000-0000-000000000000"
ddb.comment=""

[mpack]

Well, the third line of your quote seems to highlight my question. How can you have 56GB free space on a 21GB (total) partition? How exactly did you go about limiting the growth of this drive?

p.s. The VMDK size descriptor seems ok for a 21.5GB drive.

[HelterSkelter]

I know the data doesn't make sense to me either...

Originally it was a simply 9 GB hard drive, I extended it to 21 GB using the modifyhd function in the Vboxmanager tool and then of course allocated properly with Gparted. So I just expanded from 9 to 21 GB, didn't really shrink it or anything like that.

[mpack]

Originally it was a simply 9 GB hard drive, I extended it to 21 GB using the modifyhd function in the Vboxmanager tool

I don't see how that can be true. "VBoxManage modifyhd" does not and never has supported resizing of a VMDK format drive - only VDI and VHD formats are supported for that operation.

[HelterSkelter]

I should have provided more information, what I did is basically is :
1. clonehd into a VDI format
2. modifyhd (the modified VDI format) --resize 22000 (22 GB)
3. clonehd back into the VMDK format
4. use gparted to allocate space