[Solved] VDI Invalid Header

Discussions related to using VirtualBox on Windows hosts.
aquemy
Posts: 4
Joined: 5. Jan 2018, 16:16

Re: [Solved] VDI Invalid Header

Post by aquemy »

Hello again,

I ran to the closest store to buy a 1To HD, used CloneVDI and I am more than happy to announce it worked!
There is no word to express my gratitude because you basically saved me from a very difficult situation.

This VM was used to run experiments for a scientific paper. The result data are not important by themselves because I have everything to reproduce them. However, it would take weeks to obtain the results. Meanwhile the paper has been accepted in one of the most prestigious conference in my field and gained an award, under some modifications and some minor adjustments to submit before... next week. So no time to reproduce the experiments...
On top of that, the heavy modifications of the paper were all on the VM, and I did not push any of those modification to my git repo.

Without your work and fast answers I would have had to withdraw the paper and lost a unique opportunity. Thank you so much.
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: [Solved] VDI Invalid Header

Post by mpack »

I'm glad it worked for you, but if I might suggest: if the data is important then proper backups are essential, so I would always design the VM in such a way to make backups easier, mainly by making the VM a more easily manageable size and avoiding snapshots. If you wanted to you could probably shrink the partition down to something like 64GB say (*), then compact the drive using CloneVDI. CloneVDI reduces the actual size of the file, while shrinking the partition (which you should do first) ensures that it can't grow back.

(*) Using GParted or DiskPart.
aquemy
Posts: 4
Joined: 5. Jan 2018, 16:16

Re: [Solved] VDI Invalid Header

Post by aquemy »

The data were not so important (except temporarly for this exact period of time). The only important things were the paper by itself and it is totally on me not to have push to my git distant repository. Usually I have this image specially prepared to start experiments out of the box by deploying some config from different git repo and that's it.

However, I checked the size with du -h and it certainly does not sum up to 279 GB.

Code: Select all

df -h
Filesystem                    Size  Used Avail Use% Mounted on
udev                           12G     0   12G   0% /dev
tmpfs                         2,3G  9,2M  2,3G   1% /run
/dev/mapper/lubuntu--vg-root  985G   60G  876G   7% /
tmpfs                          12G   55M   12G   1% /dev/shm
tmpfs                         5,0M  4,0K  5,0M   1% /run/lock
tmpfs                          12G     0   12G   0% /sys/fs/cgroup
/dev/sda1                     236M  220M  3,7M  99% /boot
cgmfs                         100K     0  100K   0% /run/cgmanager/fs
tmpfs                         2,3G   40K  2,3G   1% /run/user/1000
What may causes the fact the VDI file is that big? Does it mean at one point of the existence of this VM it was 279GB data on it?
Anyway, I'll finish this work, delete the VM and deploy a new one for next experiments.

Thanks again!
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: [Solved] VDI Invalid Header

Post by mpack »

The size of a dynamic VDI is related to the number of non-zero sectors, not the size of current files.

That's what CloneVDI's compaction does btw: it discards data from the drive which is stored in sectors not currently mapped to any guest file. VirtualBox doesn't do that because VirtualBox doesn't know anything about guest file systems. VirtualBox is simulating the hardware of a disk drive and only cares about sectors and any data they contain. Deleting a file does not free up the sectors since sectors can't be unwritten, and that's all that VirtualBox knows about. Put another way, dynamic VDIs never shrink.

In addition, seeing that it's a Linux guest, you have probably not taken the swap partition into account. That too contains a lot of sectors, normally some small multiplier of the amount of RAM you gave to the VM.
Adan
Posts: 1
Joined: 22. Feb 2018, 18:13

Re: [Solved] VDI Invalid Header

Post by Adan »

Hi mpack,

I'm facing the same VDI: invalid header issue :(

I'm sending you the first 2097152 bytes as requested. The VDI size in bytes is: 54527000576

Thanks for your help in advance.
Attachments
adan.bin.zip
(78.14 KiB) Downloaded 6 times
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: [Solved] VDI Invalid Header

Post by mpack »

mpack wrote:Download CloneVDI and run it. The repair procedure is automated now.
Mariano0
Posts: 18
Joined: 23. Feb 2018, 04:16

Re: [Solved] VDI Invalid Header

Post by Mariano0 »

What is the reason why the header is corrupt? I am interested in knowing why I want to store important data and I am afraid that they are not safe in a virtual machine.
andyp73
Volunteer
Posts: 1631
Joined: 25. May 2010, 23:48
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Assorted Linux, Windows Server 2012, DOS, Windows 10, BIOS/UEFI emulation

Re: [Solved] VDI Invalid Header

Post by andyp73 »

Mariano0 wrote:I want to store important data
Treat it the same way as you would important data on physical hardware and make sure it is backed up. Make sure that your backup / restore procedure works in advance rather than finding out that it doesn't when something catastrophic has taken place.

You could also consider backing up the whole of the virtual machine as well. Read the tutorial Moving a VM but read it as backing up rather than moving.

-Andy.
My crystal ball is currently broken. If you want assistance you are going to have to give me all of the necessary information.
Please don't ask me to do your homework for you, I have more than enough of my own things to do.
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: [Solved] VDI Invalid Header

Post by mpack »

Mariano0 wrote:What is the reason why the header is corrupt?
That I can't answer. To me it looked like deliberate damage, e.g. from malware. I'm not aware of any VirtualBox process which would write "USUSUSUSUSUS" to a file.
NeedHelp1
Posts: 8
Joined: 13. May 2021, 20:06

Re: [Solved] VDI Invalid Header

Post by NeedHelp1 »

mpack wrote:
mpack wrote:Download CloneVDI and run it. The repair procedure is automated now.
Please, help me!
This tool cant help me, I'm trying to recover snapshot header. Tool says
"Source is a different image. Sorry, this tool cannot clone these (for now)
I have corrupted vdi snapshot file and worked backup (but it old and different size, I'm already tried copy header, but clonehd still cant recognize volume (using this tutorial mkthakral. com /2015/11/08/fix-vdi-with-corrupted-header )


Here is my first 2 098 177 bytes of vdi file ( I PMed archive password to you)
noname.7z
(40.14 KiB) Downloaded 2 times
current VDI (dynamically allocated) snapshot size is (23 880 269 824 bytes) = size from windows explorer



UPD: I can succesfully view files and file system structure i.imgur. com/7h9ZVZ4.png , using great tool DMDE, but it will be better recover my machine (I'm also plan merge snapshot and main disk file). Please, help :cry: :cry: :cry:

UPD2: Im already tried create dummy VDI with extract size in MB and replace first 512 bytes from dummy to corrupted one. I'm use Winhex, I first, cut 512 bytes from corrupted VDI, and then, paste (edit->clipboard data -> paste) offset 0 hex values(should I use normal copy in "Edit->copy block->Normally)?? from dummy.vdi . Now I cant open it via 7zip = file now corrupted ( I have backup of my corrupt file), so it just playground, before waiting for your answer
NeedHelp1
Posts: 8
Joined: 13. May 2021, 20:06

Re: [Solved] VDI Invalid Header

Post by NeedHelp1 »

I think, I'm fixed it myself. I just replace first 77 bytes ( this post helped me alot forums.virtualbox . org /viewtopic.php?p=288830#p288830 ) of my vdi snapshot and it worked, Im able merge it, boot. It fully worked again :)

Maybe Intelligent standby list cleaner cause this damage (it can empty standby list in memory) , I looked SMART for my SSD, it shows up good health.

You should add this in CloneVDI
Last edited by mpack on 14. May 2021, 10:16, edited 1 time in total.
Reason: Add link
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: [Solved] VDI Invalid Header

Post by mpack »

NeedHelp1 wrote: You should add this in CloneVDI
Perhaps it could do this specific repair, but repairing snapshots is not really practical in general. Snapshots are just fragments of the image, and fragments lack the context that allows CloneVDI to "guess" accurate replacement headers when they get corrupted. You would be well advised to avoid snapshots for this any several other reasons.
NeedHelp1
Posts: 8
Joined: 13. May 2021, 20:06

Re: [Solved] VDI Invalid Header

Post by NeedHelp1 »

mpack wrote:
NeedHelp1 wrote: You should add this in CloneVDI
Perhaps it could do this specific repair, but repairing snapshots is not really practical in general. Snapshots are just fragments of the image, and fragments lack the context that allows CloneVDI to "guess" accurate replacement headers when they get corrupted. You would be well advised to avoid snapshots for this any several other reasons.
I dont know, my snapshot have all files, when I inspect the image using DMDE. BTW instead of discontinuing use snapshots, better be use backups, as frequently as possible :D
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: [Solved] VDI Invalid Header

Post by mpack »

NeedHelp1 wrote: I dont know, my snapshot have all files
Well I do know, and snapshots do not have files. All they have are the sectors which changed relative to the parent disk. Snapshots are patch scripts, nothing more. While a snapshot (in a certain specific scenario) may contain fragments of a filesystem, including fragments large enough to recognize, they cannot in general be assumed to contain anything at all, which means you can't make enough assumptions about the content to automatically repair damaged metadata, or even recognize that it has been damaged. I'm not into providing snake oil software, so I won't be offering a repair option for corrupted snapshots.
Post Reply