VM running older linux shreddered

[unhappy2018]

Hello and Happy New Year!

I do need some assistance or my fate is sealed.

Setup is a perfectly nice running i7, opensuse 43.2 x86_64, Virtualbox is 5.1.30-39.1

There are 3 different VMs, two of them running absolutly fine, the 3rd is my concern.

This 3rd VM ("the VM") is a virtualisised, ancient SuSe Linux (8.2) with a 2.2 Kernel,
the vdi contains 2 partitions, a swap (83) and Linux(82) ext3 one.

Its purpose is to run an old but important LAMP system.
Said VM was doing fine for the last 5 or 6 years.

What happened:

I logged into the up-to-then running VM for maintaining purpose.
Starting to work I realised that the /root/bin dir was missing, ls -ali showed its name as ?bin/ (questionmarkbin)
I sync'ed and rebooted. Grub failed to boot the system.

I saved a copy of that state, added a livesystem cd-image to boot from, worked.

From the LS I tried to access the partitions of the VM vdi, worked.

Swap is fine, thanks god!
Data is corrupted, up to now beyond repair, unable to mount, superblock (and apparently all spare superblocks) is gone

I ran fsck, all files and dirs are gone, lost+found is full, was resized by fsck 12 times.

What I have:
A fine running, but old backup of the vdi.
All data backups (mysql dump) after 7. Dec. are corrupted, filled with hex data instead of ASCI

What I need:
a miracle or some hints how to proceed, I desperatly need more up-to-data data from mysql.
In addition, it would be very helpful to understand the "what and why" of this catastrophe to prevent future nervous breakdowns.

in short:

I am willing to give my soul for any hints to understand and fix the problem, especially how to restore the data.

May all the readers be spared of this situation and have a real nice NEW YEAR

[socratis]

Happy New Year to you too and everyone else!

Fact of computers: they do go down. The only way to avoid that is a good, reliable, thoroughly tested backup *and* restore method. You seem to have a backup (albeit older) and you've been doing database dumps. That's a great start, at least to recovering parts of your (virtual) computer.

From your description it seems that things stated going bad on (or around) 07/Dec., since after that there seems that a corruption happened in your database, causing all dumps to be in Hex instead of your typical/expected ASCII. That sounds like a database corruption. Most probably at the filesystem level of the VM, because it seems that thing started going worse and worse after that, to the point that you have no files left in your VM.

Data is corrupted, up to now beyond repair, unable to mount, superblock (and apparently all spare superblocks) is gone

A filesystem corruption on the VM, for me at least, implies a bad filesystem on the host. Did you check that? You definitely should. A VM's hard disk is (typically) a single file on the host. If that file starts going bad, the whole VM goes bad.

I desperatly need more up-to-data data from mysql.

You can't "invent" what you don't have in computers. That's where the backup/restore comes into play again. Your data is as good as their backup.

You got to treat your VM (especially if it's important) as another computer, and then some more. I can't emphasize how important is to have a good backup strategy, not only in the data level, but in the whole computer level. And in the case of a VM of the whole .vbox and .vdi files as well. I have 3 backups of my VMs, and they're mostly for non-important things, i.e. I wouldn't be "unhappy2018" if anything bad happened to them. Sorry that your 2018 started this way...

[unhappy2018]

Dear socratis,

thanks for your reply.

Regarding a corrupted file system on the host, it was indeed my first thought,
but as I wrote, the host is in a perfect conditions,
force fscked (because it was clean) the data partition and found no problem.

The data partition on the host is a RAID5

I only once had a similar situation with a corrupted file system long ago,
but this was due to a hardware problem on the raid controller.

Any hint or idea is still highly apprechiated!

[socratis]

There are no more hints or ideas, at least from the VirtualBox side. There's not that much you can do if the filesystem of an OS (real or virtual) is corrupt, but the VDI itself is not. Only a backup/restore can save the day.

I ran fsck, all files and dirs are gone, lost+found is full, was resized by fsck 12 times.

but as I wrote, the host is in a perfect conditions, force fscked (because it was clean) the data partition and found no problem.

Does your first description sound to you that your did an 'fsck' on the host? With no problems? It actually doesn't...

One thing you might want to try is to create a new VM and attach the VDI from the corrupt VM, but as you've said:

added a livesystem cd-image to boot from, worked. From the LS I tried to access the partitions of the VM vdi, worked. Swap is fine, thanks god! Data is corrupted, up to now beyond repair, unable to mount, superblock (and apparently all spare superblocks) is gone

So, at this point you have bupkis... BTW, I never quite understood why you were so happy about the swap partition being OK, it usually doesn't hold anything of value.

So, you got to face reality and you should start planning your restore process...

[unhappy2018]

"BTW, I never quite understood why you were so happy about the swap partition being OK"

this may be caused by your lack of humor and/or your problems recognising sarkasm.

But thanks anyway for your assistance according to your abilities.

[socratis]

this may be caused by your lack of humor and/or your problems recognising sarkasm.

Since you don't know your interlocutor's level of humor or understanding of sarcasm, it's typical (in forums, e-mails, written messages) that such emotions/intentions are followed by a smiley/emoticon. But you may be new to this whole messaging thing, so you're forgiven...

But thanks anyway for your assistance according to your abilities.

Sure thing. I really hope that at least you learned a couple of things about the importance of backups, especially for "desperately needed" data.

Sorry about what happened, and I really hope that you have a more pleasant 2018...
Frohes neues Jahr!

[mpack]

In any case this entire discussion seems to be misplaced. As far as I can tell there seems to be nothing wrong with the VM, it is entirely a software problem inside the guest OS and applications. If you have an OS crash do you ask for support from the people who made your motherboard?

I didn't answer the original question because I didn't want to be a party pooper over the holiday season, but I'm afraid the truth is that there are no fairy godmothers who can magically wish away the facts: if you have only one copy of important data and that one copy gets corrupted - then guess what, there are no solutions for that.

p.s. I hate RAID systems. I've always thought that the idea was fundamentally broken, i.e. lending people the idea that you don't need to make backups because the storage system is self repairing... except when it isn't, e.g. except when the data loss was not due to a disk failure.