Delete a snapshot when VDI is lost

This is for discussing general topics about how to use VirtualBox.
Emma2
Posts: 51
Joined: 16. Feb 2021, 11:59

Delete a snapshot when VDI is lost

Post by Emma2 »

I "somehow" lost a snapshot's VDI file. Luckily, this is the last snapshot in a chain, so I assume that I don't need that file - or do I?
However, I cannot delete that snapshot (in a Linux shell, i.e. by using vboxmanage), although I tried several "approaches",

What I am surprised of, in addition, is that it doesn't have to have any effect, if I manually edit the VBOX file:
I deleted that snapshot's XML node, but when I call "vboxmanage snapshot mymachine list" - it is still there!
Are there any "hidden" files anywhere which I have to edit or delete, too?

Or, is there any other way to get rid of that snapshot? Any help appreciated, thanks!
Last edited by Emma2 on 28. Dec 2021, 12:57, edited 2 times in total.
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: HOWTO: Delete a snapshot when VDI is lost

Post by scottgus1 »

Emma2 wrote:if I manually edit the VBOX file: I deleted that snapshot's XML node, but when I call "vboxmanage snapshot mymachine list" - it is still there!
That'll probably be because of this:
the beginning of the vbox file wrote:<?xml version="1.0"?>
<!--
** DO NOT EDIT THIS FILE.
** If you make changes to this file while any VirtualBox related application
** is running, your changes will be overwritten later, without taking effect.
** Use VBoxManage or the VirtualBox Manager GUI to make changes.
-->
Manually editing the .vbox file is a possible solution if you do it properly (so take a backup). But you can't edit the .vbox file successfully while a Virtualbox process is running. You need to reboot the host so no Virtualbox processes are running, then edit the XML. (Virtualbox has a service VboxSVC which keeps a RAM copy of all .vbox files and Virtualbox.xml read at the start of the service, and can write the RAM copy back to the disk at any time, overwriting manual changes to the files made after the service has started.)

Or restore the missing .vdi from your backups.
Emma2
Posts: 51
Joined: 16. Feb 2021, 11:59

Re: HOWTO: Delete a snapshot when VDI is lost

Post by Emma2 »

Thank you for that explanation, this explains why I didn't succeed.

Restoring the VDI from my backups is not possible, because it simply isn't there either... I assume that something during my backups went wrong:
The snapshot in question is one of those created from my backup script (which goes: stop the VM, create a snap, rsync the VM, delete the snap, restart the VM). Somehow the snap's VDI hasn't been copied properly...

Is there any way to edit the XML file without restarting the host? What about unregistering the VM, editing the XML and re-registering it after that?
(Why I don't want to restart the host? Simply because all VMs on that host are started in crontab, and I don't want to paralyze that host for too long...)
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: HOWTO: Delete a snapshot when VDI is lost

Post by scottgus1 »

Emma2 wrote:Is there any way to edit the XML file without restarting the host? What about unregistering the VM
That may work, though a suitable backup in place would be a good idea, just in case. I was able to unregister a DOS VM with 3 snapshots and reregister it. Be careful that you don't click to delete the VM's files when you unregister the VM, "Delete all files" is a one-way-no-recovery-nuke-it-from-orbit choice.
Emma2 wrote: my backup script (which goes: stop the VM, create a snap, rsync the VM, delete the snap, restart the VM)
Ah, yes, the old snapshot-enabled backup routine... It's screwed over many folks over the time.

FWIW if you're stopping the VM you don't need the snapshot. Folks try to use snapshots to back up a running VM, thinking that since the disk(s) before the snapshot are now not being written to, the disks can be backed up. Doing so leaves the VM OS in the same state as if the power were unplugged on it then it was backed up. Dirty databases, unsaved and lost data, etc. It's not a good backup routine, because the OS does not know it is being backed up, so it is not journaling its disk writes or volume shadow copying, etc.

Backups of a shutdown VM can be a simple file copy or rsync of the VM folder and any disk files residing outside of the VM folder. Backups of a running VM should be taken with VM OS or 3rd-party backup software capable of backing up the running OS. No backup routine should use a snapshot.
Emma2
Posts: 51
Joined: 16. Feb 2021, 11:59

Re: HOWTO: Delete a snapshot when VDI is lost

Post by Emma2 »

1. I'll try this and will report. (Actually, this is ony of my install test VMs, so even a total loss would not do too much harm...)
2. I do not click at all, simply because my host is a headless Ubuntu server, so I use vboxmanage.
3. After a lengthy discussion (I recall it was in this forum), I understood that I cannot properly rsync/backup a running VM, so I shut down them before snapshotting&copying them. I do not recall exactly, but I feel to remember that the discussion's final recommendation was to create s snapshot before rsyncing the (stopped!) VM... are you telling me that it were better to refrain from snapshotting entirely?
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: HOWTO: Delete a snapshot when VDI is lost

Post by scottgus1 »

Snapshots have their uses. They are more like System Restore points in a Windows OS, somewhat like for like time-traveling inside the VM to different points in the VM's life. But they make a VM more delicate, so data in them shouldn't be relied on to exist forever. Important data should be backed up outside the VM. Not to say it is definitely going to break some day, but the forums are replete with users that did something wrong with a snapshot or thought snapshots were backups, and lost their data. There's lots of opportunities for a pebcak error or an unexpected situation that the script doesn't handle. But, like most powerful tools, if they are used properly, they can be used well.

Backups should not use a snapshot. Adding delicacy to something one want's to rely on is not a good path. And once the VM is properly shut down it can be copied directly, without a snapshot.

I also like to FC file-compare and SHA256 hash my originals and backups, for LAN and offsite backup integrity checks. I had a script for the office Virtualbox server I was overseeing before I changed jobs. It would shut down all the VMs, make a local copy of the VMs' files, and FC the copies. Then it would hash the originals, then restart the VMs. The script would then copy the FC-confirmed copy to two other workstations that could double as servers for the VMs if the main server went down, to two grab-n-go USB drives, and to two offsite locations via rsync, which were double-copied offsite to USB drives. All the onsite copies were FC'd again against the confirmed first copy. At the offsite locations the rsynced files were hashed to compare with the original files' hashes. Total of 9 backups, 4 of which were offsite, with additional half-hour backups run inside the running VMs to gather changed data, sent onsite and offsite. Total destruction of the office (assuming we lived through it) would still allow a complete restart of server operations elsewhere with only a half hour max of data loss.
Emma2
Posts: 51
Joined: 16. Feb 2021, 11:59

Re: HOWTO: Delete a snapshot when VDI is lost

Post by Emma2 »

Thanks a lot so far!

1. unregistervm did the trick: I could edit the XML, delete the screenshot's node from it, re-registervm and thenrestart it properly. I possibly could check whether there are any unused VDIs in my directory, but this isn't at all urgent.
2. I will re-work my scripts to leave out the process of creatiing a snapshot before and deleting it after rsyncing the shut down VM.
3. Yes, regarding snapshots, I do NOT misuse them as backups. I actually create a snap before I do major upgrades, and I use them to store several update states of my own software.
4. Maybe two dumb questions (and I admit that this is now off-topic for this thread): what is "FC", and how do you hash and compare source and destination VMs?
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: HOWTO: Delete a snapshot when VDI is lost

Post by scottgus1 »

No, not dumb questions at all.

FC means file-compare. In Windows the actual command-line command is 'fc', and I think it is in Linux also. It does a bit-for-bit comparison to check if there are differences between two files.

I have used the MD5Deep suite for SHA256 hashing of files on Windows hosts. Linux no doubt has some similar command or program available. Run the hash program on the original and the copy, and compare the hashes. If the file copied correctly, the hashes should be the same.

FC is good when there is a high-mbps network between original and copied files. Hashing is a decent solution for off-site copies.
Emma2
Posts: 51
Joined: 16. Feb 2021, 11:59

Re: HOWTO: Delete a snapshot when VDI is lost

Post by Emma2 »

Thank!
You!
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: [SOLVED] Delete a snapshot when VDI is lost

Post by scottgus1 »

One thing I should point out:
scottgus1 wrote:Hashing is a decent solution for off-site copies.
If you are allowed to run a program on the offsite copy's computer, that is. The idea of using hashing is to allow the full mega-GB file to be checked without having to read it over the network. The file should be able to be hashed locally where the high-speed channels are available, then pass the hash back over the network for comparison.
fth0
Volunteer
Posts: 5668
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: [SOLVED] Delete a snapshot when VDI is lost

Post by fth0 »

scottgus1 wrote:FC means file-compare. In Windows the actual command-line command is 'fc', and I think it is in Linux also. It does a bit-for-bit comparison to check if there are differences between two files.
In Linux, fc (fix command) is a totally different command. You could use the cmp command instead. But if you calculate a hash anyway, what do you need the file compare for?
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: [SOLVED] Delete a snapshot when VDI is lost

Post by scottgus1 »

Linux doesn't use fc, eh? Thought for sure I'd seen that in a non-Windows context somewhere... Thanks for the info!

I would use FC for local compares, such as for copied vdi files on the same PC or over a gigabit network. Hashes would be for offsite files where internet was low-mbps and reading the vdi back for an FC would take weeks. I'd rsync dozens-of-GB vdi's over the internet to PCs in my boss's and my basements. Then run hashes remotely from the basement PCs and send the hashes back to compare with the original files' hashes. All scripted.

I'd do FC locally, because it's possible for multiple files to have the same hash, and it could theoretically happen that a corrupted copy ends up with the same hash. I considered hashes to be good enough for offsite confirmation, but FC would be more certain where bandwidth would permit it.
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: [SOLVED] Delete a snapshot when VDI is lost

Post by mpack »

scottgus1 wrote: I'd do FC locally, because it's possible for multiple files to have the same hash
Not really, not unless the files are identical or the "hash" is a really terrible one. With any relatively modern recognized hash the chances that two different files accidentally resolve to the same hash is practically zero. I.e. not quite zero, but the universe will end before you ever notice the difference.

Long before cryptographic "hashes" were a thing, my own firmware relied on 16 bit CRCs to detect changes in ROM images. That's "only" 65535:1 against a false match, but those are still long odds in practice - I've never known a false match. With a modern hash of 128+ bits... forget it.
fth0
Volunteer
Posts: 5668
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: [SOLVED] Delete a snapshot when VDI is lost

Post by fth0 »

FWIW, rsync implicitly uses MD5 hashing as part of its normal operation, so if you call it a second time with the --checksum option, it'll do all the verification for you automatically. If you use cp -r for local copies for performance reasons, also call rsync --checksum afterwards.
Emma2
Posts: 51
Joined: 16. Feb 2021, 11:59

Re: Delete a snapshot when VDI is lost

Post by Emma2 »

Unfortunally, I had un-[RESOLVED] this thread - simply because there are still open question - which I would like to answer. (As the machine in question is a test machine only which can more or less easily be recreated, I would like to be able this situation anyway - in case it might occur with another, more important, VM, too...)

First of all, my approach did not actually work: after having unregistered the VMm manually deleted the snapshot's entry, I could not register the VM again - instead I am informed that one VDI file were missing. Therefore, I did a bit of an "analysis", and I found out that

(a) I can use most, if not all, of my (about nine) snapshots snapshots (before I unregister the VM). If I do a vboxmanage list hdds, however, I can see only THREE registered VDIs, and one of them is even flagged as "inaccessible".
(b) I cannot do a vboxmanage closemedium onto this VDI - simply because the file "has child media" (which it actually has NOT - because it's the last snapshoz in its chain)
(c) Having done a web search, I found out that all registered HDDs should be listed in the file Virtualbox.xml, but they aren't.

So, my questions are:
1. Where does Virtualbox store the information on registered VDIs?
2. How can I find out which is the suspected "child media"?
3. How can I get rid of this HDD entry?
4. Where can I find information on child and parent VDIs - to possible merge them manually?

If I missed the answers on my questions somewhere in the documentation, I would be happy about a link, too.
Post Reply