Deleted snapshot images kept in Media Manager

[amk]

Hi forum,

VirtualBox 5.1.22 r115126 (Qt5.6.2) in OS X 10.11.6 experienced an issue while removing several snapshots.
Now when trying to start the VM, I end up with

Code: Select all

Failed to open a session for the virtual machine Win2000.

Could not open the medium '/Users/userid/VirtualBox VMs/Win2000/Snapshots/{07e8c4e8-2203-4715-87f3-de63db654030}.vmdk'.

VD: error VERR_FILE_NOT_FOUND opening image file '/Users/userid/VirtualBox VMs/Win2000/Snapshots/{07e8c4e8-2203-4715-87f3-de63db654030}.vmdk' (VERR_FILE_NOT_FOUND).

Result Code: NS_ERROR_FAILURE (0x80004005)
Component: MediumWrap
Interface: IMedium {4afe423b-43e0-e9d0-82e8-ceb307940dda}

The problem is that disk 07e8c4e8-2203-4715-87f3-de63db654030 was attached to a removed snapshot, and it actually is not part of the differencing chain:

Code: Select all

Mac:Snapshots userid$ vboxmanage internalcommands dumphdinfo  \{5247363e-eb53-4918-a671-eb320827d7f5\}.vmdk 
--- Dumping VD Disk, Images=1
Dumping VD image "{5247363e-eb53-4918-a671-eb320827d7f5}.vmdk" (Backend=VMDK)
Header: Geometry PCHS=8322/16/63 LCHS=522/255/63 cbSector=8388608
Header: uuidCreation={5247363e-eb53-4918-a671-eb320827d7f5}
Header: uuidModification={a1e39c3a-059b-42f3-9d14-4747c31eb0cf}
Header: uuidParent={457ffa29-c3dc-43a3-9697-9a4be405f0fd}
Header: uuidParentModification={3deb730f-7b59-4354-9d6f-03d6244ae0bc}
Mac:Snapshots userid$ vboxmanage internalcommands dumphdinfo  \{457ffa29-c3dc-43a3-9697-9a4be405f0fd\}.vmdk 
--- Dumping VD Disk, Images=1
Dumping VD image "{457ffa29-c3dc-43a3-9697-9a4be405f0fd}.vmdk" (Backend=VMDK)
Header: Geometry PCHS=8322/16/63 LCHS=522/255/63 cbSector=8388608
Header: uuidCreation={457ffa29-c3dc-43a3-9697-9a4be405f0fd}
Header: uuidModification={3deb730f-7b59-4354-9d6f-03d6244ae0bc}
Header: uuidParent={13b6bb85-c925-4d88-86b0-924f27087a81}
Header: uuidParentModification={ea6df971-f250-415d-96f8-778b7f937b10}
Mac:Snapshots userid$ vboxmanage internalcommands dumphdinfo  \{13b6bb85-c925-4d88-86b0-924f27087a81\}.vmdk 
--- Dumping VD Disk, Images=1
Dumping VD image "{13b6bb85-c925-4d88-86b0-924f27087a81}.vmdk" (Backend=VMDK)
Header: Geometry PCHS=8322/16/63 LCHS=522/255/63 cbSector=8388608
Header: uuidCreation={13b6bb85-c925-4d88-86b0-924f27087a81}
Header: uuidModification={ea6df971-f250-415d-96f8-778b7f937b10}
Header: uuidParent={baa2c3c7-a048-4060-8b44-f87b7bd11bac}
Header: uuidParentModification={59df76f9-de2c-4ff9-a767-5cd6043253cf}
Mac:Snapshots userid$ vboxmanage internalcommands dumphdinfo  \{baa2c3c7-a048-4060-8b44-f87b7bd11bac\}.vmdk 
--- Dumping VD Disk, Images=1
Dumping VD image "{baa2c3c7-a048-4060-8b44-f87b7bd11bac}.vmdk" (Backend=VMDK)
Header: Geometry PCHS=8322/16/63 LCHS=522/255/63 cbSector=8388608
Header: uuidCreation={baa2c3c7-a048-4060-8b44-f87b7bd11bac}
Header: uuidModification={bfa17c4c-baf7-4b71-9ff4-2ee2c47f2d9d}
Header: uuidParent={16f251dc-e205-409e-826e-d322464b1fa0}
Header: uuidParentModification={c3b2e057-0334-4c04-a27d-c29d14a3bce9}
Mac:Snapshots userid$ vboxmanage internalcommands dumphdinfo  ../Windows\ 2000.vmdk 
--- Dumping VD Disk, Images=1
Dumping VD image "../Windows 2000.vmdk" (Backend=VMDK)
Header: Geometry PCHS=8322/16/63 LCHS=522/255/63 cbSector=8388608
Header: uuidCreation={16f251dc-e205-409e-826e-d322464b1fa0}
Header: uuidModification={c3b2e057-0334-4c04-a27d-c29d14a3bce9}
Header: uuidParent={00000000-0000-0000-0000-000000000000}
Header: uuidParentModification={3cb43c65-d2df-4247-9c40-b34058690d9d}

At the other hand, Virtual Media Manager GUI as well as the list hdds command shows a different chain and I am unable to figure out where is this information coming from. These disk UUIDs do not appear in the VM.xml, nor in the /Users/userid/Library/VirtualBox/VirtualBox.xml file

Code: Select all

Mac:Snapshots userid$ vboxmanage list -l hdds
UUID:           16f251dc-e205-409e-826e-d322464b1fa0
Parent UUID:    base
State:          created
Type:           normal (base)
Location:       /Users/userid/VirtualBox VMs/Win2000/Windows 2000.vmdk
Storage format: VMDK
Format variant: dynamic split2G
Capacity:       4096 MBytes
Size on disk:   4085 MBytes
Encryption:     disabled
Child UUIDs:    baa2c3c7-a048-4060-8b44-f87b7bd11bac

UUID:           baa2c3c7-a048-4060-8b44-f87b7bd11bac
Parent UUID:    16f251dc-e205-409e-826e-d322464b1fa0
State:          created
Type:           normal (differencing)
Auto-Reset:     off
Location:       /Users/userid/VirtualBox VMs/Win2000/Snapshots/{baa2c3c7-a048-4060-8b44-f87b7bd11bac}.vmdk
Storage format: VMDK
Format variant: differencing default
Capacity:       4096 MBytes
Size on disk:   267 MBytes
Encryption:     disabled
In use by VMs:  Win2000 (UUID: 23d5228a-cf3a-4300-a260-7760697e1983) [Bck170331 - VMold! (UUID: 19f1d70d-0d8d-4e6c-8b59-71468284b270)]
Child UUIDs:    07e8c4e8-2203-4715-87f3-de63db654030

UUID:           07e8c4e8-2203-4715-87f3-de63db654030
Parent UUID:    baa2c3c7-a048-4060-8b44-f87b7bd11bac
State:          inaccessible
Access Error:   Could not open the medium '/Users/userid/VirtualBox VMs/Win2000/Snapshots/{07e8c4e8-2203-4715-87f3-de63db654030}.vmdk'.
VD: error VERR_FILE_NOT_FOUND opening image file '/Users/userid/VirtualBox VMs/Win2000/Snapshots/{07e8c4e8-2203-4715-87f3-de63db654030}.vmdk' (VERR_FILE_NOT_FOUND)
Type:           normal (differencing)
Auto-Reset:     off
Location:       /Users/userid/VirtualBox VMs/Win2000/Snapshots/{07e8c4e8-2203-4715-87f3-de63db654030}.vmdk
Storage format: VMDK
Format variant: dynamic default
Capacity:       0 MBytes
Size on disk:   0 MBytes
Encryption:     disabled
Child UUIDs:    377710c2-03db-4ae9-ac29-c870ce9bed81

UUID:           377710c2-03db-4ae9-ac29-c870ce9bed81
Parent UUID:    07e8c4e8-2203-4715-87f3-de63db654030
State:          inaccessible
Access Error:   Could not open the medium '/Users/userid/VirtualBox VMs/Win2000/Snapshots/{377710c2-03db-4ae9-ac29-c870ce9bed81}.vmdk'.
VD: error VERR_FILE_NOT_FOUND opening image file '/Users/userid/VirtualBox VMs/Win2000/Snapshots/{377710c2-03db-4ae9-ac29-c870ce9bed81}.vmdk' (VERR_FILE_NOT_FOUND)
Type:           normal (differencing)
Auto-Reset:     off
Location:       /Users/userid/VirtualBox VMs/Win2000/Snapshots/{377710c2-03db-4ae9-ac29-c870ce9bed81}.vmdk
Storage format: VMDK
Format variant: dynamic default
Capacity:       0 MBytes
Size on disk:   0 MBytes
Encryption:     disabled
Child UUIDs:    eadae240-1936-42bf-a9f3-88b1218d2811

UUID:           eadae240-1936-42bf-a9f3-88b1218d2811
Parent UUID:    377710c2-03db-4ae9-ac29-c870ce9bed81
State:          inaccessible
Access Error:   Could not open the medium '/Users/userid/VirtualBox VMs/Win2000/Snapshots/{eadae240-1936-42bf-a9f3-88b1218d2811}.vmdk'.
VD: error VERR_FILE_NOT_FOUND opening image file '/Users/userid/VirtualBox VMs/Win2000/Snapshots/{eadae240-1936-42bf-a9f3-88b1218d2811}.vmdk' (VERR_FILE_NOT_FOUND)
Type:           normal (differencing)
Auto-Reset:     off
Location:       /Users/userid/VirtualBox VMs/Win2000/Snapshots/{eadae240-1936-42bf-a9f3-88b1218d2811}.vmdk
Storage format: VMDK
Format variant: dynamic default
Capacity:       0 MBytes
Size on disk:   0 MBytes
Encryption:     disabled
Child UUIDs:    13b6bb85-c925-4d88-86b0-924f27087a81

UUID:           13b6bb85-c925-4d88-86b0-924f27087a81
Parent UUID:    eadae240-1936-42bf-a9f3-88b1218d2811
State:          created
Type:           normal (differencing)
Auto-Reset:     off
Location:       /Users/userid/VirtualBox VMs/Win2000/Snapshots/{13b6bb85-c925-4d88-86b0-924f27087a81}.vmdk
Storage format: VMDK
Format variant: differencing default
Capacity:       4096 MBytes
Size on disk:   74 MBytes
Encryption:     disabled
In use by VMs:  Win2000 (UUID: 23d5228a-cf3a-4300-a260-7760697e1983) [BootedNoNet (UUID: 7292c867-cc24-4ace-b60e-5336100a1482)]
Child UUIDs:    457ffa29-c3dc-43a3-9697-9a4be405f0fd

UUID:           457ffa29-c3dc-43a3-9697-9a4be405f0fd
Parent UUID:    13b6bb85-c925-4d88-86b0-924f27087a81
State:          created
Type:           normal (differencing)
Auto-Reset:     off
Location:       /Users/userid/VirtualBox VMs/Win2000/Snapshots/{457ffa29-c3dc-43a3-9697-9a4be405f0fd}.vmdk
Storage format: VMDK
Format variant: differencing default
Capacity:       4096 MBytes
Size on disk:   72 MBytes
Encryption:     disabled
In use by VMs:  Win2000 (UUID: 23d5228a-cf3a-4300-a260-7760697e1983) [BootedUpgrade VMtools (UUID: 4d74a9e8-088c-40fe-8ee9-a1c2781e911d)]
Child UUIDs:    5247363e-eb53-4918-a671-eb320827d7f5

UUID:           5247363e-eb53-4918-a671-eb320827d7f5
Parent UUID:    457ffa29-c3dc-43a3-9697-9a4be405f0fd
State:          created
Type:           normal (differencing)
Auto-Reset:     off
Location:       /Users/userid/VirtualBox VMs/Win2000/Snapshots/{5247363e-eb53-4918-a671-eb320827d7f5}.vmdk
Storage format: VMDK
Format variant: differencing default
Capacity:       4096 MBytes
Size on disk:   22 MBytes
Encryption:     disabled
In use by VMs:  Win2000 (UUID: 23d5228a-cf3a-4300-a260-7760697e1983)

Questions:

- Where is the information returned by list hdds stored/coming from? I could find these UUIDs in Win2000-1.10-macosx.xml in the VM directory, and I could not figure out what this file is for. Anyways, the VMM keeps showing these deleted disks also after removal of this file and reboot. This was attempted following viewtopic.php?f=8&t=59448&p=276543

- How to put the VMM (and list hdds) in sync with the chain in actual differencing snapshot files?

Many thanks

[socratis]

You're the second person today and the 173rd this year that's been having Snapshot problems. So, I've decided to pull our collective knowledge together and create a small Snapshot basics article. See if that helps...

[amk]

Huge thanks for taking care of the users with snapshot problems. Your article is very informative and illustrative BTW, but likely for a different target audience.

I do not think I have a problem with understanding of snapshot basics, at least in the sense explained there. It is clear for me that snapshots are not backups. My use of snapshots is to limit the amount of data copied periodically by time machine, since only the last file in chain is modified.

It is also clear to me that when a snapshot gets deleted, the sectors stored in the differencing file get merged with the next file in the chain. The parent of the removed file becomes parent of the child. This also normally works without issues.

Checking the disk files themselves I am also confident that all the sectors are available, each and every differencing file points to an expected and available parent.

My problem is with "Virtual Media Manager" where for some reason the removed disks are still visible as if it missed the update after disk merge and snapshot removal. It keeps the removed disk files in the GUI as well as in the output of "vboxmanage list hdds".

So while the disk files correctly refer to parents, I expect that something happened during the removal and the VMM is not up to date. It is actually strange because normally, in order to protect against data loss, first the data structures and pointers should be updated and only afterwards the files are deleted when nothing points to them anymore.

In my case the disk themselves are updated correctly, but the VMM is not. I expect to find the file where this information is stored and correct it manually, but so far I was not successful. I was thinking to take the virtual machine and add it to another host, but I do not think it would be possible with all snapshots. Hence I would like to know where does VMM keep its information across reloads. Would you be able to assist with a hint to locate this storage?

Can you help to understand what can be the role of VM xml file with "-1.10-macosx" inserted into name and .xml extension? I thought it would be a backup kept during upgrade of the app that is normally not used.

How can I correct/reinitialize the VMM structure? Currently it does not allow me to release the disks because it claims their child is attached to a VM snapshot, while the child itself does not refer to these disks anymore.

Thanks in advance.

[mpack]

I do not think I have a problem with understanding of snapshot basics

Except perhaps for that most basic of the basics: don't use snapshots!

[amk]

Except perhaps for that most basic of the basics: don't use snapshots!

Not funny. I am using snapshots regularly since vmware GSX in all virtualization platforms I could get access to and never experienced troubles. Here the snapshots themselves are also fine. The only issue is Virtual Media Manager out of sync. All I need is to figure out where does it store the data (should be the same place where vboxmanage list -l hdds reads from) and I should be able to fix it myself and update the thread here.

[socratis]

Can you help to understand what can be the role of VM xml file with "-1.10-macosx" inserted into name and .xml extension? I thought it would be a backup kept during upgrade of the app that is normally not used.

How old is this VM? Is it pre-VirtualBox 4.x? If it is, we have plenty of back and forth still. First, post that ".xml" file that you refer to. ZIP it first to bypass any naming restrictions.

The problem is that disk 07e8c4e8-2203-4715-87f3-de63db654030 was attached to a removed snapshot, and it actually is not part of the differencing chain

How do you know that? Did you check the whole chain hierarchy? How?

Post the output of the command (and please do not obfuscate anything):

• ls -alR "/Users/userid/VirtualBox VMs/Win2000"
  ls -alR "/Users/userid/Library/VirtualBox"

[socratis]

And since you're at 5.1.x, post the file "/Users/userid/Library/VirtualBox/VirtualBox.xml". ZIPPED.

[amk]

Yes, I checked the whole hierarchy, inside of the vmdk files as well as returned by vboxmanage internalcommands dumphdinfo - pasted in the first post.
The file VirtualBox.xml does not contain any sign of there "ghost" files either, checked by grep and the UUIDs.

[socratis]

I asked for very specific details. Instead I got bupkis. That's fine by me if you want to keep on researching this on your own. One less thread for me to worry about...

[amk]

I also keep asking for very specific detail - where does Virtual Media Manager and vboxmanage list -l hdds take their information from.

I can share the requested files as soon as I get access to that computer. At the same time I hope I am allowed to state that these are human readable xml files, I checked them very carefully and they do not refer to the images VMM is showing as inaccessible in the snapshot chain.

[mpack]

I also keep asking for very specific detail - where does Virtual Media Manager and vboxmanage list -l hdds take their information from.

The Virtual Media Manager shows the union of the local media registries of all registered VMs. It has no independant memory, so it isn't possible for it to fail to forget anything.

[socratis]

Well, that depends. The OP is talking about an XML file, not a VBOX file. That's why I keep asking for details. For some reason these are not provided. Let it be...

[amk]

I am back at the system and the issue is resolved, thanks to hints received here.

I was talking about xml file because files with vbox extension are also in fact xml files but when grepping for the missing UUIDs I missed the vbox extension.
Basically, the subtle information I was looking for was that the data is read from config files of VirtualBox itself as well as every VM, these can be named with extension .xml as well as .vbox.

For the benefit of the forum, let me explain a bit of the history, investigation process, and fix:

Trying to dig more, I have run vboxmanage list hdds under dtruss. Realized that the data printed is coming via IPC from VBoxSVC. So I stopped all VMs, and tried to debug that one. https://www.virtualbox.org/wiki/VBoxMainLogging provided some hints, tried launchctl setenv to set it, but the outcome was even less verbose. Not a big problem anyways, there was interesting (for me) confirmation that files with extension vbox are accessed, although an unexpected output is seen in VBoxSVC.log before loading VBbug.vbox.

Code: Select all

00:00:00.144948 nspr-2   Loading settings file "/Users/userid/VirtualBox VMs/Win2000/Win2000.xml" with version "1.15-macosx"
00:00:00.145955 nspr-2   ERROR [COM]: aRC=VBOX_E_OBJECT_NOT_FOUND (0x80bb0001) aIID={0169423f-46b4-cde9-91af-1e9d5b6cd945} aComponent={VirtualBoxWrap} aText={Could not find an open hard disk with UUID {baa2c3c7-a048-4060-8b44-f87b7bd11bac}}, preserve=false aResultDetail=0
00:00:00.145992 nspr-2   ERROR [COM]: aRC=NS_ERROR_FAILURE (0x80004005) aIID={b2547866-a0a1-4391-8b86-6952d82efaa0} aComponent={SnapshotMachine} aText={A differencing image of snapshot {19f1d70d-0d8d-4e6c-8b59-71468284b270} could not be found. Could not find an open hard disk with UUID {baa2c3c7-a048-4060-8b44-f87b7bd11bac}}, preserve=false aResultDetail=0
00:00:00.148130 nspr-2   Loading settings file "/Users/userid/VirtualBox VMs/VBbug/VBbug.vbox" with version "1.14-macosx"

Here comes the history: The VM originates from vmware-workstation on Linux. VMDK was imported years ago into VirtualBox 3.2 on Linux and subsequently into several versions of VirtualBox on 10.8 OSX up to current 10.11. During the first migration I encountered issue with VMDK every time disconnecting from the VM, and I created the VBbug VM as a workaround with a comment <Description>Created to keep disk of W2000 connected. otherwise it detached from its VM</Description>

So for some time that disk was connected to two VMs, nowadays the VBbug VM had no connections whatsoever and was only present idle in the list. Inside of the file VBbug.vbox there was a section <MediaRegistry>, listing all of the HardDisks I was unable to track. Once I removed this VM from VirtualBox list, this registry got transferred into Win2000.xml. I have manually removed all entries pointing to the deleted disk images, the VMM does not complain anymore and the VM now starts.

I have no idea why the registry was kept in a VM config file without the actual attachment and why it was being updated so far and is not anymore. Perhaps there was a one time glitch when I deleted the snapshots. It could be related to the "plenty of back and forth". I upgraded VirtualBox recently in an attempt to address this Win2000 VM hanging when accessing virtio NIC. This is completely different issue of course, still not working and I will have to use PcNetIII virtual NIC should I ever need to access net from that VM.

Thanks for your time, and for the great VirtualBox.

[mpack]

I have no idea why the registry was kept in a VM config file without the actual attachment

That has always been the case. Removing an attachment from a drive (or controller) is not the same as unregistering it. That's why Virtual Media Manager has two separate steps: Release (removes attachment from all VMs) and Remove (unregisters from the one VM which registered it).