virtualbox.org

Hello,

I am a novice at VB, but I had it working great for about a month. Yesterday my main guest crashed and it's now inaccessible. How to recover?

Ubuntu Server 11.04 as host
Ubuntu Server 11.04 guest
Ubuntu 11.04 desktop guest (usually off)
WinXP guest (usually off).

The ubuntu server guest is where I've configured lots of stuff so I'd like to avoid starting over if possible. I tried doing a snapshot of the ubuntu server guest while it was running and suddenly the guest crashed.

from the log:

Code: Select all

289:43:08.445 Changing the VM state from 'RUNNING' to 'RUNNING_LS'.
289:43:08.538
289:43:08.538 !!Assertion Failed!!
289:43:08.538 Expression: <NULL>
289:43:08.538 Location  : /home/vbox/vbox-4.1.2/src/VBox/VMM/VMMAll/PGMAllBth.h(1458) void pgmR3BthPAE32BitSyncPageWorkerTrackDeref(VMCPU*, PGMPOOLPAGE*, RTHCPHYS, uint16_t, RTGCPHYS)
289:43:08.538 HCPhys=000000010f616000 wasn't found!

I can't remember the exact command I used to do the snapshot. I remember I was trying to use the -desc option, but that's all I can remember. I probably typed something incorrectly.

Now vboxmanage list vms:

Code: Select all

"ubu-dsk" {e713309e-8b79-40d3-9cc2-b3e015095171}
"<inaccessible>" {d6f33087-436c-472a-a25c-e20ff27fb067}
"ubu-srv2011-08-24a" {bcb2af28-f31e-4012-aa44-f9529151958c}
"ubu-dsk2011-08-24a" {1deb9a44-2ecf-44e1-9ab8-2e3dbd50a2ef}
"win-xp_base1_dhcp" {4dfe5280-8c8a-4f12-941b-41fac17e743a}
"win-xp_ie8_dhcp" {3c06225a-7b36-474c-9874-80d7eae51312}

The inaccessible one is only one I care about. The ubu-srv2011-08-24a still works but it would take quite a bit of reconfiguring to restore from that (which is why I was trying the snapshot).

vboxmanage showvminfo {d6f33087-436c-472a-a25c-e20ff27fb067} (the inaccessible one) gives:

Code: Select all

Name:            <inaccessible!>
UUID:            d6f33087-436c-472a-a25c-e20ff27fb067
Config file:     /home/barnett/VirtualBox VMs/ubu-srv/ubu-srv.vbox
Access error details:
VBoxManage: error: Hard disk '/home/barnett/VirtualBox VMs/ubu-srv/ubu-srv.vdi' with UUID {462689d7-106b-4fb6-b838-9ef2ed6f67c3} cannot be directly attached to the virtual machine 'ubu-srv' ('/home/barnett/VirtualBox VMs/ubu-srv/ubu-srv.vbox') because it has 1 differencing child hard disks
VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005), component Machine, interface IMachine, callee

I searched the forums, found places to check, but I couldn't tell how to recover from this. It seems like it's looking for a snapshot that's not there. The vdi I was trying to take a snapshot from has not moved. It's at:
/home/barnett/Virtualbox VMs/ubu-srv/ubu-srv.vdi

Thanks .... Barnett (more info below)

When I try to restart it with vboxheadless --startvm {d6f33087-436c-472a-a25c-e20ff27fb067} I get:

Code: Select all

VBoxHeadless: error: The object functionality is limited
VBoxHeadless: error: Details: code E_ACCESSDENIED (0x80070005), component Machine, interface IMachine, callee nsISupports
Context: "LockMachine(session, LockType_Write)" at line 857 of file VBoxHeadless.cpp

This is copied out of ubu-srv.vbox:

Code: Select all

<VirtualBox xmlns="http://www.innotek.de/VirtualBox-settings" version="1.12-linux">
  <Machine uuid="{d6f33087-436c-472a-a25c-e20ff27fb067}" name="ubu-srv" OSType="Ubuntu_64" snapshotFolder="Snapshots" lastStateChange="2011-09-25T16:30:19Z" aborted="true">
    <MediaRegistry>
      <HardDisks>
        <HardDisk uuid="{462689d7-106b-4fb6-b838-9ef2ed6f67c3}" location="ubu-srv.vdi" format="VDI" type="Normal">
          <HardDisk uuid="{c7c2954c-4930-4eb7-8564-59dea17651da}" location="Snapshots/{c7c2954c-4930-4eb7-8564-59dea17651da}.vdi" format="VDI"/>
        </HardDisk>
      </HardDisks>

The Snapshots folder is empty.

My hdd isn't listed below, but the file is still there ... vboxmanage list hdds:

Code: Select all

UUID:        9757d042-72cb-421d-9a1e-5d8aa7ae489a
Parent UUID: base
Format:      VDI
Location:    /home/barnett/ubu-dsk.vdi
State:       created
Type:        normal
Usage:       ubu-dsk (UUID: e713309e-8b79-40d3-9cc2-b3e015095171) [ubu-dsk_snapshot2011-08-22 (UUID: 01b147ed-71ef-4c6c-b79e-e0ac3b383e2e)]

UUID:        58ccd62f-492f-493e-9954-e8d807d50409
Parent UUID: 9757d042-72cb-421d-9a1e-5d8aa7ae489a
Format:      VDI
Location:    /home/barnett/VirtualBox VMs/ubu-dsk/Snapshots/{58ccd62f-492f-493e-9954-e8d807d50409}.vdi
State:       created
Type:        normal
Usage:       ubu-dsk (UUID: e713309e-8b79-40d3-9cc2-b3e015095171)

UUID:        e13bf973-5a81-4dff-b58a-8e4e5dc13657
Parent UUID: base
Format:      VDI
Location:    /home/barnett/.VirtualBox/backup/ubu-srv2011-08-24a/ubu-srv2011-08-24a.vdi
State:       created
Type:        normal
Usage:       ubu-srv2011-08-24a (UUID: bcb2af28-f31e-4012-aa44-f9529151958c)

UUID:        00b51332-746f-4233-b891-1eafb1ec3fcf
Parent UUID: base
Format:      VDI
Location:    /home/barnett/.VirtualBox/backup/ubu-dsk2011-08-24a/ubu-dsk2011-08-24a.vdi
State:       created
Type:        normal
Usage:       ubu-dsk2011-08-24a (UUID: 1deb9a44-2ecf-44e1-9ab8-2e3dbd50a2ef)

UUID:        685a5fdb-b7ea-4e2f-a0e4-be45b81a7a8f
Parent UUID: base
Format:      VDI
Location:    /home/barnett/VirtualBox VMs/win-xp_base1_dhcp/win-xp_base1_dhcp.vdi
State:       created
Type:        normal
Usage:       win-xp_base1_dhcp (UUID: 4dfe5280-8c8a-4f12-941b-41fac17e743a)

UUID:        bf2103fc-006a-47f9-9b24-3c940e5ef271
Parent UUID: base
Format:      VDI
Location:    /home/barnett/VirtualBox VMs/win-xp_ie8_dhcp/win-xp_ie8_dhcp.vdi
State:       created
Type:        normal
Usage:       win-xp_ie8_dhcp (UUID: 3c06225a-7b36-474c-9874-80d7eae51312)

Maybe my post wasn't focused enough for anyone to reply. I think I've got it working again by doing this:

- cloned my original VDI that was trying to recover (pre-snapshot) to a new VDI (in case subsequent troubleshooting with it might foul it up)
- cloned a working (but not very configured) ubuntu server VM that I had
- used storageattach put the VDI I wished to recover onto the newcloned VM.
- started it up

Once I'm sure it's fully working, I'll want to remove the inaccessible VM ... which I hope won't be troublesome.

I too had exactly the same thing happen -- tried to do a snapshot (using the --description option), it crashed with an assertion error. When I tried to recover it, my VM became inaccessible.

Code: Select all

> vboxmanage snapshot winserver take ws2k8-base --description "The base WS 2008 system with a started (stalled) installation of latest software"
0%...
Message from syslogd@production at Oct 11 16:43:51 ...
 kernel:

Message from syslogd@production at Oct 11 16:43:51 ...
 kernel:!!Assertion Failed!!

Message from syslogd@production at Oct 11 16:43:51 ...
 kernel:Expression: cRefs == PGMPOOL_TD_CREFS_PHYSEXT

Message from syslogd@production at Oct 11 16:43:51 ...
 kernel:Location  : /home/vbox/vbox-4.1.2/src/VBox/VMM/VMMAll/PGMAllPool.cpp(4073) void pgmPoolTrackPhysExtDerefGCPhys(PGMPOOL*, PGMPOOLPAGE*, PGMPAGE*, uint16_t)

Message from syslogd@production at Oct 11 16:43:51 ...
 kernel:cRefs=0 pPhysPage=W--:RAM:000037426000:009b960:-0000 pPage=00007f8a5e7b15b0:{.idx=46}

Progress state: NS_ERROR_FAILURE
VBoxManage: error: Failed to take snapshot. Error message: Failed to save the machine state to '/data/gateway/winserver/Snapshots/2011-10-11T15-43-51-022102000Z.sav' (VERR_SSM_LIVE_GURU_MEDITATION)

At this point, the VM was still accessible, but in a state where it couldn't be used:

Code: Select all

> service vb-gateway status
State:           guru meditation (since 2011-10-11T15:43:52.493000000)
> service vb-gateway start
VBoxManage: error: The machine 'winserver' is already locked by a session (or being locked or unlocked)
VBoxManage: error: Details: code VBOX_E_INVALID_OBJECT_STATE (0x80bb0007), component Machine, interface IMachine, callee nsISupports
Context: "LaunchVMProcess(a->session, sessionType.raw(), env.raw(), progress.asOutParam())" at line 561 of file VBoxManageMisc.cpp
Windows gateway server "winserver" not started             [FAILED]

So I tried powering it off, and that's when it became inaccessible:

Code: Select all

> vboxmanage controlvm winserver poweroff
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
> service vb-gateway start
VBoxManage: error: Could not find a registered machine named 'winserver'
VBoxManage: error: Details: code VBOX_E_OBJECT_NOT_FOUND (0x80bb0001), component VirtualBox, interface IVirtualBox, callee nsISupports
Context: "FindMachine(Bstr(VMNameOrUuid).raw(), machine.asOutParam())" at line 2322 of file VBoxManageInfo.cpp
VBoxManage: error: Could not find a registered machine named 'winserver'
VBoxManage: error: Details: code VBOX_E_OBJECT_NOT_FOUND (0x80bb0001), component VirtualBox, interface IVirtualBox, callee nsISupports
Context: "FindMachine(Bstr(pszVM).raw(), machine.asOutParam())" at line 544 of file VBoxManageMisc.cpp
Windows gateway server "winserver" not started             [FAILED]

I can't see why it should be marked "inaccessible", as the ownerships and permissions are just the same as they always have been (previously I had problems with an inaccessible VM, and fixing the ownerships solved it).

This is serious, as the VM is a production machine running Windows Server 2k8, and obviously it's not running at the moment! I feel if I knew where to look, I should be able to recover it somehow. But everything looks OK to my untrained eye, and I'm running out of ideas...

Fixed (ie, rescued)!!

As in the OP's case, my VM was inaccessible because the attempt to take the snapshot failed, leaving an invalid child hard disk from the incomplete snapshot. (See his output from 'vboxmanage showvminfo'.) I didn't have a similar VM to clone, so I had to look for other solutions.

Well, it seems obvious now, but it took a while for me to find it. I simply edited the VM's .vbox file 'winserver.vbox' and deleted the child entry from the main hard disk entry:

Extract from 'winserver.box':

Code: Select all

    <MediaRegistry>
      <HardDisks>
        <HardDisk uuid="{1197563e-baa6-4110-92b5-f2d39ae8ce7d}" location="winserver.vdi" format="VDI" type="Normal">
          <HardDisk uuid="{ff51c007-6415-497d-a2c2-2fce05231d4a}" location="Snapshots/{ff51c007-6415-497d-a2c2-2fce05231d4a}.vdi" format="VDI"/>
        </HardDisk>
      </HardDisks>
      <DVDImages/>
      <FloppyImages/>
    </MediaRegistry>

which became, after editing:

Code: Select all

    <MediaRegistry>
      <HardDisks>
        <HardDisk uuid="{1197563e-baa6-4110-92b5-f2d39ae8ce7d}" location="winserver.vdi" format="VDI" type="Normal">
        </HardDisk>
      </HardDisks>
      <DVDImages/>
      <FloppyImages/>
    </MediaRegistry>

Simple, eh? But it solved the problem, my VM was accessible again, and what's more, it started up cleanly first time.

(Remember that I had powered down the VM when it was in the 'guru meditation' state after the failed snapshot, and this was what resulted in its becoming inaccessible. So it was starting up from a powered-off state, which may have made the restart cleaner - your experience may vary.)

Now, does anyone have any idea why our snapshots might have failed? I'm still scared of trying to take another snapshot, and I know I really should be able to do that...
[My setup: Host OS: RHEL6 x64; VirtualBox: version 4.1.2; Guest OS: Windows Server 2008 SE x64]

caisys, I just registered to thank you for helping me out at work today. I wound up in the exact same situation as you, where the VM was inaccessible after trying to snapshot the machine while running. Removing the snapshot lines from the .vbox XML file also fixed it up no problem! Thanks again!

I think you forgot '--pause'

Same issue over here. Thanks for the heads-up on how to recover without destroying the VM.

I found that the snapshot process consistently crashed with this "VERR_SSM_LIVE_GURU_MEDITATION" error if the VM was active (running or paused) during the snapshot process. Specifically, if it has to create the /saved machine state/ file.

Creating a snapshot on a stopped VM works. However, since vboxmanage was my primary API (I'm using that term loosly), I could easily encounter this issue if I didn't sleep after the poweroff and give the VM a chance to settle:

Code: Select all

#!/bin/sh
vboxmanage controlvm $vm poweroff
sleep 0.5
vboxmanage snapshot $vm take $name

This is virtualbox 4.1.18 on Ubuntu Linux 32bit

Thanks caisys. Real lifesaver. I'd tried other solutions with creating new machines and booting from the old VDI file, and they didn't work, but modifying the .vbox files (and .vbox-prev in my case) to remove the snapshot listing worked great.

for me editing the .vbox file didn't work. My solution was to open the vBox Manager, open mass storage, click controller (SATA in my case) and check wheter the vdi file is there. In my case it was not there. I had an empty hard disk there and I removed it. Then I added the right vdi file using the plus sign next to "controller". Then everything was fine. I was able to delete the broken snapshot and the VM started whithout errors.

"the vdi"?

A snapshot VM can hold a whole chain of VDIs. If all you do is reattach the base VDI then you are liable to lose all your work from the moment the very first snapshot was created. Your solution only works if the problem occurs on the first snapshot.

A more sensible approach IMHO is to avoid snapshots altogether: no snapshots == no possibility of snapshot related data corruption. Do proper VM backups to keep your data safe (snapshots are not backups). A VM that doesn't use snapshots has a much simpler control file structure that can easily be reconstituted - if something goes wrong and you don't have a recent enough backup.

it's true, I had only a single snapshot.
Up until now I haven't had lots of problems with snapshots and I don't use the often (for the reasons you mentioned). I think this time I made the mistake to try to remove the snapshot while the VM was running ( I got an error message). The problem with the snapshot occured after that.
Anyways, thanks for your comment...

virtualbox.org

snapshot crashed vm, now inaccessible

snapshot crashed vm, now inaccessible

Re: snapshot crashed vm, now inaccessible

Re: snapshot crashed vm, now inaccessible

Re: snapshot crashed vm, now inaccessible

Re: snapshot crashed vm, now inaccessible

Re: snapshot crashed vm, now inaccessible

Re: snapshot crashed vm, now inaccessible

Re: snapshot crashed vm, now inaccessible

Re: snapshot crashed vm, now inaccessible

Re: snapshot crashed vm, now inaccessible

Re: snapshot crashed vm, now inaccessible