PhysicalDisk and Snapshot Disaster - Pls, I need help.

This is for discussing general topics about how to use VirtualBox.

PhysicalDisk and Snapshot Disaster - Pls, I need help.

Postby GregorK » 8. Jul 2019, 20:20

Hi,

I have a virtual box image, running linux on a windows host.
I have attached 3 physical hard drives to the image (made 3 raw vmdk files referencing to the physical drives).
10 months ago I made a snapshot. After some time I recognized that the virtual machine uses more and more disk space of the host system. The issue is that any changes made on the physical drive are stored in a separated snapshot file from that time I made the snapshot.

How can I fix that? I want to persist the snapshot data to the corresponding physical hard drive.
How can I merge/copy the data from Snapshots/{56cdf3a5-9751-40bf-88e4-8254dfcfa3b2}.vmdk to my hdd/hdd1.vmdk?

Below is the hard disks definition of my virtual machine (.vbox).

For instance:
Hard disk uuid={4b241091-f8ee-460b-834f-649db62ac100} points to hdd/hdd1.vmdk which references to the physical hard drive.
Due to the snapshot any changes are stored to the snapshot file Snapshots/{56cdf3a5-9751-40bf-88e4-8254dfcfa3b2}.vmdk instead to the physical hard drive.

.vbox - HardDisks
Code: Select all   Expand viewCollapse view
      <HardDisks>
        <HardDisk uuid="{f401eb5d-1f9c-42c5-81be-29472cd7b5e1}" location="OpenSuse.vdi" format="VDI" type="Normal">
          <HardDisk uuid="{a7eca8af-da1b-47ab-a9db-301c08061096}" location="Snapshots/{a7eca8af-da1b-47ab-a9db-301c08061096}.vdi" format="VDI">
            <HardDisk uuid="{8db439c7-b5e6-4cd3-ac65-09b92e217322}" location="Snapshots/{8db439c7-b5e6-4cd3-ac65-09b92e217322}.vdi" format="VDI"/>
          </HardDisk>
        </HardDisk>
        <HardDisk uuid="{4b241091-f8ee-460b-834f-649db62ac100}" location="hdd/hdd1.vmdk" format="VMDK" type="Normal">
          <HardDisk uuid="{56cdf3a5-9751-40bf-88e4-8254dfcfa3b2}" location="Snapshots/{56cdf3a5-9751-40bf-88e4-8254dfcfa3b2}.vmdk" format="VMDK"/>
        </HardDisk>
        <HardDisk uuid="{00058bb3-e0e9-4487-89c7-c46b50859f38}" location="hdd/hdd2.vmdk" format="VMDK" type="Normal">
          <HardDisk uuid="{77e654cb-50b2-4536-9be3-533a45012f14}" location="Snapshots/{77e654cb-50b2-4536-9be3-533a45012f14}.vmdk" format="VMDK"/>
        </HardDisk>
        <HardDisk uuid="{b38696c1-c662-4de5-9b07-efcca82d76c8}" location="hdd/hdd3.vmdk" format="VMDK" type="Normal">
          <HardDisk uuid="{064b6e62-074e-42e0-8a75-24ff884875b3}" location="Snapshots/{064b6e62-074e-42e0-8a75-24ff884875b3}.vmdk" format="VMDK"/>
        </HardDisk>
      </HardDisks>


hdd1.vmdk
Code: Select all   Expand viewCollapse view
# Disk DescriptorFile
version=1
CID=0a18555f
parentCID=ffffffff
createType="fullDevice"

# Extent description
RW 7814037168 FLAT "\\.\PhysicalDrive0" 0

# The disk Data Base
#DDB

ddb.virtualHWVersion = "4"
ddb.adapterType="ide"
ddb.geometry.cylinders="16383"
ddb.geometry.heads="16"
ddb.geometry.sectors="63"
ddb.uuid.image="4b241091-f8ee-460b-834f-649db62ac100"
ddb.uuid.parent="00000000-0000-0000-0000-000000000000"
ddb.uuid.modification="1a4036df-5a58-48a9-aa8e-6777d096a27a"
ddb.uuid.parentmodification="00000000-0000-0000-0000-000000000000"
ddb.geometry.biosCylinders="1024"
ddb.geometry.biosHeads="255"
ddb.geometry.biosSectors="63"



Disk Descriptor of the Snapshots/{56cdf3a5-9751-40bf-88e4-8254dfcfa3b2}.vmdk
Code: Select all   Expand viewCollapse view
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
# Disk DescriptorFile
version=1
CID=83376400
parentCID=ffffffff
createType="monolithicSparse"

# Extent description
RW 7814037168 SPARSE "{56cdf3a5-9751-40bf-88e4-8254dfcfa3b2}.vmdk"

# The disk Data Base
#DDB

ddb.virtualHWVersion = "4"
ddb.adapterType="ide"
ddb.geometry.cylinders="16383"
ddb.geometry.heads="16"
ddb.geometry.sectors="63"
ddb.geometry.biosCylinders="1024"
ddb.geometry.biosHeads="255"
ddb.geometry.biosSectors="63"
ddb.uuid.image="56cdf3a5-9751-40bf-88e4-8254dfcfa3b2"
ddb.uuid.parent="4b241091-f8ee-460b-834f-649db62ac100"
ddb.uuid.modification="be19e12c-84b9-4223-a64d-bf40b6cd07ed"
ddb.uuid.parentmodification="1a4036df-5a58-48a9-aa8e-6777d096a27a"
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@... more binary data

Please, please I need you help guys.
Last edited by GregorK on 8. Jul 2019, 21:30, edited 1 time in total.
GregorK
 
Posts: 3
Joined: 8. Jul 2019, 19:53

Re: PhysicalDisk and Snapshot Disaster - Pls, I need help.

Postby scottgus1 » 8. Jul 2019, 20:56

Here is my suggestion after playing around with some snapshots in a throwaway guest. You should make a throwaway guest and experiment with snapshots yourself, too, so you know what may happen.

I was able to have two snapshots on a DOS guest, and by restoring the snapshots, I was able to toggle back and forth restoring one or the other snapshot and have a certain text file on the disk or not have it. Then I deleted snapshots, starting with the end one, and got back to a state with no snapshots. (I also ended back at the state where my file wasn't there, as I had when I started the experiment, so strong precautions about backups apply to your case.)

So I think there is a way to get you back to having no snapshots, but you will have to be careful.

First, I would back up the guest's data with disk imaging software within the guest. Install something in a throwaway guest and learn to use it well, both taking and restoring images in the scrap guest. You may want to boot the 3-raw-disk guest with the imaging software's restore disk to see if the software is compatible with what you did with the 3 raw disks. Try doing a backup and restore while booted from the restore disk so your data is not live while the imaging is taking place.

Attach a new virtual disk to your snapshotted-3-raw-disk guest, with the virtual disk file stored on a separate physical disk. Use the imaging software either in the running guest or booted from the software's restore disk while the guest data is cold, to take a full disk image of the guest's contents. For safety's sake, attach that virtual disk to another throwaway guest and see if it restores properly. I would make two disk images on two different physical disks as a safety buffer and confirm they can both restore properly to fresh new guests.

Now, you can delete the snapshot on your 3-raw-disk guest. This delete may merge the data from the snapshot back into the 3-raw-disk set and get rid of the snapshot. Or it may just get rid of the snapshot and not merge the snapshot's data. If data is missing after the snapshot file is gone, use the disk images to restore the full data set back into the 3-raw-disk guest.

FWIW, you may want to wait a little and see if someone more knowledgeable than I shows up....
Human government is like that crazy uncle who hides a quarter in his fist behind his back, then asks you to guess which fist the quarter is in...
No matter which side you choose, Left or Right, both Sides are empty.
scottgus1
Volunteer
 
Posts: 4969
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Win7

Re: PhysicalDisk and Snapshot Disaster - Pls, I need help.

Postby scottgus1 » 8. Jul 2019, 21:23

I just did another experiment that made things turn out a bit better. I tried again with the snapshot but just did one snapshot. After I made the snapshot I made a special text file on the guest. I was able to reboot the guest and see the special file. Then I deleted the snapshot. On reboot I found the special file merged over.

So deleting does seem to merge. That said, take lots of restorable backups....
Human government is like that crazy uncle who hides a quarter in his fist behind his back, then asks you to guess which fist the quarter is in...
No matter which side you choose, Left or Right, both Sides are empty.
scottgus1
Volunteer
 
Posts: 4969
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Win7

Re: PhysicalDisk and Snapshot Disaster - Pls, I need help.

Postby GregorK » 8. Jul 2019, 21:34

scottgus1 wrote:I just did another experiment that made things turn out a bit better. I tried again with the snapshot but just did one snapshot. After I made the snapshot I made a special text file on the guest. I was able to reboot the guest and see the special file. Then I deleted the snapshot. On reboot I found the special file merged over.

So deleting does seem to merge. That said, take lots of restorable backups....


Thank you very much for your help and reply.
Have you tested it with a RAW-VMDK referencing to a physical drive?
Something like that
Code: Select all   Expand viewCollapse view
VBoxManage internalcommands createrawvmdk -filename "d:\hdd\hdd1.vmdk" -rawdisk \\.\PhysicalDrive0
GregorK
 
Posts: 3
Joined: 8. Jul 2019, 19:53

Re: PhysicalDisk and Snapshot Disaster - Pls, I need help.

Postby scottgus1 » 8. Jul 2019, 23:10

I have done raw-disk before, but not with a snapshot. Take lots of backups.
Human government is like that crazy uncle who hides a quarter in his fist behind his back, then asks you to guess which fist the quarter is in...
No matter which side you choose, Left or Right, both Sides are empty.
scottgus1
Volunteer
 
Posts: 4969
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Win7

Re: PhysicalDisk and Snapshot Disaster - Pls, I need help.

Postby GregorK » 9. Jul 2019, 08:39

I copied all files of my virtual machine to another ssd. I rewrited the VirtualBox.xml located in the user directory in order to read the VM from the new destination.
I removed any snapshot which took some time due to the merging. Next I started my system which failed due to some file system issues. I could fixed them during the night and now my VM is running without any snapshot.

I don't know what exactly happened with the snapshots corresponding to the physical drives. I am not sure whether a merge happened which sounds impossible for me because the host is not able to read the file system of the physical drives (ext4) and the VM is not running during the merge process. Somehow I haven't lost any data as far as I know.

Thank you Scott for your help!
GregorK
 
Posts: 3
Joined: 8. Jul 2019, 19:53

Re: PhysicalDisk and Snapshot Disaster - Pls, I need help.

Postby scottgus1 » 9. Jul 2019, 13:15

Glad to help! As long as the data is there, that's the important part. I noticed a snapshot file in my guest's Snapshots folder while I had a snapshot, and then after I deleted the snapshot the snapshot file was gone. So if your guest's Snapshots folder is empty then it looks like you're back to no snapshots.

If I read right, the snapshots are changed disk sectors that don't care about the filesystem of the data recorded therein. An ext4 filesystem is the same as an NTFS or a FAT32 to a disk sector. So the host OS doesn't have to understand the guest filesystem to merge the disk sectors.
Human government is like that crazy uncle who hides a quarter in his fist behind his back, then asks you to guess which fist the quarter is in...
No matter which side you choose, Left or Right, both Sides are empty.
scottgus1
Volunteer
 
Posts: 4969
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Win7

Re: PhysicalDisk and Snapshot Disaster - Pls, I need help.

Postby socratis » 10. Jul 2019, 09:14

scottgus1 wrote:So the host OS doesn't have to understand the guest filesystem to merge the disk sectors
Correct. Snapshots, and virtual disks in general can be anything, the host doesn't need to know the filesystem. It would be impossible to run a DOS VM on an OSX host, or an OSX VM on a Linux host (on Apple hardware of course).

I wanted to comment on this thread for quite some time, but I don't have enough time at the moment.
GregorK wrote:Due to the snapshot any changes are stored to the snapshot file Snapshots/{56cdf3a5-9751-40bf-88e4-8254dfcfa3b2}.vmdk instead to the physical hard drive.
It was a really BadIdea™ to have a snapshot on a raw disk. If something goes wrong (and it's easier than you think), your disk could be left inaccessible. See Snapshot Basics.

It would be a *really* BadIdea™ if you were storing the snapshots on the same medium. You're supposed to "freeze" the medium with a snapshot, how can you keep on modifying a frozen medium? That's a clear recipe for a disaster! You're lucky that either by ignorance or by accident you didn't change the default Snapshot Folder in your settings. I think it's the latter, because your knowledge of manipulating the XML files shows that you have some know-how.

GregorK wrote:Have you tested it with a RAW-VMDK referencing to a physical drive?
I have. With a USB stick that I couldn't care less if it was lost, not with my actual hard drive which (let me guess) you didn't have a sector-by-sector backup. I did it, noticed the behavior, noticed where/how things were changing. Enough to know how this feature works, and enough to know to avoid it like the plague. And mind you, this is coming from someone that supports the use of (limited) snapshots. ;)

GregorK wrote:I don't know what exactly happened with the snapshots corresponding to the physical drives ... Somehow I haven't lost any data as far as I know
I honestly think that you got really lucky and were able to restore your setup to something that didn't mainly have corrupted filesystem indexes. Glad you got it going without any obvious side-effects, marking as [Resolved].
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
socratis
Site Moderator
 
Posts: 27451
Joined: 22. Oct 2010, 11:03
Location: Greece
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Win(*>98), Linux*, OSX>10.5


Return to Using VirtualBox

Who is online

Users browsing this forum: mpack and 13 guests