Using Cloning to Merge Snapshots

[jnl2]

I'm a relative newb to managing VBox especially in terms of utilizing snapshots over time.

I have a VBox VM environment where over 15 snapshots have been taken over time, some while the client was running, most while the client was off.

My goal, to back up a few states of the VM for obvious record keeping purposes and to off-load these copies to save disk space.

The end result desired is two fold:

1. A few early states/versions of the client backed up and off-load. (The formal documentation seem clear that this could be achieved by rolling back the VM to earlier states to achieve this goal.)

2. One current consolidated state running on the client.

An article found on the SuperUsers blog site describes an intriguing approach that I might be able to utilize toward my ends. (giving credit to that author, their suggestion can be found here):
https://superuser.com/questions/590968/ ... wer-684128

Here's a technique (tested on VirtualBox 4.3.4) that uses the 'Clone' command to merge and compact all the snapshots to a new VM containing a single VDI file:

Rename the VM so you can reuse its current name for cloned VM (you can change the VM name from the 'Basic' tab of the 'General' settings).

Right-click on the VM and run the 'Clone...' command:
Set the name to the VM's original name.
Accept the remaining defaults i.e. Clone type: 'Full clone'; Snapshots: 'Current machine state'.

Delete the old VM (right-click on the VM and run the 'Remove...' command).

In order to validate this approach, does our community support the notion to use cloning to merge snapshots in order for me to achieve my ultimate goals?

Thanks very much for your help.

[mpack]

Most experienced users don't use snapshots at all, so the best method of merging them doesn't arise: though of course the method mentioned in the link will work since it's the same method copied from the tutorials on this site.

When the goal is to maintain one self consistent state, with backups of current and previous states, then I'd propose to do exactly that. There's no mystery to it: just use the host OS to copy the VM folder to secondary storage every time you want to make a backup.

[jnl2]

Thank you again mpack for your continued input and guidance.

Originally, I didn't think this particular client would have last this long. I gather that most folks use a client as a 'use and dispose off' platform. My client was been in use for over a year and has had many resources added to it. Backing it up initially was not in the original plans, but now I'm trying to figure out ways to both continue to use the client with it possessing a smaller footprint (disk space consumed) as well as producing a few earlier copies of the client environment in order to bloom those version later should I want.

Just to be clear with one of your last comments, you said:

... just use the host OS to copy the VM folder to secondary storage every time you want to make a backup.

The "VM folder" referred to, is that the directory titled Snapshots? Or are you referring to the next higher directory to be backed-up/copied?

Have a great day.

[socratis]

The "VM folder" referred to, is that the directory titled Snapshots? Or are you referring to the next higher directory to be backed-up/copied?

The latter. Snapshots are not backups!

See the FAQ "Moving a VM" and interpret it as "Backing up a VM". For a complete VirtualBox backup, you should also back up the "VirtualBox.xml" file. The location of that file depends on the host, see ch. 10.1. Where VirtualBox stores its files of the User Manual.

[jnl2]

Not clear on the response to the question posed seeking clarity on mpack's suggestion above in terms of backing up ones VM:

...just use the host OS to copy the VM folder to secondary storage every time you want to make a backup.

Utilizing this screenshot image as reference of a directory on my host, is he suggesting copying/off-loading/backing up this entire "VBox Win" (my naming) directory?
Link: https://drive.google.com/file/d/1zVMOlU ... sp=sharing

Thanks!

[mpack]

is he suggesting copying/off-loading/backing up this entire "VBox Win" (my naming) directory?

Yes. The VM folder.

[jnl2]

Uh oh..... in just the last month it appears that my system's HD has gotten into a position of being nearly full. Yes, I know I should have been more aware of the HD's health and frankly, I'm not quite sure how it got to this state other than capturing and retaining about 12 snaphots of the VM over time.

I seem to recall setting up the VM's environment to used half of the system's HD's space; 400GB for the VM, 400GB for my host. This, my first ever installation of a VM, took place in 2016. In retrospect, I "may" have set up the virtual disk to be dynamic, but maybe I didn't or maybe I did something in error.

A screenshot image depicting VM's folder on the host is attached for illustration purposes: https://drive.google.com/open?id=1ctF0l ... Hr1Ce68s2f
Additionally, a screenshot image of the Snapshot folder on the host is provided: https://drive.google.com/open?id=1cuhD8 ... A6lB-Tacbq
Lastly, a screenshot image of the client's HD is also provided: https://drive.google.com/open?id=1Kw37M ... mkJzEvcKDj

As one can see, the VM folder takes up ~760GB in total on the host HD with the .vdi file and snapshots folder consuming ~420GB and ~340GB respectively. But within the client, the HD depicts that only 55GB are being used.

My approaches to reduce the client has been to attempt to delete snapshots out right, make a clone of the current VM and move the clone to a new virtual drive that is for sure small initially as well as being dynamic coupled with deleting the original virtual harddrive and its VM. However, the host does not like any of these approaches now due to not having enough memory.

Is there anything I can do to reduce the VM's footprint while retaining the current state at the least short of buying an expensive +1TB external HD to export the VM to (not clear on exporting as a process and as a useful tool in this case) or just plain wiping the client off the host and starting an arduous process of trying to rebuild a facsimile of the VM again?

Aside, simpler steps of off-loading and deleting files from the client were already taken recently.

How screwed am I?

[socratis]

I merged your question with the previous thread that you had opened. It's the same issue, and it give it more context. Plus there is a solution in the previous thread/answers, so I'm not sure what the problem is now...

Also, check out the thread "Out of space on HD. How can I get rid of snapshots". It seems that everybody these days are skipping over checking their available HD space...

[jnl2]

Yes, I read that thread as well as a few others on this board and across a few additional sites for other VM tools.

The final conclusion appears to be something that I was trying to avoid, additional costs. Being un-employed/under-employed, spending ~$100 for a 2TB external DAS to download and clone, condense, what have you my VM was not something that I was aware of as a possible outcome when VBox was introduced to me. Never did my acquaintance that introduced me to VMs nor did the literature initially read to set up a VBox VM was it mention that it's generally not good practice to hold a set of snapshots on your host nor was it made apparent the relationship between the size of snapshots and the free disk space needed to perform rollbacks, deletions or clones.

So, I'd say that after wrestling with this issue for a week, investing two hours a day, I'm disappointed with the VBox experience and now have a maxed out HD. I'm faced with now dumping more time into deleting VBox entirely off my system and re-installing software and files onto a host only system that would apparently only consume 100GB out of my 900GB HD or spend money for an external HD and more time learning to perform exports/imports, cloning, etc when I'm not being paid to perform these gyrations.

Somewhere in the discourse of whether or not to use a VBox VM and/or definitely in the literature for utilizing snapshots it should be mentioned in bold print some basics the importance for managing ones total VM footprint with relation to their overall HD space.

[socratis]

I'm a born optimist, so I see this as an opportunity; an opportunity to learn a lesson to keep an eye on your system's resources.

when I'm not being paid to perform these gyrations.

You could make a suggestion to Oracle to hire/pay beta testers (other users too while you're at it), for their free VirtualBox offering. You never know, they have deep pockets...

[jnl2]

I'm not a fan of "corporate speak" nor is it considerate to offer that valed jab.

For volunteers to give their time freely, that is highly commendable.

I'll stand with the sentiment offered earlier, it should be boldly marked in the documentation the necessity for certain amounts of free memory relative to snapshot(s) size(s) to be available on the host in order to perform roll-back, clones, etc.


key: not enough memory to perform clone roll-back delete snapshots snap shot

[socratis]

it should be boldly marked in the documentation the necessity for certain amounts of free memory relative to snapshot(s) size(s) to be available on the host in order to perform roll-back, clones, etc.

You mean like the following from ch. "5.5 Differencing images"? Emphasis mine:

If you later delete a snapshot in order to free disk space, for each disk attachment, one of the differencing images becomes obsolete. In this case, the differencing image of the disk attachment cannot simply be deleted. Instead, VirtualBox needs to look at each sector of the differencing image and needs to copy it back into its parent; this is called “merging” images and can be a potentially lengthy process, depending on how large the differencing image is. It can also temporarily need a considerable amount of extra disk space, before the differencing image obsoleted by the merge operation is deleted.

Someone will then complain about the length of time required, someone will complain about the previous image becoming obsolete, etc. And before you know it, the whole manual is going to be in bold. Which defeats the purpose...