Cloning a VM with multiple snapshots?

This is for discussing general topics about how to use VirtualBox.
Corey Murtagh
Posts: 4
Joined: 28. Jun 2010, 04:11
Primary OS: MS Windows 2003
VBox Version: VirtualBox+Oracle ExtPack
Guest OSses: MS Windows 2003

Re: Cloning a VM with multiple snapshots?

Post by Corey Murtagh »

The original VM is a test environment in development with snapshots for various parts of its progress. We're still refining the upgrade process for a major piece of software, and having those snapshots available for testing individual steps will make my life MUCH simpler in the long run. Space is not an issue (I have a 1TB drive devoted to this project), so having snapshots taking up space is no problem.

The boss decided that I needed to clone at a specific point of the process so that the end users could test functionality to that point, which might end up changing my process. If not for that I wouldn't have been too worried about it.

There are many reasons not to worry about restoring snapshots, but occasionaly it's simpler to have them than not. Main reason I spent the time on it is because I goofed and needed a way to undo the goof. Now that this thread has shown me a better way I can avoid such problems in future... but to err is human, and no doubt I'll have plenty of opportunities to err again before this project is completed. And if anyone else goofs the same way and is stuck for a solution, this might be of some minor assistance.
mpack
Site Moderator
Posts: 39134
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: VirtualBox+Oracle ExtPack
Guest OSses: Mostly XP

Re: Cloning a VM with multiple snapshots?

Post by mpack »

You may have missed my main point. I don't doubt that you have good reason to make backups, my question is why you would take the risk of storing your backups in such an insecure way, especially when you say that disk space is not an issue, and you've been caught by snapshot fragility at least once already?

You have found that snapshots make recovery difficult, otherwise your Excel tool would not be needed - but you seem to have drawn the wrong lesson from it: instead of concluding that snapshots cause increased risk and difficulty for no obvious benefit, you seem to have decided that an extra tool will make the increased risk and difficulty more acceptable. That choice seems odd to me!

It's like people have a blind spot: VirtualBox provides a snapshot feature therefore they have to use it, despite its many faults. And use it for important backups too!
Last edited by mpack on 29. Jun 2010, 14:23, edited 1 time in total.
bdforbes
Posts: 1
Joined: 24. Mar 2010, 13:05
Primary OS: Ubuntu other
VBox Version: OSE other
Guest OSses: WinXp, Win7

Re: Cloning a VM with multiple snapshots?

Post by bdforbes »

Is it possible to clone snapshots without flattening them? I.e. retaining the entire hierarchy of snapshots? It seems that the only issue is changing the UUID of the differencing VDIs. Can I manually go in and change the UUID stored in the VDI?
mpack
Site Moderator
Posts: 39134
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: VirtualBox+Oracle ExtPack
Guest OSses: Mostly XP

Re: Cloning a VM with multiple snapshots?

Post by mpack »

The only way to clone a snapshot tree is to copy the whole lot to another host. There is no tool provided to change the UUIDs in a snapshot tree, and in any case that would not be enough - the settings xml must also be structured correctly for a VM which has used snapshots, basically the VM.xml is significantly more complex. It too would have to be cloned, and make reference to the correct disk image UUIDs.
Corey Murtagh
Posts: 4
Joined: 28. Jun 2010, 04:11
Primary OS: MS Windows 2003
VBox Version: VirtualBox+Oracle ExtPack
Guest OSses: MS Windows 2003

Re: Cloning a VM with multiple snapshots?

Post by Corey Murtagh »

mpack wrote:You may have missed my main point. I don't doubt that you have good reason to make backups, my question is why you would take the risk of storing your backups in such an insecure way, especially when you say that disk space is not an issue, and you've been caught by snapshot fragility at least once already?
I come from a VMWare background, so finding that snapshots are not as simple in VirtualBox is something I'm still coming to terms with.
mpack wrote:You have found that snapshots make recovery difficult, otherwise your Excel tool would not be needed - but you seem to have drawn the wrong lesson from it: instead of concluding that snapshots cause increased risk and difficulty for no obvious benefit, you seem to have decided that an extra tool will make the increased risk and difficulty more acceptable. That choice seems odd to me!
Far from it. I understand now - or at least better comprehend - that snapshots on VirtualBox are not as user-friendly as I once thought. They are still useful things to my mind if for no other reason than I can backstep through the development of the system to identify the point at which things have broken. There are other reasons to use them, and if I had it to do over again (as I likely will in the future) I would likely do the same.

The quick-n-nasty tool to rebuild the HardDrive entries in my VirtualBox.xml file is not a reason to support using snapshots, or to excuse their faults, but simply a way to get out of a jam I got into because of imperfect understanding of issues with VirtualBox's snapshot handling. I'm used to a VM being a fairly self-contained thing, with all of the required information stored in a single place. Finding that this wasn't the case I needed to undo my goof, and the VBA script helped with that.
mpack wrote:It's like people have a blind spot: VirtualBox provides a snapshot feature therefore they have to use it, despite its many faults. And use it for important backups too!
The blind spot, if you want to call it that, is more along the lines of expecting that VirtualBox would act in a fashion similar to other virtualisation environments with regard to snapshots. Had I done on VMWare what I did with VirtualBox I wouldn't have had the issue I did. Having encountered the problem I found a solution, and thought it might be of use to others.

In the last week or so I've found plenty of references to the problem with snapshots and cloning, and there's some great advice available on the subject(s). This would tend to indicate that others have used snapshots fairly extensively. Would I do so in a production environment? No. But my test system is just that, and snapshotting it is damn handy.

I really don't want to be confrontational mpack, but I don't believe that your opinion on the issue is widely shared.
Corey Murtagh
Posts: 4
Joined: 28. Jun 2010, 04:11
Primary OS: MS Windows 2003
VBox Version: VirtualBox+Oracle ExtPack
Guest OSses: MS Windows 2003

Re: Cloning a VM with multiple snapshots?

Post by Corey Murtagh »

bdforbes wrote:Is it possible to clone snapshots without flattening them? I.e. retaining the entire hierarchy of snapshots? It seems that the only issue is changing the UUID of the differencing VDIs. Can I manually go in and change the UUID stored in the VDI?
Short Answer:
No. Don't do it.

Long Answer:
You need to modify not only the VDI files - change UUID for each VDI, and for each child/parent VDI - but tinker with the snapshot entries in the VM's xml file and modify VirtualBox.xml to reflect all of the changes. As there are no tools to do this, you will probably find that it's going to require a lot of learning about the file formats, interdependencies and so on.

With all that in mind, I would suggest that you think carefully before trying something like this. Unless of course you are a code wizard with a real desire to hack on the VirtualBox source to ferret out all the details of VDIs and snapshots in general. Everyone else (myself included) should probably stand well clear of the whole thing :?
jnygaard
Posts: 6
Joined: 27. Oct 2010, 23:55
Primary OS: Ubuntu other
VBox Version: OSE other
Guest OSses: Windoze

Re: Cloning a VM with multiple snapshots?

Post by jnygaard »

The amazing thing is that nobody has made a tool (or command in the GUI) for doing exactly this. Collecting all these more or less cryptic steps needed, into a simple to use command shouldn't be that hard.
Perryg
Site Moderator
Posts: 34369
Joined: 6. Sep 2008, 22:55
Primary OS: Linux other
VBox Version: OSE self-compiled
Guest OSses: *NIX

Re: Cloning a VM with multiple snapshots?

Post by Perryg »

I am sure everyone will praise your name as soon as you get it done and posted. When do you think you will have it ready for testing?
jnygaard
Posts: 6
Joined: 27. Oct 2010, 23:55
Primary OS: Ubuntu other
VBox Version: OSE other
Guest OSses: Windoze

Re: Cloning a VM with multiple snapshots?

Post by jnygaard »

Hehe... As they always say... it'll be ready when it's ready... :-)
Perryg
Site Moderator
Posts: 34369
Joined: 6. Sep 2008, 22:55
Primary OS: Linux other
VBox Version: OSE self-compiled
Guest OSses: *NIX

Re: Cloning a VM with multiple snapshots?

Post by Perryg »

Ah and I thought since you said it "should not be that hard" you could just whip it up tonight and by tomorrow we would be testing it.
The current source code is always posted here in case you change your mind.
mpack
Site Moderator
Posts: 39134
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: VirtualBox+Oracle ExtPack
Guest OSses: Mostly XP

Re: Cloning a VM with multiple snapshots?

Post by mpack »

jnygaard wrote:The amazing thing is that nobody has made a tool (or command in the GUI) for doing exactly this
No, the really amazing thing to me is that anyone would want to do it at all, with or without an automated tool. The downsides of snapshots are not eliminated just because you hide them behind a GUI.
jnygaard
Posts: 6
Joined: 27. Oct 2010, 23:55
Primary OS: Ubuntu other
VBox Version: OSE other
Guest OSses: Windoze

Re: Cloning a VM with multiple snapshots?

Post by jnygaard »

Ok, now I find myself in a situation where I think I need this cloning-snapshorts-thingy. Hopefully I'm wrong, and somebody can point me in the direction of another solution...

I have installed a guest and a lot of tools. In order to be able to go back a step or two if I mess up things under installation, I have made snapshots after each major work item. For example, I made a snapshot after installing VS 2010, in case I had to redo it due to some unforeseen sub-package I had forgot to install or something. This leaves me with a nicely working VB and about 20 snapshots, on which I can do my development tasks.

However, now I need to test a software piece I built on this VB, but preferably on a "clean" machine not containing my development settings and tools. This corresponds to my snapshot number 7.

How do I do this? Discard (i.e. merge) snapshots 1-7, then clone, then delete remaining snapshots for one of the clones (to become the "clean" test bed) on which I can do my testing? Except that this will not work since I cannot clone a machine with snapshots, right?

J.
mpack
Site Moderator
Posts: 39134
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: VirtualBox+Oracle ExtPack
Guest OSses: Mostly XP

Re: Cloning a VM with multiple snapshots?

Post by mpack »

"How you do this" was answered by Mark Cranness's post earlier in the thread.
daveparrish
Posts: 3
Joined: 24. Mar 2011, 18:52
Primary OS: Ubuntu other
VBox Version: OSE other
Guest OSses: CentOS, Ubuntu, Debian, Windows

Re: Cloning a VM with multiple snapshots?

Post by daveparrish »

MarkCranness,

Thank you for the excellent post on how to clone a virtual hard drive from a snapshot. The steps were almost exactly what I had to do. The only difference was that the file I needed to look through for the snapshot harddrive image UUID was <Machine Name>.vbox now. And the location of this file was was ~/VirtualBox VMs/<Machine Name>/<Machine Name>.vbox

This was exactly what I need to know to make backups and send the VM to other people. Thanks again. :D

BTW, I'm using Ubuntu 10.10 with Virtualbox 4.0.4.
alpha_YbSXf
Posts: 5
Joined: 1. Mar 2018, 20:32

Re: Cloning a VM with multiple snapshots?

Post by alpha_YbSXf »

To clone a VM while retaining the snapshots from the commandline, use the following command:

`C:\Program Files\Oracle\VirtualBox\VBoxManage.exe clonevm old_VM_name --mode all new_VM_name --basefolder "C:\Users\username\VirtualBox VMs" --register`


Heres the explanation

`--mode all`: states that all snapshots and folders should be copied.

`--basefolder "C:\Users\username\VirtualBox VMs"` : mentions where the VM folder should be located

`--register`: registers the VM so that you can use it immediately after the cloning process from the VirtualBox GUI or CLI
Locked