CloneVDI tool - Discussion & Support

Discussions related to using VirtualBox on Windows hosts.
Stamimail
Posts: 93
Joined: 17. Aug 2011, 11:53
Primary OS: MS Windows 7
VBox Version: OSE other
Guest OSses: W7

Re: CloneVDI tool - Discussion & Support

Post by Stamimail »

Can we have also a CloneVM tool?

Compacting a VM with Snapshots is not an easy proccess.
I don't know if keeping the structure of the VM (Base VDI + Snapshots) can be made when trying to compact a VM with Snapshots, but at least, it seems that the Current State can be cloned to a new single VDI. Can we have auto-compacting while VM cloning process?
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: CloneVDI tool - Discussion & Support

Post by mpack »

Stamimail wrote:Can we have also a CloneVM tool?
Compacting a VM with Snapshots is not an easy proccess.
It's actually quite easy. 1. Clone the VM using VirtualBox, do not preserve snapshots. Then 2. Clone and compact the VDI in place (same name for source and destination VDI, +Compact, +Keep UUID).

Compacting a VM while maintaining the snapshot structure is non trivial, and not really useful either.

I consider maintaining VirtualBox xml structures to be beyond the CloneVDI remit, so I won't be offering a clone VM feature in CloneVDI, nor will I be deleting snapshots from a VM xml: as the release notes tell you, if you clone a snapshot VDI you must use the clone in a new VM, not the existing one.

Final note: I've said it many times, the easiest way to avoid difficulties raised by snapshots is to stop using snapshots. There are IMHO no good long term reasons to use them.
Stamimail
Posts: 93
Joined: 17. Aug 2011, 11:53
Primary OS: MS Windows 7
VBox Version: OSE other
Guest OSses: W7

Re: CloneVDI tool - Discussion & Support

Post by Stamimail »

mpack wrote:It's actually quite easy. 1. Clone the VM using VirtualBox, do not preserve snapshots. Then 2. Clone and compact the VDI in place (same name for source and destination VDI, +Compact, +Keep UUID).
Yes, but currently it means there are 2 processes of cloning, and I'm asking to do it by only one process of cloning.
Final note: I've said it many times, the easiest way to avoid difficulties raised by snapshots is to stop using snapshots. There are IMHO no good long term reasons to use them.
I'm using snapshots for:
1. fast return to a new clean OS.
2. as a fast BackUp for the base VDI. Only when I'm sure that the changes I've made in the new state are desired, I merging the snapshot with the base vdi, and straight afterwards I making a new snapshot to preserve the Base VDI from changes.
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: CloneVDI tool - Discussion & Support

Post by mpack »

I can "fast" return to a new clean OS without using snapshots. I simply mentally disregard the clone and return my attention to the original VM. However, this topic is not a forum for discussing snapshots, so let's leave it there.
Stamimail
Posts: 93
Joined: 17. Aug 2011, 11:53
Primary OS: MS Windows 7
VBox Version: OSE other
Guest OSses: W7

Re: CloneVDI tool - Discussion & Support

Post by Stamimail »

1. You did not answer the main question/request: VM with snapshots - Can it be cloned and compacted to a new compacted VM/VDI by a single cloning?
2. If I want to keep the VM with the snapshots: cloning and compacting just the base VDI by CloneVDI tool - can it cause damage for the snapshots?
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: CloneVDI tool - Discussion & Support

Post by mpack »

Yes I did answer (1) already - see third paragraph of my first response: but if that wasn't clear then here it is again. The answer to "can CloneVDI clone VMs" is no, and nor do I plan to add that feature in future. You can use CloneVDI to clone VDIs (including snapshot VDIs). You cannot use it to clone VMs.

As to (2), I answered that already also. I pointed out that the release notes tell you explicitly not to do this. Presumably that is a sufficient indication that it isn't something I encourage you to do.

If you clone a snapshot VDI using CloneVDI then the result is a merged, stand alone VDI - not a snapshot VDI. Putting this non snapshot VDI into a snapshot chain may appear to work (in the short term), but things will break downstream when you later try to do snapshot things (like revert or merge) to a file that isn't a snapshot. DON'T DO IT! I don't know how much clearer I can be on this.
Stamimail
Posts: 93
Joined: 17. Aug 2011, 11:53
Primary OS: MS Windows 7
VBox Version: OSE other
Guest OSses: W7

Re: CloneVDI tool - Discussion & Support

Post by Stamimail »

If you clone a snapshot VDI using CloneVDI then the result is a merged, stand alone VDI - not a snapshot VDI.
What did you mean?
I tried now to clone a snapshot VDI {b97920cd-31ab-4206-9ed9-6fb994ddb29f}.vdi and I got Error:
"Source is a difference image. Sorry, this tool cannot clone these (for now)"
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: CloneVDI tool - Discussion & Support

Post by mpack »

Stamimail wrote:
If you clone a snapshot VDI using CloneVDI then the result is a merged, stand alone VDI - not a snapshot VDI.
What did you mean?
Look, it irritates me when people ask me to repeat myself, when I'm pretty sure I was clear the first time. Plus it's there in writing, so it's not like you didn't hear. Adjust your preconceptions, absorb what I said: CloneVDI can read snapshot VDI files, but it never creates them. So you must not use the output of CloneVDI in a snapshot chain. If the original was a snapshot then the clone must be used in a new VM.
Stamimail wrote: I tried now to clone a snapshot VDI {b97920cd-31ab-4206-9ed9-6fb994ddb29f}.vdi and I got Error:
"Source is a difference image. Sorry, this tool cannot clone these (for now)"
You get this error if you select a snapshot VDI and CloneVDI can't see the rest of the snapshot chain in the same folder.

I think you probably need to do some more reading to understand what snapshots are. They are not stand-alone files, they are more like patch scripts.
  • You start off with a base VDI. This file is frozen when you create your first snapshot.
  • After you create your first snapshot, then every time you write to the chain [[base]first] the changes (at a sector level) get written to the [first] file. In effect [first] is like a script which can be used to patch [base] in order to get a complete and current disk image. Notice how you need to think of the chain as your disk image. Individual snapshot element files are not useful on their own.
  • If you again create a new snapshot then [first] is frozen and the new chain is [[[base]first]second].
When CloneVDI is asked to clone the [second] state it will reads data from all elements in the chain so that the clone is a merged/flattened image. It is not a patch file, not a difference state. If you were able to insert the clone into a VM in place of [second] then of course it has all of the same data available as the [[[base]first]second] chain, so the VM will boot and it will appear to work, but it will break the first time you do anything snapshotty to it, because it isn't a snapshot. This is over and above the problems you'll have due to the UUID links being wrong.

Snapshots are inherently unsafe. If you damage [base] then it potentially damages all descendant elements too - patch scripts are meaningless if they don't have the correct input to patch. It was the unsatisfactory nature of snapshots and (at the time) poor support for cloning that made me develop CloneVDI in the first place, to give people an alternative, so you can understand I hope that I'm not keen to spend much time in this topic helping someone use snapshots!
Stamimail
Posts: 93
Joined: 17. Aug 2011, 11:53
Primary OS: MS Windows 7
VBox Version: OSE other
Guest OSses: W7

Re: CloneVDI tool - Discussion & Support

Post by Stamimail »

I think I understood you right. Sorry that things went this way.
I asked What did you mean? because you said we can use CloneVDI to clone snapshot VDIs, But the CloneVDI Error said: "Source is a difference image. Sorry, this tool cannot clone these (for now)"
>>
You get this error if you select a snapshot VDI and CloneVDI can't see the rest of the snapshot chain in the same folder.
This is the answer that was missing and caused the misunderstanding, since in my specific case my (existing) Base Vdi is located outside its VM folder.
The Error message didn't talk about such scenario.
So I copied my Base Vdi file into its VM folder, and tried again to use CloneVDI to clone the snapshot, then I saw it works. :)
The result looks like it indeed merged the Base Vdi and the snapshot file, as you said, but it also looks like it really requires only one process for cloning merging and compacting. But you wrote:
It's actually quite easy. 1. Clone the VM using VirtualBox, do not preserve snapshots. Then 2. Clone and compact the VDI in place (same name for source and destination VDI, +Compact, +Keep UUID).
Why did you write it requires two processes. Are you sure we can't get cloning merging and compaction by only one process = by cloning the (desired state) snapshot VDI? (+Compact is checked)
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: CloneVDI tool - Discussion & Support

Post by mpack »

Step 1: clone (or create anew) the original VM, using VirtualBox or VBoxManage.
Step 2: clone and compact the VDI to use in the VM, using CloneVDI.

I make that two steps. I even numbered the steps the same way in the text you quoted. It's certainly been awhile since someone queried my ability to count to 2.
doc_jochim
Posts: 10
Joined: 2. Sep 2016, 17:44

Re: CloneVDI tool - Discussion & Support

Post by doc_jochim »

@mpack :D

Hi there,

I have to use some older programs that require WindowsXP. So I put my old system into a VHD-container using disk2vhd-utility. The created VHD-image can be run under VirtualBox, but the image-size is growing step by step, although I already uninstalled many programs.

I found the information that there is the possibility to shrink the size by using vboxmanage following the parameters 'modifyhd --compact DRIVE:\PATH\IMAGE.vdi', but...:

a) If I type 'vboxmanage /?', I only find modifymedium. Does the parameter modifyhd really exist in acutal virtualbox?

b) I should have a vdi-container. Compacting an image seems not to work in vhd-files.

So I tried to use clonevdi 2.51, which should have the ability to read vhd-files and to compact the image while building a vdi-copy of the original vhd-container.

But after starting the process by using the GUI's button 'proceed', clonevdi-utility seems to stop living... No more reactions, spinning circle while moving mouse-cursor over the utility's window and 'no response' written at the top of the utility-window!? :?

Task-manager shows a clonevdi-cpu-load of appr. 26%.

Is there anything wrong? How can I find out if the program is still doing something?

Thanks for a short information

Yours

André

PS: sorry about my bad english, but school is already a long time ago... :roll:
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: CloneVDI tool - Discussion & Support

Post by mpack »

This topic is for supporting CloneVDI, not VBoxManage etc, but I'll try to answer your questions briefly.
doc_jochim wrote: I have to use some older programs that require WindowsXP.
This statement is usually wrong. XP offers apps the Win32 api, and that API is still available in Windows 10. CloneVDI itself was "designed for XP", but no-one fails to assume that it'll work anywhere! I'd ask you to explain this assertion, but here is not the right topic for it.
doc_jochim wrote:a) If I type 'vboxmanage /?', I only find modifymedium. Does the parameter modifyhd really exist in acutal virtualbox?
The modifyhd command has been renamed, but the modifyhd form still works. In fact I prefer the less verbose command, even if not technically accurate (it handles more than hds).
doc_jochim wrote:b) I should have a vdi-container. Compacting an image seems not to work in vhd-files.
VBoxManage compaction works on VHDs, but you have to run sdelete first to zero unused blocks. CloneVDI doesn't require this.
doc_jochim wrote:But after starting the process by using the GUI's button 'proceed', clonevdi-utility seems to stop living... Task-manager shows a clonevdi-cpu-load of appr. 26%.
I guess that means you have a quad core CPU, and the core running CloneVDI has locked up, or at least is very slow. I've never seen or had reported that CloneVDI itself does this, so I'd guess your source or destination drive is either VERY slow, or defective.
doc_jochim
Posts: 10
Joined: 2. Sep 2016, 17:44

Re: CloneVDI tool - Discussion & Support

Post by doc_jochim »

This statement is usually wrong. XP offers apps the Win32 api, and that API is still available in Windows 10. CloneVDI itself was "designed for XP", but no-one fails to assume that it'll work anywhere! I'd ask you to explain tbis assertion, but this is not the right topic for it.
As far as I know, 16-bit programs do not run in 64-bit Operating-Systems like Win10pro64bit or Win7pro64bit. I already had to install the VirtualXP-mode offered by Win7 to use the old 16-bit-applications under Win7.
The modifyhd command has been renamed, but the modifyhd form still works.
ok.
VBoxManage compaction works on VHDs, but you have to run sdelete first to zero unused blocks. CloneVDI doesn't require this.
That's why I wanted to use CloneVDI.
I guess that means you have a quad core CPU
Intel Core i5-2500 CPU @ 3.30GHz
the core running CloneVDI has locked up, or at least is very slow.
Why could a core lock up? My CPU should be fast enough!?
I've never seen or had reported that CloneVDI itself does this
Well, I report it the first time ;-)
I'd guess your source or destination drive is either VERY slow, or defective.
System-HD is an Crucial-SSD, Data-HD is a Toshiba-DT01ACA100 SATA3 - seems not to be too slow !? Crystel-Disk-Info doesn't report any defects.

Do you have any idea, what could be the reason for CloneVDI to stop living after starting the copy-operation?

Are there any limitations concerning the size of the vhd-container. My vhd-file ist about 148GB. Does CloneVDI read and explore the whole file at once first, before starting any copy-action - and is perhaps unreachable during this time? Do I just have to wait for the program to give me any feedback, that it is doing something?

André
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: CloneVDI tool - Discussion & Support

Post by mpack »

If it's a 16 bit app then it was designed for Windows 3, not Windows XP. It requires any OS which supports the Win16 API, which is true of every 32bit Windows version, including Windows 10 (32bit). So, as predicted the statement was incorrect: the app did not require XP.
Why could a core lock up? My CPU should be fast enough!?
Speed isn't the issue. The core would lock up because some task is not completing, either at all or in a reasonable time. This would be because of an I/O problem as I already mentioned, and not a CPU problem. The most common I/O problems are corruption of the file or the filesystem, or bad sectors. So try an MD5 checksum test on your VHD file. Do a filesystem check of connected drives.

Everybody who uses CloneVDI uses the file copy function, so I don't accept that you are the first (in 7 years and hundreds of thousands of downloads) to notice what would be a very obvious bug in its primary function.
doc_jochim
Posts: 10
Joined: 2. Sep 2016, 17:44

Re: CloneVDI tool - Discussion & Support

Post by doc_jochim »

If it's a 16 bit app [...] it requires any OS which supports the Win16 API, which is true of every 32bit Windows version, including Windows 10 (32bit). So, as predicted the statement was incorrect: the app did not require XP.
ok - so I can also use any 32bit OS inside the virtualbox. But the advantage of my virtual XP ist the already existing license. To install another more actual OS, I first have to get a license.
Do a filesystem check of connected drives.
Done - no errors.
So try an MD5 checksum test on your VHD file.
Which tool do I have to use to get the MD5-checksum? Or ist there an internal function of windows? Deep inside the dark shadows of my mind I remember that perhaps 7-zip has got such a funktion. I'll search for this feature.
Everybody who uses CloneVDI uses the file copy function
I just tried to uncheck the 'compact drive while copying'-funktions. After clicking on 'proceed' the software immediately started copying and shows a progess-bar.

But if the option 'compact drive while copying' is checked the tool goes to nirwana...

Is there perhaps a correlation between converting from vhd to vdi while simultaneously compacting the virtual drive?

Do I first have to convert from vhd to vdi without compacting and then to copy to another clone without changing fileformat using the compacting-option?

André
Last edited by doc_jochim on 4. Sep 2016, 13:28, edited 1 time in total.
Post Reply