CloneVDI tool - Discussion & Support

Discussions related to using VirtualBox on Windows hosts.

CloneVDI tool - Discussion & Support

Postby mpack » 15. Sep 2009, 23:13

Many Windows users dislike the fact that you currently have to use the Command Prompt to clone VDI files (*). Also, questions often arise related to VDI files such as "how do I increase the maximum size of a VDI?" or "how can I make my dynamic VDI file small again?". As a recent vacation project I decided to try to write a nice simple tool that provided a good answer to all of these questions. Hopefully I succeeded.

(*) [edit: VirtualBox v4.1 finally added GUI support for cloning]

Attached to this message is a zip file containing the software tool that I wrote, which I call "CloneVDI" (currently at version 2.10). Features as of this release include:

  • Reads VDI,VHD,VMDK,Parallels HDD,RAW files and physical drives, writes VDI.
  • A simple GUI Interface.
  • A CLI interface too (for people who like that kind of thing!)
  • Ability to compact (discard unused blocks from) the clone, with no need to run sdelete/zerofree first.
  • Ability to enlarge the maximum size of a virtual disk, optionally enlarging the main partition too.
  • Merge snapshot VDI chain into a single VDI (VHD and VMDK snapshot chains supported too).
  • User choice to either keep the old UUID or generate a new one for the clone.
  • Some useful diagnostics about the source VDI, such as the partition map and the internal fileystem.
  • Better feedback while cloning: a progress bar and an estimated time to completion.
  • It seems to be significantly faster than "VBoxManage clonehd".
  • Open source.
To install, simply create a folder and unpack the zip contents into it. Optionally, you can create a shortcut to link your desktop to the CloneVDI.exe file in the target folder. The zip file includes release notes which hopefully provides answers to all of your questions.

Note that this tool is not simply a GUI front end for VBoxManage. Instead it uses my own code to read and write VDI files (and other formats), and therefore works in a rather different way, with different features and capabilities.

Enjoy. Feel free to ask any questions here, but read the release notes first.

[ Edit: 12th Sep 2012. CloneVDI attachment updated to v2.10. Bugfixes plus support for VMDK and VHD snapshots. ]
[ Edit: 20th June 2011. CloneVDI attachment updated to v2.05. Bugfix release for broken VHD support. ]
[ Edit: 16th May 2011. CloneVDI attachment updated to v2.04. Fix to NTFS partition enlargement. Simplified snapshot cloning. ]
[ Edit: 14th Oct. CloneVDI attachment updated to v2.03. Minor tweak to recognize VirtualBox 3.2.10 (Oracle tagged) VDI files. ]
[ Edit: 1st Mar. CloneVDI attachment updated to v2.02. Fixed another bug in "enlarge partition" feature, completed French language support. ]
[ Edit: 8th Feb. CloneVDI attachment updated to v2.01. Fixed bug in "enlarge partition" feature, added French language support. ]
[ Edit: 31st Jan 2010. CloneVDI attachment updated to v2.00. This version includes several new features and some tidy-ups ]
[ Edit: 10th Dec. CloneVDI attachment updated to v1.43. This version fixes a problem when enlarging drives smaller than 8GB ]
[ Edit: 12th Nov. CloneVDI attachment updated to v1.42. This version adds support for Dutch and German languages ]
[ Edit: 31st Oct. CloneVDI attachment updated to v1.41. This version has several internal tidy-ups, and adds the CLI feature. ]
[ Edit: 14th Oct. CloneVDI attachment updated to v1.40. This version adds RAW read support and a "sector viewer" feature. ]
[ Edit: 07th Oct. CloneVDI attachment updated to v1.30. This version adds VMDK read support. ]
[ Edit: 21st Sep. CloneVDI attachment updated to v1.22. The previous version STILL couldn't handle VHDs bigger than 4GB. ]
[ Edit: 21st Sep. CloneVDI attachment updated to v1.21. The previous version didn't like VHDs bigger than 4GB. ]
[ Edit: 20th Sep. CloneVDI attachment updated to v1.20. This version adds VHD read support. ]
[15th Sep 2009. First public release (CloneVDI v1.10). ]
------
v2.10 source code can now be found attached to the next message.
------
Attachments
CloneVDI-exe-2.10.zip
CloneVDI v2.10, executable with release notes.
(118.2 KiB) Downloaded 50032 times
Last edited by mpack on 14. Sep 2012, 14:32, edited 35 times in total.
mpack
Site Moderator
 
Posts: 13169
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows XP
VBox Version: PUEL
Guest OSses: Mostly XP

Re: CloneVDI tool - Discussion & Support

Postby mpack » 15. Sep 2009, 23:30

Reliability is an obvious concern with an unproven tool such as this. All I can tell you is that I have been using it on my own VDI's, which I would not do if I had any doubts! Also, the tool has been made available to forum volunteers for a week or two now, and none of them have reported a single bug. In fact the tool should be quite reliable: its' basic "modus operandi" is to make a copy of your source VDI and to only ever modify the copy, never the original. So, if you intend to delete the original then you should be quite safe, provided you give the clone a thorough test first. It does extensive validation of both the original VDI and the copy process, which again should lead to solid reliability.

But, it's fast too. :-)

[Edit: source code attached this message, to avoid confusion in the root message. ]
Attachments
CloneVDI-src-v2.10.zip
CloneVDI v2.10, source code.
(236.9 KiB) Downloaded 4352 times
Last edited by mpack on 20. Jun 2011, 21:33, edited 3 times in total.
mpack
Site Moderator
 
Posts: 13169
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows XP
VBox Version: PUEL
Guest OSses: Mostly XP

Re: CloneVDI tool - Discussion & Support

Postby mpack » 15. Sep 2009, 23:32

Please be aware that this tool currently supports VDI files only (not VHD or VMDK). If there is sufficient demand then I may implement support for other formats in the future.

Oh - and while this tool is intended primarly for Windows hosts, I am told that it does work under Wine (Linux users take note).

[edit: VHD/VMDK and RAW read support was added in later versions. ]
Last edited by mpack on 14. Oct 2009, 11:05, edited 3 times in total.
mpack
Site Moderator
 
Posts: 13169
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows XP
VBox Version: PUEL
Guest OSses: Mostly XP

Re: CloneVDI tool - Discussion & Support

Postby wmeyer » 16. Sep 2009, 15:30

I have been toying with writing a gui front end for VBoxManage, simply because I do not have occasion to use it often enough to remember all its capabilities. I was interested to see what you built, but as I use many VHD files (my coworkers are on Virtual PC 2007), I do need VHD support.

One thing which has stopped me cold is that the command which would allow me to copy a VHD to a VDI has not worked for me at all. It is not satisfied with the arguments I pass, and its error messages are not sufficiently illuminating to let me discover why it is not satisfied.

Yes, I am a Windows developer, but no, I am by no means allergic to command line utilities, and use them nearly every day. However, when the syntax spec is all that is available, the clarity is not always great. I could wish for examples in the documentation.

Lately, my work schedule has consumed enough of my energies that I have not found time to work on my own tools. When that situation changes, I will get back to it, I hope.
wmeyer
 
Posts: 20
Joined: 14. May 2009, 18:42
Primary OS: MS Windows 7
VBox Version: OSE other
Guest OSses: Windows 7

Re: CloneVDI tool - Discussion & Support

Postby mpack » 16. Sep 2009, 16:27

Implementing VHD read support should not be too hard. VHD is badly designed (a few wacky decisions, like big endian byte order and a "footer" that has to move every time the file grows), but apart from that, its general structure is similar to VDI. I'm reluctant to add write support.
mpack
Site Moderator
 
Posts: 13169
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows XP
VBox Version: PUEL
Guest OSses: Mostly XP

Re: CloneVDI tool - Discussion & Support

Postby vbox4me2 » 16. Sep 2009, 19:35

I used it today and I am :D good work :!:
vbox4me2
Volunteer
 
Posts: 5222
Joined: 21. Nov 2008, 20:27
Location: Rotterdam

Re: CloneVDI tool - Discussion & Support

Postby mpack » 16. Sep 2009, 21:56

vbox4me2 wrote:I used it today and I am :D good work :!:

Pleased to be of service! :D
mpack
Site Moderator
 
Posts: 13169
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows XP
VBox Version: PUEL
Guest OSses: Mostly XP

Re: CloneVDI tool - Discussion & Support

Postby wmeyer » 16. Sep 2009, 22:59

My interest is in converting VHD to VDI, far more than the other way.
wmeyer
 
Posts: 20
Joined: 14. May 2009, 18:42
Primary OS: MS Windows 7
VBox Version: OSE other
Guest OSses: Windows 7

Re: CloneVDI tool - Discussion & Support

Postby ID4170 » 17. Sep 2009, 02:56

I do have a couple questions for you Mpack, can I just copy my vdi to any additional hard drive directly from virtual box folder like drag and drop to a backup hd, without using the clone or any other software? and use a later time? or I have to use a software and why?
Why do I have to use the clone? Also using the clone I will not lose any part of my data? is possible to check that?
I using the clone right now and will post my results tomorrow.
Thank You Very Much!
ID4170
 
Posts: 9
Joined: 16. Sep 2009, 19:53
Primary OS: MS Windows XP
VBox Version: PUEL
Guest OSses: WINDOWS 2003

Re: CloneVDI tool - Discussion & Support

Postby ID4170 » 17. Sep 2009, 03:04

Mpack here is the results:

I copy my hd.vdi the size is 4.86gb
Took me under 5 min very nice the new size is 4.44gb why is the difference also I ask to compact drive that's why?
My big concern is not to lose any data.

Thank You Again. :lol:
ID4170
 
Posts: 9
Joined: 16. Sep 2009, 19:53
Primary OS: MS Windows XP
VBox Version: PUEL
Guest OSses: WINDOWS 2003

Re: CloneVDI tool - Discussion & Support

Postby mpack » 17. Sep 2009, 10:46

Yes, the "Compact" option tries to make the clone smaller. This is described in the release notes. In your case it seems to have recovered 400MB of disk space, nothing to sniff at :)

You cannot lose data unless and until you delete your original. So, provided you check the clone carefully for errors first, then you should be quite safe. Of course if you have no intention of deleting the original then you are safe.

As part of my testing of the compact feature I ran various disk check untilities inside a clone of each filesystem type. If I found an error I then restored the original and repeated the check with that. So far I have not found any disk errors in the clone that were not also present in the original.
mpack
Site Moderator
 
Posts: 13169
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows XP
VBox Version: PUEL
Guest OSses: Mostly XP

Re: CloneVDI tool - Discussion & Support

Postby mpack » 17. Sep 2009, 10:56

ID4170 wrote:can I just copy my vdi to any additional hard drive directly from virtual box folder like drag and drop to a backup hd, without using the clone or any other software? and use a later time? or I have to use a software and why?

Yes, you can copy a VDI to a backup folder without using this software. A simple copy of course will not be optimized or compacted, and it will have the same UUID as before, meaning it cannot be registered on the same VirtualBox host as the original. If you intend that the backup would replace the original in the event of a mishap then all of this is fine.

ID4170 wrote:Also using the clone I will not lose any part of my data? is possible to check that?

As mentioned elsewhere, you can't lose data unless you delete the original. Feel free to run any kind of disk check utility you like inside the cloned guest. If you see any errors, do make sure to check the original as well before concluding that CloneVDI caused the error.
mpack
Site Moderator
 
Posts: 13169
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows XP
VBox Version: PUEL
Guest OSses: Mostly XP

Re: CloneVDI tool - Discussion & Support

Postby ID4170 » 17. Sep 2009, 17:57

Thanks Mpack the software is great also you have answer all my questions. :lol:
ID4170
 
Posts: 9
Joined: 16. Sep 2009, 19:53
Primary OS: MS Windows XP
VBox Version: PUEL
Guest OSses: WINDOWS 2003

Re: CloneVDI tool - Discussion & Support

Postby hangerglide » 18. Sep 2009, 04:30

I've never tried to clone a vbox vdi using the command line tools. I was able to use this tool to copy a vdi but I'm not sure how to introduce the new vm to the vbox user interface? What steps do I take to get vbox to recognize the new file as a vm?

Thanks
hangerglide
 
Posts: 78
Joined: 29. Mar 2009, 00:53
Primary OS: MS Windows 7
VBox Version: OSE Debian
Guest OSses: ubuntu 8.10

Re: CloneVDI tool - Discussion & Support

Postby undefined » 18. Sep 2009, 05:35

It worked great, thanks! With your program I expanded my 10 GB virtual drive to a 30 GB virtual drive (and I used gparted to expand the existing partition).
undefined
 
Posts: 1
Joined: 18. Sep 2009, 04:30
Primary OS: MS Windows XP
VBox Version: PUEL
Guest OSses: Win XP

Next

Return to VirtualBox on Windows Hosts

Who is online

Users browsing this forum: Google [Bot] and 31 guests