CloneVDI tool - Discussion & Support

[mpack]

Many Windows users dislike the fact that you currently have to use the Command Prompt to clone VDI files (*). Also, questions often arise related to VDI files such as "how do I increase the maximum size of a VDI?" or "how can I make my dynamic VDI file small again?". As a recent vacation project I decided to try to write a nice simple tool that provided a good answer to all of these questions. Hopefully I succeeded.

(*) [edit: VirtualBox v4.1 finally added GUI support for cloning]

Attached to this message is a zip file containing the software tool that I wrote, which I call "CloneVDI" (currently at version 3.01). Features as of this release include:

• Reads VDI,VHD,VHDX,VMDK,Parallels HDD,RAW files and physical drives, writes VDI.
• A simple GUI Interface.
• A CLI interface too (for people who like that kind of thing!)
• Ability to compact (discard unused blocks from) the clone, with no need to run sdelete/zerofree first.
• Ability to repair common forms of header corruption.
• Ability to enlarge the maximum size of a virtual disk, optionally enlarging the main partition too.
• Merge snapshot VDI chain into a single VDI (VHD and VMDK snapshot chains supported too).
• User choice to either keep the old UUID or generate a new one for the clone.
• Some useful diagnostics about the source VDI, such as the partition map and the internal fileystem.
• Better feedback while cloning: a progress bar and an estimated time to completion.
• It seems to be significantly faster than "VBoxManage clonehd".

To install, simply create a folder and unpack the zip contents into it. Optionally, you can create a shortcut to link your desktop to the CloneVDI.exe file in the target folder. The zip file includes release notes which hopefully provides answers to all of your questions.

Note that this tool is not simply a GUI front end for VBoxManage. Instead it uses my own code to read and write VDI files (and other formats), and therefore works in a rather different way, with different features and capabilities.

Enjoy. Feel free to ask any questions here, but read the release notes first.

[ Edit: 24th Dec 2017. CloneVDI attachment updated to v3.01. Tweak to header repair, icon that looks better on hidpi displays. ]
[ Edit: 5th Jul 2017. CloneVDI attachment updated to v3.00. Auto-header repair, shrinking VDIs, EXT4-64bit compatibility. ]
[ Edit: 4th Dec 2015. CloneVDI attachment updated to v2.51. Fixed a bug affecting VDIs created with >256GB logical size. ]
[ Edit: 10th Oct 2015. CloneVDI attachment updated to v2.50. Bugfixes plus 1MB header alignment plus VHDX support. ]
[ Edit: 12th Sep 2012. CloneVDI attachment updated to v2.10. Bugfixes plus support for VMDK and VHD snapshots. ]
[ Edit: 20th June 2011. CloneVDI attachment updated to v2.05. Bugfix release for broken VHD support. ]
[ Edit: 16th May 2011. CloneVDI attachment updated to v2.04. Fix to NTFS partition enlargement. Simplified snapshot cloning. ]
[ Edit: 14th Oct. CloneVDI attachment updated to v2.03. Minor tweak to recognize VirtualBox 3.2.10 (Oracle tagged) VDI files. ]
[ Edit: 1st Mar. CloneVDI attachment updated to v2.02. Fixed another bug in "enlarge partition" feature, completed French language support. ]
[ Edit: 8th Feb. CloneVDI attachment updated to v2.01. Fixed bug in "enlarge partition" feature, added French language support. ]
[ Edit: 31st Jan 2010. CloneVDI attachment updated to v2.00. This version includes several new features and some tidy-ups ]
[ Edit: 10th Dec. CloneVDI attachment updated to v1.43. This version fixes a problem when enlarging drives smaller than 8GB ]
[ Edit: 12th Nov. CloneVDI attachment updated to v1.42. This version adds support for Dutch and German languages ]
[ Edit: 31st Oct. CloneVDI attachment updated to v1.41. This version has several internal tidy-ups, and adds the CLI feature. ]
[ Edit: 14th Oct. CloneVDI attachment updated to v1.40. This version adds RAW read support and a "sector viewer" feature. ]
[ Edit: 07th Oct. CloneVDI attachment updated to v1.30. This version adds VMDK read support. ]
[ Edit: 21st Sep. CloneVDI attachment updated to v1.22. The previous version STILL couldn't handle VHDs bigger than 4GB. ]
[ Edit: 21st Sep. CloneVDI attachment updated to v1.21. The previous version didn't like VHDs bigger than 4GB. ]
[ Edit: 20th Sep. CloneVDI attachment updated to v1.20. This version adds VHD read support. ]
[15th Sep 2009. First public release (CloneVDI v1.10). ]
------
v2.10 source code can now be found attached to the next message.
------

[mpack]

Reliability is an obvious concern with an unproven tool such as this. All I can tell you is that I have been using it on my own VDI's, which I would not do if I had any doubts! Also, the tool has been made available to forum volunteers for a week or two now, and none of them have reported a single bug. In fact the tool should be quite reliable: its' basic "modus operandi" is to make a copy of your source VDI and to only ever modify the copy, never the original. So, if you intend to delete the original then you should be quite safe, provided you give the clone a thorough test first. It does extensive validation of both the original VDI and the copy process, which again should lead to solid reliability.

But, it's fast too.

[Edit: source code attached this message, to avoid confusion in the root message. ]

[mpack]

Please be aware that this tool currently supports VDI files only (not VHD or VMDK). If there is sufficient demand then I may implement support for other formats in the future.

Oh - and while this tool is intended primarly for Windows hosts, I am told that it does work under Wine (Linux and OS X users take note).

[edit: VHD/VMDK and RAW read support was added in later versions. ]

[wmeyer]

I have been toying with writing a gui front end for VBoxManage, simply because I do not have occasion to use it often enough to remember all its capabilities. I was interested to see what you built, but as I use many VHD files (my coworkers are on Virtual PC 2007), I do need VHD support.

One thing which has stopped me cold is that the command which would allow me to copy a VHD to a VDI has not worked for me at all. It is not satisfied with the arguments I pass, and its error messages are not sufficiently illuminating to let me discover why it is not satisfied.

Yes, I am a Windows developer, but no, I am by no means allergic to command line utilities, and use them nearly every day. However, when the syntax spec is all that is available, the clarity is not always great. I could wish for examples in the documentation.

Lately, my work schedule has consumed enough of my energies that I have not found time to work on my own tools. When that situation changes, I will get back to it, I hope.

[mpack]

Implementing VHD read support should not be too hard. VHD is badly designed (a few wacky decisions, like big endian byte order and a "footer" that has to move every time the file grows), but apart from that, its general structure is similar to VDI. I'm reluctant to add write support.

[vbox4me2]

I used it today and I am good work

[mpack]

I used it today and I am good work

Pleased to be of service!

[wmeyer]

My interest is in converting VHD to VDI, far more than the other way.

[ID4170]

I do have a couple questions for you Mpack, can I just copy my vdi to any additional hard drive directly from virtual box folder like drag and drop to a backup hd, without using the clone or any other software? and use a later time? or I have to use a software and why?
Why do I have to use the clone? Also using the clone I will not lose any part of my data? is possible to check that?
I using the clone right now and will post my results tomorrow.
Thank You Very Much!

[ID4170]

Mpack here is the results:

I copy my hd.vdi the size is 4.86gb
Took me under 5 min very nice the new size is 4.44gb why is the difference also I ask to compact drive that's why?
My big concern is not to lose any data.

Thank You Again.

[mpack]

Yes, the "Compact" option tries to make the clone smaller. This is described in the release notes. In your case it seems to have recovered 400MB of disk space, nothing to sniff at

You cannot lose data unless and until you delete your original. So, provided you check the clone carefully for errors first, then you should be quite safe. Of course if you have no intention of deleting the original then you are safe.

As part of my testing of the compact feature I ran various disk check untilities inside a clone of each filesystem type. If I found an error I then restored the original and repeated the check with that. So far I have not found any disk errors in the clone that were not also present in the original.

[mpack]

can I just copy my vdi to any additional hard drive directly from virtual box folder like drag and drop to a backup hd, without using the clone or any other software? and use a later time? or I have to use a software and why?

Yes, you can copy a VDI to a backup folder without using this software. A simple copy of course will not be optimized or compacted, and it will have the same UUID as before, meaning it cannot be registered on the same VirtualBox host as the original. If you intend that the backup would replace the original in the event of a mishap then all of this is fine.

Also using the clone I will not lose any part of my data? is possible to check that?

As mentioned elsewhere, you can't lose data unless you delete the original. Feel free to run any kind of disk check utility you like inside the cloned guest. If you see any errors, do make sure to check the original as well before concluding that CloneVDI caused the error.

[ID4170]

Thanks Mpack the software is great also you have answer all my questions.

[hangerglide]

I've never tried to clone a vbox vdi using the command line tools. I was able to use this tool to copy a vdi but I'm not sure how to introduce the new vm to the vbox user interface? What steps do I take to get vbox to recognize the new file as a vm?

Thanks

[undefined]

It worked great, thanks! With your program I expanded my 10 GB virtual drive to a 30 GB virtual drive (and I used gparted to expand the existing partition).