Question about --variant options (clonehd)

This is for discussing general topics about how to use VirtualBox.
Locked
sirius
Posts: 13
Joined: 26. Feb 2011, 01:45
Primary OS: MS Windows 7
VBox Version: OSE other
Guest OSses: Windows, Linux, OS X

Question about --variant options (clonehd)

Post by sirius »

I frequently use the "VBoxManage clonehd" command to create duplicate VMs. I've been using the "--variant Fixed" option, as that was what I found when I first googled for how to duplicate a VM, but I don't actually know what the "fixed" variant (or the others) mean.

The documentation has this to say about the --variant option:
variant
Allow to choose a file format variant for the output file. It is a comma-separated list of variant flags. Not all combinations are supported, and specifying inconsistent flags will result in an error message.
That's less than helpful though, as it says nothing about what the different variant options (Standard, Fixed, Split2G, Stream, ESX) are for.

Can someone please explain them for me?

Thanks.
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: Question about --variant options (clonehd)

Post by mpack »

It means that many of the supported image formats (VDI, VMDK, VHD) have more than one type (a.k.a. variant). You can select which type you want if you know enough about what you're doing to express a preference. For example the "--variant Fixed" type means that you prefer the pointlessly disk space wasting variants offered by the VDI, VMDK or VHD formats, instead of the more usual "use only as much host disk space as you really need" default. Split2G is a variant offered by VMDK only, which means that image is split into 2GB segments, making it compatible with older filesystems such as FAT, at a cost in performance. Stream (VMDK only) means that the image is optimized for downloading (i.e. it's compressed), needing to be unpacked before use (OVF appliances may use this form). ESX is another VMDK option, it has something to do with VMWare's server products, and more than that I've never needed to know.

Standard means to use the default, which is the same as what happens if you express no variant preference at all.
sirius
Posts: 13
Joined: 26. Feb 2011, 01:45
Primary OS: MS Windows 7
VBox Version: OSE other
Guest OSses: Windows, Linux, OS X

Re: Question about --variant options (clonehd)

Post by sirius »

Thanks. I figured that's what "fixed" meant (and I'm working with fixed files anyhow since drive space isn't a factor but performance is). Since "dynamic" wasn't one of the options though I wasn't certain about the rest.

Thanks for the info, it'd be nice if they documented that somewhere.
Sasquatch
Volunteer
Posts: 17798
Joined: 17. Mar 2008, 13:41
Primary OS: Debian other
VBox Version: PUEL
Guest OSses: Windows XP, Windows 7, Linux
Location: /dev/random

Re: Question about --variant options (clonehd)

Post by Sasquatch »

You could say that it's not documented that well because those who have heard about the different variants know what they mean and do. People who haven't heard about them usually don't enter such optional parameter.

I do wonder why you think that fixed performs better than dynamic. There have been benchmarks here in the past and the difference is not worth mentioning.
Read the Forum Posting Guide before opening a topic.
VirtualBox FAQ: Check this before asking questions.
Online User Manual: A must read if you want to know what we're talking about.
Howto: Install Linux Guest Additions
Howto: Use Shared Folders on Linux Guest
See the Tutorials and FAQ section at the top of the Forum for more guides.
Try searching the forums first with Google and add the site filter for this forum.
E.g. install guest additions site:forums.virtualbox.org

Retired from this Forum since OSSO introduction.
tbfh1955
Posts: 3
Joined: 10. Sep 2011, 13:46
Primary OS: Ubuntu other
VBox Version: PUEL
Guest OSses: windows, linux, osx

Re: Question about --variant options (clonehd)

Post by tbfh1955 »

You could say that it's not documented that well because those who have heard about the different variants know what they mean and do. People who haven't heard about them usually don't enter such optional parameter.
You could also say it's not documented well because "Standard vs. Fixed" is painfully counter-intuitive, whereas "Dynamic vs. Fixed" is obvious and easily inferred given the content of the documentation. As it is, I can't tell whether "Standard" actually means "Dynamic", or if "Dynamic" is actually the default, and "Standard" means something else entirely. As it stands, "not documented well" is too generous. It's actually documented very poorly. If --variant refers only to VMWare, it should be stated as such; as it is, --variant appears to refer to both .vdi and .vmdk, but gives no indication of the distinction.
I do wonder why you think that fixed performs better than dynamic. There have been benchmarks here in the past and the difference is not worth mentioning.
If you happen to be creating your .vdi on an NTFS filesystem, claiming a fixed amount of space up front is a prudent way of insuring that your dynamic allocations are not scattered about the partition, and hence leading to increased latency over time. It could also facilitate organizing a set of virtual machines with different caching schemas, and tracking performance or prioritizing certain applications, like video capture for example.

Virtualbox is proving to be an outstanding product, and questions arising from incomplete or ambiguous documentation provide the development team with valuable feedback which should enable them to correct those problems. I do wonder why there is such an excess of un-helpful 'attitude' on the part of some in the Vbox forums who seem to prefer ridiculing those who post questions, rather than simply providing the most useful answers possible. If you don't like the form or the content of the question, why bother posting a response?
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: Question about --variant options (clonehd)

Post by mpack »

tbfh1955 wrote:You could also say it's not documented well because "Standard vs. Fixed" is painfully counter-intuitive, whereas "Dynamic vs. Fixed" is obvious and easily inferred given the content of the documentation.
Nonsense. Standard vs Fixed is your own straw man, there is no such choice given in the manual. Standard vs <Custom> is the true choice, just like it is (for example) with any application installer. Choosing a custom mode is a declaration by you that your knowledge is superior to that encapsulated in the wizard: and if it isn't then you only have yourself to blame for taking that road.
tbfh1955 wrote:If you happen to be creating your .vdi on an NTFS filesystem, claiming a fixed amount of space up front is a prudent way of insuring that your dynamic allocations are not scattered about the partition, and hence leading to increased latency over time.
Yes, at the cost of potentially fragmenting every other file you subsequently write to that drive, due to increased congestion. People who use this technique will often unconsciously limit the size of the drive too, therefore causing congestion and fragmentation inside the guest.

Really, what's the big deal? If you have an actual fragmentation problem then run a defragger overnight. End of story. Hogging 30GB of disk space for a file that only really needs 4GB is definitely not IMHO a "prudent" use of resources.
tbfh1955
Posts: 3
Joined: 10. Sep 2011, 13:46
Primary OS: Ubuntu other
VBox Version: PUEL
Guest OSses: windows, linux, osx

Re: Question about --variant options (clonehd)

Post by tbfh1955 »

Nonsense. Standard vs Fixed is your own straw man, there is no such choice given in the manual. Standard vs <Custom> is the true choice, just like it is (for example) with any application installer.
Not sure what manual you're looking at. Here's vbox/ch08.html:
This command creates a new virtual hard disk image. The syntax is as follows:
VBoxManage createhd --filename <filename> --size <megabytes>
[--format VDI|VMDK|VHD] (default: VDI)
[--variant Standard,Fixed,Split2G,Stream,ESX]
where the parameters mean:
filename
Allows to choose a file name. Mandatory.
size
Allows to define the image capacity, in 1 MiB units. Mandatory.
format
Allows to choose a file format for the output file different from the file format of the input file.
variant
Allows to choose a file format variant for the output file. It is a comma-separated list of variant flags. Not all combinations are supported, and specifying inconsistent flags will result in an error message.
There it is: "Standard,Fixed". I don't see "Standard|<Custom>" anywhere, but maybe you can provide me a link to the manual you're referring to. The term 'Standard' is about as vague a choice as you could make. In the manual's overview of Virtual Storage (Chapter 5), the reference is to 'fixed' & 'dynamic':
Irrespective of the disk capacity and format, as briefly mentioned in the section called “Creating your first virtual machine”, there are two options of how to create a disk image: fixed-size or dynamically allocated.
The consistent use of terminology is what I would call "Technical Writing 101", absolutely essential if it is to serve it's purpose. A statement like the following:
Choosing a custom mode is a declaration by you that your knowledge is superior to that encapsulated in the wizard: and if it isn't then you only have yourself to blame for taking that road.
...is a classic example of what I would call "sophomoric hacker arrogance". '...encapsulated in the wizard'? Wizards are for people who don't want to read reference manuals -- completely irrelevant to this thread. The guy had a simple question: what do the words referenced under '--variant' mean. He couldn't find it documented anywhere, and neither could I. I finally got the answer simply by making one...but why not just call it what it is: 'Dynamic'? Problem solved, no need for this thread.

A well organized, consistent and comprehensive reference manual is essential for serious application development using a product as complex as Virtualbox. In this case, a mature designer would likely have adopted an established CLI convention, such as the one used by the mkfs command:

mkfs [-V] [-t fstype] [fs-options] filesys [blocks]

for example:

createhd [--type Dynamic(default)|Fixed] [--variant VMDK <optionlist>]

Given the earlier reference in Chapter 5, most people wouldn't need to consult the docs to know how to use the command.
Yes, at the cost of potentially fragmenting every other file you subsequently write to that drive, due to increased congestion. People who use this technique will often unconsciously limit the size of the drive too, therefore causing congestion and fragmentation inside the guest.
Of course you're making the broad assumption that you know how a given user intends to utilize a very large partition with potentially large numbers of reusable guests. If the context is well-defined (virtual desktops for a hospital as an example) dynamic allocation could become a nightmare with potentially hundreds of vms, all of which could be quite comfortably configured with a standard fixed-length 6GB .vdi.
Really, what's the big deal? If you have an actual fragmentation problem then run a defragger overnight. End of story. Hogging 30GB of disk space for a file that only really needs 4GB is definitely not IMHO a "prudent" use of resources.
Maybe you're unfamiliar with the NTFS format, but cluster allocation is based on the size of the request, and that .vdi could be severely fragmented as it grows (unless Virtualbox supports an NTFS specific handler capable of managing how NTFS allocates space...does it?). Theres a very reasonable argument for pre-allocating fixed storage if it's necessary to put virtual volumes on NTFS (or any other filesystem for that matter). It all depends on how the guest OS's are intended to be used.

More to the point: your statement "If you have an actual fragmentation problem, then run a defragger overnight. End of story.", is once again, arrogant and sophomoric. It suggests you have no experience whatsoever with enterprise-level system planning or real-world business applications. Defragmenting an NTFS volume is non-trivial, depending upon the number of fixed system-related objects which cannot be relocated. There are tools that can fully defragment and optimize an NTFS volume, but the process can take a long time, and believe it or not, there are businesses who may want to operate arrays of virtual machines 24x7.

The Open Source non-Windows software community has suffered for years from so-called support forums populated by (and moderated by) people who are more concerned about demonstrating how "smart" they are, than helping people succeed in using whatever application the forum is supposed to support. I've been watching the development of Virtualbox for a number of years, and it is finally reaching the point where it is being considered for adoption by mainstream business enterprises. Next to providing thorough, well-organized reference docs, operating forums which encourage the success of people who seek out help is a far more productive policy than trying to find ways to make them feel stupid.
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: Question about --variant options (clonehd)

Post by mpack »

Sorry, I don't have the time, energy or interest to to read all of that. Look up the word "standard" in a dictionary if you are unsure of its meaning. Obviously the opposite of standard is non-standard, a.k.a. custom, hence why I say that standard vs <custom> is the choice being offered, with "<custom>" being a placeholder for the other options listed in the manual. I don't know why you fixated on only one of the latter.
tbfh1955
Posts: 3
Joined: 10. Sep 2011, 13:46
Primary OS: Ubuntu other
VBox Version: PUEL
Guest OSses: windows, linux, osx

Re: Question about --variant options (clonehd)

Post by tbfh1955 »

Sorry, I don't have the time, energy or interest to to read all of that. Look up the word "standard" in a dictionary if you are unsure of its meaning. Obviously the opposite of standard is non-standard, a.k.a. custom, hence why I say that standard vs <custom> is the choice being offered, with "<custom>" being a placeholder for the other options listed in the manual. I don't know why you fixated on only one of the latter.
Thank you for proving my point so succinctly (predictably, you missed it entirely). Let me spell it out for you: since you list yourself as a "volunteer" in this forum, perhaps you should consider whether or not you have the capacity to make constructive contributions, and if not, find some other venue to demonstrate your pedestrian insights (try 'Perl Monks' or 'Doom9'...much more receptive to people of your ilk). If I were moderating this forum, you'd be excluded permanently.

PS: FBO, moderators: I only registered here because I had gotten tired of reading "mpack's" gratuitously demeaning responses to other folks attempting to get constructive help using Vbox. If you want to see an example of a forum that maintains a much higher standard of comity amongst it's members (IMHO), take a look at Ubuntu's. People like this aren't tolerated there, hence it's standing as one of the most useful on-line resources for open-source development.
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: Question about --variant options (clonehd)

Post by mpack »

Ah, I see we have another self appointed defender of the innocent. Your insights, while all valuable I'm sure, are off-topic here. See the "Feedback & comment to the Administrators and Moderators" topic in the Suggestions forum. You might also want to review the last several messages here (from before your first post) and see which ones of us are here to help, and which came here looking to pick a fight.
xerces8
Posts: 76
Joined: 27. Mar 2008, 12:48

Re: Question about --variant options (clonehd)

Post by xerces8 »

For Christs sake, if dynamic is the "standard" variant then simply put a word of two stating that into the docs.
Is that so hard?

I googled this issue for the exact same reason. I wanted a dynamic disk image, but it is nowhere documented.
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: Question about --variant options (clonehd)

Post by mpack »

Instead of complaining about the current wording - and being rightfully ignored - why not make a concrete suggestion and offer it to the devs? They have showed themselves perfectly willing in the recent past to accept good alternatives to help and manual text.

However it's strange that several people, nominally English speakers evidently, seem to be unaware of the meaning of basic English words. "Standard" means to use the most common form of <whatever>. Dynamic does not mean anything like that. In this context dynamic means to use a specific variant which may not exist for the selected type <whatever>. So IMHO, "dynamic" would not be a suitable substitute for "standard". There may however be an argument for adding "Dynamic" to the list of variant terms.
Perryg
Site Moderator
Posts: 34369
Joined: 6. Sep 2008, 22:55
Primary OS: Linux other
VBox Version: OSE self-compiled
Guest OSses: *NIX

Re: Question about --variant options (clonehd)

Post by Perryg »

And the once it is changed to dynamic just wait for the users to complain that it is not something else.
This is a no win scenario. The user manual specifically explains it but that never gets looked at.
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: Question about --variant options (clonehd)

Post by mpack »

This thread seems to have answered the OPs question long ago, and now only seems to serve as a debating forum, with the self righteous in some cases registering for that reason alone. I'm going to lock it I think.
Locked