virtualbox.org

Ok, well, is there a way of not getting it to zero all the clusters, or maybe one of the other HD formats for preallocation?

mpack wrote:The marginal difference caused by internal block order fragmentation can be eliminated at any time by cloning the drive, especially convenient if done using CloneVDI with the "keep UUID" option set.

I'm not talking about how the image file is internally organized, I'm talking about how it's stored on disk. A file that grows over time is far more likely to be fragmented on the host filesystem. I'm also not sure how many users are able and/or willing to run CloneVDI or something similar regularly.

Of course a humungous fixed size drive image is more likely to be fragmented as a file on the host drive, so IMHO most users will see no difference in net effect. In fact it would pay to defragment the VDI, then defragment the host using a different tool - after the file size has stablilized of course. That should achieve the perfect ideal of compactness and performance

A humongous file created as a unit is far less likely to be fragmented, unless the host filesystem sucks. I agree that letting an image grow and defragmenting it later is a better use of resources, but I disagree that it's something users are likely to do. Just saying.

I'll also add a caveat that I'm not sure how much fragmentation matters with SSDs. I assume it's significantly less than with rotational media.

adrianh wrote:Ok, well, is there a way of not getting it to zero all the clusters, or maybe one of the other HD formats for preallocation?

So I'm guessing I'm going to have to roll my own?

adrianh wrote:
adrianh wrote:Ok, well, is there a way of not getting it to zero all the clusters, or maybe one of the other HD formats for preallocation?
So I'm guessing I'm going to have to roll my own?

Well, due to the lack of response, I'm rolling my own. This message is quite useful up to a point. Looks like nobody checked the math or confirmed the results:

TerryE wrote: In the case of a dynamic VDI (Type 1) as “new” 1Mbyte blocks in the virtual HDD are written to, these are mapped to new 1Mbyte blocks which are allocated at the end of the VDI file; that is a dynamic VDI is initially ordered in chronological not physical order. The initial size of an N Mbyte dynamic VDI is 512+4N+F bytes with all N image map entries set to -1. F = (508+4N) mod 512, that is the packing necessary to round up to a sector boundary.

In the case of a static (Type 2) VDI all blocks are pre-allocated so no dynamic allocation is necessary. The create utility also writes zeros to the entire file because many file systems themselves adopt a just-in-time allocation policy. The initial size of an N Mbyte static VDI is 512+4N+F+1048576N bytes, where F is as in the dynamic case.

Creating a 10M file I get this:

Code: Select all

$ vboxmanage createhd --filename c.vdi --size 10 --format VDI --variant Standard
$ ls -l c.vdi
-rwxr-xr-x 1 Adrian None 8192 Jun 19 11:49 c.vdi

Now, according to the info, the file size should be: 512+4*10+(508+4*10) mod 512 = 588 which is not even close to the actual result of 8192.

If this was right in the past, the version number hasn't changed, so what's up?

EDIT: Through experiment, I've confirmed that the allocation is done on 4K boundaries, which seems reasonable as 4K is used a lot in terms of virtual memory.

The actual equation should be:

Code: Select all

(int(4(N-1)/4096)+1)*4096

or equivalent, where the +1 represents the header's space in its own 4K block even though it is only 512 bytes in size.

There's no need to experiment. The structure of a VDI is given in the source code.

mpack wrote:There's no need to experiment. The structure of a VDI is given in the source code.

Yes, I could go through the code to track it down. But I'm new to the project and there is little documentation there too and no overview document requiring a lot of guessing where things are and why they are named the way they are. And I also hate it when people say, "but its common sense!" because it is common only to those on the project after a good long time studying and delving into the code.

Why is it that there is a fairly good document on the structure of VDIs but nobody bothered to corrected the pages? Can nobody modify them? Do I really want to go through a bunch of code where there should be a spec? No! Going through code is a tedious endeavour and documentation makes it easier and faster to get up to speed. It gets me angry when I see such things, and is usually why I stay away from other's projects unless I really have to.

This is my rant for the day. Please don't take it personally. Would you direct me to the correct header/source files that would get me the information that I need? Thanks.

Well now. That should get you somewhere. Smack down the people that know and insult them. I for one am outa here.

Perryg wrote:Well now. That should get you somewhere. Smack down the people that know and insult them. I for one am outa here.

Well Perryg, where did you see an insult? I merely stated my feelings about this and many other projects. Did I hurt your feelings by stating that it's documentation is poor? It is most likely not your fault, it is just a fact, mostly due to no leadership, time and/or drive. I've been in this game for a while and this is not the first and will not be the last time when I see a poorly documented system.

If however, you wish to take upon yourself to feel insulted for something beyond your control, you are free to do so. But IMHO that doesn't show any strength of character.

virtualbox.org

Why does creating a fixed size VDI take so long?

Re: Why does creating a fixed size VDI take so long?

Re: Why does creating a fixed size VDI take so long?

Re: Why does creating a fixed size VDI take so long?

Re: Why does creating a fixed size VDI take so long?

Re: Why does creating a fixed size VDI take so long?

Re: Why does creating a fixed size VDI take so long?

Re: Why does creating a fixed size VDI take so long?

Re: Why does creating a fixed size VDI take so long?