Big VDI file vs VMWare VMDK split files

[AntiMatter]

Hi,

I have finished preparing a big Dev VM. The size of the VDI file is 22 GB. Given this VM for preview to a fellow who is more used to VMWare. He told me that Virtualbox is good but what bothers him is the big VDI file which makes the defrag (in the host) difficult to impossible.

Is it a justified argument ? If yes, if there anything we can do to reduce or eliminate the inconveniences of the big file size?

Thanks in advance for any help.

[mpack]

It sounds like nonsense to me. If anything I'd expect the opposite to be true, i.e. that lots of little files scattered over your host HD can make it more likely that any new big files are fragmented. That's a pretty weak scenario, and even then, defragmentation is easy. I'd like to hear a technical explanation of how he came to this conclusion.

[AntiMatter]

Because, may be the OS needs a free space of equal size to the file to be defragged? So if you don't have enough free space, you cannot defrag?

EDIT: I have asked the co-worker and the reason is mainly practical user experience. He told me that when he changed from a monolithic big VMDK file to a 2GB split version, he felt a big improvement in maintenance. Mainly defrag, file copy / backup.

[Sasquatch]

If you use a smart program to defrag your drive, it doesn't matter how much free space you have. If you want a continuous block for the VDI, defrag your drive first, then put the VDI on it and analyse the drive. You'll see that it will not be fragmented. Else some small files are filling the gaps between it and a run or two will get rid of them. Move small files out of the way, then the big file can be made one block.

[AntiMatter]

Move small files out of the way, then the big file can be made one block.

Make sense. And what if the VDI itself increase in size overtime? You may start with 10 GB which increases gradually to 22 GB over may be 6 months? Would the split file version provide a kind of containment which "divide" the defrag effects by chunk of of 2GB?

[Sasquatch]

I wonder if VMWare creates full 2 GB chunks, or only chunks up to 2 GB, allowing blocks of even a few MB. That would be more sensitive to fragmentation than a VDI IMO. Think about it, would it help if chunks 2, 4 and 6 are at the beginning of a drive, and 1, 3 and 5 at the end for better performance, or would one big file be better? I think the latter, because the hard drive doesn't need to go back and forth, but can do a continuous sweep over the data. Of course, when it's fully defragmented.

[AntiMatter]

I wonder if VMWare creates full 2 GB chunks, or only chunks up to 2 GB, allowing blocks of even a few MB.

The chunk of 2GB grows dynamically in size and maxes at 2 GB. For example when a new chunk is created, it could have a small size like 5 MB.

[Sasquatch]

In that case I think that a VDI is better for the overall performance and if you put the files on a different hard drive with other big files, you would get better results than with VMDK split files.

[mpack]

In that case I think that a VDI is better for the overall performance and if you put the files on a different hard drive with other big files, you would get better results than with VMDK split files.

Exactly. What exactly is fragmentation? It's your file broken into bits, right? Ok, next question is: what is a split VMDK?

Best performance IMHO would come from a single file, optimized and defragmented at reasonable intervals.

[smilo]

A useful reason for split files would be when the virtual hard drives are on FAT32 external drives. FAT32 external drives are preferred if you need to attached to Windows or Mac for read/write access. I mostly use VMWare Workstation and Fusion, so it is convenient to be able to run a given virtual machine on both hosts.

[mpack]

No one in this discussion is saying that split files are never useful. This discussion was about whether split files bring performance advantages on the host by reducing fragmentation. And the answer is, as far as I'm concerned, that a large file should not cause host fragmentation, but OTOH having the disk image as a single, large, unfragmented image on an internal drive should give the best guest disk I/O performance.

I'm not sure if you were saying that you actually run your split VMDK images directly off a removable drive: that is often a bad idea, as it leads too easily to corruption of the disk image (eg. if you unplug the drive while the VM is still running).