virtualbox.org

Hello community,
• Host: Windows 10 (1909, x64), VirtualBox: Version 6.1.16 r140961 (Qt5.6.2)
• Guest – MT-Z2726VM03: Windows 10 (1909, x64), VirtualBox Guest Additions: Version 6.1.16 r140961
• VBoxHardeing.log is attached as ZIP
Problem:
On my host system, I have a SSD with 1TB Size and this was running out of memory due to the size of a VDI in one of my guest systems (MT-Z2726VM03).
< HDD > (dynamic allocated)
UUID: b5a02a4f-f2cf-4585-90c7-884deb8c1915
Parent UUID: base
State: created
Type: normal (base)
Location: E:\Rechner für Treiber Signatur\MT-Z2726VM03\MT-Z2726VM03\MT-Z2726VM03.vdi
Storage format: VDI
Capacity: 819200 Mbytes
Encryption: disabled
</ HDD >
If I start the guest the size of the content ist displayed: Used Space 121 GB, Capacity 798 GB

I found the suggestion to reduce / shrink the disk size (https://www.howtogeek.com/312883/how-to ... isk-space/), but it do not work. I did it 1st time as user and 2nd time as administrator, but the size of the disk was not reduced.
Could you please give me a hint to solve my problem?
But the data on the guest system should stay as they are.

TIA Alexander Sailer

This process is called compacting. The easiest method is to use Mpack's CloneVDI.

There is also the 'vboxmanage modifymedium "drive:\path\to\disk.file" --compact' command, which has prerequisite processes, see https://www.virtualbox.org/manual/ch08. ... difymedium.

Note that the disk file can get big again, all the way back to 798GB. After the compact, go in the VM's OS and reduce the partitions inside the OS to a final desired size. The disk file will only increase to the total of the internal partition sizes, plus a few MB.

Hello scottgus1,

scottgus1 wrote:This process is called compacting. The easiest method is to use Mpack's CloneVDI.
...

Your suggestion don't work as expected ...but I found a workaround with the following steps:
1.) In the guest use the manage disks from control panel and shrink the volume as suggested from the os to the possible minimum size.
2.) Use SDelete (https://www.howtogeek.com/312883/how-to ... isk-space/)
3.) Use Mpack's CloneVDI (see above) with the options: "Keep old UUID", "Compact drive while copying"

This leads me to an acceptable solution. Have in mind to have enough space on the target and use the CopyTool NOT from VirtualBox application.

Kind regards
Alexander

On step 2: if you use mpack's CloneVDI with compaction enabled then the sdelete step was superfluous. CloneVDI does not care if the blocks are zero or not.

mpack wrote:On step 2: if you use mpack's CloneVDI with compaction enabled then the sdelete step was superfluous. CloneVDI does not care if the blocks are zero or not.

Thanks for this information. In this step I was worried about the results and took the long way to the aim.

Kind regards
Alexander

In nearly every tutorial/explanation reg. compacting with vboxmanage.exe it is said that at first a defragmentation has to be done. Sometimes this rule is more specified that a defragmentation of the free space has to be done. After this as second step a zero-writing with sdelete.exe (in Windows-VM) has to be done.
mpack has explained above that "CloneVDI does not care if the blocks are zero or not".

Now I wonder whether the defragmentation (of the free space) is mandatory or recommended for best results/maximal compacting, either with vboxmanage.exe or with CloneCDI or both. In the VB-manual only zero-writing is mentioned, no word about defragmentation.

That depends on the result you expect. Compacting will certainly reduce the size of the .vdi, even if you don't zero out the used space. If you use CloneVDI, you don't need to use sdelete first.

Defragmentation is not really a problem I consider worth worrying about these days, and certainly not in a vm. I think you will find mpack thinks much the same.

Did we not have this conversation already?

In VBoxManage, compacting is the process of eliminating blocks that have been zeroed by sdelete or some other secure erase tool. VBoxManage needs this because it doesn't know anything about guest filesystems.

In CloneVDI, compacting is the process of eliminating unused blocks from the guest filesystem. It doesn't need help from sdelete to identify those blocks.

In theory, defragging will group unused sectors together, increasing the chance that entire 1MB blocks will be unused. In practice most modern drives are not fragmented, so this step makes the process take about 5 times as long and only recovers a few additional percent of disk space... which will be almost instantly grabbed again next time I use the VM (by definition, almost any write to a compacted VDI will make it grow).

I only ever compact a drive if some error has made it grow unreasonably, or if I'm about to clone/archive the VM or copy it to a drive for transport. I don't get anal about the extra couple of percent I could get from defragging.

I understood that the compacting of CloneVDI is different from the compacting of VBoxManage. I am asking because:
I want to use an existing old Win7-32-installation as VM in VB (under Win10-32), both running on a SSD 128GB. I have separated data from software so that only the software (with some minor exceptions) is part of the VM.
After converting the Win7-32-installation to a VM in the format VHD (because so I can get access to the VM als Windows-VM, too, in cases of emergency) I realized that the space the VM-VHD-files takes on the SSD is significant larger than it´s "real" content. And due to the limitations of the SSD is it a significant difference whether the VM-file takes 50GB or 80GB.

After deleting the no longer needed software in the VM and cloning and compacting the VHD to VDI with CloneVDI the VDI-VM was around 59GB. Due to the mentioned reason I converted the VDI to VHD and made a copy in the SSD for testing; the size increased for some GB.
Later, after reading the manual reg. compacting with VBoxManage, with the compacted "original" VHD-VM (not it´s copy in the SSD) I did a defragmentation of free space (with mydefrag) and zero-writing with sdelete. This increases the size of the VHD-VM to around 95GB. After conversion to VDI I run VBoxManage for compacting and the result was a VDI-VM with only around 45GB.
Strange.

Now I took the VHD-VM of the SSD and cloned and compacted it with CloneVDI to a VDI; the result was not surprisingly a VDI-VM with nearly 60GB.
Then I did a defragmentation for SSD (with mydefrag) only (no zero-writing) with the VHD-VM of the SSD and again cloned and compacted it with CloneVDI to a VDI. The result was a VDI-VM with around 46GB, which is nearly the "real" content of the VM.
The difference in size of around 14GB, which is nearly 25% less, is so large that obviously a defragmentation is recommend for compacting with CloneVDI, too.
The final test will be taking the "original" VDI-VM (the result of the first compacting with CloneVDI, nearly 100% identical with the VHD-VM of the SSD), do a zero-writing only and compacting with VBoxManage. The result will show whether also VBoxManage needs a previous defragmentation of the VM for optimal compacting.

It sounds like that disk is a special case. IMO it doesn't change the general rule.

Quite clearly: defragging can only have a large effect on a fragmented drive. IME fragmentation is rarely a problem these days.

Either way, the choice is yours.

I did some tests - this took a lot of time ...
Unfortunately I had deleted the original VM-file of the Win7-32-installation already. Only the .vdi-VM-file as result of it´s first test-cloning&compacting with CloneVDI - but w/o previous defragmentation - after deleting the no longer needed software from the VM was left. This file has 59,3GB and was the base of all my further tests. Let´s call it origin.vdi. For defragmentation I used mydefrag, for zero-writing I used sdelete. I did the following:

1. (mydefrag in the SSD-mode (with defragmentation of free space at the end of the disk))
a)
- mydefrag to origin.vdi,result origin2.vdi 78,3GB
- CloneVDI with compact to origin2.vdi,result 45,8GB
- VBoxManage with compact to origin2.vdi, result 78,3GB
b)
- sdelete to origin2.vdi, result origin3.vdi 78,3GB
- CloneVDI with compact to origin3.vdi, result 44,8GB
- VBoxManage with compact to origin3.vdi, result 44,8GB

2. As 1. but mydefrag in the "normal" free-space-defrag-mode:
nearly the same results

3.
a)
- sdelete to origin.vdi, result origin4.vdi 60,3GB
- VBoxManage with compact to origin4.vdi, result origin5.vdi 59,4GB
b) Defragmentation with Defraggler, adv.option freespace & fragmentation allowed
- Defraggler to origin5.vdi,result origin6.vdi 78,8GB
- sdelete to origin6.vdi, result origin7.vdi 78,9GB
- VBoxManage with compact to origin7.vdi, result 52,6GB

I doubt that this Win32-installation/VM is really a very special case. Defraggler reportet a fragmentation of around 13% - whatever this information means.This Win7-32-installation is running since 2005, until separating data from software for converting into VM the software and data are mixed in the usual way, a lot of copying, deleting, installation and deinstallation in all the years - and so of course the HDD/filesystem was fragmented. But this is the normal condition after years of usage.
I assume that this would be different when data and software is separated from the beginning, what is recommended and may be typical when using a SSD.

Anyway - the result of my test is, that in both ways of shrinking/compacting a VM-file it can be advantegous doing at first a defragmentation of free space - with identical results. Of course it depends on the usage of the system in the past. But due to the fact that the growth of the VM-file is a result of significant adding and deleting files IMHO this growth typical also increases fragmentation of the free space. And so a previous defragmentation of free space will increase/optimize the benefit of shrinking/compacting.

A different matter is the effect of fragmentation in regard of speed. I don´t know whether this is no more problem today or the same problem as in the past while using a HDD (in regard of speed).
I know the common opinion that fragmentation does not slow down the usage of a SSD. I don´t know enough about the internal working of a SSD for having a resilient own opinion about this, although it sounds reasonable that the need of additonal operations when reading fragmented files (or writing fragments) will slow down a not so fast PC. This is different to the strategy of wear leveling in the firmware/hardware of the SSD which results in fragmented files because the fragmentation we talk about here is a matter of the file-system, not the internal working of the SSD.

Anyway - beside the matter of speed: If significant fragmentation is a disadvantage when shrinking/compacting a VM-file (and my tests show this result) and if it is wanted or necessary to keep the space of the VM-file small, then doing a defragmentation of free space before shrinking/compacting is mandatory for an optimal result.

The common opinion says that defragmentation should not be done when using a SSD because this would counteract the wear-leveling-mechanismen. If so this operation(s) should be done with a HDD-copy of the VM-file only. I doubt that "double-compacting" has any advantage, but because the procedure must start with copying the VM-file (which is located in the SSD) this can be done with CloneVDI with compact-option, so that at least less space is needed on the HDD. Then opening this VM-copy (VM-file-copy ) , running mydefrag (or similar software with free-space-defragmentation), closing and doing again a copy&compact with CloneVDI, now with the SSD as destination. The results of VBoxManage (with previous sdelete) and CloneVDI are identical, so that CloneVDI has the advantage of less time consumption.