vdi space compact

Discussions about using Linux guests in VirtualBox.

vdi space compact

Postby alfredopacino » 31. Aug 2016, 14:54

Hi,
I'm working on an Hadoop project. The cluster runs on Ubuntu 14.04.4 VBox VMs.
The problem I found is Hadoop with its tmp files expand the vdi to the maximum (100GB). Even If I stop the Hadoop Job the vdi stays 100GB.
If I search for the 10 biggest files/dir using
Code: Select all   Expand viewCollapse view
du -hsx * | sort -rh | head -10

I get 2-3GB tops, that's why I'm pretty sure the 100GB vdi comes from 'dirty' free space. How can I compact the size of the vdi?
The command
Code: Select all   Expand viewCollapse view
VBoxManage.exe modifyhd "MYVDI.vdi" --compact

does nothing about that.
alfredopacino
 
Posts: 2
Joined: 31. Aug 2016, 14:46

Re: vdi space compact

Postby mpack » 31. Aug 2016, 15:07

What makes this a "Windows Guests" question? Are the VMs running Windows?

"VBoxManage compact" eliminates zeroed blocks, so you first have to zero the blocks using a zeroing tool appropriate to the guest OS and filesystem. For Windows guests that would be "sdelete".

Alternatively, use the CloneVDI tool to compact the VDI. CloneVDI has knowledge of popular filesystems and hence doesn't require that "zero fill" preprocessing step.

Alternatively, if the guest OS is modern enough to support SSDs, then change the hard disk to an SSD in the VM settings, then issue a trim command from the guest OS whenever you want to shrink the VDI. I'm not entirely convinced by this idea because, although very convenient, the feature is quite new, would intuitively therefore be prone to bugs, and besides which is an "in place" operation - you don't get a backup from before the change, which you get (say) from CloneVDI.
mpack
Site Moderator
 
Posts: 34727
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: vdi space compact

Postby alfredopacino » 31. Aug 2016, 19:28

yes VMs running on windows 10 without admin permissions, do I have misunderstood the aim of this section?
I tried sdelete and then VBoxManage compact, nothing changed on my 100GB vdi..

The machines aren't on SSD. I can't use CloneVDI easily, there is no enough space on win10 partition to clone the vdi (I should clone on another device..).
alfredopacino
 
Posts: 2
Joined: 31. Aug 2016, 14:46

Re: vdi space compact

Postby mpack » 1. Sep 2016, 09:09

alfredopacino wrote:yes VMs running on windows 10 without admin permissions, do I have misunderstood the aim of this section?

VMs running "on" Windows 10? That would make Windows 10 your host, not your guest. For now I'll assume that you've given me bad information, and move you to "Linux Guests".

alfredopacino wrote:I tried sdelete and then VBoxManage compact, nothing changed on my 100GB vdi..

You ran the zero fill tool in the guest right? Because obviously, running it on the host is going to do nothing for a guest filesystem. Except now you'd have to clarify how you managed to run sdelete in a Linux guest.

alfredopacino wrote:The machines aren't on SSD.

You mean the host? We are discussing the configuration of a virtual machine, the technology used by your physical host is not directly relevant. VirtualBox doesn't require the presence of a physical SSD in order to simulate a virtual one.

alfredopacino wrote:I can't use CloneVDI easily, there is no enough space on win10 partition to clone the vdi (I should clone on another device..).

Yes, you could clone to an external drive or network folder.
mpack
Site Moderator
 
Posts: 34727
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: vdi space compact

Postby Bubba198 » 15. Sep 2016, 07:33

So is this so hard to admit or publish somewhere? Shrinking VDI is simply not supported? Yes the VDI is dynamic and yes I deleted all partitions just for the test... blank space; no partitions defined..

Code: Select all   Expand viewCollapse view
vboxmanage modifyhd 3805d7a1-166e-41f7-a7b9-8626d9c46459 --resize 43000
0%...
Progress state: VBOX_E_NOT_SUPPORTED
VBoxManage: error: Resize medium operation for this format is not implemented yet!
Bubba198
 
Posts: 6
Joined: 21. Jul 2014, 19:03
Primary OS: Ubuntu 12.12
VBox Version: OSE Debian
Guest OSses: Ubuntu 14 LTS

Re: vdi space compact

Postby socratis » 15. Sep 2016, 12:05

Bubba198 wrote:
VBoxManage: error: Resize medium operation for this format is not implemented yet!

That means that your "VDI" is not a VDI that can be compacted. Please post the output of the command:
    VBoxManage showmediuminfo 3805d7a1-166e-41f7-a7b9-8626d9c46459
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
socratis
Site Moderator
 
Posts: 27690
Joined: 22. Oct 2010, 11:03
Location: Greece
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Win(*>98), Linux*, OSX>10.5

Re: vdi space compact

Postby mpack » 15. Sep 2016, 12:12

Bubba198 wrote:and yes I deleted all partitions just for the test... blank space; no partitions defined..

Even if you get beyond the error message, this still won't work, because you have not understood how a virtual disk works. Or even a physical disk for that matter.

A partition map is merely a data structure stored on the disk, part of the management information used by the guest OS. It has no meaning to the hardware. Do sectors on your physical hard disk stop storing data because you deleted a partition? Of course not. On the contrary, all that happens is that the partition map is overwritten with new data. The partitioned sectors remain where they are, with the same data as before.

As already mentioned above, VBoxManage compaction causes ZEROED blocks to be eliminated from the VDI (they are replaced in the block table by a code which indicates a zeroed block). Deleting a partition entry does not cause the partitions sectors to be zero filled.
mpack
Site Moderator
 
Posts: 34727
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP


Return to Linux Guests

Who is online

Users browsing this forum: esalvesen and 17 guests