Virtual Disk Size Much Greater than File System Disk Size

Discussions related to using VirtualBox on Windows hosts.
Post Reply
WhataWheasel
Posts: 6
Joined: 3. Mar 2013, 17:04
Primary OS: MS Windows 8
VBox Version: OSE other
Guest OSses: Win 2012 Server, Win 8, Linux Ubuntu, Linux CentOS

Virtual Disk Size Much Greater than File System Disk Size

Post by WhataWheasel »

Hi all - I've installed Ubuntu 64 Bit on a Windows 8 host using VirtualBox and am finding discrepancies between the hard disk space used in the Ubuntu file system vs the Windows or VirtualBox file system. When creating the machine, I chose a 100GB dynamic disk which grows in size as more data is added. After a few weeks of using, I'm noticing the vbox file growing, but the space used within the Ubuntu file system is not increasing much.

Ubuntu Disk Analyzer: Main Volume shows 8.4GB / 104.6 GB
VirtualBox Machine Settings: Shows Virtual Size: 100GB, Actual Size: 54.27GB, Details: Dynamically allocated storage.
Windows file system: VDI file in the machine's directory shows: 56,903,060KB

Should I assume that the actual used space is what Ubuntu is showing, or what Windows/VirtualBox is showing? What will happen if the vdi file gets to be larger than 100GB in this case? Is the dynamicly allocated storage option the best to use or should I use a set disk size? Is there any way to defrag / optimize the vbox image so its reduced to actual size again?

Thanks for your feedback on this.
Perryg
Site Moderator
Posts: 34369
Joined: 6. Sep 2008, 22:55
Primary OS: Linux other
VBox Version: OSE self-compiled
Guest OSses: *NIX

Re: Virtual Disk Size Much Greater than File System Disk Siz

Post by Perryg »

I would run the command df-h in a terminal on the guest and see what the actual usage is.
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: Virtual Disk Size Much Greater than File System Disk Siz

Post by mpack »

In fact, dynamic disks do not grow because data is added. They grow when sectors on the disk are written to, which can happen for many reasons other than adding what the user might think of as data. For example, moving a file from one folder to another adds no data to the system, but it may involve a bunch of new sectors being written to.

Since there is no concept of un-writing a sector it means that dynamic disks can only grow, never shrink: at least not unless you run a compaction tool on it, such as CloneVDI. It's also best to avoid Snapshots if you want to minimize wasted disk space.

So: the guest will show you what capacity the disk has, and how much data has been stored. VirtualBox on the other hand will tell you the disk capacity and the amount of it which has been "used", i.e. which has been written to even once.
WhataWheasel
Posts: 6
Joined: 3. Mar 2013, 17:04
Primary OS: MS Windows 8
VBox Version: OSE other
Guest OSses: Win 2012 Server, Win 8, Linux Ubuntu, Linux CentOS

Re: Virtual Disk Size Much Greater than File System Disk Siz

Post by WhataWheasel »

Thanks for the replys. That explanation of how the dynamic disk operates helped greatly. If VirtualBox is reporting the amount of the disk that has been "used" (written to once), will the disk still become "full" at capacity or will VirtualBox continue to write to the volume until the guest shows the disk as full? Perhaps a fixed size disk will be better for my circumstance. I am running an application on that box that routinely downloads large amounts of data, analyzes to form new files, and then deletes the originally downloaded data. Will a fixed size disk be better for this use case?

Find attached some images for reference showing disk usage as seen by the Windows file system, VirtualBox, Ubuntu Disk Analyser, and Ubuntu Terminal.
Windows System Disk Usage
Windows System Disk Usage
Windows System Disk Usage.PNG (13.37 KiB) Viewed 22063 times
VirtualBox Settings Disk Usage
VirtualBox Settings Disk Usage
VirtualBox Settings Disk Usage.PNG (17.88 KiB) Viewed 22063 times
Ubuntu Disk Usage
Ubuntu Disk Usage
Ubuntu Disk Usage.PNG (17.68 KiB) Viewed 22063 times
WhataWheasel
Posts: 6
Joined: 3. Mar 2013, 17:04
Primary OS: MS Windows 8
VBox Version: OSE other
Guest OSses: Win 2012 Server, Win 8, Linux Ubuntu, Linux CentOS

Re: Virtual Disk Size Much Greater than File System Disk Siz

Post by WhataWheasel »

Ubuntu Terminal Disk Usage Screenshot:
Ubuntu Terminal Disk Usage
Ubuntu Terminal Disk Usage
Ubuntu Terminal Disk Usage.PNG (12.46 KiB) Viewed 22060 times
Perryg
Site Moderator
Posts: 34369
Joined: 6. Sep 2008, 22:55
Primary OS: Linux other
VBox Version: OSE self-compiled
Guest OSses: *NIX

Re: Virtual Disk Size Much Greater than File System Disk Siz

Post by Perryg »

What does df -h in a guests terminal say?

Sorry you posted while I was replying.

You have available exactly what the df -h says no matter what the usage shows in the VBox GUI.. As Don said VBox only knows that the disk has been written to. You should not change to fixed. There are ways to compact the guest but IMHO it is not necessary in a Linux guest.
WhataWheasel
Posts: 6
Joined: 3. Mar 2013, 17:04
Primary OS: MS Windows 8
VBox Version: OSE other
Guest OSses: Win 2012 Server, Win 8, Linux Ubuntu, Linux CentOS

Re: Virtual Disk Size Much Greater than File System Disk Siz

Post by WhataWheasel »

Ok, I think I've got it then... Based on your reply, I'd expect the VirtualBox interface to eventually show "Virtual Size" and "Actual Size" of 100.00 GB (while never going above 100 GB) while the Windows file system also shows a 100 GB file size. I'd also assume that at this point, the file will not continue to grow in the Windows file system... Of course, Ubuntu will continue to show the disk's used space increasing as more data is written to the disk. Could you verify that I'm understanding this correctly? Thanks so much for your help.
Perryg
Site Moderator
Posts: 34369
Joined: 6. Sep 2008, 22:55
Primary OS: Linux other
VBox Version: OSE self-compiled
Guest OSses: *NIX

Re: Virtual Disk Size Much Greater than File System Disk Siz

Post by Perryg »

The dynamic size and the VBox actual size reported really does not matter that much. It is how much free space you have in the Linux guest. Use df -h to check on occasion.

If the actual size reported by VBox really is a bother to you then compact the guests drive. Just make sure you keep good backups. Other than that it will not really make a difference.
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: Virtual Disk Size Much Greater than File System Disk Siz

Post by mpack »

Sounds like your understanding is fine. Just as long as you remember that the guest's opinion of "used space" can only be a guide as to what space the file takes on the host. And, if host disk space is an issue for you, then look into CloneVDI to compact the host file, i.e. down to the min size needed to hold the guests files, discarding space used by deleted files.

But, as Perry says, there's really no need to be alarmed. When a VM is new then almost every write to the disk will be to a previously unused sector, so a dynamic drive will appear to grow in size alarmingly. However after not very long the chances of hitting a new sector will reduce to almost nothing, and the dynamic drive will have hit the perfect size for your system. However, beware of any kind of guest process which involves writing to every sector of the disk, e.g. full disk surface checks. The latter are completely pointless in a VM, but people have done it anyway!
WhataWheasel
Posts: 6
Joined: 3. Mar 2013, 17:04
Primary OS: MS Windows 8
VBox Version: OSE other
Guest OSses: Win 2012 Server, Win 8, Linux Ubuntu, Linux CentOS

Re: Virtual Disk Size Much Greater than File System Disk Siz

Post by WhataWheasel »

Great, thanks. Host disk space is not a problem, but I'm using a script to shut the Virtual Machines down, backup and restart, so there are some backup considerations since I'm keeping a number of images for archive purposes. Eventually, the machine will use much more of the 100 GB space allocated though, so I'm fine with the host file system reporting 100 GB even as the drive fills up.
Armando
Posts: 101
Joined: 26. May 2012, 06:50

Re: Virtual Disk Size Much Greater than File System Disk Size

Post by Armando »

I found this old thread, which mostly matches the question I wanted to ask, so I'm appending here instead of opening a new topic.

I understand (and I agree) that there is no need to be "alarmed" when the VDI file dimensions get quite bigger than the actual usage of a dynamic disk.

However I would like to understand why my Ubuntu guest's VDI file keeps being "too big" even after I cloned/compacted it with CloneVdi (rel 2.10), which usually I successfully use to compact Windows guests VDIs from time to time (or after removing much stuff inside the guest), so that VM backups keep as neat and quick as possible.

The Ubuntu VDI was about 32 GB, while the Linux df command reported just 9GB of used disk space. That was ok, as I had "cleaned" the guest by uninstalling and deleting quite a lot.
After cloning/compacting, however, the VDI file did not shrink to about 10GB as I expected (and usually happens with Windows guests).
It certainly got smaller, but "only" to about 23GB.

Am I missing something?
Is there something I have to do inside the guest before cloning?
Thanks in advance for any advice.
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: Virtual Disk Size Much Greater than File System Disk Size

Post by mpack »

You are probably forgetting about the swap partition - which Ubuntu guests have and Windows guests don't. Last time I checked, the Ubuntu install usually set aside 2-3 times the VM RAM as a swap partition. This space is outside the filesystem so the guest doesn't report it as filesystem usage. Nor can it be compacted by CloneVDI since it doesn't use a known filesystem.

Windows doesn't use a swap partition, it uses a dynamically sized swap file within the normal filesystem. So it may take up the same space, but it's reported normally inside the guest OS.
Post Reply