Why don't VMs release their unused memory to the host ?

This is for discussing general topics about how to use VirtualBox.
Post Reply
HulkySulky
Posts: 29
Joined: 28. Jan 2020, 06:24

Why don't VMs release their unused memory to the host ?

Post by HulkySulky »

I'm aware that once a VM has used some memory, that memory stays with the VM, and is never de-allocated (never returned to the host). Through ballooning VMs can share a common memory pool, but once again, once physical memory from the host has been requested by a VM, it stays in that memory pool. VB's documentation is very clear about that (https://docs.oracle.com/en/virtualizati ... dd-balloon):
The memory made available through the ballooning mechanism is only available for reuse by Oracle VM VirtualBox. It is not returned as free memory to the host. Requesting balloon memory from a running guest will therefore not increase the amount of free, unallocated memory on the host.
My question is : why that choice ? I mean, why not allow VM's to tell the host "hey, currently I'm using only x amount of RAM, if you allocated more then you can free some of it" ? Isn't that the kind of information the guest additions could provide to the host ?

I have a 8 GiB laptop, with a 4 GiB VM. Sometimes the VM only uses 1 GiB, yet all 4GiB are allocated on the host, because the VM requested it before. And if I need to do some work on the host that needs more memory, I have to shut the VM down and restart it, in order to de-allocated its unused memory.

Why such a choice ?
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: Why don't VMs release their unused memory to the host ?

Post by mpack »

What should it do, crash? It has been allocated that RAM by the host. If it gives it up there is no guarantee that it can get it back.
HulkySulky
Posts: 29
Joined: 28. Jan 2020, 06:24

Re: Why don't VMs release their unused memory to the host ?

Post by HulkySulky »

mpack wrote:What should it do, crash ?
Yes, crash. Or swap.

I mean, that's already the case. Let's say I start my 4 GiB VM, and it uses 1 GiB to run initally. Then I do some work on my 8GiB host, until its RAM is used at 7GiB out of 8. Then I go back to my VM, do some work, e.g. open a lot of tabs in a browser, when the host memory reaches 8GiB everything crashes, both host and guests (providing there is no swap enabled). That's expected, if you don't want it to crash then you have to watch your memory on the host.

So, how is that different from allowing the VM to return unused memory to the host ?
mpack wrote:If it gives it up there is no guarantee that it can get it back
Again, in my scenario there was no guarantee either that the memory was available to the guest when it needed it, and if fact in wasn't and it crashed. No difference there.

Returning unused memory to the host would avoid to have to restart the VMs in order to reclaim unused memory on the host.

It seems (don't take my word for it, haven't tried) that other hypervisors, like Hyper-V or KVM, implement such a feature :

viewtopic.php?f=3&t=101406
WSL 2 (Windows Subsystem for Linux) uses the Dynamic Memory feature of Hyper-V, which is configured with multiple RAM sizes. If you would use WSL 2, you'd configure Startup RAM and Minimum RAM to 4 GB, and Maximum RAM to 20 GB. The VM would then statically reserve 4 GB RAM from the host, and dynamically use 16 GB on demand. If the host was low on physical memory, the dynamic memory could even be moved to the pagefile, severely slowing down the VM (but better than crashing).
https://pve.proxmox.com/wiki/Dynamic_Memory_Management
Memory ballooning (KVM only) allows you to have your guest dynamically change it’s memory usage by evicting unused memory during run time. It reduces the impact your guest can have on memory usage of your host by giving up unused memory back to the host.

The Proxmox VE host can loan ballooned memory to a busy VM. The VM decides which processes or cache pages to swap out to free up memory for the balloon. The VM (Windows or Linux) knows best which memory regions it can give up without impacting performance of the VM.
I guess there's a good reason why VirtualBox doesn't do it, I'm just trying to understand why. Is it just not yet implemented ? Has it something to do with the fact that VB is a type2 hypervisor ? Do VB developers think it's a bad idea ? If yes, why ? Because it can crash ? Again, it already crashes if a VM requires more memory that the host has, no difference.
fth0
Volunteer
Posts: 5668
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: Why don't VMs release their unused memory to the host ?

Post by fth0 »

The answer to most of your questions boils down to one thing: Performance.

Generally speaking, memory management of modern OSes is quite a complex beast, and a comprehensive description would fill a thick book. In consequence, it's difficult to give easy answers.

Your example indicates that you're coming from a very high level, measuring memory with only a single size parameter. In reality, modern OSes have Physical Memory and Virtual Memory layers, and the virtual memory layer (*) handles between 5 and 10 different types and categories of memory (for Windows, see screenshots of Sysinternals RAMMap and VMMap to get a first impression). And adding VMs to the topic, where the host and guest OSes can be different, ...

(*) Just in case, do not confuse this virtual memory layer with paging files or swap files.
HulkySulky
Posts: 29
Joined: 28. Jan 2020, 06:24

Re: Why don't VMs release their unused memory to the host ?

Post by HulkySulky »

fth0 wrote:And adding VMs to the topic, where the host and guest OSes can be different, ...
That made me realize that of all the hypervisors I mentioned, VBox is the only one that is multi-OS at the host level (Linux, Mac, Windows), whereas KVM is Linux specific (maybe Mac?) and Hyper-V Windows specific. That may be the reason why those 2 other hypervisors managed to implemented dynamic ballooning, it's probably easier when there's only one host OS.
fth0
Volunteer
Posts: 5668
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: Why don't VMs release their unused memory to the host ?

Post by fth0 »

AFAIU, Hyper-V is the only hypervisor that really gives the memory back to the host (Hyper-V Dynamic Memory).

VirtualBox, KVM and Proxmox all use the memory ballooning technique, where a hypervisor's driver inside the guest OS allocates guest physical memory and lets the hypervisor provide that memory to other VMs. Note that from the host OS'es perspective, the host physical memory does not return in this case.
HulkySulky
Posts: 29
Joined: 28. Jan 2020, 06:24

Re: Why don't VMs release their unused memory to the host ?

Post by HulkySulky »

fth0 wrote:VirtualBox, KVM and Proxmox all use the memory ballooning technique
OK thanks, good to know. Proxmox' quote is a bit misleading then when they state that they give "unused memory back to the host". I guess they mean they give it back to the common pool of VM memory on the host, not back to the host's OS per se.

But a difference between Proxmox and VBox seems to be that Proxmox does automatic ballooning, according again to their quote, whereas in VBox it has to be done manually.
HulkySulky
Posts: 29
Joined: 28. Jan 2020, 06:24

Re: Why don't VMs release their unused memory to the host ?

Post by HulkySulky »

fth0 wrote:AFAIU, Hyper-V is the only hypervisor that really gives the memory back to the host (Hyper-V Dynamic Memory).
FYI I just asked on the Proxmox forum, and it seems that the memory taken back by the host can indeed be for any tasks, even non-Proxmox/KVM related ones. So it behaves more like Hyper-V than like VBox.

https://forum.proxmox.com/threads/clari ... ge.113293/
fth0
Volunteer
Posts: 5668
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: Why don't VMs release their unused memory to the host ?

Post by fth0 »

Interesting. This probably means that Proxmox implemented the host-side part themselves, because AFAIK, plain QEMU/KVM only supports manual ballooning. There's an old experimental KVM project for Automatic Ballooning that didn't make it into QEMU (at least not until 2019).
multiOS
Volunteer
Posts: 800
Joined: 14. Sep 2019, 16:51
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: WIN11,10, 7, Linux (various)
Location: United Kingdom

Re: Why don't VMs release their unused memory to the host ?

Post by multiOS »

That's understandable as Proxmox is closely allied to the use of KVM (and Qemu) on Linux-based OSs and, as KVM is built into the Linux Kernel, it operates as something of an hybrid which is difficult to classify in that it can demonstrate both a 'Type 1' and 'Type 2' hypervisor behaviour.

But, in questioning why VirtualBox memory management is designed/programmed the way it is, it may be relevant to consider that VMWare as developers of a similar virtualisation product also reached the same memory management decision in designing and programming their Type 2 desktop application(s).

Apart from the positive performance management reasons for it being the way it is, you should also probably consider that the present design decision means that the User remains wholly in control and is therefore responsible for any overcommitment of available resources and any consequent OS crashes; whereas, if User decisions on VM resource allocations were were to be overridden via programming techniques, simply because a VM wasn't making immediate use some of those resources, then the programmers would be directly responsible for any consequent VM OS crashes.
HulkySulky
Posts: 29
Joined: 28. Jan 2020, 06:24

Re: Why don't VMs release their unused memory to the host ?

Post by HulkySulky »

multiOS wrote:the present design decision means that the User remains wholly in control and is therefore responsible for any overcommitment of available resources and any consequent OS crashes
Interesting point. Although one could argue that once the user checked some "automatic ballooning" and/or "dynamic memory" boxes, he explicitly agreed to hand over the control to the Hypervisor
multiOS wrote:it may be relevant to consider that VMWare as developers of a similar virtualisation product also reached the same memory management decision in designing and programming their Type 2 desktop application(s)
An other similarity to note it that VMWare's type 2 desktop hypervisor, like VBox, runs on both Linux and Windows, whereas KVM and Hyper-V are both mono-OS. Memory management is probably easier to implement when there's only one family of host OS involved
Post Reply