More RAM for VM = much slower Windows guest boot time ?
-
- Posts: 26
- Joined: 26. Mar 2011, 00:52
- Primary OS: MS Windows 7
- VBox Version: OSE other
- Guest OSses: CentOS OS/2
More RAM for VM = much slower Windows guest boot time ?
I have a host Windows machine with 32 GB of RAM. Task manager shows there are still many gigs available - ie. it doesn't get into a paging situation.
I have experimented with different RAM amounts for a Windows 10 VM.
Here are the boot times.
2GB RAM : 42 seconds
6GB RAM : 55 seconds
16GB RAM : 4 minutes and 43 seconds ! And yes, I'm certain my host wasn't paging even in this case.
I am measuring boot time from the moment I press "start" in virtualbox to when I hear the Windows startup sound.
I thought perhaps Windows 10 memory compression was causing this, so I tried disabling it using "Disable-MMagent -mc" in Powershell.
Unfortunately, the boot time results were essentially unchanged. RAM actually used varied, due to compression.
I had similar results with Windows 11 - lower RAM amount is better, though I only tried 6GB and 16GB.
With an Ubuntu 20 guest, there was no difference in boot time whether I assigned 2GB or 20GB of RAM to the VM. It was 42 seconds in both cases. This is fortunate, since I really do need the high RAM amount for development (Chrome compilation).
What's the reason for this massive difference here for the Windows guest boot time as a function of RAM assigned to the VM ?
FYI, my host is an i7-5820k running Windows 10.
I have experimented with different RAM amounts for a Windows 10 VM.
Here are the boot times.
2GB RAM : 42 seconds
6GB RAM : 55 seconds
16GB RAM : 4 minutes and 43 seconds ! And yes, I'm certain my host wasn't paging even in this case.
I am measuring boot time from the moment I press "start" in virtualbox to when I hear the Windows startup sound.
I thought perhaps Windows 10 memory compression was causing this, so I tried disabling it using "Disable-MMagent -mc" in Powershell.
Unfortunately, the boot time results were essentially unchanged. RAM actually used varied, due to compression.
I had similar results with Windows 11 - lower RAM amount is better, though I only tried 6GB and 16GB.
With an Ubuntu 20 guest, there was no difference in boot time whether I assigned 2GB or 20GB of RAM to the VM. It was 42 seconds in both cases. This is fortunate, since I really do need the high RAM amount for development (Chrome compilation).
What's the reason for this massive difference here for the Windows guest boot time as a function of RAM assigned to the VM ?
FYI, my host is an i7-5820k running Windows 10.
-
- Site Moderator
- Posts: 39134
- Joined: 4. Sep 2008, 17:09
- Primary OS: MS Windows 10
- VBox Version: PUEL
- Guest OSses: Mostly XP
Re: More RAM for VM = much slower Windows guest boot time ?
Please provide a VM log file. Make sure the VM is fully shut down, then right click it in the manager UI. Select "Show Log" and save "VBox.log" (no other file) to a zip file. Attach the zip here.
-
- Posts: 26
- Joined: 26. Mar 2011, 00:52
- Primary OS: MS Windows 7
- VBox Version: OSE other
- Guest OSses: CentOS OS/2
Re: More RAM for VM = much slower Windows guest boot time ?
Here you go.mpack wrote:Please provide a VM log file. Make sure the VM is fully shut down, then right click it in the manager UI. Select "Show Log" and save "VBox.log" (no other file) to a zip file. Attach the zip here.
I have provided two logs for my Win11 guest VM.
One with 4GB. It took 32 seconds until the sound played back. I shut it down at 41 seconds.
One with 16GB. It took 2 minutes 59 seconds until the sound played back. I shut it down soon after at 3 mins 19 seconds.
I ran these back to back. The only thing I changed was the RAM assigned in the VM properties.
I had memory compression disabled in the Windows 10 host, but I don't believe that affects the result meaningfully. A massive boot time difference remains depending on RAM assigned, and it boots much faster with less RAM.
- Attachments
-
- ram.zip
- (69.13 KiB) Downloaded 16 times
-
- Volunteer
- Posts: 5677
- Joined: 14. Feb 2019, 03:06
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Linux, Windows 10, ...
- Location: Germany
Re: More RAM for VM = much slower Windows guest boot time ?
VirtualBox only allocates the host memory when it is accessed by the guest OS. When starting up, a Linux OS usually only accesses the memory that it needs, so that the total host memory allocation gets distributed over the first hours of runtime. A Windows OS usually accesses all the available memory during startup, so the total host memory allocation only gets distributed over the startup phase.
Although the following may seem counter-intuitive, please disable the use of large memory pages and repeat your measurement:
I'm really curious about what you'll find out.
Although the following may seem counter-intuitive, please disable the use of large memory pages and repeat your measurement:
Code: Select all
cd "C:\Program Files\Oracle\VirtualBox"
VBoxManage modifyvm "Win 11" --largepages off
-
- Posts: 26
- Joined: 26. Mar 2011, 00:52
- Primary OS: MS Windows 7
- VBox Version: OSE other
- Guest OSses: CentOS OS/2
Re: More RAM for VM = much slower Windows guest boot time ?
Thanks ! That made a dramatic difference.fth0 wrote:VirtualBox only allocates the host memory when it is accessed by the guest OS. When starting up, a Linux OS usually only accesses the memory that it needs, so that the total host memory allocation gets distributed over the first hours of runtime. A Windows OS usually accesses all the available memory during startup, so the total host memory allocation only gets distributed over the startup phase.
Although the following may seem counter-intuitive, please disable the use of large memory pages and repeat your measurement:
I'm really curious about what you'll find out.Code: Select all
cd "C:\Program Files\Oracle\VirtualBox" VBoxManage modifyvm "Win 11" --largepages off
The startup time with 4GB dropped to 27 seconds.
The startup time with 16GB dropped to 49 seconds.
What's the downside of this setting ? And why isn't it the default for Windows 10/11 guests ?
Last edited by madbrain on 13. Oct 2021, 21:43, edited 1 time in total.
-
- Volunteer
- Posts: 5677
- Joined: 14. Feb 2019, 03:06
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Linux, Windows 10, ...
- Location: Germany
Re: More RAM for VM = much slower Windows guest boot time ?
On x86 CPUs, memory is organized in pages (4 kB), large pages (2 MB) and huge pages (1 GB) (and even bigger pages (512 GB)). When VirtualBox needs to allocate 1 GB (in smaller memory blocks), it can either request 512 large pages or 512 * 512 = 262144 (small) pages from the host OS. Therefore, it seems natural to assume that using large pages is more efficient.madbrain wrote:What's the downside of this setting ?
That being said, the memory management of a modern host OS is quite a complex beast, and although I know quite a bit about it, I cannot really answer your question in general. You should simply disable the use of large pages for your VirtualBox VMs, and if it suits you, all is fine.
-
- Posts: 26
- Joined: 26. Mar 2011, 00:52
- Primary OS: MS Windows 7
- VBox Version: OSE other
- Guest OSses: CentOS OS/2
Re: More RAM for VM = much slower Windows guest boot time ?
Yes, I'm a developer as well, and it makes sense to me that large pages should be faster than small pages as well, but perhaps there is a bug somewhere, or an optimization that needs to be done. Surely the Virtualbox developers have done some benchmarking when they implemented the large pages feature and decided to make it the default.fth0 wrote:On x86 CPUs, memory is organized in pages (4 kB), large pages (2 MB) and huge pages (1 GB) (and even bigger pages (512 GB)). When VirtualBox needs to allocate 1 GB (in smaller memory blocks), it can either request 512 large pages or 512 * 512 = 262144 (small) pages from the host OS. Therefore, it seems natural to assume that using large pages is more efficient.madbrain wrote:What's the downside of this setting ?
That being said, the memory management of a modern host OS is quite a complex beast, and although I know quite a bit about it, I cannot really answer your question in general. You should simply disable the use of large pages for your VirtualBox VMs, and if it suits you, all is fine.
I'm wondering if others are seeing this dramatic difference too ...
-
- Volunteer
- Posts: 5677
- Joined: 14. Feb 2019, 03:06
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Linux, Windows 10, ...
- Location: Germany
Re: More RAM for VM = much slower Windows guest boot time ?
As far as can be guesstimated from VirtualBox users posting here in the forums, the majority of free VirtualBox users only provide a few GB of memory to their VMs, so they usually won't notice.madbrain wrote:I'm wondering if others are seeing this dramatic difference too ...
When you did your initial measurements, what was the uptime of the Windows host? Does it make a considerable difference, when the Windows host has just been started before the measurements?
-
- Posts: 26
- Joined: 26. Mar 2011, 00:52
- Primary OS: MS Windows 7
- VBox Version: OSE other
- Guest OSses: CentOS OS/2
Re: More RAM for VM = much slower Windows guest boot time ?
It was measured in hours. I know there are a lot of things that load on startup. That wasn't the issue. The problem is entirely repeatable by changing the RAM assigned to the VM back and forth. Assigning more RAM always equals slower boot time, but much more so with large pages on.fth0 wrote:When you did your initial measurements, what was the uptime of the Windows host? Does it make a considerable difference, when the Windows host has just been started before the measurements?madbrain wrote:I'm wondering if others are seeing this dramatic difference too ...
-
- Volunteer
- Posts: 5677
- Joined: 14. Feb 2019, 03:06
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Linux, Windows 10, ...
- Location: Germany
Re: More RAM for VM = much slower Windows guest boot time ?
One possible theory behind the slowness is that although the Windows host OS still has plenty of free memory, it already has considerable additional work to do to provide large pages to VirtualBox. AFAIK, it hasn't been proven yet ... but there have been VIrtualBox users that didn't run into the issue when starting their VM right after starting the Windows host OS.
-
- Posts: 26
- Joined: 26. Mar 2011, 00:52
- Primary OS: MS Windows 7
- VBox Version: OSE other
- Guest OSses: CentOS OS/2
Re: More RAM for VM = much slower Windows guest boot time ?
I set largepages back to on, rebooted my host Windows OS, and still saw the very slow startup of the VM. So, that definitely doesn't solve the issue here.fth0 wrote:One possible theory behind the slowness is that although the Windows host OS still has plenty of free memory, it already has considerable additional work to do to provide large pages to VirtualBox. AFAIK, it hasn't been proven yet ... but there have been VIrtualBox users that didn't run into the issue when starting their VM right after starting the Windows host OS.
-
- Volunteer
- Posts: 5677
- Joined: 14. Feb 2019, 03:06
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Linux, Windows 10, ...
- Location: Germany
Re: More RAM for VM = much slower Windows guest boot time ?
Thanks for the feedback. Can you perform another experiment for me?
Reboot the Windows host, start the VM with large pages enabled, wait 5 minutes, and shut down the VM from within the guest OS. Reboot the Windows host, start the VM with large pages disabled, wait 5 minutes, and shut down the VM from within the guest OS. Provide VBox.log and VBox.log.1.
Reboot the Windows host, start the VM with large pages enabled, wait 5 minutes, and shut down the VM from within the guest OS. Reboot the Windows host, start the VM with large pages disabled, wait 5 minutes, and shut down the VM from within the guest OS. Provide VBox.log and VBox.log.1.