Memory configuration

Discussions related to using VirtualBox on Solaris hosts.
Post Reply
bauer40
Posts: 37
Joined: 14. Sep 2008, 09:16

Memory configuration

Post by bauer40 »

Hi Gurus!

I wonder about a few memory settings for a server running VirtualBox and simultaniously acting as a small file server.

The server has 24GB memory, running Solaris 11.3. The sum of RAM for the running VMs is 11.25GB, and the sum of the processes rss (and vss) is close to this value. However, my kernel memory seems to be exhausted:

Code: Select all

root@vmhost:~# echo "::memstat" | mdb -k
Page Summary                            Pages             Bytes  %Tot
---------------------------- ----------------  ----------------  ----
Kernel                                5601305             21.3G   89%   <-------- Wow!!!!
Defdump prealloc                       171182            668.6M    3%
ZFS                                     82094            320.6M    1%
Anon                                   165536            646.6M    3%
Exec and libs                            5788             22.6M    0%
Page cache                              94703            369.9M    2%
Free (cachelist)                          239              956k    0%
Free (freelist)                        168349            657.6M    3%
Total                                 6289196             23.9G
There is a documentation (https://docs.oracle.com/cd/E53394_01/ht ... gnwox.html) on setting user_reserve_hint_pct to limit zfs arc cache to grow too large and throw "application" memory out. But I'm not sure if VBox allocates "normal" memory, or utilizes kernel memory.

So could somebody clarify a few points for me, please?
1) do Virtual Machines started with VBoxHeadless allocate the per-guest configured memory as kernel memory, or is it "regular" memory? (of course I know a few MB kernel memory will be used by the network drivers, I care more about those 4GB virtual RAM for a VM)
2) is this memory locked in RAM, so it can't be paged out?
3) should I limit the arc cache using the mentioned procedure to make sure my VMs are held in memory?

Thank you for your precious feedback,

Peter
bauer40
Posts: 37
Joined: 14. Sep 2008, 09:16

Re: Memory configuration

Post by bauer40 »

Addendum:

When I stop a VM rated with 4GB, the kernel memory allocation shrinks:

Code: Select all

root@infra1:~# echo "::memstat" | mdb -k
Page Summary                            Pages             Bytes  %Tot
---------------------------- ----------------  ----------------  ----
Kernel                                4537441             17.3G   72%     <--------------- shrunk by the size of the stopped VM
Defdump prealloc                       171182            668.6M    3%
ZFS                                     91644            357.9M    1%
Anon                                   155966            609.2M    2%
Exec and libs                            5821             22.7M    0%
Page cache                              93864            366.6M    1%
Free (cachelist)                         7365             28.7M    0%
Free (freelist)                       1225913              4.6G   19%
Total                                 6289196             23.9G
Starting up this again, the kernel memory allocation grows by the size of the VM.

Code: Select all

root@infra1:~# echo "::memstat" | mdb -k
Page Summary                            Pages             Bytes  %Tot
---------------------------- ----------------  ----------------  ----
Kernel                                5502694             20.9G   87%     <--------------- grew again
Defdump prealloc                       171182            668.6M    3%
ZFS                                     85196            332.7M    1%
mdb: found fewer ZFS pages than expected (2m)
Anon                                   170958            667.8M    3%
Exec and libs                            5773             22.5M    0%
Page cache                              96573            377.2M    2%
Free (cachelist)                         4100             16.0M    0%
Free (freelist)                        253232            989.1M    4%
Total                                 6289196             23.9G
Obvisously, there is some correllation between kernel memory allocation and running VMs
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: Memory configuration

Post by mpack »

I've never used Solaris, but I'd be surprised if disk cache is allocated out of kernel RAM. In fact I'd be surprised if VirtualBox allocates disk cache at all. Normally the OS does that. VirtualBox does have a lazy / asynchronous write feature which will use RAM, but AFAIK that is allocated by a user level task (VBoxSVC) so should not be allocated out of kernel RAM.

I do wonder if I'm misunderstanding "kernel RAM" in the Solaris context though, seeing as you have 17GB allocated to it. AIUI in Windows this simply means the first 4GB of RAM, and the only thing special about it is that it's addressable by both 32bit and 64bit code, which makes it useful as a shared pool for all system processes. Btw I am not an expert on Windows kernel processes either, as I've never needed to be (not much call for it in embedded MCU work).
bauer40
Posts: 37
Joined: 14. Sep 2008, 09:16

Re: Memory configuration

Post by bauer40 »

Hi mpack,

no, I don't refer to memory allocation for Disks, I talk about memory allocation for a guest's RAM. This seems to be kernel memory, and I start asking myself wether or not Solaris maps kernel Memory to a processes virtual address space ... no, that can't be. Can it ...? :?
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: Memory configuration

Post by mpack »

I think I'll have to leave you to a dev if he happens to pass by. I know too little about Solaris and what "kernel memory" means there.
Ramshankar
Oracle Corporation
Posts: 793
Joined: 7. Jan 2008, 16:17

Re: Memory configuration

Post by Ramshankar »

bauer40 wrote:1) do Virtual Machines started with VBoxHeadless allocate the per-guest configured memory as kernel memory, or is it "regular" memory? (of course I know a few MB kernel memory will be used by the network drivers, I care more about those 4GB virtual RAM for a VM)
2) is this memory locked in RAM, so it can't be paged out?
3) should I limit the arc cache using the mentioned procedure to make sure my VMs are held in memory?
1) The memory is allocated in the kernel but there is also parts of the VM process that use userspace allocated memory but that should be a tiny fraction in comparison to the RAM allocation of a typical VM.
2) This memory is locked down so it cannot be swapped or paged out.
3) The ZFS arc cache needs to be adjusted as recommended in the manual if required (https://www.virtualbox.org/manual/ch12.html#idm11558)
Oracle Corp.
bauer40
Posts: 37
Joined: 14. Sep 2008, 09:16

Re: Memory configuration

Post by bauer40 »

Well, thank you, but ... there is not only a little part of "normal" memory in use.

I have a guest which uses 2GB of virtual RAM - named exchange. Here is the virtual process size:

Code: Select all

root@infra1:~# grep -i memory /vbox_ufs/exchange64/exchange64.vbox
      <Memory RAMSize="2048"/>

root@infra1:~# ps -efo "vsz rss pid args"|grep exchange
2257756 2155940 21323 /opt/VirtualBox/amd64/VBoxHeadless -s exchange64
You can see, the virtual process size is 2 GB, and, correct me if I'm wrong, kernel kemory is not mapped to a process.

Looking into the processes memory allocation, it looks like this:

Code: Select all

root@infra1:~# pmap -x 21323 
21323:  /opt/VirtualBox/amd64/VBoxHeadless -s exchange64
         Address     Kbytes        RSS       Anon     Locked Mode   Mapped File
0000000000400000         36         36          -          - r-x----  VBoxHeadless
0000000000418000          4          4          -          - rw-----  VBoxHeadless
0000000000419000      53000       3448       2284          - rw-----    [ heap ]
FFFF80FEB5E00000       2048       2048       2048          - rw-----    [ anon ]
FFFF80FEB6200000       2048       2048       2048          - rw-----    [ anon ]
FFFF80FEB6600000       2048       2048          -          -rw-s--- 
FFFF80FEB6A00000       2048       2048          -          -rw-s--- 
FFFF80FEB6E00000       2048       2048          -          -rw-s--- 
FFFF80FEB7200000       2048       2048          -          -rw-s--- 
FFFF80FEB7600000       2048       2048          -          -rw-s--- 
FFFF80FEB7A00000       2048       2048          -          -rw-s--- 
FFFF80FEB7E00000       2048       2048          -          -rw-s--- 
FFFF80FEB8200000       2048       2048          -          -rw-s--- 
FFFF80FEB8600000       2048       2048          -          -rw-s--- 
FFFF80FEB8A00000       2048       2048          -          -rw-s--- 
FFFF80FEB8E00000       2048       2048          -          -rw-s--- 
FFFF80FEB9200000       2048       2048          -          -rw-s--- 
FFFF80FEB9600000       2048       2048          -          -rw-s--- 
FFFF80FEB9A00000       2048       2048          -          -rw-s--- 
FFFF80FEB9E00000       2048       2048          -          -rw-s--- 
FFFF80FEBA200000       2048       2048          -          -rw-s--- 
FFFF80FEBA600000       2048       2048          -          -rw-s--- 
FFFF80FEBAA00000       2048       2048          -          -rw-s--- 
FFFF80FEBAE00000       2048       2048          -          -rw-s--- 
FFFF80FEBB200000       2048       2048          -          -rw-s--- 
FFFF80FEBB600000       2048       2048          -          -rw-s--- 
FFFF80FEBBA00000       2048       2048          -          -rw-s--- 
FFFF80FEBBE00000       2048       2048          -          -rw-s--- 
FFFF80FEBC200000       2048       2048          -          -rw-s--- 
FFFF80FEBC600000       2048       2048          -          -rw-s--- 
FFFF80FEBCA00000       2048       2048          -          -rw-s--- 
FFFF80FEBCE00000       2048       2048          -          -rw-s--- 
FFFF80FEBD200000       2048       2048          -          -rw-s--- 
FFFF80FEBD600000       2048       2048          -          -rw-s--- 
FFFF80FEBDA00000       2048       2048          -          -rw-s--- 
FFFF80FEBDE00000       2048       2048          -          -rw-s--- 
FFFF80FEBE200000       2048       2048          -          -rw-s--- 
FFFF80FEBE600000       2048       2048          -          -rw-s--- 
FFFF80FEBEA00000       2048       2048          -          -rw-s--- 
FFFF80FEBEE00000       2048       2048          -          -rw-s--- 
FFFF80FEBF200000       2048       2048          -          -rw-s--- 
FFFF80FEBF600000       2048       2048          -          -rw-s--- 
FFFF80FEBFA00000       2048       2048          -          -rw-s--- 
FFFF80FEBFE00000       2048       2048          -          -rw-s--- 
FFFF80FEC0200000       2048       2048          -          -rw-s--- 
FFFF80FEC0600000       2048       2048          -          -rw-s--- 
FFFF80FEC0A00000       2048       2048          -          -rw-s--- 
FFFF80FEC0E00000       2048       2048          -          -rw-s--- 
FFFF80FEC1200000       2048       2048          -          -rw-s--- 
FFFF80FEC1600000       2048       2048          -          -rw-s--- 
FFFF80FEC1A00000       2048       2048          -          -rw-s--- 
FFFF80FEC1E00000       2048       2048          -          -rw-s--- 
FFFF80FEC2200000       2048       2048          -          -rw-s--- 
There are roughly 1024 2k blocks, which sum up to 2GB.

Is this a double memory allocation, or are these non-mapped/unmarked memory blocks kernel memory?
michaln
Oracle Corporation
Posts: 2973
Joined: 19. Dec 2007, 15:45
Primary OS: MS Windows 7
VBox Version: PUEL
Guest OSses: Any and all
Contact:

Re: Memory configuration

Post by michaln »

bauer40 wrote:You can see, the virtual process size is 2 GB, and, correct me if I'm wrong, kernel kemory is not mapped to a process.
I'll correct you -- you're wrong :) The VM's RAM is allocated as "kernel" memory on the host, but it is also mapped into the VM's process address space.
Post Reply