Memory configuration

Discussions related to using VirtualBox on Solaris hosts.

Memory configuration

Postby bauer40 » 30. Oct 2017, 19:30

Hi Gurus!

I wonder about a few memory settings for a server running VirtualBox and simultaniously acting as a small file server.

The server has 24GB memory, running Solaris 11.3. The sum of RAM for the running VMs is 11.25GB, and the sum of the processes rss (and vss) is close to this value. However, my kernel memory seems to be exhausted:

Code: Select all   Expand viewCollapse view
root@vmhost:~# echo "::memstat" | mdb -k
Page Summary                            Pages             Bytes  %Tot
---------------------------- ----------------  ----------------  ----
Kernel                                5601305             21.3G   89%   <-------- Wow!!!!
Defdump prealloc                       171182            668.6M    3%
ZFS                                     82094            320.6M    1%
Anon                                   165536            646.6M    3%
Exec and libs                            5788             22.6M    0%
Page cache                              94703            369.9M    2%
Free (cachelist)                          239              956k    0%
Free (freelist)                        168349            657.6M    3%
Total                                 6289196             23.9G


There is a documentation (https://docs.oracle.com/cd/E53394_01/ht ... gnwox.html) on setting user_reserve_hint_pct to limit zfs arc cache to grow too large and throw "application" memory out. But I'm not sure if VBox allocates "normal" memory, or utilizes kernel memory.

So could somebody clarify a few points for me, please?
1) do Virtual Machines started with VBoxHeadless allocate the per-guest configured memory as kernel memory, or is it "regular" memory? (of course I know a few MB kernel memory will be used by the network drivers, I care more about those 4GB virtual RAM for a VM)
2) is this memory locked in RAM, so it can't be paged out?
3) should I limit the arc cache using the mentioned procedure to make sure my VMs are held in memory?

Thank you for your precious feedback,

Peter
bauer40
 
Posts: 34
Joined: 14. Sep 2008, 09:16

Re: Memory configuration

Postby bauer40 » 30. Oct 2017, 19:47

Addendum:

When I stop a VM rated with 4GB, the kernel memory allocation shrinks:

Code: Select all   Expand viewCollapse view
root@infra1:~# echo "::memstat" | mdb -k
Page Summary                            Pages             Bytes  %Tot
---------------------------- ----------------  ----------------  ----
Kernel                                4537441             17.3G   72%     <--------------- shrunk by the size of the stopped VM
Defdump prealloc                       171182            668.6M    3%
ZFS                                     91644            357.9M    1%
Anon                                   155966            609.2M    2%
Exec and libs                            5821             22.7M    0%
Page cache                              93864            366.6M    1%
Free (cachelist)                         7365             28.7M    0%
Free (freelist)                       1225913              4.6G   19%
Total                                 6289196             23.9G


Starting up this again, the kernel memory allocation grows by the size of the VM.

Code: Select all   Expand viewCollapse view
root@infra1:~# echo "::memstat" | mdb -k
Page Summary                            Pages             Bytes  %Tot
---------------------------- ----------------  ----------------  ----
Kernel                                5502694             20.9G   87%     <--------------- grew again
Defdump prealloc                       171182            668.6M    3%
ZFS                                     85196            332.7M    1%
mdb: found fewer ZFS pages than expected (2m)
Anon                                   170958            667.8M    3%
Exec and libs                            5773             22.5M    0%
Page cache                              96573            377.2M    2%
Free (cachelist)                         4100             16.0M    0%
Free (freelist)                        253232            989.1M    4%
Total                                 6289196             23.9G


Obvisously, there is some correllation between kernel memory allocation and running VMs
bauer40
 
Posts: 34
Joined: 14. Sep 2008, 09:16

Re: Memory configuration

Postby mpack » 30. Oct 2017, 20:26

I've never used Solaris, but I'd be surprised if disk cache is allocated out of kernel RAM. In fact I'd be surprised if VirtualBox allocates disk cache at all. Normally the OS does that. VirtualBox does have a lazy / asynchronous write feature which will use RAM, but AFAIK that is allocated by a user level task (VBoxSVC) so should not be allocated out of kernel RAM.

I do wonder if I'm misunderstanding "kernel RAM" in the Solaris context though, seeing as you have 17GB allocated to it. AIUI in Windows this simply means the first 4GB of RAM, and the only thing special about it is that it's addressable by both 32bit and 64bit code, which makes it useful as a shared pool for all system processes. Btw I am not an expert on Windows kernel processes either, as I've never needed to be (not much call for it in embedded MCU work).
mpack
Site Moderator
 
Posts: 25435
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: Memory configuration

Postby bauer40 » 1. Nov 2017, 08:01

Hi mpack,

no, I don't refer to memory allocation for Disks, I talk about memory allocation for a guest's RAM. This seems to be kernel memory, and I start asking myself wether or not Solaris maps kernel Memory to a processes virtual address space ... no, that can't be. Can it ...? :?
bauer40
 
Posts: 34
Joined: 14. Sep 2008, 09:16

Re: Memory configuration

Postby mpack » 1. Nov 2017, 11:08

I think I'll have to leave you to a dev if he happens to pass by. I know too little about Solaris and what "kernel memory" means there.
mpack
Site Moderator
 
Posts: 25435
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: Memory configuration

Postby Ramshankar » 2. Nov 2017, 13:15

bauer40 wrote:1) do Virtual Machines started with VBoxHeadless allocate the per-guest configured memory as kernel memory, or is it "regular" memory? (of course I know a few MB kernel memory will be used by the network drivers, I care more about those 4GB virtual RAM for a VM)
2) is this memory locked in RAM, so it can't be paged out?
3) should I limit the arc cache using the mentioned procedure to make sure my VMs are held in memory?


1) The memory is allocated in the kernel but there is also parts of the VM process that use userspace allocated memory but that should be a tiny fraction in comparison to the RAM allocation of a typical VM.
2) This memory is locked down so it cannot be swapped or paged out.
3) The ZFS arc cache needs to be adjusted as recommended in the manual if required (https://www.virtualbox.org/manual/ch12.html#idm11558)
Oracle Corp.
Ramshankar
Oracle Corporation
 
Posts: 774
Joined: 7. Jan 2008, 16:17

Re: Memory configuration

Postby bauer40 » 2. Nov 2017, 19:19

Well, thank you, but ... there is not only a little part of "normal" memory in use.

I have a guest which uses 2GB of virtual RAM - named exchange. Here is the virtual process size:

Code: Select all   Expand viewCollapse view
root@infra1:~# grep -i memory /vbox_ufs/exchange64/exchange64.vbox
      <Memory RAMSize="2048"/>

root@infra1:~# ps -efo "vsz rss pid args"|grep exchange
2257756 2155940 21323 /opt/VirtualBox/amd64/VBoxHeadless -s exchange64


You can see, the virtual process size is 2 GB, and, correct me if I'm wrong, kernel kemory is not mapped to a process.

Looking into the processes memory allocation, it looks like this:

Code: Select all   Expand viewCollapse view
root@infra1:~# pmap -x 21323
21323:  /opt/VirtualBox/amd64/VBoxHeadless -s exchange64
         Address     Kbytes        RSS       Anon     Locked Mode   Mapped File
0000000000400000         36         36          -          - r-x----  VBoxHeadless
0000000000418000          4          4          -          - rw-----  VBoxHeadless
0000000000419000      53000       3448       2284          - rw-----    [ heap ]
FFFF80FEB5E00000       2048       2048       2048          - rw-----    [ anon ]
FFFF80FEB6200000       2048       2048       2048          - rw-----    [ anon ]
FFFF80FEB6600000       2048       2048          -          -rw-s---
FFFF80FEB6A00000       2048       2048          -          -rw-s---
FFFF80FEB6E00000       2048       2048          -          -rw-s---
FFFF80FEB7200000       2048       2048          -          -rw-s---
FFFF80FEB7600000       2048       2048          -          -rw-s---
FFFF80FEB7A00000       2048       2048          -          -rw-s---
FFFF80FEB7E00000       2048       2048          -          -rw-s---
FFFF80FEB8200000       2048       2048          -          -rw-s---
FFFF80FEB8600000       2048       2048          -          -rw-s---
FFFF80FEB8A00000       2048       2048          -          -rw-s---
FFFF80FEB8E00000       2048       2048          -          -rw-s---
FFFF80FEB9200000       2048       2048          -          -rw-s---
FFFF80FEB9600000       2048       2048          -          -rw-s---
FFFF80FEB9A00000       2048       2048          -          -rw-s---
FFFF80FEB9E00000       2048       2048          -          -rw-s---
FFFF80FEBA200000       2048       2048          -          -rw-s---
FFFF80FEBA600000       2048       2048          -          -rw-s---
FFFF80FEBAA00000       2048       2048          -          -rw-s---
FFFF80FEBAE00000       2048       2048          -          -rw-s---
FFFF80FEBB200000       2048       2048          -          -rw-s---
FFFF80FEBB600000       2048       2048          -          -rw-s---
FFFF80FEBBA00000       2048       2048          -          -rw-s---
FFFF80FEBBE00000       2048       2048          -          -rw-s---
FFFF80FEBC200000       2048       2048          -          -rw-s---
FFFF80FEBC600000       2048       2048          -          -rw-s---
FFFF80FEBCA00000       2048       2048          -          -rw-s---
FFFF80FEBCE00000       2048       2048          -          -rw-s---
FFFF80FEBD200000       2048       2048          -          -rw-s---
FFFF80FEBD600000       2048       2048          -          -rw-s---
FFFF80FEBDA00000       2048       2048          -          -rw-s---
FFFF80FEBDE00000       2048       2048          -          -rw-s---
FFFF80FEBE200000       2048       2048          -          -rw-s---
FFFF80FEBE600000       2048       2048          -          -rw-s---
FFFF80FEBEA00000       2048       2048          -          -rw-s---
FFFF80FEBEE00000       2048       2048          -          -rw-s---
FFFF80FEBF200000       2048       2048          -          -rw-s---
FFFF80FEBF600000       2048       2048          -          -rw-s---
FFFF80FEBFA00000       2048       2048          -          -rw-s---
FFFF80FEBFE00000       2048       2048          -          -rw-s---
FFFF80FEC0200000       2048       2048          -          -rw-s---
FFFF80FEC0600000       2048       2048          -          -rw-s---
FFFF80FEC0A00000       2048       2048          -          -rw-s---
FFFF80FEC0E00000       2048       2048          -          -rw-s---
FFFF80FEC1200000       2048       2048          -          -rw-s---
FFFF80FEC1600000       2048       2048          -          -rw-s---
FFFF80FEC1A00000       2048       2048          -          -rw-s---
FFFF80FEC1E00000       2048       2048          -          -rw-s---
FFFF80FEC2200000       2048       2048          -          -rw-s---


There are roughly 1024 2k blocks, which sum up to 2GB.

Is this a double memory allocation, or are these non-mapped/unmarked memory blocks kernel memory?
bauer40
 
Posts: 34
Joined: 14. Sep 2008, 09:16

Re: Memory configuration

Postby michaln » 7. Nov 2017, 10:25

bauer40 wrote:You can see, the virtual process size is 2 GB, and, correct me if I'm wrong, kernel kemory is not mapped to a process.

I'll correct you -- you're wrong :) The VM's RAM is allocated as "kernel" memory on the host, but it is also mapped into the VM's process address space.
michaln
Oracle Corporation
 
Posts: 2841
Joined: 19. Dec 2007, 15:45
Primary OS: MS Windows 7
VBox Version: PUEL
Guest OSses: Any and all


Return to VirtualBox on Solaris Hosts

Who is online

Users browsing this forum: No registered users and 1 guest