hello,
I have a dual core intel based desktop that i run ubuntu 10.04 LTS on.
I have recently installed virtual box 4.2.8 r83876 and installed a solaris 11.1 guest.
I first noticed that simple bash ( 4.1.11 or 4.2) scripts would dump core in the guest eg
while :
do
/bin/true
done
A bit of debugging showed that bash forked and then tripped over a freed malloc buffer.
A bit more debugging showed that the buffer was freed in the bash parent process after the fork() had returned.
That shouldnt happen.
I then wrote a little test gcc case that ..
1) mallocs a small buffer
2) writes a 1 into the buffer
3) forks
4) the parent immediately writes a 2 into the buffer
5) the child checks buffer to see if its 1 ( correct) or 2 ( incorrect).
Occasionally ( 1 in 250) its incorrect. Running on bare metal ubuntu or solaris 11.1 bare metal it works perfectly.
The way I think Solaris works is that pages in the malloc buffer are allocated from a segvn segment, and will have their hardware PTE mapping changed to read-only in the fork() system call whilst the corresponding segvn segment permissions remain read/write , this allows segvn_fault() to recognise that a Copy On Write ( COW) pagefault has occurred on a private mapping when the #e pagefault trap comes in and the page needs duplicating - first process to write to the mapping gets a new page with a copy of the old pages data all of their own.
A bit of dtrace and its clear that the fork() code is doing the right thing and changing the PTE from 0x39827067 newpte 0x39827065 ( losing the write bit) and then finally to 0x39827005 so its lost the mod and ref bits as well. the page frame number matches what i get when i hand walk the mapping tables and the PTE what i see in the lowest level table at the expected offset for the malloc pages buffer.
But shortly after the fork() when we write a 2 to our malloc buffer in the parent, the 249 times it works we see a segvn_fault for our buffers address as it handles a COW pagefault , but the times when the child sees 2 not 1 in its copy of the malloc buffer we dont see the COW pagefault and we see that both child and parent are sharing the same physical page of memory which isnt right after a fork().
So not knowing much about virtualbox i am a bit stumped on how to debug it further - why does the parent program not get a pagefault when it writes to the malloc buffer page given the PTE is read only?
thanks
tim