Allocating over 4gb of RAM for a vm guest

[rpmurray]

The bitness of the VirtualBox app follows the kernel bitness, on OS X and all other platforms. Linux, Windows, and Solaris all had 64-bit kernels in 2004-2005, with OS X it took until 2009. And the algorithm to decide which systems will run the 64-bit kernel and which won't is, shall we say, interesting.

Yes, and I was making the point that on his system with a 32-bit kernel that VirtualBox does not run as a 64-bit app. So while I have a ton of apps running as 64-bit while I'm booted with the 32-bit kernel on my Mac Pro, VirtualBox is not one of them.

[michaln]

So while I have a ton of apps running as 64-bit while I'm booted with the 32-bit kernel on my Mac Pro, VirtualBox is not one of them.

Correct. And as I explained, the reason for that is that there's a significant chunk of VirtualBox which runs in kernel mode and is thus restricted to 32-bit on OS X (with 32-bit kernel). We can only wish Apple hadn't made things so interesting.

[hinto]

I'd chalk it up as a bug. But I don't think I'll get much traction. Most modern virtualization engines at the minimum support 64 bit guest with more than 4 gigs of ram, as long as the 32 bit PAE kernel is available on the host. OSX is a slightly different beast than say XP (with PAE) or Linux (with PAE), but in both the Windows and Linux platforms, I can get the configuration I need. I think that Virtual Box's algorithm needs to be tweaked.
-H

[michaln]

I think that Virtual Box's algorithm needs to be tweaked.

In case that wasn't clear - that's not a business case either.

[hinto]

I didn't think it would get much traction.
There are other options that I can use.
Thanks for the info.
-H

[michaln]

You're welcome - I hope it's now clearer what's going on. One of the big reasons why extending the support for 32-bit OS X kernels is extremely unappealing is that they're going away, so it would be a project with very rapidly diminishing returns.

For what it's worth, if you're serious about virtualization, get something better than a Core 2. I have extensively used both Core 2 Duo/Quad class CPUs and newer Nehalem and Sandy Bridge processors. The VT-x implementation in Core 2 is... not very good. It's almost hard to believe how much better the newer CPUs are for virtualization.

[hinto]

I'll make the business justification;)
Currently, what I'll do is keep my vm guest on Linux and rdesktop to it, rather than hosting it on the Mac.
-Hinto

[waldorfm]

So while I have a ton of apps running as 64-bit while I'm booted with the 32-bit kernel on my Mac Pro, VirtualBox is not one of them.

Correct. And as I explained, the reason for that is that there's a significant chunk of VirtualBox which runs in kernel mode and is thus restricted to 32-bit on OS X (with 32-bit kernel). We can only wish Apple hadn't made things so interesting.

I hope your are not insinuating that Apple is behind technology, including some of your previous replies. Apple provided the first consumer grade desktop system worth mentioning using a 64-bit RISC CPU, before switching to Intel. Unfortunately, the x86_64 CPU concept allowed MS Windows and other 3rd party software vendors to continue with their CISC software junk. It was the Intel/PC world that was years behind technology and required strange solutions like PAE kernel just to address more than 3.5 GB of RAM. Now that everyone's is finally calling for native 64-bit, the x86_64 CPU design is actually obsolete and the whole story somewhat pathetic. It seems the IA-64 is pretty much dead in terms of software and OS support, so I guess Intel can come with yet another "new" CPU design. For what it's worth, Apple OSX can be switched to 64-bit kernel or system mode and to my experience there are no compatibility issues with any Apple provided software. Unfortunately the same is not true for 3rd party kernel drivers.

Btw, why do I have to switch the kernel to 64-bit mode to address more than 3.5 GB of RAM in VirtualBox? The Mac OSX kernel and other apps have apparently no trouble to address more 4 GB of RAM even in "32-bit" mode, similar to Windows and Linux using the PAE "36-bit" kernel. Intel versions of Mac OS X support PAE. Sorry, I think the Mac OSX 32-bit kernel is not a valid argument to justify the VirtualBox RAM limits. The Mac OSX kernel might be a strange beast for Oracle or Sun VirtualBox, which falls out of the general "PC" norms, but then the solution lies upon Oracle platform support.

[michaln]

I hope your are not insinuating that Apple is behind technology, including some of your previous replies.

I'm not insinuating anything. It's easy to find out when Linux, Windows, Solaris, and OS X added support for the AMD64 architecture. The fact that Apple had 64-bit PowerPC support long before 64-bit Intel support is not relevant when discussing VirtualBox.

Apple provided the first consumer grade desktop system worth mentioning using a 64-bit RISC CPU, before switching to Intel.

I hoped that by now everyone would have realized that the whole RISC vs CISC nonsense is utterly meaningless...

Btw, why do I have to switch the kernel to 64-bit mode to address more than 3.5 GB of RAM in VirtualBox?

Because we don't see why we should be writing extra code that is becoming more and more obsolete as we speak. Apple has made it clear that the hybrid 32-bit kernel + 64-bit userland scenario is not going to last for much longer, and it is already on the way out.

The Mac OSX kernel and other apps have apparently no trouble to address more 4 GB of RAM even in "32-bit" mode, similar to Windows and Linux using the PAE "36-bit" kernel.

You don't appear to appreciate the difference between using more than 4GB in a system through technologies such as PAE and addressing more than 4GB within a single process, which is quite difficult with 32-bit pointers. Since the VirtualBox kexts must be 32-bit when 32-bit kernel is used, the 32-bit limitation poses significant difficulties.

It's not technically impossible to support > 4GB VMs on the 32-bit OS X kernel, but so far no one has provided any argument for why we should do that (and no, "because I want it" doesn't count). Is it really more important than, say, supporting Mountain Lion?

[hinto]

From what I've read, you can hack the 32-bit EFI for the MacPro1,1 by changing it to the 64-bit EFI, and thereby load the 64-bit kernel. It may be an interesting exercise, but for me, it's just too easy to remote desktop to a Win7 VM on a non-OSX host.
-Hinto

[waldorfm]

It's not technically impossible to support > 4GB VMs on the 32-bit OS X kernel, but so far no one has provided any argument for why we should do that (and no, "because I want it" doesn't count). Is it really more important than, say, supporting Mountain Lion?

You are right, some was not necessarily relevant to VirtualBox, but I think neither was your statement "We can only wish Apple hadn't made things so interesting" and other previous replies, which I find provoking. I guess the bottom line is that you may get applause for criticizing Microsoft, which users typically swear at the system, but not Apple or Oracle, which users typically swear by their computers and software. Well, for what it's worth, I think users often get and use what they deserve

I run OSX in 64-bit mode and it seems fine, apart from VirtualBox crashing the machine from time to time, but that's a different story and apparently, according to other users experiencing the same issue, something that started with VirtualBox 4.1. The reason why I need more than 4 GB of RAM in a guest OS is because I'm running and testing Oracle products like Oracle Linux, 11gR2, etc. Unfortunately, Oracle pulled the plug on Mac OSX after 10g.

Are you providing support for more than 4 GB RAM in VirtualBox under Windows PAE? But that's more like a rhetoric question because I have absolutely no intention whatsoever to run Windows on any of my computers at home, never did, unless for fooling around or running some crappy stuff.

[michaln]

You are right, some was not necessarily relevant to VirtualBox, but I think neither was your statement "We can only wish Apple hadn't made things so interesting" and other previous replies, which I find provoking.

I'm not sure why it's provocative. I use Macs every day and I'm quite happy with them (I've been using Intel Macs since 2006), but that doesn't mean I'm going to pretend that running 64-bit apps on top of a 32-bit kernel is somehow state of the art. It's not, and I'm glad Apple is pushing all new systems (and many existing ones) towards 64-bit kernels.

Are you providing support for more than 4 GB RAM in VirtualBox under Windows PAE?

No. Again, it's technically doable but not worth the effort, in part because Windows server OSes no longer exist in 32-bit versions.

[waldorfm]

You are right, some was not necessarily relevant to VirtualBox, but I think neither was your statement "We can only wish Apple hadn't made things so interesting" and other previous replies, which I find provoking.

I'm not sure why it's provocative. I use Macs every day and I'm quite happy with them (I've been using Intel Macs since 2006), but that doesn't mean I'm going to pretend that running 64-bit apps on top of a 32-bit kernel is somehow state of the art. It's not, and I'm glad Apple is pushing all new systems (and many existing ones) towards 64-bit kernels.

Are you providing support for more than 4 GB RAM in VirtualBox under Windows PAE?

No. Again, it's technically doable but not worth the effort, in part because Windows server OSes no longer exist in 32-bit versions.

Ok, I understand that providing PAE support for 32-bit kernels is no longer a killer feature. And I wonder when the 32-bit CISC (x86) stuff inside the x86_64 CPU is going to be obsolete too.

[hinto]

^Apparently it already is
-H

[michaln]

For what it's worth, it appears that Mountain Lion is going to remove the 32-bit kernel (the same thing Solaris did last year) and 64-bit will be the only option. The kernel is thus going to be a single-architecture binary, just like in the old days.