virtualbox.org

I did a performance benchmark between VBox and VMWare server.
http://ezjd.blogspot.com/2009/08/virtua ... -part.html

It turns out that VBox and VMWare sever have very similar performance except for
-- Apache test
-- Compiling
-- Threaded IO

I noticed that the first 2 test actually don't involve much IO operations.

But I don't understand why they have very bad performance at compiling code.

Anyone has some ideas to improve?

Whatever the test says, I have always found windows guests more responsive in VB than in vmware.
But maybe with linux guests is different.

And both suck pretty much the same when assigning more than one core

I assume that you had VBox Guest Additions and VMware tools installed in the respective VMs. VB GA focuses on video integration and acceleration. VMwareTools has disk and network acceleration. This would explain the different strengths re compilation and apache performance.

No I didn't install addon/tools. But It is good to know their difference
What concerned me most is building performance as I have to use VM to do embedded dev.

Have a look at the second VMware Paper referenced on this General Reading topic.

Thank you for great report!

At this picture - http://4.bp.blogspot.com/_kXtvRBukDlc/S ... 00-h/9.png I see that VMWare even faster than native host. This is nonsense, of course and could be in case when time measurement is based on guest clock. Guest clock could go slower under heavy load, so test measurement can lie - show faster results. I observed this in Win guest under VBox/Linux host.

Probably other results also suffer from this issue. Do not trust them.

TerryE wrote:Have a look at the second VMware Paper referenced on this General Reading topic.

Thanks. I understand the overhead added with IO. That probably is the direction of HW virtualization improvement.

Regarding compiling, it isn't all about IO. Actually, I did notice that there isn't many disk operations during the test like IO testing.
And I found if dual CPU enabled, compiling performance got improved when IO performance dropped a lot. So I would think compiling test is more CPU benchmarking.

I will post the dual-CPU result tomorrow.

vuser1 wrote:Thank you for great report!

At this picture - http://4.bp.blogspot.com/_kXtvRBukDlc/S ... 00-h/9.png I see that VMWare even faster than native host. This is nonsense, of course and could be in case when time measurement is based on guest clock. Guest clock could go slower under heavy load, so test measurement can lie - show faster results. I observed this in Win guest under VBox/Linux host.

Probably other results also suffer from this issue. Do not trust them.

Yes, I found that the result can't be used for more detailed and accurate analysis too for many reasons like this. I really wanted to get an idea of how VBox guest runs and I did get some. The original reason behind this is to evaluate VBox guest as development platform, and I am disappointed

Unfortunately, I am unable to get reliable KVM guest running and it has similar problem, for example, apache performance of KVM guest is better than native.

Out of interest, when you talk of a "development platform" this can mean many different things. For example the use case profiles tend to be very different for an MS Visual Studio based environment vs. a Windows Eclipse vs. a Linux Eclipse vs. a more "roll-your-own scenario". Are you benchmarking mega makes or what? I used to use a VM for doing OOo builds (roughly 2M lines of source). As long as you had ccache loaded this was fine for incrementals, but the clean rebuild was an overnight job. Also how you access your sources is important -- for example using host filesharing seems to suck performance-wise.

If you are using a GUI intensive dev life cycle with decent incremental caching then the Video integration of VBox is actually more important than the raw sped.

As VMWare have always stated, you can't really benchmark a virtual system, especially if you're using the guest clock, and yes sometimes you end up with stupid results like the guest being faster than the host!

I'd say KVM should be the fastest of the three as its more of a hypervisor than the other two.

VirtualBox sucks for compilation, mainly due to the speed of disk access it seems, and multiple cores doesn't help; this has been discussed before. I've tried Visual Studio and gcc and they are horrendously slow at compiling things like XBMC or the Linux kernel.

I didn't do much GUI development, but I agree that GUI dev has more than compiling. What I did a lot is just gcc/make.

Per this build-apache test, as I said before, no much disk/IO operations involved, so I still think something isn't good enough with CPU virtualization or something I don't know yet. I hate the feeling that building a bunch of software for a couple of hours in native machine while it took a few times more in guest OS (in that more real world building case, I believe bad IO performance plays a big role too).

I have seen this (very slow compiles) with vmware-server and virtual box. For a long time I was anticipating the arrival of SMP with virtualbox so I could ditch the slow vmware server compiles however now that I have tested both vmware server appears to be a little faster at compiles with both of which taking > 2 times the amount of time it would take on a native machine with the same hardware.

sej7278 wrote:As VMWare have always stated, you can't really benchmark a virtual system, especially if you're using the guest clock, and yes sometimes you end up with stupid results like the guest being faster than the host!

Right, that's the common mistake people make.

sej7278 wrote:I'd say KVM should be the fastest of the three as its more of a hypervisor than the other two.

KVM is no different from VMWare or VirtualBox. Confused with Xen perhaps?

sej7278 wrote:VirtualBox sucks for compilation, mainly due to the speed of disk access it seems, and multiple cores doesn't help; this has been discussed before. I've tried Visual Studio and gcc and they are horrendously slow at compiling things like XBMC or the Linux kernel.

The bottleneck lies elsewhere. Nested paging shows close to host performance in uni and smp VMs. (see the benchmark post)

sandervl wrote:The bottleneck lies elsewhere. Nested paging shows close to host performance in uni and smp VMs. (see the benchmark post)

Is this a hardware limit/problem? bad(or missing) vt-x implementation on cpu level? the Host hogging paging too much?

vbox4me2 wrote:

sandervl wrote:The bottleneck lies elsewhere. Nested paging shows close to host performance in uni and smp VMs. (see the benchmark post)

Is this a hardware limit/problem? bad(or missing) vt-x implementation on cpu level? the Host hogging paging too much?

No, it's a guest paging issue where our code is too inefficient. Already managed to get a 10% improvement, but that's still not enough.