VMWare server 1.0.6 37% faster than Virtual Box 1.6.2

Discussions related to using VirtualBox on Linux hosts.
bifferos
Posts: 19
Joined: 16. May 2008, 16:29

VMWare server 1.0.6 37% faster than Virtual Box 1.6.2

Post by bifferos »

I've converted a Windows XP Virtual Box image to vmware by installing the VMWare Converter software (http://www.vmware.com/download/converter/) to export to VMDKs (I found this to be easier than messing about with Windows IDE drivers, although it took 2 hours...).

I've found build times for a series of Visual Studio 8.0 projects to be about 37% faster under VMWare Server (1.0.6). I believe that this could be in part due to the fact that the converter exports to SCSI disks and my VB image uses IDE, but I don't see where 37% comes from - build time is reduced from 3.5 hours to just over 2 hours, so this is a *big* difference.

I don't think Virtual Box can be complete sh*t, since interactively it does *seem* to be faster, and indeed, when I tried running two guests simultaneously Virtual Box wins hands-down. On the same hardware VMWare Server was virtually unusable like this.

So I'm looking for suggestions about how to speed the V.B. system up, or why there should be such a difference. In both cases I'm using expanding disks, with the build happening on a slave disk. I've verified on a second machine and came up with similar results. It does seem like build times suck under Virtual Box :-(.

thanks for any help.
biff.
sandervl
Volunteer
Posts: 1064
Joined: 10. May 2007, 10:27
Primary OS: MS Windows Vista
VBox Version: PUEL
Guest OSses: Windows, Linux, Solaris

Post by sandervl »

Try using SATA instead of IDE. It's not so easy with an XP guest though. Use the search function to find some useful descriptions on this forum.
TerryE
Volunteer
Posts: 3572
Joined: 28. May 2008, 08:40
Primary OS: Ubuntu other
VBox Version: PUEL
Guest OSses: Ubuntu 10.04 & 11.10, both Svr&Wstn, Debian, CentOS
Contact:

Post by TerryE »

I've done a few posts related to this. The VMware tools include a paravirtualisation driver for disk I/O, and VBox doesn't so if you are hammering the disk (as Visual Studio does) then VBox is slower then VMware server. On the other hand the Vbox GA drivers for mouse and display are a lot learner and faster then VMware, so the VBox GUI experience if more satisfying.

Try using clonezilla to copy the VMDKs to VDIs and see if that speeds things up.
Read the Forum Posting Guide
Google your Q site:VirtualBox.org or search for the answer before posting.
bifferos
Posts: 19
Joined: 16. May 2008, 16:29

Post by bifferos »

Thanks for the suggestions. There is a topic covering this:
http://forums.virtualbox.org/viewtopic.php?t=7177

But it is little help. I've downloaded the intel SATA drivers, but I get "delayed write failed" messages, with the drive eventually going offline. I haven't dared to convert the boot disk to SATA yet.

Note that these messages only come up from time to time, with plenty of disk activity in between, however they are sufficient to make my build fail.
bifferos
Posts: 19
Joined: 16. May 2008, 16:29

Post by bifferos »

Just for info, I raised a ticked and this has now been fixed.

http://www.virtualbox.org/ticket/1873

This may bring some relief to others suffering the dreaded "delayed write failed" messages with SATA under Windows XP. It seems the problem only affects single CPU machines, hence many people not seeing it on the forum.

Looking forward to the next release - VB developers rock :-).
bifferos
Posts: 19
Joined: 16. May 2008, 16:29

Post by bifferos »

VirtualBox 1.6.4 Fixes the issues I was having with SATA, however for me, using SATA makes no difference whatsoever to the speed of any build (Yes, I have disabled the extra logging with the 'nofile' env. variable). This is with write-caching turned on on both disks. It seems that (for the single CPU case, at least) SATA provides no performance improvement whatsoever.

Can someone tell me what I might be doing wrong? SATA operating in IDE compatibility mode? How would I know?

thanks
Biff.
TerryE
Volunteer
Posts: 3572
Joined: 28. May 2008, 08:40
Primary OS: Ubuntu other
VBox Version: PUEL
Guest OSses: Ubuntu 10.04 & 11.10, both Svr&Wstn, Debian, CentOS
Contact:

Post by TerryE »

I haven't got my head fully around this yet, but from the various quick and dirty Benchmarks that we've done around the forum, the slowdown of Linux hosts is a lot worse that that on the NT OSs. It seems to be about a factor of 2 in disk throughput guest performance compared to host, and it doesn't really matter whether you are using static or dynamic VDIs or even raw partitions.

I suspect that some subtle serialisation is occurring in the VMM so that we are loosing our DMA concurrency, but I haven't worked out what yet.. To do this I'd really need to compile the OSE and add some real-time BITE. The trouble with the current diagnostics is that it logs to the FS rather than tracing to an in RAM wrap-around buffer, so it is really only useful for functional testing and not performance testing. This would end up as 10-day project and I don't have the time to spare at the moment. Sorry.
Read the Forum Posting Guide
Google your Q site:VirtualBox.org or search for the answer before posting.
bifferos
Posts: 19
Joined: 16. May 2008, 16:29

Post by bifferos »

I think I will keep re-posting my benchmarks here for new editions of VirtualBox (until someone complains that is :-)).

I have finally managed to get VB 2.0.2 working for long enough to re-run the compilation test, and on the first run, it is taking approximately 12% longer than 1.6.2 did.

This would imply VB 2.0.2 is now 42% slower than VMWare 1.0.6 for this task, when using SATA disks.

I wanted to repeat the experiment, however I've been unable to keep 2.0.2 up long enough to do this, as the VM session keeps getting aborted at random times through the compilation for some reason.
bifferos
Posts: 19
Joined: 16. May 2008, 16:29

Just tested 2.0.6

Post by bifferos »

The task, machine spec and host OS all remain unchanged over the previous tests in this thread (naturally...).

With all disks as SATA I am now seeing almost 4% speed improvement over VB 1.6.2, so it seems there really were some significant SATA performance improvements put in 2.0.6, as detailed in the changelog. It is impossible for me to tell whether this speed change is due to SATA or other improvements - I am simply running a 3 hour clean compilation job and timing it.

Unfortunately, VMWare 1.0.7 still far outperforms VB, being approximately 34% faster at running the test, but at least there is now finally a reason to upgrade from 1.6.2 -> 2.0.6 if you are paranoid about losing performance. The penalty is that you'll have to stop using IDE disks. I haven't re-tested IDE disks with 2.0.6 but I imagine the performance will be inferior to 1.6.2.
RGummi
Posts: 7
Joined: 13. Mar 2008, 20:45

Post by RGummi »

You could use a virtual disk (write through!) for sourcecode and compiler generated files. I do this and performance is ok.
TerryE
Volunteer
Posts: 3572
Joined: 28. May 2008, 08:40
Primary OS: Ubuntu other
VBox Version: PUEL
Guest OSses: Ubuntu 10.04 & 11.10, both Svr&Wstn, Debian, CentOS
Contact:

Post by TerryE »

Yup, but I am not sure how this impacts bifferos's original point. I believe that he is already comparing VMware with its native VMDK virtual drives to VirtualBox with its native VDI virtual drives.

I don't believe that write-through materially changes I/O performance, or are you claiming it does?
Read the Forum Posting Guide
Google your Q site:VirtualBox.org or search for the answer before posting.
bifferos
Posts: 19
Joined: 16. May 2008, 16:29

Post by bifferos »

RGummi wrote:You could use a virtual disk (write through!) for sourcecode and compiler generated files. I do this and performance is ok.
The problem is, I am not looking for 'ok' performance!

For continuous integration, and any other automated task, the speed of execution of the task becomes critical. The time taken to arrive at the conclusion is critical, and it doesn't matter how fast the GUI is, or that VB allows multiple VMs to execute concurrently without slowing your machine to a crawl. VMWare Server allows one to reach the conclusion much faster than VB, so VB is currently unsuitable for this task.

Anyhow, I have just re-tested with write-through and the difference is "in the noise", so that ain't going to get me back my 30%.

regards,
biff.
TerryE
Volunteer
Posts: 3572
Joined: 28. May 2008, 08:40
Primary OS: Ubuntu other
VBox Version: PUEL
Guest OSses: Ubuntu 10.04 & 11.10, both Svr&Wstn, Debian, CentOS
Contact:

Post by TerryE »

Biff, I sounds like you and I are very much of a mind when it comes to comparisons between VMware and VirtualBox when it comes to I/O performance. (Note that this is strangely only the case for *nix hosts hosts as VB is faster than VMware on Windows.)

Where we differ is on the overall conclusion:
  • You feel that this is a killer when is comes to comparing the two products.
  • I on the other hand regard it as a shame, and I hope that the dev team could address it sometime. I/O performance, or the lack thereof, is really only a "killer" in limited cases. Yes VMware can outperform VBox on Linux on I/O intensive tasks such as bulk compile, but overall all in terms of GUI integration it is a lot better. VMware server has a crude GUI integration; VMware player is a pain to use for developers and is slow; In these domains VBox's true competition is VMware Workstation and this is expensive, not free. Very rarely does this drop in I/O performance become significant for me. OK recompiling OOo is an overnight job, but its an overnight job either way.
Read the Forum Posting Guide
Google your Q site:VirtualBox.org or search for the answer before posting.
bifferos
Posts: 19
Joined: 16. May 2008, 16:29

Post by bifferos »

TerryE wrote:You feel that this is a killer when is comes to comparing the two products.
I admit there is a somewhat gloomy tone to my posts, but I actually never said that!

I actually use VB every day for development, so it's far from a killer. What it *does* do is prevent me from evangelising it for wider adoption though.

regards,
biff.
bifferos
Posts: 19
Joined: 16. May 2008, 16:29

Post by bifferos »

TerryE wrote:(Note that this is strangely only the case for *nix hosts hosts as VB is faster than VMware on Windows.)
This has prompted me to do another test. I re-tried with VB 2.0.6 on Intel OpenSolaris. The result was a whopping 25% slower than Linux, although to be fair, I had problems to figure out how to add my second IDE disk to the ZFS pool, so both OS and virtual images were on the same physical disk for this test. The host disks were of comparable speeds (both IDE).

Judging by the fact that OpenSolaris 2008.11 seemed to respond like molasses on my P4 with 1GB ram, I'd say the extra disk probably wouldn't make much difference.

I will have a go at Windows XP host with VB 2.0.6 next to try to put some real figures to your statement above.
Post Reply