Status: VirtualBox Wide-Scale Stability Test Matrix

[Technologov]

I decided once in a while to make a serious Wide-Scale stability and OS coverage testing of VirtualBox to discover rough edges.

Host: Windows XP, 32-bit, Core 2 Q6600, VBox 3.1.0, VT-x=on.

Most serious problems uncovered:
1. Fedora8/X64 FAILS to install Bug #5410
FIXED in 3.2.0
2. openSUSE 10.3/X64 FAILS to install Bug #5457
FIXED in 3.2.0
3. openSUSE Guest Additions seriously broken in 3.1.0. Told to Michael.
FIXED in 3.2.0
4. Fedora 7 and 9 have very serious bugs. Not sure what we can do about it. Told to Michael.
Those are Fedora bugs; nothing can be done on VBox part.
5. RHEL 3/4 needs more love. Told to Michael.
FIXED in 3.2.0
6. Windows GA: Many executables refuse to run fom Shared Folders Bug #5732
Not fixed.
7. Multiple VMs with Sound: Running Sound from multiple VMs and switching between VMs fails sound on them all Bug #5733
8. OpenGL on Fedora 12/x64 segfaults (after deactivating SELinux). Told to Michael.
This problem gone, but some other problems still remain with OpenGL.
9. Fedora 10/11/12 cannot install successfully from DVD, unless user manually ejects DVD. Bug #2680
Fedora bug. Maybe VBox can workaround it by changing default boot order to HDD->CD/DVD
10. old SUSE Linux refuse to start from CD-ROM if VT-x enabled Bug #5407
Not fixed.

update: Forums disallows to post any Excel or ODS files. Added ZIP file. Cleared non-topic answers.

-Technologov

[rasker]

Technologov: Great work, very comprehensive. I am wondering if the table can be recreated in the virtualbox wiki using restructured text? This might be a good place for people to collaborate on updating it as things change. I couldn't find any software to do it automatically (although twiki has a plugin to import excel spreadsheets). I don't really know enough about restructured text (I know it can do tables) but it must be possible, no?

cheers
R

[Technologov]

The wiki idea is good one and may work... but only if people that update the table put a version in the field, like v3.1.2.

The biggest problem is that I don't see how can we edit the wiki.

[rasker]

Yeah you seem to be right! It is a trac wiki as far as I can tell and I have a login on the virtualbox site but I don't think it actually allows me to do anything. Going from the howtos page in the user docs it seems that one has to email the user mailing list to get something added to the trac wiki documentation pages.

Another option I guess is to use one of the free wiki sites to create a wiki specifically about compatibility of the different versions. Maybe wikispaces or perhaps even better wikia which is mediawiki based?

[Technologov]

Yes... 3rd party wiki may be needed. If you want .. go for it ! ...or maybe Google has some web base collaborative spreadsheets for us ?

[AntiMatter]

Hi Techno,

That was a huge job you have done there. Thanks very much for carrying out this extensive test grid and for sharing the documentation.

[_litz]

Please add to your grid (for a Centos5 host):

1) SMP unusable on RH EL4/Centos 4 guests (os detects 2nd CPU as having 0 'bogomips') due to speed issues -- runs very slow
2) RH EL4/Centos 4 very slow in general even using 1 cpu. CPU/hdd intensive operations like a DB query to a LOCAL database (not even networked) run about 5% normal speed (yes FIVE percent) with guest OS cpu usage maxed @ 100%.
3) SATA is completely nonfunctional for a RH EL4/Centos 4 guest.

(I have had bugs opened for the above for weeks, so far they have been completely ignored)

- _litz

[Technologov]

Test Results Updated.

This tests VirtualBox 3.1.2 vs. VMware 7.0.0

[angussf]

Great job . I'd *_really*_ like to see the reverse tests as well -- Linux hosts of various types (Ubuntu, Fedora, CentOS, openSUSE) hosting for various Windows and *nix guests just to see what makes the best platform on which to run VirtualBox .

[Technologov]

Installing a lot of different _host OSes_ is too time consuming, because I can't parallelize this job on my PC.

If I had a lab with 4 identical PCs, I could make it happen.

IMO - the best platform to run on, is the one *you* know best. This will save much more time than actual performance differences.
(example: on DOS computer with 486 CPU *I work* much faster than on an i-phone that has more powerful CPU)

[Technologov]

Added some benchmark tests: (Disk I/O)
viewtopic.php?f=3&t=31465
Network I/O:
viewtopic.php?f=15&t=30660

[abcuser]

I hope some of Oracle employee see this matrix, you should get the job at Oracle in testing lab!!!

[Technologov]

crash0veride wrote very interesting tests here:
viewtopic.php?f=7&t=31770