Ubuntu Guest Performance Better Than Win Host Native

[Thell]

Hello,

Let me start of by saying thank you, thank you, and thank you!

I've been a long long time user of VMWare Workstation (since Win2k) on both Win and *nix hosts and have dabbled with VB, VPC, and others on and off. Ironically about that same time period is when I stopped getting the latest and greatest hardware, so I've just accepted that performance wouldn't ever be 'my thing' when doing dev work. Recently I was lucky enough to upgrade to a core i7 with VT-x along with 24GB memory on a laptop. It didn't take long to rule out the idea of a linux host system being that the systems F keys aren't supported (and I like my backlit keyboard! ). But before truly ruling it out some benchmarking was needed...

I do CPU intensive tasks which sometimes beatup the memory too, so after running the one-size fits all benchmarks (nqueens and such) and seeing both VMware and VB taking a performance hit of about 50% off of a native Ubuntu LiveCD I was shocked (to say the least). It couldn't be! This 4th gen i7 with that much memory shouldn't incur a hit like that when both should give 'near native' performance.

So... I ran a real world benchmark of CPU/Memory intensive tasks of the kind I need written in R. The specific times won't mean much to anyone but you should know that in every test of any reasonable magnitude the VirtualBox machine beat the native host. That means single threaded, and parallel (that's four VM processors vs 4 w/hyperthreading - but since these are CPU bottle-necked the hyperthreads aren't of any value anyway)!

You guys have done a fantastic job!

* It would be nice to have multi-touch horizontal scrolling over a synaptics touchpad passed to the VM ( I looked and there is _no_ activity represented on the USB icon in the status bar or in xev, with any of the three pointer types ), but I'm still happy!

Thank you very much,
Thell


ps - If you have any interest here are the testing results the fifth column is time in seconds.

Code: Select all

Hardware - Toshiba P75-A7200 4700MQ core i7 w/ 24GB
Host Windows 8.1 core
LiveCD - Ubuntu 13.10
VB Guest - 4 processors (vtx, pae/nx), 12GB mem, storage pre-allocated, 128MB vid w/no acceleration
[ tested both ICH9 and piiX3, with video and headless ]

               R on LiveCD      FORK4     1000000     5    3.717    1.000     2.278    0.337      0.000     0.000
         RStudio within VM      FORK8     1000000     5    3.790    1.000     2.398    0.497      0.000     0.000
         RStudio within VM      FORK4     1000000     5    3.816    1.007     2.273    0.633      0.000     0.000
               R within VM      FORK4     1000000     5    3.980    1.000     2.341    0.390      0.000     0.000
               R on LiveCD      FORK8     1000000     5    4.040    1.087     2.229    0.397      0.000     0.000
   RStudio Svr Headless VM      FORK4     1000000     5    4.234    1.000     2.058    1.414      0.000     0.000
               R within VM      FORK8     1000000     5    4.399    1.105     2.390    0.412      0.000     0.000
   RStudio Svr Headless VM     LAPPLY     1000000     5    4.770    1.127     4.677    0.071      0.000     0.000
               R on LiveCD     LAPPLY     1000000     5    4.965    1.336     4.968    0.000      0.000     0.000
         RStudio within VM     LAPPLY     1000000     5    5.240    1.383     5.178    0.071      0.000     0.000
               R within VM     LAPPLY     1000000     5    5.252    1.320     5.249    0.001      0.000     0.000
   RStudio Svr Headless VM      FORK8     1000000     5    5.265    1.244     2.008    2.253      0.000     0.000
               R on Win8.1     LAPPLY     1000000     5    5.33     1.000     5.31     0.00         NA        NA 
               R on LiveCD     PSOCK4     1000000     5    6.980    1.878     2.410    0.185      0.007     0.082
         RStudio within VM     PSOCK4     1000000     5    7.581    2.000     2.444    0.397      0.013     0.040
               R on Win8.1     PSOCK4     1000000     5    8.92     1.674     2.50     0.11         NA        NA 
               R within VM     PSOCK4     1000000     5    7.514    1.888     2.415    0.341      0.006     0.059
   RStudio Svr Headless VM     PSOCK4     1000000     5    7.688    1.816     1.963    1.239      0.004     0.788
               R on LiveCD     PSOCK8     1000000     5    9.842    2.648     2.533    0.209      0.007     0.161
               R on Win8.1     PSOCK8     1000000     5   13.03     2.445     2.55     0.27         NA        NA 

               R on LiveCD      FORK4     3000000     5    9.688    1.000     6.023    0.726      0.000     0.000
               R on LiveCD      FORK8     3000000     5    9.874    1.019     5.949    0.809      0.000     0.000
               R within VM      FORK8     3000000     5    9.868    1.000     6.474    0.801      0         0.000
   RStudio Svr Headless VM      FORK8     3000000     5    9.966    1.000     6.499    1.024      0.000     0.000
         RStudio within VM      FORK8     3000000     5   10.236    1.000     6.555    1.011      0.000     0.000
               R within VM      FORK4     3000000     5   10.641    1.078     6.535    0.831      0         0.000
         RStudio within VM      FORK4     3000000     5   11.404    1.114     6.635    1.073      0.000     0.000
   RStudio Svr Headless VM      FORK4     3000000     5   11.751    1.179     6.887    0.914      0.000     0.000
               R on LiveCD     PSOCK4     3000000     5   14.739    1.521     6.464    0.360      0.003     0.115
               R on LiveCD     LAPPLY     3000000     5   16.144    1.666    16.105    0.045      0.000     0.000
               R within VM     PSOCK4     3000000     5   16.383    1.660     6.427    0.951      0         0.098
   RStudio Svr Headless VM     PSOCK4     3000000     5   16.489    1.655     6.420    1.091      0.011     0.087
               R on Win8.1     LAPPLY     3000000     5   16.60     1.000    16.51     0.08         NA        NA
               R within VM     LAPPLY     3000000     5   16.663    1.689    16.505    0.156      0         0.000
         RStudio within VM     PSOCK4     3000000     5   16.768    1.638     6.672    1.048      0.006     0.109
   RStudio Svr Headless VM     LAPPLY     3000000     5   17.088    1.715    16.824    0.278      0.000     0.000
               R on Win8.1     PSOCK4     3000000     5   17.19     1.036     6.59     0.50         NA        NA
         RStudio within VM     LAPPLY     3000000     5   17.375    1.697    17.244    0.140      0.000     0.000
               R on LiveCD     PSOCK8     3000000     5   17.514    1.808     6.472    0.526      0.008     0.204
               R on Win8.1     PSOCK8     3000000     5   21.52     1.296     6.60     0.55         NA        NA

               R on LiveCD      FORK8    20000000     5   81.966    1.000    38.880    5.839      0.000     0.000
             RStudio in VM      FORK4    20000000     5   83.493    1.000    43.147    5.665      0.000     0.000
             RStudio in VM      FORK8    20000000     5   87.665    1.050    42.245    6.185      0.000     0.000
               R on LiveCD      FORK4    20000000     5   88.109    1.075    38.745    5.719      0.000     0.000
               R on LiveCD     PSOCK4    20000000     5   96.501    1.177    41.711    5.107      0.003     0.427
               R on LiveCD     PSOCK8    20000000     5   97.765    1.193    41.878    5.344      0.000     0.821
             RStudio in VM     PSOCK4    20000000     5  101.783    1.219    42.132    6.454      0.007     0.481
               R on Win8.1     PSOCK4    20000000     5  110.54     1.000    44.90    10.62         NA        NA
             RStudio in VM     LAPPLY    20000000     5  113.668    1.361   113.229    0.533      0.000     0.000
               R on Win8.1     PSOCK8    20000000     5  114.14     1.033    46.01    11.72         NA        NA
               R on LiveCD     LAPPLY    20000000     5  159.745    1.949   156.492    3.301      0.000     0.000
               R on Win8.1     LAPPLY    20000000     5  181.72     1.644   173.17     8.52         NA        NA

               R on LiveCD      FORK8    30000000     5  122.798    1.000    58.790    9.178      0.000     0.000
               R on LiveCD      FORK4    30000000     5  127.231    1.036    58.254    9.277      0.000     0.000
               R on LiveCD     PSOCK4    30000000     5  139.749    1.138    61.324    6.731      0.004     0.528
             RStudio in VM      FORK4    30000000     5  147.699    1.000    63.744    9.223      0.000     0.000
             RStudio in VM      FORK8    30000000     5  151.262    1.024    63.226    9.169      0.000     0.000
             RStudio in VM     PSOCK4    30000000     5  158.602    1.074    62.665   10.233      0.023     0.626
               R on Win8.1     PSOCK4    30000000     5  171.19     1.000    66.72    18.09         NA        NA
               R on Win8.1     PSOCK8    30000000     5  173.95     1.016    69.19    18.85         NA        NA
               R on LiveCD     LAPPLY    30000000     5  241.446    1.966   238.165    3.390      0.000     0.000
             RStudio in VM     LAPPLY    30000000     5  251.008    1.699   249.835    1.209      0.000     0.000
               R on Win8.1     LAPPLY    30000000     5  273.39     1.597   260.58    12.75         NA        NA
  
Results from VM using ICH9 chipset instead of pii3x with host power set to Balanced instead of High Performance in R console...

                                FORK8    30000000     5  155.356    1.000    70.221   17.008      0.000    0.000
                                FORK4    30000000     5  156.960    1.010    76.344   18.133      0.000    0.000
                               PSOCK4    30000000     5  169.027    1.088    73.215   20.437      0.011    0.613
                               LAPPLY    30000000     5  253.239    1.630   250.339    2.780      0.000    0.000

Results from Ubuntu 14.04 VM using ICH9 chipset instead of pii3x with host power set to Balanced instead of High Performance in R console...

                                FORK4    30000000     5  199.665    1.000    73.432   33.156      0.000    0.000
                                FORK8    30000000     5  201.893    1.011    75.451   34.347      0.000    0.000
                               PSOCK4    30000000     5  219.379    1.099    79.698   30.190      0.017    0.679
                               LAPPLY    30000000     5  259.034    1.297   215.755   43.096      0.000    0.000

RStudio...

							   LFORK4    30000000      5 154.285    1.000    68.343   11.040      0.000    0.000
							   LFORK8    30000000      5 158.987    1.030    68.009   10.973      0.000    0.000
							  LPSOCK4    30000000      5 167.582    1.086    69.355    8.035      0.016    0.512
							   LAPPLY    30000000      5 254.430    1.649   250.710    3.882      0.000    0.000

On installed Ubuntu 13.10

                                FORK8    30000000     5  126.897    1.000    57.665    8.515      0.000    0.000
                                FORK4    30000000     5  127.708    1.006    56.956    7.882      0.000    0.000
                               PSOCK4    30000000     5  141.113    1.112    60.028    7.172      0.003    0.518
                               LAPPLY    30000000     5  231.241    1.822   227.688    3.630      0.000    0.000

[/size]

[mpack]

Tell me you didn't rely on virtual timers for these benchmarks...

[Thell]

Tell me you didn't rely on virtual timers for these benchmarks...

Well, no, and yes, and no...

no:
Initial testing was done by 'feel' that's when it became totally obvious that a VM with anything less than 'Balanced Power' on my laptop was totally unusable with a GUI... After bumping to 'High Performance' the VM completed the compiles and unit tests of a few of my projects ( remember I'm building up a new system here, which, as you know, means a lot of rework ) in a noticeable difference of time by the clock (like shaving off 5 minutes from a validation run). I didn't figure that could even be close to right, but I checked and the results where identical. Since this is a new setup I was really focusing on the VM vs LiveCD to see if the loss of the laptop functionality that's built-in and working for Windows would be worth a dual-boot setup. After seeing how fast my base compiles and unit tests completed I decided to do the benchmarks.

yes:
The results posted are output from rbenchmark which essentially wraps system.time and until I read your reply I really didn't consider that the VM could possibly vary so much as to give a 20 second difference in a single test.

no:
After reading a bit about some of the controversy of benchmarking on Windows 8 today, it seemed that answer would be to just hook up the video camera, put a stop watch next to the screen and run the tests within the VM and on the Win8.1 host, then play it back in VLC and slow it down to get the times. Guess what... The VM is still faster.

By the stopwatch on just the single threaded command ( lapply in the listing ) @ 30000000 result on the VM 2:39 compared to the Host @ 2:54.

That is default install of Windows without a bunch of tools added and just the cruft from Toshiba/Norton/etc... removed compared to the VM of the default of Ubuntu with only a few shell tweaks. It could all come down to the R being compiled differently for Windows, or in how the memory is managed, or whatever...

The point is I didn't do anything special to the setup in either place and just by using the VM instead of the Host I'll get my results faster and still get to use the working functionality of the latop. I sincerely appreciate that.