VirtualBox vs Hyper-V AMD/Intel big performance difference

Discussions about using Windows guests in VirtualBox.

VirtualBox vs Hyper-V AMD/Intel big performance difference

Postby Petr Vones » 26. Jul 2021, 10:21

I use Windows 10 x64 host and guest (MSDN) for development (Visual Studio, MS SQL Server and so on). I am aware that Hyper-V is a different kind of software, but this my finding are strange anyway. I converted the VirtualBox VDI to VHD and tried in Hyper-V on several machines. I used max. recommended settings of CPU and RAM in VirtualBox guests and then the same settings in Hyper-V. Performace tests were made by running unit tests (mostly CPU bound rather than I/O) in Visual Studio that takes several minutes. Laptops were always AC powered to get maximum performance. No software was running on hosts, no 3rd party antivirus crap, Windows Defender disabled in guests. Hyper-V was uninstalled and disabled for VirtualBox tests, of course.

AMD Ryzen 5 3600, 32 GB RAM, Samsung SSD 980 Pro - no significant performance difference between VirtualBox and Hyper-V, approximately 5% slower in VirtualBox as expected
Intel Core i5-8265U, 8 GB RAM, WDC SN720 SSD - double performance in Hyper-V compared to VirtualBox
Intel Core i5 2540M, 8 GB RAM, Intel 535 SSD - double performance in Hyper-V compared to VirtualBox

Is there any explanation why there is no significant performance difference on AMD PC, but it runs two times faster in Hyper-V on both Intel laptops? One thing i noticed the CPU clock speed reported in Task Manager in VirtualBox guests is lower than in host and Hyper-V guest.
Petr Vones
 
Posts: 87
Joined: 27. Dec 2012, 01:20
Location: Czech Republic
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows 10 64-bit

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Postby fth0 » 26. Jul 2021, 16:38

Petr Vones wrote:max. recommended settings of CPU

There is no such thing. The red color of the CPU slider shows what is "not recommended", and the green color of the CPU slider shows what is "not not recommended", which is not the same as being recommended. For most use cases, providing two vCPUs is recommended.

Regarding the performance, can you provide some values? Can you also provide a VBox.log file from a VM run on the i5-8265U? (I chose that because I have the same CPU.)



On a general note:

Performance measurement is a highly complex topic by itself (*), and analyzing virtual machine performance has even greater complexity. From my own experience, a thorough investigation could quickly take days or weeks. How devoted are you to investigate the reason for the performance difference?

(*) Have you ever heard the term Active Benchmarking?
fth0
Volunteer
 
Posts: 2400
Joined: 14. Feb 2019, 03:06
Location: Germany
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Postby Petr Vones » 26. Jul 2021, 18:26

Speaking of the i5-8265U machine, I assigned 5 GB RAM and tried 2 and 4 CPU (in both VirtualBox and Hyper-V). The performance is measured by typical usage scenarios: start Visual Studio, open solution, build it, run unit tests etc. where I can see at the first glance the Hyper-V is faster in overall. To get some numbers I run a project with ~70 unit tests that takes about 5 minutes to complete and provides durations. I run it multiple times to get a better picture of the values and the Hyper-V was two times faster than the VirtualBox (except of the AMD).
Petr Vones
 
Posts: 87
Joined: 27. Dec 2012, 01:20
Location: Czech Republic
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows 10 64-bit

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Postby scottgus1 » 26. Jul 2021, 18:36

A possible suggestion from someone who doesn't understand the nuances: The physical CPU shows through to the VM OS; in fact it is the only physical hardware that the VM OS sees.

Might an OS be tuned to an Intel or AMD CPU during install, and therefore need re-installation if the processor manufacturer is changed?

To put it another way, if you make fresh VMs on each host, one for Hyper-V and one for Virtualbox, a total of 6 fresh VMs, and install the same test environment in each, does the same timing come out?
scottgus1
Site Moderator
 
Posts: 13057
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Postby Petr Vones » 26. Jul 2021, 18:53

scottgus1 wrote:Might an OS be tuned to an Intel or AMD CPU during install, and therefore need re-installation if the processor manufacturer is changed?

This particular virtual machine was originally created in VirtualBox on the Intel i5-8265U CPU, later moved to the AMD. So it actually shows the exact opposite issue :) It runs faster on the "non-original" AMD than the Intel. Some manufaturers also provides ready-to-made virtual machines that are CPU vendor agnostic so I don't think it is a valid scenario.

scottgus1 wrote:To put it another way, if you make fresh VMs on each host, one for Hyper-V and one for Virtualbox, a total of 6 fresh VMs, and install the same test environment in each, does the same timing come out?

I don't have a free week for this, sorry. To install and setup the complete environment takes two days, that's why I am saving my time by moving the virtual machines between physical computers. And again, the same virtual machine works faster in Hyper-V on those two laptops. Would running CPU-Z test on the Intel machines in both VirtualBox and Hyper-V make sense?

One thing that may explain why there is no significant performance loss on the AMD PC is that the 980 Pro SSD is superfast, 10 times faster than the SSD on the laptop. But the operations aren't too much I/O bound, small files read/write only. And I tried both Host I/O cache options, no significant difference.

I could easily move to Hyper-V and forget, but I still like the VirtualBox user interface more and the networking is more flexible. I wouldn't care if there was a 10 or 20 percent performance loss using the VirtualBox over the Hyper-V, but 50 percent is too much and weird.
Last edited by Petr Vones on 26. Jul 2021, 19:07, edited 1 time in total.
Petr Vones
 
Posts: 87
Joined: 27. Dec 2012, 01:20
Location: Czech Republic
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows 10 64-bit

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Postby fth0 » 26. Jul 2021, 19:42

Thanks for describing your performance measurement method in some detail.

Petr Vones wrote:Would running CPU-Z test on the Intel machines in both VirtualBox and Hyper-V make sense?

It would enable me to do similar tests on my Lenovo T590 (Intel Core i5-8265U) for comparison, which is a dual-boot system (Windows 10 21H1, Linux Mint 20) with VirtualBox 6.1.24 and VMs with the same OSs (in all 4 combinations). Please also run the CPU-Z test on your host for comparison.

Additionally, please provide a (zipped) VBox.log file, so that I can compare our VM settings and/or test if the differences play a role.
fth0
Volunteer
 
Posts: 2400
Joined: 14. Feb 2019, 03:06
Location: Germany
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Postby Petr Vones » 26. Jul 2021, 19:50

Ok, but I'll use the Core i5 2540M only as switching the VirtualBox/Hyper-V isn't easy (I need the 8265U for work) I am aware it is older CPU but the difference is most significant threre.
Petr Vones
 
Posts: 87
Joined: 27. Dec 2012, 01:20
Location: Czech Republic
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows 10 64-bit

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Postby Petr Vones » 26. Jul 2021, 21:16

The CPU-Z test is enough to see the issue on Intel Core i5 2540M: HyperV - 244/447, VirtualBox - 138/282. Both machines had the same settings, 2 CPU and 5 GB RAM. And the in I/O performance in a real world usage scenarios probably makes the bigger overall difference.

Screenshots and log file attached.
Attachments
Test.zip
(62.88 KiB) Downloaded 2 times
Petr Vones
 
Posts: 87
Joined: 27. Dec 2012, 01:20
Location: Czech Republic
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows 10 64-bit

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Postby Petr Vones » 26. Jul 2021, 22:07

On the AMD Ryzen 5 3600 the results are close: Hyper V - 502/972, VirtualBox - 471/840. Logs attached.
Attachments
TestAMD.zip
(61.2 KiB) Downloaded 2 times
Petr Vones
 
Posts: 87
Joined: 27. Dec 2012, 01:20
Location: Czech Republic
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows 10 64-bit

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Postby Petr Vones » 26. Jul 2021, 23:34

And the most surprising result is when the Hyper-V is enabled, so the VirtualBox runs in the native api 'turtle mode' on Intel Core i5 2540M. It actually runs the CPU operations faster than the VT-x engine, but I/O performance is worse. CPU-Z reports 210/459 that is similar to the Hyper-V performance. Log attached.

It seems as the VT-x engine isn't utilizing maximum Intel CPU performance.
Attachments
TestIntelNativeApi.zip
(46.27 KiB) Downloaded 1 time
Petr Vones
 
Posts: 87
Joined: 27. Dec 2012, 01:20
Location: Czech Republic
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows 10 64-bit

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Postby fth0 » 27. Jul 2021, 01:56

I've taken a first look into the results you provided. I haven't seen anything peculiar in the first VBox.log file (Intel CPU w/o Hyper-V), so that's good for starters. I'll do my own tests with CPU-Z at some time in the next evenings.

Can you also provide the CPU-Z test results for the native host OS as a baseline for comparison (as already requested)? At least the single-core results on the Intel CPU(s) may prove important.
fth0
Volunteer
 
Posts: 2400
Joined: 14. Feb 2019, 03:06
Location: Germany
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Postby Petr Vones » 27. Jul 2021, 09:49

Results for Intel Core i5 2540M host (Hyper-V turned off): 2 threads (similar to vm guests) - 260/451, 4 threads (default) - 280/751. So the Hyper-V guest runs on full power as the results 244/447 posted earlier indicates.

The CPU-Z results for the Intel Core i5 2540M summarized:

Machine             Single Thread  Multi Thread
Host 260 451
Guest VB VT-x 138 282
Guest VB Native Api 210 459
Guest Hyper-V 244 447
Petr Vones
 
Posts: 87
Joined: 27. Dec 2012, 01:20
Location: Czech Republic
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows 10 64-bit

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Postby fth0 » 27. Jul 2021, 17:16

Example CPU-Z results for my ThinkPad T590 with Intel Core i5-8265U, Windows 10 21H1 host, VirtualBox 6.1.24, Windows 10 21H1 guest with 2 vCPUs and 2 GB RAM:

Single Thread  Multi Thread  Machines                     
425 830 Windows Host
390 790 Windows Host + Windows VM
385 750 Windows Host + Windows VM (*)
415 820 Linux Host + Windows VM

(*) For a special test procedure, I ran the CPU-Z benchmark on the Windows host, left the program idling, started the Windows VM, ran the benchmark inside the Windows VM, and the multi-thread results were negatively affected! Welcome to Active Benchmarking. ;)

The results above are consistent with other benchmarks I did a few times in the past on my Linux VMs, where the CPU performance generally was between 85% and 95% of the hosts CPU performance.

Considering our (yours and mine) test results so far, IMHO we can concentrate on CPU-Z benchmarks on the Windows host alone and on the Windows host plus VirtualBox plus the Windows VM. It would be beneficial, if you could do just one pair of tests on your i5-8265U based laptop for comparison with the values in my table above (e.g. your host results alone could already significantly differ from mine, in both directions). In the meantime, I'll think about ways to combine the CPU-Z benchmarking with the VirtualBox statistics output ...
fth0
Volunteer
 
Posts: 2400
Joined: 14. Feb 2019, 03:06
Location: Germany
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Postby Petr Vones » 27. Jul 2021, 18:22

Intel Core i5-8265U, 2 CPU. It performs better there.

Machine             Single Thread  Multi Thread
Host 442 838
Guest VB VT-x 399 818
Guest Hyper-V 423 823
Petr Vones
 
Posts: 87
Joined: 27. Dec 2012, 01:20
Location: Czech Republic
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows 10 64-bit

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Postby fth0 » 27. Jul 2021, 19:41

Hmm. In your initial post, you stated that your unit test benchmarking routine is CPU bound and showed significantly lower performance than a Hyper-V VM (from which I'd assume that it would not be much faster than the host without Hyper-V) on the i5-8265U. The CPU-Z benchmarking routine is also expected to be CPU bound, but doesn't show a significant performance difference between the host and the guest. What are we overlooking? Do you have any good ideas to explain that?

Some general remarks:

When doing benchmarking, one should start from the same situation each time (guest and host pre-configured and powered off), at least until you're sure that some pre-conditions do not play a role.

I think we cannot easily compare the i5-8265U and the i5-2540M, because the former is seven years newer than the latter.
fth0
Volunteer
 
Posts: 2400
Joined: 14. Feb 2019, 03:06
Location: Germany
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...

Next

Return to Windows Guests

Who is online

Users browsing this forum: No registered users and 18 guests