VirtualBox vs Hyper-V AMD/Intel big performance difference

Discussions about using Windows guests in VirtualBox.
Petr Vones
Posts: 89
Joined: 27. Dec 2012, 01:20
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows 10 64-bit
Location: Czech Republic

VirtualBox vs Hyper-V AMD/Intel big performance difference

Post by Petr Vones »

I use Windows 10 x64 host and guest (MSDN) for development (Visual Studio, MS SQL Server and so on). I am aware that Hyper-V is a different kind of software, but this my finding are strange anyway. I converted the VirtualBox VDI to VHD and tried in Hyper-V on several machines. I used max. recommended settings of CPU and RAM in VirtualBox guests and then the same settings in Hyper-V. Performace tests were made by running unit tests (mostly CPU bound rather than I/O) in Visual Studio that takes several minutes. Laptops were always AC powered to get maximum performance. No software was running on hosts, no 3rd party antivirus crap, Windows Defender disabled in guests. Hyper-V was uninstalled and disabled for VirtualBox tests, of course.

AMD Ryzen 5 3600, 32 GB RAM, Samsung SSD 980 Pro - no significant performance difference between VirtualBox and Hyper-V, approximately 5% slower in VirtualBox as expected
Intel Core i5-8265U, 8 GB RAM, WDC SN720 SSD - double performance in Hyper-V compared to VirtualBox
Intel Core i5 2540M, 8 GB RAM, Intel 535 SSD - double performance in Hyper-V compared to VirtualBox

Is there any explanation why there is no significant performance difference on AMD PC, but it runs two times faster in Hyper-V on both Intel laptops? One thing i noticed the CPU clock speed reported in Task Manager in VirtualBox guests is lower than in host and Hyper-V guest.
fth0
Volunteer
Posts: 5668
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Post by fth0 »

Petr Vones wrote:max. recommended settings of CPU
There is no such thing. The red color of the CPU slider shows what is "not recommended", and the green color of the CPU slider shows what is "not not recommended", which is not the same as being recommended. For most use cases, providing two vCPUs is recommended.

Regarding the performance, can you provide some values? Can you also provide a VBox.log file from a VM run on the i5-8265U? (I chose that because I have the same CPU.)

On a general note:

Performance measurement is a highly complex topic by itself (*), and analyzing virtual machine performance has even greater complexity. From my own experience, a thorough investigation could quickly take days or weeks. How devoted are you to investigate the reason for the performance difference?

(*) Have you ever heard the term Active Benchmarking?
Petr Vones
Posts: 89
Joined: 27. Dec 2012, 01:20
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows 10 64-bit
Location: Czech Republic

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Post by Petr Vones »

Speaking of the i5-8265U machine, I assigned 5 GB RAM and tried 2 and 4 CPU (in both VirtualBox and Hyper-V). The performance is measured by typical usage scenarios: start Visual Studio, open solution, build it, run unit tests etc. where I can see at the first glance the Hyper-V is faster in overall. To get some numbers I run a project with ~70 unit tests that takes about 5 minutes to complete and provides durations. I run it multiple times to get a better picture of the values and the Hyper-V was two times faster than the VirtualBox (except of the AMD).
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Post by scottgus1 »

A possible suggestion from someone who doesn't understand the nuances: The physical CPU shows through to the VM OS; in fact it is the only physical hardware that the VM OS sees.

Might an OS be tuned to an Intel or AMD CPU during install, and therefore need re-installation if the processor manufacturer is changed?

To put it another way, if you make fresh VMs on each host, one for Hyper-V and one for Virtualbox, a total of 6 fresh VMs, and install the same test environment in each, does the same timing come out?
Petr Vones
Posts: 89
Joined: 27. Dec 2012, 01:20
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows 10 64-bit
Location: Czech Republic

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Post by Petr Vones »

scottgus1 wrote:Might an OS be tuned to an Intel or AMD CPU during install, and therefore need re-installation if the processor manufacturer is changed?
This particular virtual machine was originally created in VirtualBox on the Intel i5-8265U CPU, later moved to the AMD. So it actually shows the exact opposite issue :) It runs faster on the "non-original" AMD than the Intel. Some manufaturers also provides ready-to-made virtual machines that are CPU vendor agnostic so I don't think it is a valid scenario.
scottgus1 wrote:To put it another way, if you make fresh VMs on each host, one for Hyper-V and one for Virtualbox, a total of 6 fresh VMs, and install the same test environment in each, does the same timing come out?
I don't have a free week for this, sorry. To install and setup the complete environment takes two days, that's why I am saving my time by moving the virtual machines between physical computers. And again, the same virtual machine works faster in Hyper-V on those two laptops. Would running CPU-Z test on the Intel machines in both VirtualBox and Hyper-V make sense?

One thing that may explain why there is no significant performance loss on the AMD PC is that the 980 Pro SSD is superfast, 10 times faster than the SSD on the laptop. But the operations aren't too much I/O bound, small files read/write only. And I tried both Host I/O cache options, no significant difference.

I could easily move to Hyper-V and forget, but I still like the VirtualBox user interface more and the networking is more flexible. I wouldn't care if there was a 10 or 20 percent performance loss using the VirtualBox over the Hyper-V, but 50 percent is too much and weird.
Last edited by Petr Vones on 26. Jul 2021, 19:07, edited 1 time in total.
fth0
Volunteer
Posts: 5668
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Post by fth0 »

Thanks for describing your performance measurement method in some detail.
Petr Vones wrote:Would running CPU-Z test on the Intel machines in both VirtualBox and Hyper-V make sense?
It would enable me to do similar tests on my Lenovo T590 (Intel Core i5-8265U) for comparison, which is a dual-boot system (Windows 10 21H1, Linux Mint 20) with VirtualBox 6.1.24 and VMs with the same OSs (in all 4 combinations). Please also run the CPU-Z test on your host for comparison.

Additionally, please provide a (zipped) VBox.log file, so that I can compare our VM settings and/or test if the differences play a role.
Petr Vones
Posts: 89
Joined: 27. Dec 2012, 01:20
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows 10 64-bit
Location: Czech Republic

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Post by Petr Vones »

Ok, but I'll use the Core i5 2540M only as switching the VirtualBox/Hyper-V isn't easy (I need the 8265U for work) I am aware it is older CPU but the difference is most significant threre.
Petr Vones
Posts: 89
Joined: 27. Dec 2012, 01:20
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows 10 64-bit
Location: Czech Republic

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Post by Petr Vones »

The CPU-Z test is enough to see the issue on Intel Core i5 2540M: HyperV - 244/447, VirtualBox - 138/282. Both machines had the same settings, 2 CPU and 5 GB RAM. And the in I/O performance in a real world usage scenarios probably makes the bigger overall difference.

Screenshots and log file attached.
Attachments
Test.zip
(62.88 KiB) Downloaded 11 times
Petr Vones
Posts: 89
Joined: 27. Dec 2012, 01:20
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows 10 64-bit
Location: Czech Republic

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Post by Petr Vones »

On the AMD Ryzen 5 3600 the results are close: Hyper V - 502/972, VirtualBox - 471/840. Logs attached.
Attachments
TestAMD.zip
(61.2 KiB) Downloaded 12 times
Petr Vones
Posts: 89
Joined: 27. Dec 2012, 01:20
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows 10 64-bit
Location: Czech Republic

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Post by Petr Vones »

And the most surprising result is when the Hyper-V is enabled, so the VirtualBox runs in the native api 'turtle mode' on Intel Core i5 2540M. It actually runs the CPU operations faster than the VT-x engine, but I/O performance is worse. CPU-Z reports 210/459 that is similar to the Hyper-V performance. Log attached.

It seems as the VT-x engine isn't utilizing maximum Intel CPU performance.
Attachments
TestIntelNativeApi.zip
(46.27 KiB) Downloaded 7 times
fth0
Volunteer
Posts: 5668
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Post by fth0 »

I've taken a first look into the results you provided. I haven't seen anything peculiar in the first VBox.log file (Intel CPU w/o Hyper-V), so that's good for starters. I'll do my own tests with CPU-Z at some time in the next evenings.

Can you also provide the CPU-Z test results for the native host OS as a baseline for comparison (as already requested)? At least the single-core results on the Intel CPU(s) may prove important.
Petr Vones
Posts: 89
Joined: 27. Dec 2012, 01:20
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows 10 64-bit
Location: Czech Republic

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Post by Petr Vones »

Results for Intel Core i5 2540M host (Hyper-V turned off): 2 threads (similar to vm guests) - 260/451, 4 threads (default) - 280/751. So the Hyper-V guest runs on full power as the results 244/447 posted earlier indicates.

The CPU-Z results for the Intel Core i5 2540M summarized:
Machine             Single Thread  Multi Thread
Host                          260           451
Guest VB VT-x                 138           282
Guest VB Native Api           210           459
Guest Hyper-V                 244           447
fth0
Volunteer
Posts: 5668
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Post by fth0 »

Example CPU-Z results for my ThinkPad T590 with Intel Core i5-8265U, Windows 10 21H1 host, VirtualBox 6.1.24, Windows 10 21H1 guest with 2 vCPUs and 2 GB RAM:
Single Thread  Multi Thread  Machines                     
          425           830  Windows Host
          390           790  Windows Host + Windows VM
          385           750  Windows Host + Windows VM (*)
          415           820  Linux   Host + Windows VM
(*) For a special test procedure, I ran the CPU-Z benchmark on the Windows host, left the program idling, started the Windows VM, ran the benchmark inside the Windows VM, and the multi-thread results were negatively affected! Welcome to Active Benchmarking. ;)

The results above are consistent with other benchmarks I did a few times in the past on my Linux VMs, where the CPU performance generally was between 85% and 95% of the hosts CPU performance.

Considering our (yours and mine) test results so far, IMHO we can concentrate on CPU-Z benchmarks on the Windows host alone and on the Windows host plus VirtualBox plus the Windows VM. It would be beneficial, if you could do just one pair of tests on your i5-8265U based laptop for comparison with the values in my table above (e.g. your host results alone could already significantly differ from mine, in both directions). In the meantime, I'll think about ways to combine the CPU-Z benchmarking with the VirtualBox statistics output ...
Petr Vones
Posts: 89
Joined: 27. Dec 2012, 01:20
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows 10 64-bit
Location: Czech Republic

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Post by Petr Vones »

Intel Core i5-8265U, 2 CPU. It performs better there.
Machine             Single Thread  Multi Thread
Host                          442           838
Guest VB VT-x                 399           818
Guest Hyper-V                 423           823
fth0
Volunteer
Posts: 5668
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: VirtualBox vs Hyper-V AMD/Intel big performance difference

Post by fth0 »

Hmm. In your initial post, you stated that your unit test benchmarking routine is CPU bound and showed significantly lower performance than a Hyper-V VM (from which I'd assume that it would not be much faster than the host without Hyper-V) on the i5-8265U. The CPU-Z benchmarking routine is also expected to be CPU bound, but doesn't show a significant performance difference between the host and the guest. What are we overlooking? Do you have any good ideas to explain that?

Some general remarks:

When doing benchmarking, one should start from the same situation each time (guest and host pre-configured and powered off), at least until you're sure that some pre-conditions do not play a role.

I think we cannot easily compare the i5-8265U and the i5-2540M, because the former is seven years newer than the latter.
Post Reply