VirtualBox 6.0 and Hyper-V

Discussions related to using VirtualBox on Windows hosts.
Arny1
Posts: 2
Joined: 26. Jun 2020, 16:42

Re: VBox 6.1 / WMware / Hyper-V - some experiences

Post by Arny1 »

As already others experienced, also for me Virtualbox 6.1 machines do not run reliable with the Hyper-V hypervisor.
At least in my Win10 installation #1.

I have two Windows 10 installations on my harddisk, which do not see each other (partitions differently encrypted).
In my Win10 installation #2, VBox 6.1 runs well, but I am not sure if Hyper-V hypervisor is or is not running (details below).

My test setups: virtual machines with two cores, around 2400 MB Ram. No harddisk, but one virtual DVD drive containing KNOPPIX_V8.6.1-2019-10-14-DE.iso
The test:
  1. boot from the Knoppix ISO (which is a live Linux OS)
  2. and then within the virtual machine, from a command prompt, I have the MD5 hash be computed for the complete Knoppix DVD
  3. and then I have the MD5 computation repeated in order to find an MD5 discrepancy which would mean that something is wrong
The test command to measure MD5 computation time and show the MD5 result is: time md5sum /dev/sr0.

Test results in Win10 installation #1:

Code: Select all

VBox 6.1.10                    22 sec, but MD5 checksums different on every test run (Alas!)
VMWare Player 15.5.6     71 sec, MD5 checksum always the same
Hyper-V Manager            47/33 sec first/second test run, MD5 checksum always the same
Installation #1 is somehow controlled by my organization. They have Defender ATP installed, and probably other stuff too (e.g. Credential Guard?, Device Guard?), which I don't know much about. When I set "hypervisorlaunchtype Off" in the BCD store, this seems not to have any effect on VBox 6.1. BTW, HWiNFO64.exe always claims that some hypervisor is running. I upgraded from Windows 1803 to 1909 to 2004. This did not help; the MD5 discrepancy in my VBox test persists.

Installation #2 has been installed by me, with no extra protection software. I have Hyper-V stuff and Windows-Hyper-V-Platform installed just for testing how Virtualbox cooperates with it, and I have no problem at all. For some strange reason, HWiNFO64.exe also claims there is a Hypervisor running, even when the Hyper-V is not installed (but I need to check this again; sorry, I prematurely pressed Submit so this post got submitted too early). Update: HWINFO64.exe is correct; Hyper-V is enforced on installation #2 because installation #1 set some EFI variables which enforce Hyper-V usage even for installation #2.

Update 2020-07-03 - More internet researching, and testing by me shows:

If a Win 10 host is running on top of Hyper-V, this always makes VirtualBox become unreliable. Then do not use VirtualBox!!!

For detecting if the Win 10 host is running on top of Hyper-V, the preinstalled console program %windir%\system32\systeminfo.exe can be used. External program HWiNFO64.exe is not needed. At the end of its output, systeminfo either tells that Hyper-V is active, or the requirements which need to be met in order to activate Hyper-V.

When booting, there are several factors where every single one enforces Hyper-V usage:
  1. The well known "hypervisorlaunchtype Auto" setting in the BCD store
  2. Some so called "EFI variable" settings in the firmware. These seem to be: Kernel_Lsa_Cfg_Flags, VbsPolicy of the variable group GUID {77FA9ABD-0359-4D32-BD60-28F4E78F784B}
  3. (claimed somewhere else, not verified by me) Always when Secure Boot is turned on in the firmware setup
Only the Lsa_Cgf_Flags variable is visible within a running operating system; the Vbs variables have an attribute assigned which makes them accessible during booting only (e.g. by bootmgfw or Grub). Note that "Vbs" means "Virtualization Based Security".

It seems that under certain conditions (e.g. Credential Guard installed and configured appropriately) Windows 10 sets the EFI variable Kernel_Lsa_Cfg_Flags. This causes some EFI program running during the boot process (bootmgfw.efi, bootmgr.efi, WinLoad.efi, ...?) to create and set the EFI variable VbsPolicy, which determines that Hyper-V should be used for running Win 10. Note that possibly this whole EFI variable setting and Hyper-V enforcing becomes effective only after a 2nd or 3rd boot.
Last edited by Arny1 on 3. Jul 2020, 13:57, edited 12 times in total.
bendem
Posts: 1
Joined: 2. Jul 2020, 16:24

Re: VirtualBox 6.0 and Hyper-V

Post by bendem »

I have been trying to get this to work as well to test ansible playbooks using molecule-vagrant and I have two problems:

* about half the time, the VM hangs during boot, I just reset it and it works again, this I mostly ignore
* about 85% of the time, stuff I download from the network inside the guest system is corrupted. That, I can't work with.

I opened the bug 19695 about that (can't post links yet).
KenUnix
Posts: 9
Joined: 12. Apr 2020, 03:01

Re: VirtualBox 6.0 and Hyper-V

Post by KenUnix »

The latest Windows upgrade to 2004 created problems for me.

I use WSL and Oracle Virtualbox. After the upgrade running "X" VcXsrv for WSL Linux fails. I had to convert Debian & Ubuntu & Kali back to WSL-1 and set WSL 1 as default using Powershell

wsl --set-default-version 1

Then for each Linux distro

wsl --set-version Debian 1 as an example ( takes awhile)

To see what distros are installed and their WSL level do

wsl -l --verbose

After completing the above go to "Windows features" and turn off Virtual machine and Hyper-V. Reboot P.C.

Oracle virtualbox had problems. A VM would not start. It would hang at the loading screen

Start Virtualbox and a VM and select settings

Open Display

Set display to VboxVGA set memory to 128MB turn off 3D.

Now srart the distro and it should come up OK.

While this is not ideal it works as a workaround until the problems are fixed.
BillG
Volunteer
Posts: 5100
Joined: 19. Sep 2009, 04:44
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows 10,7 and earlier
Location: Sydney, Australia

Re: VirtualBox 6.0 and Hyper-V

Post by BillG »

Sounds about right. WSL2 uses hardware virtualization so it causes the same problems as Hyper-V, Credential Guard, Device Guard etc.

Sometimes turning off all the Hyper-V stuff doesn't clear all the problems. You need to stop the hypervisor from loading using the bcdedit command.
Bill
dtpoirot
Posts: 28
Joined: 12. Nov 2018, 18:57

Re: VirtualBox 6.0 and Hyper-V

Post by dtpoirot »

...so we are still at 'VirtualBox doesn't run with Hyper-V', regardless of what anyone says?

I have never been able to get it to work, even in slow-turtle mode.
HisMessengerDAB
Posts: 1
Joined: 17. Jul 2020, 16:12

Re: VirtualBox 6.0 and Hyper-V

Post by HisMessengerDAB »

My concern is this: I am trying to run Docker for Windows and to be able to run VMs on VB but I cannot do both at this time. I would really like to be able to do both. Docker for Windows requires HyperV but when I try to run my 64bit Ubuntu boxes while having HyperV enabled, I get the this error that has been shown by others:

Code: Select all

VMMR0_DO_NEM_INIT_VM failed: VERR_NEM_MISSING_KERNEL_API_2 (VERR_NEM_MISSING_KERNEL_API_2).
VT-x is not available (VERR_VMX_NO_VMX).

Result Code: 
E_FAIL (0x80004005)
Component: 
ConsoleWrap
Interface: 
IConsole {872da645-4a9b-1727-bee2-5585105b9eed}
I have scoured the web for an answer on how to resolve this and this forum thread seems to be my best hope. Does **anyone** have any suggestions on how to make it so that I can run both, Docker for Windows AND my Virtualbox VMs? Please help! Is there somewhere earlier in this thread that has an answer for this problem that I just missed?
multiOS
Volunteer
Posts: 797
Joined: 14. Sep 2019, 16:51
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: WIN11,10, 7, Linux (various)
Location: United Kingdom

Re: VirtualBox 6.0 and Hyper-V

Post by multiOS »

Docker for Windows uses Hyper-V, but having Hyper-V enabled prevents VirtualBox from working effectively, i.e. it's extremely slooooooow. and will remain so until the developers find a more effective solution than currently exists.

What you want to do is simply not possible at this time, because Hyper-V has to be completely disabled for VirtualBox VMs to run reliably.
RAMChYLD
Posts: 4
Joined: 8. Aug 2020, 09:16

Re: VirtualBox 6.0 and Hyper-V

Post by RAMChYLD »

Hi, I'm having a perculiar problem in that I'm able to get Virtualbox 6.1 running alongside Hyper-V on two Ryzen machines, but not on a Ryzen Threadripper machine.

The Threadripper machine hardware specs is as follows:
- Ryzen Threadripper 2990WX
- 64GB RAM
- Radeon Pro WX7100 GPU
- Gigabyte X399 Aorus Gaming 7 motherboard

I've already ensured that Windows Hypervisor Platform and Hyper-V are installed. I'm running Windows 20H1, and the latest chipset and GPU drivers are installed.

EDIT: The error given is as follows:
Failed to open a session for the virtual machine XMode.

Code: Select all

VMMR0_DO_NEM_INIT_VM failed: VERR_NEM_MISSING_KERNEL_API_2 (VERR_NEM_MISSING_KERNEL_API_2).

AMD-V is not available (VERR_SVM_NO_SVM).

Result Code: E_FAIL (0x80004005)
Component: ConsoleWrap
Interface: IConsole {872da645-4a9b-1727-bee2-5585105b9eed}
Which is wrong because Hyper-V wouldn't be working otherwise. Also, crosschecking with the BIOS indicates that SVM is enabled.

Can anyone please explain why I cannot run Virtualbox alongside Hyper-V on this machine when it's running well on two other normal Ryzen machines? The Threadripper looks like the ideal CPU to do networking simulations on due to the number of cores and RAM support, but ultimately is giving false readings to VirtualBox for some reason.
svr
Posts: 3
Joined: 8. Jun 2015, 09:28

Re: VirtualBox 6.0 and Hyper-V

Post by svr »

RAMChYLD wrote:Hi, I'm having a perculiar problem in that I'm able to get Virtualbox 6.1 running alongside Hyper-V on two Ryzen machines, but not on a Ryzen Threadripper machine.

The Threadripper machine hardware specs is as follows:
- Ryzen Threadripper 2990WX
- 64GB RAM
- Radeon Pro WX7100 GPU
- Gigabyte X399 Aorus Gaming 7 motherboard

I've already ensured that Windows Hypervisor Platform and Hyper-V are installed. I'm running Windows 20H1, and the latest chipset and GPU drivers are installed.

EDIT: The error given is as follows:
Failed to open a session for the virtual machine XMode.

Code: Select all

VMMR0_DO_NEM_INIT_VM failed: VERR_NEM_MISSING_KERNEL_API_2 (VERR_NEM_MISSING_KERNEL_API_2).

AMD-V is not available (VERR_SVM_NO_SVM).

Result Code: E_FAIL (0x80004005)
Component: ConsoleWrap
Interface: IConsole {872da645-4a9b-1727-bee2-5585105b9eed}
Which is wrong because Hyper-V wouldn't be working otherwise. Also, crosschecking with the BIOS indicates that SVM is enabled.

Can anyone please explain why I cannot run Virtualbox alongside Hyper-V on this machine when it's running well on two other normal Ryzen machines? The Threadripper looks like the ideal CPU to do networking simulations on due to the number of cores and RAM support, but ultimately is giving false readings to VirtualBox for some reason.
Virtualbox doesn't run alongside Hyper-V very well. At the moment you better disable Hyper-V if you want to use Virtualbox.
Andrew Brehm
Posts: 42
Joined: 28. Mar 2012, 17:19

Re: VirtualBox 6.0 and Hyper-V

Post by Andrew Brehm »

HisMessengerDAB wrote:My concern is this: I am trying to run Docker for Windows and to be able to run VMs on VB but I cannot do both at this time. I would really like to be able to do both. Docker for Windows requires HyperV
Docker for Windows does not require Hyper-V. Run your containers with docker run --isolation process and they will run on your Windows environment instead of launching a container-supporting Windows VM in Hyper-V.

I have scoured the web for an answer on how to resolve this and this forum thread seems to be my best hope. Does **anyone** have any suggestions on how to make it so that I can run both, Docker for Windows AND my Virtualbox VMs? Please help! Is there somewhere earlier in this thread that has an answer for this problem that I just missed?
If you also require Linux containers, which Docker wants to run in a Linux VM in Hyper-V, you can install Docker on your Linux VM in VirtualBox.
deaton64
Posts: 1
Joined: 7. Sep 2020, 14:56

Re: VirtualBox 6.0 and Hyper-V

Post by deaton64 »

Hi,

Well this as been going on for so long I thought it would never get resolved and I sort of understand why, but also don't.

Anyway, I thought I'd give it another go as I need to be able to run Vagrant VMs using Virtualbox and want to fire up Docker containers in WSL.

I almost can do what I'd like to be able to do, but something just isn't quite right. I think it might be with Virtualbox Network, as I'm not getting the IP I've specified on the VM, but I think that could be a Vagrant/Virtualbox/Hyper V Networking issue.

I'm running:
Windows 10.0.19041
Virtualbox 6.1.14r140239(Qt5.6.2)
Vagrant 2.2.10
Hyper V
WSL - Ubuntu 18.04 & 20.04
Docker Desktop

I can run docker containers and have Virtualbox VMs at the same time. There's no fancy installs, the last thing I did was, uninstall an older version of Virtual box, enable Hyper V, Virtual Machine Platform & Windows Hypervisor Platform, which had been disabled so I could use Virtualbox. After the reboot I installed the latest version of Vagrant & Virtualbox.

As I said, a slight Networking issue, but it does work for me :D
aeichner
Oracle Corporation
Posts: 193
Joined: 31. Aug 2007, 19:12

Re: VirtualBox 6.0 and Hyper-V

Post by aeichner »

The VERR_NEM_MISSING_KERNEL_API_2 error should be fixed with 6.1.14

Regarding the seemingly random memory corruption (wrong checksums, etc.):
We identified the issue but the fix went in too late for 6.1.14. We uploaded a new 6.1 testbuild to https://www.virtualbox.org/wiki/Testbuilds , make sure you are using the build with revision >= r140270. Feedback is greatly appreciated!
aeichner
Oracle Corporation
Posts: 193
Joined: 31. Aug 2007, 19:12

Re: VirtualBox 6.0 and Hyper-V

Post by aeichner »

Have to correct myself, there was another issue lurking in the Hyper-V backend which also shows the same symptoms, new 6.1 testbuild is up on https://www.virtualbox.org/wiki/Testbuilds , make sure you are using the build with revision >= r140284.
svr
Posts: 3
Joined: 8. Jun 2015, 09:28

Re: VirtualBox 6.0 and Hyper-V

Post by svr »

Using r140351 at the moment. So far it's looking good. The VMs don't generate random errors on network and are running stable. Thx for the efforts!

There is still a performance hit though... I see high CPU load and low disk IO on a process that normally goes pretty fast when directly using VT-X (so normally high disk IO)... But with Hyper-V enabled, it takes a long time... I know it's a little vague :)
hnyman
Posts: 9
Joined: 30. Apr 2011, 10:50
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Ubuntu 21.10

Re: VirtualBox 6.0 and Hyper-V

Post by hnyman »

aeichner wrote:make sure you are using the build with revision >= r140284.
Looks like the recent commits have really fixed something.

I used 6.1.15 r140351 to run Ubuntu 20.04 in Windows 10 to compile OpenWrt. It finally worked (unlike a few weeks ago).

There is maybe 5-10% performance hit, as the compilation that also involves gigabytes of disk action takes maybe 17-18 minutes instead of the normal 15-16 minutes (when allocating 4 cores of i7-9700 for Virtualbox). The performance difference is rather small compared to the earlier experiences that tended to crash before completion.

Could Virtualbox really finally be usable with Hyper-V ?

Ps. Interesting to see from source code that some of the fixed Hyper-V bugs have been there since early 2018...
E.g.
bug fixed by https://www.virtualbox.org/changeset/86067/vbox
originates in https://www.virtualbox.org/changeset/72403/vbox in May 2018
Locked