Questions on Nested Virtualization

[Technologov]

hi all!

Congratulations to Oracle for releasing the next gen v6. 1 BETA!

Question :
"Virtualization core: support for nested hardware-virtualization on Intel CPUs (starting with 5th generation Core i, codename Broadwell)"

1. I have heard rumors, that there was experimental support for AMD CPUs as well? Does it exist? Is it stable? (rumors from v6.0)

2. Why Broadwell instead of Haswell?
I mean those CPU architectures are very similar, brothers, yet Haswell is more common among users. Any new instructions that are specifically required? Which? Is it simple to support Haswell also?
(Broadwell = Core i7 5000-series and Xeon v4 vs. Haswell = Core i7 4000-series and Xeon v3)

Thanks,
-Technologov

[klaus]

1. Rumors about nested virtualization for AMD CPUs? Already present in 6.0, and should work quite nicely. Was much less complicated than the nested VT-x.

2. Broadwell, because not all Haswell (the rather late ones would be OK) have the "VMCS shadowing" feature of VT-x, which helps a lot with achieving decent performance. We went for this in the changelog, because performance with older CPUs (bare minimum requirement is actually "unrestricted execution", which some 1st gen Core i and all 2nd gen Core i CPUs should have) can be unsatisfactory.

And yes, we know... BETA1 contains a bug which keeps the "nested virtualization" checkbox in the GUI disabled. Not a GUI bug, the API doesn't report the feature to the GUI. The Windows build has this already fixed (which reflects the "bare minimum requirement"), but for all others this wasn't possible to include...

[Technologov]

"VMCS shadowing" according to Google exists in all Haswell CPUs.

https://blog.bjornhouben.com/2013/04/29 ... alization/

Nested Virtualization for AMD was never declared 'stable', would be nice to change the v6.1 description to supports AMD and Intel CPUs, because some users may think that only Intel are supported.

[socratis]

@Technologov
There are other articles that claim otherwise. For example, in https://searchservervirtualization.tech ... -shadowing I see (emphasis mine):

Intel currently deploys VMCS shadowing in certain fourth-generation Intel Core vPro processors, such as i5 vPro and i7 vPro, used in desktop and notebook systems, as well as the Xeon E5-2600, E5-1600 and E3-1200 processor families for enterprise servers.

@ALL
For those analyzing the logs, here's what's required. Looking in the "VT-x features" section:

00:00:00.987730 DMI Product Name: MacBookPro11,5
...
00:00:01.163288 CPUM: Matched host CPU INTEL 0x6/0x46/0x1 Intel_Core7_Haswell with CPU DB entry 'Intel Core i7-5600U' (INTEL 0x6/0x3d/0x4 Intel_Core7_Broadwell)
...
00:00:01.397175 Full Name: "Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz"
...
00:00:01.397194 *********************** VT-x features ***********************
00:00:01.397195 Nested hardware virtualization - VMX features
00:00:01.397195   Mnemonic - Description                        = guest (host)
00:00:01.397212   UnrestrictedGuest - Unrestricted guest        = 0 (1)
00:00:01.397215   VmcsShadowing - VMCS shadowing                = 0 (1)

whereas on my older MBP:

00:00:06.744806 DMI Product Name: MacBookPro8,3
00:00:07.111203 CPUM: Matched host CPU INTEL 0x6/0x2a/0x7 Intel_Core7_SandyBridge with CPU DB entry 'Intel Core i7-2635QM' (INTEL 0x6/0x2a/0x7 Intel_Core7_SandyBridge)
00:00:08.650086 Full Name: "Intel(R) Core(TM) i7-2820QM CPU @ 2.30GHz"
00:00:08.650145   UnrestrictedGuest - Unrestricted guest                  = 0 (1)
00:00:08.650151   VmcsShadowing - VMCS shadowing                          = 0 (0)

@Technologov

Nested Virtualization for AMD was never declared 'stable'

You should tell that to the following features, all snippets from the User Manual when searching for experimental:

• Oracle VM VirtualBox 3.2 added experimental support for Mac OS X guests
• Note that the ICH9 support is experimental and not recommended for guest OSes which do not require it.
• Oracle VM VirtualBox includes experimental support for the Extensible Firmware Interface (EFI)
• The Oracle VM VirtualBox Guest Additions contain experimental hardware 3D support for Windows, Linux, and Oracle Solaris guests.
• both OpenGL and Direct3D 8/9 are supported on an experimental basis.
• As an experimental feature, for additional capabilities, it is possible to give the guest direct access to the CD/DVD host drive by enabling passthrough mode.
• Starts a VM with a detachable UI. Technically, it is a headless VM with user interface in a separate process. This is an experimental feature...
• When running on Linux hosts with a kernel version later than 2.6.31, experimental host PCI devices passthrough is available.
• Oracle VM VirtualBox 4.3 includes an experimental feature which enables a guest to use a host webcam.
• As an experimental feature, Oracle VM VirtualBox enables access to an iSCSI target
• Another extension pack called VNC is available. This extension pack is open source and replaces the previous integration of the VNC remote access protocol. This is experimental code...
• As an experimental feature, primarily due to being limited to Linux host only and unknown Linux distribution coverage, Oracle VM VirtualBox supports passing through the PC speaker to the host.
• Oracle VM VirtualBox can be used on a Windows host where Hyper-V is running. This is an experimental feature.

or nicely summed up in ch. 14.1. Experimental Features. I guess the only thing you can "charge" them with is that they didn't include "nested virtualization on AMD CPUs"

[michaln]

"VMCS shadowing" according to Google exists in all Haswell CPUs.

Would you believe me if I told you that the Internet is full of lies?

It's possible that back in 2013, someone thought all Haswell CPUs will have VMCS shadowing (it's even possible that Intel gave that impression). The reality is that they don't. Some Haswell CPUs do have VMCS shadowing; as far as we know Haswell server CPUs do have it, and we know some Haswell desktop CPUs do (i7-4990 for example), but not all of them. Most non-server Haswell CPUs probably don't have it. See for example here showing that i7-4770K doesn't.

If you show us CPUID + VMX feature dumps (like the ones in VBox.log) covering a range of Haswell CPUs and indicating that they all have VMCS shadowing, that would be convincing. What some random person on the Internet said before the Haswell CPUs were even out is not.

[andyp73]

For those analyzing the logs, here's what's required. Looking in the "VT-x features" section:

00:00:01.397194 *********************** VT-x features ***********************
00:00:01.397195 Nested hardware virtualization - VMX features
00:00:01.397195   Mnemonic - Description                        = guest (host)
00:00:01.397212   UnrestrictedGuest - Unrestricted guest        = 0 (1)
00:00:01.397215   VmcsShadowing - VMCS shadowing                = 0 (1)

The existing logs on my iMac with i5-7500 (Kaby Lake) has all the right ticks in all the right places. Time to unleash v6.1 and see what chaos I can create!

-Andy.

[socratis]

Time to unleash v6.1 and see what chaos I can create!

Don't be so sure about it. The only test I tried (need to try some more) was:

• OSX 10.11.6 host, VirtualBox 6.1.0b1
  • Win10-64 VM, VirtualBox 6.1.0b1
    • FreeDOS VM.

The Win10 VM froze every time.

[ManWithNoName]

That's a good news !!!
I look forward to test the INTEL nested virtualisation feature of the final v6.1 release.
Thanks to the dev team

[ManWithNoName]

Hi everyone !

I'm using a computer with an INTEL Core i5-7200U and I have some issues by using Nested Virtualisation since the v6.1.0 vbox release.
I can't check the checkbox, it's greyed out.

Greyed out pb. (before command line executed)

cfg_greyed_pb - with_arrow.PNG (27.49 KiB) Viewed 32412 times

After waiting severals updates (now the v6.1.6 release), the issue is always here.

I have checked my CPU features to be sure that it's compatible.
INTEL Core i5-7200U -> 7th gen. Kabylake
So it is well > 5th gen. Broadwell

After viewing some forum post, I have finally try to activate the feature by the command line and...
IT WORKS FINE !!

Code: Select all

VBoxManage modifyvm YourVirtualBoxName --nested-hw-virt on

Greyed out pb. (after command line executed)

cfg_greyed_pb - solved by cli usage.PNG (29.21 KiB) Viewed 32412 times

It is very strange.
I'm asking if the problem occurs ONLY by updating vbox on a computer with an existing installation or if it also occurs on a new installation.

My OP is Windows 10 [version 10.0.18362.778]

[klaus]

Please provide VBox.log... I suspect that Intel has decided to not give your CPU model (but pretty much all others of the same generation) a key feature which makes the nested virtualization performance significantly better. This is what the GUI uses as the criteria for offering the feature. The VirtualBox engine can do nested virtualization without this key feature, but at a pretty high performance cost.

[therock247uk]

Nested virtulzation is greyed out here to and im using a i7 cpu no idea why this feature is so restricted.

[fth0]

a key feature which makes the nested virtualization performance significantly better

If you're talking about VMCS Shadowing, I believe that's restricted to Intel CPUs with Intel vPro. The latter is the keyword to look for inside the Intel CPU specifications on https://ark.intel.com. For example, the (quite old) desktop CPU Intel i5-4690 has this feature, while the (much newer) mobile CPU Intel i5-8265U doesn't have it.

[therock247uk]

vmware's nested works fine with no performance issues any idea why virtualbox is so strict?

[ManWithNoName]

Please provide VBox.log... I suspect that Intel has decided to not give your CPU model (but pretty much all others of the same generation) a key feature which makes the nested virtualization performance significantly better. This is what the GUI uses as the criteria for offering the feature. The VirtualBox engine can do nested virtualization without this key feature, but at a pretty high performance cost.

My post is updated with the VBox.log.
Thanks for your reply.

[fth0]

Please take a look at my previous post (2nd above yours) and at Intel Core i5-7200U. I'm pretty sure that I'm right.