Trying to enable Docker Desktop

[poflynn]

Host: Mint 19.3
Guests: Tried clean installs of Win 10 Home + Pro, 2004 + all latest updates
VirtualBox: 6.1.14
Enabled Nested VT-x/AMD-V on both Windows guests

Docker Desktop in Windows gives this error on both VMs:

"System.InvalidOperationException:
Failed to deploy distro docker-desktop to C:\Users\pofly\AppData\Local\Docker\wsl\distro: exit code: -1
stdout: Please enable the Virtual Machine Platform Windows feature and ensure virtualization is enabled in the BIOS."

Speccy gives this CPU info from within the guest:

Cores 1
Threads 4
Name AMD Ryzen 5 PRO
Code Name Picasso
Package Socket AM4 (1331)
Technology 12nm
Specification AMD Ryzen 5 PRO 3400GE w/ Radeon Vega Graphics
Family F
Extended Family 17
Model 8
Extended Model 18
Stepping 1
Instructions MMX (+), SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, SSE4A, AMD 64, NX, VMX, AES, AVX, AVX2
Virtualization Supported, Disabled
Hyperthreading Supported, Enabled

Note that "Virtualization" is "Supported" but "Disabled". My guess is "Enabled Nested VT-x/AMD-V" isn't "taking", ie isn't available within the guest? I've been googling for two days but it's difficult to google and 90% of the hits are folks using Windows as a host and trying to run VB + Hyper-V at the same time etc etc. Any suggestions? Thanks in advance!

Paul

[scottgus1]

For all intents and purposes you are trying nested virtualization.

Start the guest from full power off, not save-state. Run until you see the problem happen, then shut down the guest from within the guest OS if possible. If not possible, close the Virtualbox window for the guest with the Power Off option set.

Please right-click the guest in the main Virtualbox window's guest list, choose Show Log. Bring up the far left tab's log.

Now search the log file for these items I have shamelessly copied from forum guru fth0's post here on CPU features necessary to handle nested virtualization: viewtopic.php?f=1&t=98708&p=478598#p478598

00:00:01.017650 *********************** VT-x features ***********************
00:00:01.017651 Nested hardware virtualization - VMX features
00:00:01.017651   Mnemonic - Description                          = guest (host)
00:00:01.017651   VMX - Virtual-Machine Extensions                = 1 (?)
00:00:01.017665   Ept - Extended Page Tables                      = 0 (?)
00:00:01.017668   UnrestrictedGuest - Unrestricted guest          = 0 (?)
00:00:01.017671   VmcsShadowing - VMCS shadowing                  = 0 (?)

All the results shown as (?) for the (host) column must be (1) for your guest to have full nested virtualization capability to handle a second guest layer. If you have a (0) in place of a (?) in the (host) column, then your host is not fully capable of nested virtualization.

[Pernat1y]

AFAIK, Docker Desktop requires Hyper-V, which, most likely, will not work in guest OS.

[poflynn]

Thanks for the replies folks, especially the helpful link @scottgus1 (although I suspect the included info applies only to Intel). I have pasted various bits out of my log below. Right off the bat you'll see VMX=0 on the host at the beginning, which doesn't bode well although elsewhere we see NestedHWVirt being available. I would have thought my CPU has the features needed from what I can tell via googling but it's hard to say for sure. I suspect I'm fighting a losing battle here

Paul

Code: Select all

00:00:01.230253 Features
00:00:01.230253   Mnemonic - Description                                  = guest (host)
00:00:01.230254   FPU - x87 FPU on Chip                                   = 1 (1)
00:00:01.230255   VME - Virtual 8086 Mode Enhancements                    = 1 (1)
00:00:01.230256   DE - Debugging extensions                               = 1 (1)
00:00:01.230257   PSE - Page Size Extension                               = 1 (1)
00:00:01.230258   TSC - Time Stamp Counter                                = 1 (1)
00:00:01.230259   MSR - Model Specific Registers                          = 1 (1)
00:00:01.230260   PAE - Physical Address Extension                        = 1 (1)
00:00:01.230261   MCE - Machine Check Exception                           = 1 (1)
00:00:01.230262   CX8 - CMPXCHG8B instruction                             = 1 (1)
00:00:01.230263   APIC - APIC On-Chip                                     = 1 (1)
00:00:01.230263   SEP - SYSENTER and SYSEXIT Present                      = 1 (1)
00:00:01.230264   MTRR - Memory Type Range Registers                      = 1 (1)
00:00:01.230265   PGE - PTE Global Bit                                    = 1 (1)
00:00:01.230266   MCA - Machine Check Architecture                        = 1 (1)
00:00:01.230267   CMOV - Conditional Move instructions                    = 1 (1)
00:00:01.230267   PAT - Page Attribute Table                              = 1 (1)
00:00:01.230268   PSE-36 - 36-bit Page Size Extension                     = 1 (1)
00:00:01.230269   PSN - Processor Serial Number                           = 0 (0)
00:00:01.230270   CLFSH - CLFLUSH instruction                             = 1 (1)
00:00:01.230271   DS - Debug Store                                        = 0 (0)
00:00:01.230272   ACPI - Thermal Mon. & Soft. Clock Ctrl.                 = 0 (0)
00:00:01.230273   MMX - Intel MMX Technology                              = 1 (1)
00:00:01.230274   FXSR - FXSAVE and FXRSTOR instructions                  = 1 (1)
00:00:01.230274   SSE - SSE support                                       = 1 (1)
00:00:01.230275   SSE2 - SSE2 support                                     = 1 (1)
00:00:01.230276   SS - Self Snoop                                         = 0 (0)
00:00:01.230277   HTT - Hyper-Threading Technology                        = 1 (1)
00:00:01.230278   TM - Therm. Monitor                                     = 0 (0)
00:00:01.230279   PBE - Pending Break Enabled                             = 0 (0)
00:00:01.230280   SSE3 - SSE3 support                                     = 1 (1)
00:00:01.230281   PCLMUL - PCLMULQDQ support (for AES-GCM)                = 1 (1)
00:00:01.230282   DTES64 - DS Area 64-bit Layout                          = 0 (0)
00:00:01.230283   MONITOR - MONITOR/MWAIT instructions                    = 0 (1)
00:00:01.230283   CPL-DS - CPL Qualified Debug Store                      = 0 (0)
00:00:01.230284   VMX - Virtual Machine Extensions                        = 0 (0)
00:00:01.230285   SMX - Safer Mode Extensions                             = 0 (0)
00:00:01.230286   EST - Enhanced SpeedStep Technology                     = 0 (0)
00:00:01.230286   TM2 - Terminal Monitor 2                                = 0 (0)
00:00:01.230287   SSSE3 - Supplemental Streaming SIMD Extensions 3        = 1 (1)
00:00:01.230288   CNTX-ID - L1 Context ID                                 = 0 (0)
00:00:01.230289   SDBG - Silicon Debug interface                          = 0 (0)
00:00:01.230290   FMA - Fused Multiply Add extensions                     = 0 (1)
00:00:01.230291   CX16 - CMPXCHG16B instruction                           = 1 (1)
00:00:01.230291   TPRUPDATE - xTPR Update Control                         = 0 (0)
00:00:01.230292   PDCM - Perf/Debug Capability MSR                        = 0 (0)
00:00:01.230293   PCID - Process Context Identifiers                      = 0 (0)
00:00:01.230294   DCA - Direct Cache Access                               = 0 (0)
00:00:01.230295   SSE4_1 - SSE4_1 support                                 = 1 (1)
00:00:01.230295   SSE4_2 - SSE4_2 support                                 = 1 (1)
00:00:01.230296   X2APIC - x2APIC support                                 = 0 (0)
00:00:01.230297   MOVBE - MOVBE instruction                               = 1 (1)
00:00:01.230298   POPCNT - POPCNT instruction                             = 1 (1)
00:00:01.230299   TSCDEADL - Time Stamp Counter Deadline                  = 0 (0)
00:00:01.230300   AES - AES instructions                                  = 1 (1)
00:00:01.230301   XSAVE - XSAVE instruction                               = 1 (1)
00:00:01.230301   OSXSAVE - OSXSAVE instruction                           = 0 (1)
00:00:01.230302   AVX - AVX support                                       = 1 (1)
00:00:01.230303   F16C - 16-bit floating point conversion instructions    = 0 (1)
00:00:01.230304   RDRAND - RDRAND instruction                             = 1 (1)
00:00:01.230305   HVP - Hypervisor Present (we're a guest)                = 1 (0)

snip

00:00:01.052017 [/CPUM/] (level 1)
00:00:01.052020   GuestCpuName       <string>  = "host" (cb=5)
00:00:01.052026   NestedHWVirt       <integer> = 0x0000000000000001 (1)
00:00:01.052029   PortableCpuIdLevel <integer> = 0x0000000000000000 (0)
00:00:01.052030   SpecCtrl           <integer> = 0x0000000000000000 (0)

snip

00:00:01.052559 [/HM/] (level 1)
00:00:01.052561   64bitEnabled       <integer> = 0x0000000000000001 (1)
00:00:01.052562   EnableLargePages   <integer> = 0x0000000000000000 (0)
00:00:01.052563   EnableNestedPaging <integer> = 0x0000000000000001 (1)
00:00:01.052564   EnableUX           <integer> = 0x0000000000000001 (1)
00:00:01.052565   EnableVPID         <integer> = 0x0000000000000001 (1)
00:00:01.052567   Exclusive          <integer> = 0x0000000000000001 (1)
00:00:01.052568   HMForced           <integer> = 0x0000000000000001 (1)
00:00:01.052569   IBPBOnVMEntry      <integer> = 0x0000000000000000 (0)
00:00:01.052570   IBPBOnVMExit       <integer> = 0x0000000000000000 (0)
00:00:01.052571   L1DFlushOnSched    <integer> = 0x0000000000000001 (1)
00:00:01.052572   L1DFlushOnVMEntry  <integer> = 0x0000000000000000 (0)
00:00:01.052573   MDSClearOnSched    <integer> = 0x0000000000000001 (1)
00:00:01.052574   MDSClearOnVMEntry  <integer> = 0x0000000000000000 (0)
00:00:01.052575   SpecCtrlByHost     <integer> = 0x0000000000000000 (0)
00:00:01.052576   UseNEMInstead      <integer> = 0x0000000000000000 (0)

snip

00:00:01.053970 HM: HMR3Init: AMD-V w/ nested paging
00:00:01.054033 MM: cbHyperHeap=0x240000 (2359296)
00:00:01.055563 CPUM: fXStateHostMask=0x7; initial: 0x7; host XCR0=0x7
00:00:01.057581 CPUM: Matched host CPU AMD 0x17/0x18/0x1 AMD_Zen_Ryzen with CPU DB entry 'AMD FX-8150 Eight-Core' (AMD 0x15/0x1/0x2 AMD_15h_Bulldozer)
00:00:01.057862 CPUM: MXCSR_MASK=0x2ffff (host: 0x2ffff)
00:00:01.057878 CPUM: Microcode revision 0x08108109
00:00:01.057892 CPUM: MSR fudge: 0x00000017 IA32_PLATFORM_ID
00:00:01.057900 CPUM: MSR fudge: 0x000001a0 IA32_MISC_ENABLE
00:00:01.057905 CPUM: MSR fudge: 0x0000002c P4_EBC_FREQUENCY_ID
00:00:01.057927 CPUM: Allocating 20 pages for the nested-guest SVM MSR and IO permission bitmaps
00:00:01.058617 PGM: Host paging mode: AMD64+PGE+NX

snip

00:00:01.228389 HM: Using AMD-V implementation 2.0
00:00:01.228389 HM: Max resume loops                  = 8192
00:00:01.228390 HM: AMD HWCR MSR                      = 0x9000011
00:00:01.228391 HM: AMD-V revision                    = 0x1
00:00:01.228391 HM: AMD-V max ASID                    = 32768
00:00:01.228392 HM: AMD-V features                    = 0x1bcff
00:00:01.228392 HM:   NESTED_PAGING
00:00:01.228393 HM:   LBR_VIRT
00:00:01.228393 HM:   SVM_LOCK
00:00:01.228393 HM:   NRIP_SAVE
00:00:01.228393 HM:   TSC_RATE_MSR
00:00:01.228394 HM:   VMCB_CLEAN
00:00:01.228394 HM:   FLUSH_BY_ASID
00:00:01.228394 HM:   DECODE_ASSISTS
00:00:01.228395 HM:   PAUSE_FILTER
00:00:01.228395 HM:   PAUSE_FILTER_THRESHOLD
00:00:01.228395 HM:   AVIC
00:00:01.228395 HM:   VIRT_VMSAVE_VMLOAD
00:00:01.228396 HM:   VGIF
00:00:01.228396 HM:   Reserved bit 11
00:00:01.228405 HM: Enabled SVM
00:00:01.228405 HM:   Enabled nested paging
00:00:01.228410 CPUM: SetGuestCpuIdFeature: Enabled SYSENTER/EXIT
00:00:01.228410 CPUM: SetGuestCpuIdFeature: Enabled SYSCALL/RET
00:00:01.228411 CPUM: SetGuestCpuIdFeature: Enabled PAE
00:00:01.228411 CPUM: SetGuestCpuIdFeature: Enabled LONG MODE
00:00:01.228411 CPUM: SetGuestCpuIdFeature: Enabled NX
00:00:01.228412 CPUM: SetGuestCpuIdFeature: Enabled LAHF/SAHF
00:00:01.228412 HM: Disabled TPR patching
00:00:01.228412 HM: Guest support: 32-bit and 64-bit
00:00:01.228412 HM: VT-x/AMD-V init method: Global

snip

00:00:01.230454 SVM Feature Identification (leaf A):
00:00:01.230455   NP - Nested Paging                                      = 0 (1)
00:00:01.230456   LbrVirt - Last Branch Record Virtualization             = 0 (1)
00:00:01.230457   SVML - SVM Lock                                         = 0 (1)

snip

00:06:56.916297 VCPU[3] hardware virtualization state:
00:06:56.916298 fLocalForcedActions          = 0x0
00:06:56.916298 SVM hwvirt state:
00:06:56.916299   fGif                       = true 
00:06:56.916300   uMsrHSavePa                = 0x0
00:06:56.916301   GCPhysVmcb                 = 0x0000000000000000
00:06:56.916301   VmcbCtrl:
00:06:56.916302     CRX-read intercepts        = 0x0
00:06:56.916302     CRX-write intercepts       = 0x0
00:06:56.916303     DRX-read intercepts        = 0x0
00:06:56.916303     DRX-write intercepts       = 0x0
00:06:56.916304     Exception intercepts       = 0x0
00:06:56.916304     Control intercepts         = 0x0
00:06:56.916304     Pause-filter threshold     = 0x0
00:06:56.916305     Pause-filter count         = 0x0
00:06:56.916305     IOPM bitmap physaddr       = 0x0
00:06:56.916306     MSRPM bitmap physaddr      = 0x0
00:06:56.916306     TSC offset                 = 0x0
00:06:56.916306     TLB Control
00:06:56.916307       ASID                       = 0x0
00:06:56.916307       TLB-flush type             = 0
00:06:56.916308     Interrupt Control
00:06:56.916308       VTPR                       = 0x0 (0)
00:06:56.916309       VIRQ (Pending)             = false
00:06:56.916309       VINTR vector               = 0x0
00:06:56.916310       VGIF                       = 0
00:06:56.916310       VINTR priority             = 0x0
00:06:56.916311       Ignore TPR                 = false
00:06:56.916311       VINTR masking              = false
00:06:56.916312       VGIF enable                = false
00:06:56.916312       AVIC enable                = false
00:06:56.916312     Interrupt Shadow
00:06:56.916313       Interrupt shadow           = false
00:06:56.916313       Guest-interrupt Mask       = false
00:06:56.916314     Exit Code                  = 0x0
00:06:56.916314     EXITINFO1                  = 0x0
00:06:56.916314     EXITINFO2                  = 0x0
00:06:56.916315     Exit Interrupt Info
00:06:56.916315       Valid                      = false
00:06:56.916315       Vector                     = 0x0 (0)
00:06:56.916316       Type                       = 0
00:06:56.916316       Error-code valid           = false
00:06:56.916317       Error-code                 = 0x0
00:06:56.916317     Nested paging and SEV
00:06:56.916318       Nested paging              = false
00:06:56.916318       SEV (Secure Encrypted VM)  = false
00:06:56.916318       SEV-ES (Encrypted State)   = false
00:06:56.916319     Event Inject
00:06:56.916319       Valid                      = false
00:06:56.916319       Vector                     = 0x0 (0)
00:06:56.916320       Type                       = 0
00:06:56.916320       Error-code valid           = false
00:06:56.916321       Error-code                 = 0x0
00:06:56.916321     Nested-paging CR3          = 0x0

snip

00:06:56.916366 Guest paging mode (VCPU #3):  AMD64+NX (changed 3 times), A20 enabled (changed 0 times)
00:06:56.916368 Shadow paging mode (VCPU #3): Nested-AMD64
00:06:56.916369 Host paging mode:             AMD64+G+NX

[poflynn]

An interesting link I found: https://www.cyberciti.biz/faq/linux-xen ... v-support/
From here I learned that the AMD equivalent of VMX is SVM and EPT = NPT. My CPU has both as you can see below. Interestingly there is no mention of NPT anywhere in my guest log, so maybe VB simply doesn't look for this or support it altho everything I read online seemed to indicated that this was better supported on AMD than Intel by VB.

Ran the below cmd on the host:

Code: Select all

root@fun:~# cat /proc/cpuinfo 
processor	: 0
vendor_id	: AuthenticAMD
cpu family	: 23
model		: 24
model name	: AMD Ryzen 5 PRO 3400GE w/ Radeon Vega Graphics
stepping	: 1
microcode	: 0x8108109
cpu MHz		: 1815.220
cache size	: 512 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb hw_pstate sme ssbd sev ibpb vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 xsaves clzero irperf xsaveerptr arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif overflow_recov succor smca
bugs		: sysret_ss_attrs null_seg spectre_v1 spectre_v2 spec_store_bypass
bogomips	: 6587.70
TLB size	: 2560 4K pages
clflush size	: 64
cache_alignment	: 64
address sizes	: 43 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate eff_freq_ro [13] [14]

[fth0]

helpful link [...] (although I suspect the included info applies only to Intel)

Yes, that's correct. Note that I always write about VT-x only and do not mention AMD-V, with the sole exception when naming the checkbox in the VM configuration. The background for this is that I don't know much about AMD CPUs, and I try to be as specific as possible.

You've found out correctly that you have to look for SVM instead of VMX. The equivalent of EPT could also be named RVI or SLAT. Unrestricted Guest doesn't have and need an equivalent AFAIK. But I don't know what would be the equivalent of VMCS Shadowing, and that's what makes the difference in the Intel CPU case. I'd take a look at the VirtualBox source code to come to an educated guess, but maybe not today. Please post a complete, zipped VBox.log file, if you want to support my little research.

[poflynn]

Thanks for the offer of help! Full log attached.

[fth0]

00:00:01.230412 Ext Features
00:00:01.230412   Mnemonic - Description                                  = guest (host)
00:00:01.230437   SVM - AMD Secure Virtual Machine extensions             = 1 (1)
00:00:01.230454 SVM Feature Identification (leaf A):
00:00:01.230455   NP - Nested Paging                                      = 0 (1)
00:00:01.230465   VMSAVEVirt - VMSAVE and VMLOAD Virtualization           = 0 (1)

All needed AMD CPU features seem to be available: SVM is the counterpart to VMX, NP is the counterpart of EPT, and VMSAVEVirt is the counterpart of VMCS Shadowing. For a test, you could install VirtualBox inside the guest OS, and create and start a nested VirtualBox VM.

Note that in general, the hypervisor manufactures (e.g. VMware, Microsoft, Oracle) do not explicitly support mixed environments for nested virtualization. One exception is that VirtualBox allows KVM as nested hypervisor (at least according to the VirtualBox Manual).

[poflynn]

All needed AMD CPU features seem to be available:

That was my guess, but those "0"s under "Guest" indicate these are turned off for that guest which I assume is bad?

For a test, you could install VirtualBox inside the guest OS, and create and start a nested VirtualBox VM.

Happy to try this this weekend when I have time.

Note that in general, the hypervisor manufactures (e.g. VMware, Microsoft, Oracle) do not explicitly support mixed environments for nested virtualization. One exception is that VirtualBox allows KVM as nested hypervisor (at least according to the VirtualBox Manual).

Docker Desktop requires Hyper-V which does not use KVM I believe, won't this be a problem?

Thx!

[fth0]

those "0"s under "Guest" indicate these are turned off for that guest which I assume is bad?

Maybe, maybe not. There could be implicit ways of providing the features to the inner VirtualBox. FWIW, I have tried a nested setup some weeks ago, where the log messages indicated EPT not being provided to the outer guest, but the inner guest was nearly as fast as the outer guest.

Docker Desktop requires Hyper-V which does not use KVM I believe, won't this be a problem?

Yes. I'd either use Docker directly on your Linux host, or use KVM (instead of VirtualBox) on your Linux host with a Windows VM containing Docker Desktop for Windows, but expect a performance impact with the latter setup.

[poflynn]

Yes. I'd either use Docker directly on your Linux host, or use KVM (instead of VirtualBox) on your Linux host with a Windows VM containing Docker Desktop for Windows, but expect a performance impact with the latter setup.

Well I spent most of today messing with KVM + a Win 10 guest but the results are basically the same, in fact, the o/p of speccy in the win guest is identical, ie the same subset of CPU flags available + "virtualization supported, disabled" . I'm probably giving up at this point, which is disappointing, I didn't think it would be this hard. Thx to everyone on this thread.

Paul