Host: Ubuntu20.04.2 VM 6.1.16_Ubuntu r140961; Win7 not starting with VERR_SVM_ILLEGAL_EFER_MSR.

Discussions related to using VirtualBox on Linux hosts.
Post Reply
Dochsavage
Posts: 8
Joined: 3. Jun 2021, 22:21

Host: Ubuntu20.04.2 VM 6.1.16_Ubuntu r140961; Win7 not starting with VERR_SVM_ILLEGAL_EFER_MSR.

Post by Dochsavage »

Hi folks!

I've been running this VM for years over multiple versions of Ubuntu, and I've made no drastic recent changes.

Upon starting the VM, it immediately failed with reference to chip architecture features.
  1. VERR_SVM_ILLEGAL_EFER_MSR.
  2. AMD K6 EFER - Extended Feature Enable Register.
  3. MSR - Model-Specific Registers
My best hypothesis is that something about the last kernel update changed how the processor features are reported.
  • Ubuntu 20.04.2 LTS
My objective is to recover this VM and have it start normally.
Is there a set of buttons-and-levers that might either unearth more information about the root-cause?
Has anyone seen something similar? I found essentially nothing in my searches.

The startup logs will be attached, both with current error and from a previously successful start.
Win7image-error.log
VM log with current errors
(3.38 KiB) Downloaded 10 times
Win7image-error.log
VM log with current errors
(3.38 KiB) Downloaded 10 times
Many thanks for your brainpower!
Doch
Attachments
Win7image-working-abridged.log
Head of VM log from previously successful start
(3.74 KiB) Downloaded 8 times
fth0
Volunteer
Posts: 5668
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: Host: Ubuntu20.04.2 VM 6.1.16_Ubuntu r140961; Win7 not starting with VERR_SVM_ILLEGAL_EFER_MSR.

Post by fth0 »

Please provide a zipped copy of the complete log of the successful run.
Dochsavage
Posts: 8
Joined: 3. Jun 2021, 22:21

Re: Host: Ubuntu20.04.2 VM 6.1.16_Ubuntu r140961; Win7 not starting with VERR_SVM_ILLEGAL_EFER_MSR.

Post by Dochsavage »

Full log as requested.
Win7image-working-b.zip
(24.27 KiB) Downloaded 7 times
fth0
Volunteer
Posts: 5668
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: Host: Ubuntu20.04.2 VM 6.1.16_Ubuntu r140961; Win7 not starting with VERR_SVM_ILLEGAL_EFER_MSR.

Post by fth0 »

In the full log, I didn't see any unusual log messages regarding the AMD-V/SVM-related MSRs and CPUID information (although I must admit that I'm not as proficient regarding the AMD CPUs as regarding the Intel CPUs), so I cannot draw any helpful conclusions from that.

Regarding the VERR_SVM_ILLEGAL_EFER_MSR error, VirtualBox first checks if AMD-V/SVM is disabled in the BIOS. If it is not disabled, VirtualBox activates AMD-V/SVM and checks if the activation was successful. This last check fails. Typical reasons would be either a bug in the BIOS, or another hypervisor already running. Do you perhaps have KVM running (lsmod | grep -i kvm)?
Dochsavage
Posts: 8
Joined: 3. Jun 2021, 22:21

Re: Host: Ubuntu20.04.2 VM 6.1.16_Ubuntu r140961; Win7 not starting with VERR_SVM_ILLEGAL_EFER_MSR.

Post by Dochsavage »

Thanks for your initial analysis.

> lsmod | grep -i kvm
returns nothing, so KVM is not running.

I've restarted my machine and I found that the appropriate setting in the BIOS is enabled.
Does anyone know of a way to retrieve the actual EFER binary value?
Dochsavage
Posts: 8
Joined: 3. Jun 2021, 22:21

Re: Host: Ubuntu20.04.2 VM 6.1.16_Ubuntu r140961; Win7 not starting with VERR_SVM_ILLEGAL_EFER_MSR.

Post by Dochsavage »

For reference, the precise kernel that I'm running is:
  1. Ubuntu 5.4.0-74.83-generic 5.4.114
There appears to be a kernel changelog here:
There's nothing obvious to my eye here, though. I was looking for something about EFER I/O corrections or changes.
fth0
Volunteer
Posts: 5668
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: Host: Ubuntu20.04.2 VM 6.1.16_Ubuntu r140961; Win7 not starting with VERR_SVM_ILLEGAL_EFER_MSR.

Post by fth0 »

Dochsavage wrote:Does anyone know of a way to retrieve the actual EFER binary value?
On a Ubuntu Linux host, you can use the following commands:

Code: Select all

sudo apt install msr-tools
sudo modprobe msr
sudo rdmsr 0xc0000080
Dochsavage
Posts: 8
Joined: 3. Jun 2021, 22:21

Re: Host: Ubuntu20.04.2 VM 6.1.16_Ubuntu r140961; Win7 not starting with VERR_SVM_ILLEGAL_EFER_MSR.

Post by Dochsavage »

Many thanks!

Results:

Code: Select all

$ sudo rdmsr 0xc0000080
d01
Converted to a binary switch list: 110100000001

Also:

Code: Select all

$ sudo rdmsr 0xC0010114 #EFER
8
Converted to a binary switch list: 000000001000
(!) This seems to indicate that "Bit 3" is set.


From article: viewtopic.php?p=176873
Checking whether hardware virtualization is permanently (in current session) disabled
For AMD CPU to check whether you can use hardware virtualization, i.e. it wasn't blocked by BIOS, etc. you have to look at SVME_DISABLE bit (4th, counting from 0) in VM_CR (MSR 0xC0010114).

Per the AMD64 Architecture Manual, vol 2:
LOCK—Bit 3. When this bit is set, writes to LOCK and SVMDIS are silently ignored. When this bit is clear, VM_CR bits 3 and 4 can be written. Once set, LOCK can only be cleared using the SVM_KEY MSR (See Section 15.31.) This bit is not affected by INIT or SKINIT.

SVMDIS—Bit 4. When this bit is set, writes to EFER treat the SVME bit as MBZ. When this bit isclear, EFER.SVME can be written normally. This bit does not prevent CPUID from reporting that SVM is available. Setting SVMDIS while EFER.SVME is 1 generates a #GP fault, regardless of the current state of VM_CR.LOCK. This bit is not affected by SKINIT. It is cleared by INIT when LOCK is cleared to 0; otherwise, it is not affected.

From the reference to Section 15.31 above for LOCK bit:
15.31.1 SVM_KEY MSR (C001_0118h)
The write-only SVM_KEY MSR is used to create a password-protected mechanism to clear VM_CR.LOCK.
When VM_CR.LOCK is zero, writes to SVM_KEY MSR set the 64-bit SVM Key value.
When VM_CR.LOCK is one, writes to SVM_KEY MSR compare the written value to the SVM Key value; if the values match and are non-zero, the VM_CR.LOCK bit is cleared.

If the values mismatch or the SVM Key value is zero, the write to SVM_KEY is ignored, and VM_CR.LOCK is unmodified.
Software should read VM_CR.LOCK after writing SVM_KEY to determine whether the unlock succeeded.
If SVM Key is zero when VM_CR.LOCK is one, VM_CR.LOCK can only be cleared by a processor reset.
To preserve the security of the SVM key, reading the SVM_KEY MSR always returns zero.

So, I have more information, but the implications of these bits on my situation remain clear as mud.
Still, if I get this correctly, "Bit 3" could be a problem? If so, I'd need to clear it...probably by resetting the processor? By powering down for a 1 min+ period?

Let me know your thoughts.
Doch
fth0
Volunteer
Posts: 5668
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: Host: Ubuntu20.04.2 VM 6.1.16_Ubuntu r140961; Win7 not starting with VERR_SVM_ILLEGAL_EFER_MSR.

Post by fth0 »

On your host, VM_CR.LOCK == 1 and VM_CR.SVMDIS == 0. That's exactly the combination that you want, indicating that SVM was not disabled by the BIOS and cannot be disabled without powering off the CPU (or using the SVM_KEY, if someone (e.g. the BIOS) has set a key before locking). Usually, this combination is set by the BIOS, and depends on the BIOS configuration, so you cannot change it later on.

In this situation, anyone should be able to enable SVM by setting EFER.SVME = 1. VirtualBox does this and afterwards reads it back, getting the unexpected EFER.SVME == 0. Since WRMSR is a serializing CPU instruction, the subsequent RDMSR cannot be too early (if the AMD documentation is correct in this regard).

In consequence, the situation looks as if VirtualBox is not accessing the real EFER.SVME. This would be the case if Virtualbox was running inside another hypervisor, that's why I asked for KVM.
Dochsavage
Posts: 8
Joined: 3. Jun 2021, 22:21

Re: Host: Ubuntu20.04.2 VM 6.1.16_Ubuntu r140961; Win7 not starting with VERR_SVM_ILLEGAL_EFER_MSR.

Post by Dochsavage »

I hear you, and follow your logic.
Nonetheless, I'm running VirtualBox directly on Ubuntu20.04.2 on a AMD chipset.
Dochsavage
Posts: 8
Joined: 3. Jun 2021, 22:21

Re: Host: Ubuntu20.04.2 VM 6.1.16_Ubuntu r140961; Win7 not starting with VERR_SVM_ILLEGAL_EFER_MSR.

Post by Dochsavage »

My next step is to perform a stone-cold reboot.
Let me know if you have a better idea.
--Doch
fth0
Volunteer
Posts: 5668
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: Host: Ubuntu20.04.2 VM 6.1.16_Ubuntu r140961; Win7 not starting with VERR_SVM_ILLEGAL_EFER_MSR.

Post by fth0 »

I don't know what a "stone-cold boot" is, but starting the host from a power-off state sounds like a natural step when trying to start from a reproducible initial state. I'd also check the BIOS settings for virtualization-related settings, when you're at it.
Dochsavage
Posts: 8
Joined: 3. Jun 2021, 22:21

Re: Host: Ubuntu20.04.2 VM 6.1.16_Ubuntu r140961; Win7 not starting with VERR_SVM_ILLEGAL_EFER_MSR.

Post by Dochsavage »

In this case, a stone-cold boot is a process where one:
* powers down the machine
* unplugs the machine from line power
* waits for 3 to 5 minutes
* re-plugs the machine into line power
* starts the machine

And in this case, it worked.

I'm now able to start my VMs again, as previously.

Many thanks for your analysis and aid!
--Doch
fth0
Volunteer
Posts: 5668
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: Host: Ubuntu20.04.2 VM 6.1.16_Ubuntu r140961; Win7 not starting with VERR_SVM_ILLEGAL_EFER_MSR.

Post by fth0 »

You're welcome, and thanks for reporting back.
Post Reply