Windows 10 update causing boot failure. wrmsr(0x122,0x0`00000003) (#19169)

Discussions about using Windows guests in VirtualBox.
doezel
Posts: 7
Joined: 12. Dec 2019, 21:42

Re: Can't run any vm on my new laptop

Post by doezel »

Update:

I was looking at this thread: viewtopic.php?f=3&t=93990&start=15 which also mentioned a Lenovo, same as the thread here with the Windows Guests; [ModEdit; threads merged]

So i downgraded VirtualBox to version 6.0.6 including the Extension pack for that version.

Now my win10 VM's work!

So, I presume there's something between the Lenovo hardware, The latest windows updates and the latest VirtualBox version that causes my issues.
socratis
Site Moderator
Posts: 27330
Joined: 22. Oct 2010, 11:03
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Win(*>98), Linux*, OSX>10.5
Location: Greece

Re: Can't run any vm on my new laptop

Post by socratis »

doezel wrote:as for the 2 mice: it's a thinkpad with a trackpoint and a touchpad. and sometimes a wired or a bluetooth mous
No, that's not it. Remeber that we're talking about a Virtual Machine, one that has no notion of what's attached on your real computer. The two "mice" comes from the fact that you've selected "USB Multi-Touch Tablet" in the VM Settings » System » Motherboard » Pointing Device. Got to remember that...
[quote=""win10.vbox""]
<HID Pointing="USBMultiTouch"/>
[/quote][quote=""win10-2019-12-13-11-42-30.log""]
00:00:01.858810 [/USB/HidMouse/0/Config/] (level 4)
00:00:01.858811   Mode <string>  = "absolute" (cb=9)
...
00:00:01.858817 [/USB/HidMouse/1/Config/] (level 4)
00:00:01.858818   Mode <string>  = "multitouch" (cb=11)
[/quote]

On the freezing front...
00:00:16.405986 IEM: wrmsr(0x122,0x0`00000003) -> #GP(0)
00:00:45.278022 Changing the VM state from 'RUNNING' to 'SUSPENDING'
doezel wrote:So, I presume there's something between the Lenovo hardware
Something is wrong all right, but I don't think you can say it's the "Lenovo" part, because just today I saw another similar issue: [ModEdit; threads merged], where the OP has a "Dell Precision 7540". And while you have a:
00:00:02.361270 Full Name: "Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz"
the other OP has a:
00:00:02.451294 Full Name: "Intel(R) Xeon(R) E-2286M  CPU @ 2.40GHz"
Both of the CPUs are not your everyday ones, they're rather rare. My gut feeling? When they took out the "rawmode", something didn't go as planned. Will keep gathering information...
doezel wrote:i downgraded VirtualBox to version 6.0.6 ... win10 VM's work!
Can you please try with 6.0.14, the current release as of this writing of the 6.0.x series?
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
mga
Posts: 6
Joined: 28. Apr 2019, 08:05

Re: Windows 10 update causing boot failure

Post by mga »

Thanks for sharing, I'm gonna try the downgrade.
whith The upgrade to 6.1 i have the same issue on my thinkpad
fth0
Volunteer
Posts: 5661
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: Windows 10 Guests do not start after Bios Update

Post by fth0 »

socratis wrote:
00:00:07.800331 IEM: wrmsr(0x122,0x0`00000003) -> #GP(0)
Something MSR related again... :?
The IA32_TSX_CTRL MSR (0x122) is only available in some quite new Intel CPUs and not yet documented in the Intel Software Architecture Manual. The technical details can be found on an Intel web page Deep Dive: Intel® Transactional Synchronization Extensions (Intel® TSX) Asynchronous Abort and in a Linux kernel patch x86/msr: Add the IA32_TSX_CTRL MSR.

@socratis: A VirtualBox developer should be notified about this thread. The topic would be TSX Asynchronous Abort mitigation.
socratis
Site Moderator
Posts: 27330
Joined: 22. Oct 2010, 11:03
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Win(*>98), Linux*, OSX>10.5
Location: Greece

Re: Windows 10 update causing boot failure

Post by socratis »

00:05:46.382438 IEM: wrmsr(0x122,0x0`00000003) -> #GP(0)
Yet another case of MSR 0x122, we need to find the common denominator.

'mga' can you try the latest of the 6.0.x series (6.0.14) instead/in addition to 6.0.6 suggested by 'doezel'?
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
fth0
Volunteer
Posts: 5661
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: Windows 10 update causing boot failure

Post by fth0 »

socratis wrote:Yet another case of MSR 0x122, we need to find the common denominator.
The common denominator is a quite new CPU having the IA32_TSX_CTRL MSR. See [ModEdit] above.
fth0
Volunteer
Posts: 5661
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: Windows 10 update causing boot failure

Post by fth0 »

fth0 wrote:The common denominator is a quite new CPU having the IA32_TSX_CTRL MSR.
I'll have to refine my statement a little bit: The IA32_TSX_CTRL MSR is being added by a quite new CPU microcode update to quite new CPUs. I don't know the exact date of the microcode update, but CVE-2019-11135 has been disclosed about 4 weeks ago.
socratis
Site Moderator
Posts: 27330
Joined: 22. Oct 2010, 11:03
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Win(*>98), Linux*, OSX>10.5
Location: Greece

Re: Windows 10 update causing boot failure

Post by socratis »

Instead of playing ping-pong between three threads, I merged all three of them. In summary, this is what I see as a common denominator, if you have any other ideas let me know. I've also highlighted the differences in the CPU:

sgadsby
00:00:01.517355 VirtualBox VM 6.0.14 r133895 win.amd64 (Oct 10 2019 19:44:31) release log
00:00:01.517362 OS Product: Windows 10
00:00:01.517363 OS Release: 10.0.18363
00:00:01.645496 Guest OS type: 'Windows10_64'
00:00:02.208314 Full Name:   "Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz"
00:05:46.382438 IEM: wrmsr(0x122,0x0`00000003) -> #GP(0)
doezel
00:00:01.714972 VirtualBox VM 6.1.0 r135406 win.amd64 (Dec 10 2019 14:34:16) release log
00:00:01.714978 OS Product: Windows 10
00:00:01.714979 OS Release: 10.0.18363
00:00:01.839993 Guest OS type: 'Windows10_64'
00:00:02.361270 Full Name:   "Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz"
00:00:16.405986 IEM: wrmsr(0x122,0x0`00000003) -> #GP(0)
VBoxxy
00:00:01.610613 VirtualBox VM 6.1.0 r135406 win.amd64 (Dec 10 2019 14:34:16) release log
00:00:01.610628 OS Product: Windows 10
00:00:01.610629 OS Release: 10.0.18363
00:00:01.774836 Guest OS type: 'Windows10_64'
00:00:02.451294 Full Name:   "Intel(R) Xeon(R) E-2286M  CPU @ 2.40GHz"
00:00:07.800331 IEM: wrmsr(0x122,0x0`00000003) -> #GP(0)
I need to know if 'VBoxxy' and 'doezel' can repeat the problem with 6.0.14. I don't need the logs, just look for the key entry "wrmsr(0x122,0x0`00000003)".
 Edit: While editing 'doezel' sent me a PM about VirtualBox 6.0.14 failing. See above... 
If it's in the logs, then I would like *all three reporters* to start downgrading your VirtualBox until you reach 6.0.6, that according to 'doezel' works. If this is something that's happening in the 6.0.x series, we'd like to know when *exactly* it started happening, the devs will love you... ;)

Also, of paramount importance, will be if the "2019-11 Cumulative Update for Windows 10 Version 1909 for x64-based Systems (KB4524570)" has been installed on the Guests, and what happens if you remove/uninstall it.
fth0 wrote:The IA32_TSX_CTRL MSR (0x122) is only available in some quite new Intel CPUs
If downgrading VirtualBox or uninstalling a Win update fixes it, then this is something that needs to be addressed in VirtualBox (methinks).
fth0 wrote:@socratis: A VirtualBox developer should be notified about this thread. The topic would be TSX Asynchronous Abort mitigation.
1) The developers will be notified tomorrow Monday, 2) I don't want to change the individual posts' titles, because we'd lose track who came from which of the three merged threads. I only changed the 1st post's title to include the MSR info...

So... conditions that seem to be required so far:
  1. A Windows 10 (10.0.18363) Host.
  2. A Win10-64 guest.
  3. A new Intel CPU. 2019 and later?
  4. 6.0.6 < VirtualBox_version < 6.0.14/6.1.0
Further investigation needed to determine:
  1. Which exact VirtualBox_version this started happening with.
  2. If KB4524570 has a role to play.
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
socratis
Site Moderator
Posts: 27330
Joined: 22. Oct 2010, 11:03
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Win(*>98), Linux*, OSX>10.5
Location: Greece

Re: Windows 10 update causing boot failure. wrmsr(0x122,0x0`00000003)

Post by socratis »

fth0 wrote:... but CVE-2019-11135 has been disclosed about 4 weeks ago.
And that's one of the CVEs that Microsoft's KB4524570 targets specifically. Hmm...
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
mga
Posts: 6
Joined: 28. Apr 2019, 08:05

Re: Windows 10 update causing boot failure

Post by mga »

Hello,

I 've made some tests. Hope it can help.
ON HP Zbook (Intel(R) Core(TM) i7-4700MQ CPU @ 2.40GHz ) : No problem

ON Lenovo thinkpad with Processor : Intel(R) Core(TM) i7-9850H CPU @ 2.60GHz problems

Methodology of test:
tests are running within a clean install of w10 1909
- start VM
- Dowload and KB4524570 from Catalog microsoft update.
- Snaptshot
- Install KB4524570
- if fails,
- Restore snapchot
- restart VM
- shutdown vm
- go tho next VB Version.

6.1 : Crash
6.0.14 : crash
6.0.12 : crash
6.0.10 : crash
6.0.8 : crash
6.0.6 : no crash, vm is booting normally.
VBoxxy
Posts: 7
Joined: 12. Dec 2019, 19:35

Re: Windows 10 update causing boot failure. wrmsr(0x122,0x0`00000003)

Post by VBoxxy »

Hello Team,

GREAT JOB, the hint to downgrade to Version 6.0.6 fix my Problem!

My Windows 10 Guests WORKS!!!

I you need any additional Information, Logging, ... please Contact me.

Merry Chrismal for all.
fth0
Volunteer
Posts: 5661
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: Windows 10 update causing boot failure. wrmsr(0x122,0x0`00000003)

Post by fth0 »

socratis wrote:
fth0 wrote:@socratis: A VirtualBox developer should be notified about this thread. The topic would be TSX Asynchronous Abort mitigation.
I don't want to change the individual posts' titles
Ah, I see now that my wording gives room to interpretation: I didn't mean to suggest a topic for this thread, but a topic for the notification (as a direct hint to the background of the problem).

At the moment I cannot tell if the problem is rather in VirtualBox or in the guest OSs. The Intel microcode update is either from 2019-09-18 or newer.

Edit: The Intel microcode update adding the IA32_TSX_CTRL MSR (0x122) is from 2019-11-12.
Last edited by fth0 on 15. Dec 2019, 15:24, edited 1 time in total.
VBoxxy
Posts: 7
Joined: 12. Dec 2019, 19:35

Re: Windows 10 update causing boot failure. wrmsr(0x122,0x0`00000003)

Post by VBoxxy »

Hello Team,

some additional Information:

I install the same Windows-Updates on both machines and use the Same Version of Virtualbox (6.0.14)

On the PC with NO Bios Update, VirtualBox work well.

Only the Bios Update with the following changes produce the Problem:
Dell Precision 7540 and 7740 System BIOS 1.5.1

This package contains the Dell system BIOS update. BIOS is a firmware that is embedded on a small memory chip on the system board. It controls the keyboard, monitor, disk drives, and other devices. This update addresses the Intel Security Advisories INTEL-SA-00219, INTEL-SA-00220, INTEL-SA-00270, and INTEL-SA-00241. A security advisory is a statement when a product is impacted by a security vulnerability and a remedy is available.

Fixes & Enhancements
Fixes:
  • Fixed the issue where the Ethernet disconnects when the system is connected to a dock.
  • Firmware updates to address Intel security advisories INTEL-SA-00219 (CVE-2019-0117), INTEL-SA-00220 (CVE-2019-0123), and INTEL-SA-00270 (CVE-2019-11135) [ModEdit; emphasis added]
  • Firmware updates to address security advisory INTEL-SA-00241 (CVE-2019-0169, CVE-2019-11105, CVE-2019-11104, CVE-2019-11103, CVE-2019-11090, CVE-2019-0165, CVE-2019-0168, CVE-2019-11087, CVE-2019-11101, CVE-2019-11106, CVE-2019-11108, CVE-2019-11147, CVE-2019-11088, CVE-2019-0131, CVE-2019-0166, CVE-2019-11100, CVE-2019-11107, CVE-2019-11086, CVE-2019-11132, and CVE-2019-11131)
  • Firmware updates to address Intel security advisories INTEL-SA-00240 (CVE-2019-0151) and INTEL-SA-00164 (CVE-2019-0184)
  • Firmware updates to address Intel security advisories INTEL-SA-00220 (CVE-2019-0124), INTEL-SA-00260 (CVE-2019-0154), and INTEL-SA-00254 (CVE-2019-0185)
  • Fixed a BIOS Setup configuration issue that occurs after clearing the CMOS.
  • Fixed an issue with preboot TPM detection and error logging.
  • Removed the IpSec driver and application.
Enhancements:
  • Added an enhancement to extend the BIOS password configuration measurements into the TPM.
  • This update integrates the BIOSConnect feature into Dell SupportAssist OS Recovery. It connects the system to the Dell image server to download and recover the operating system.
  • Updated the Embedded Controller Engine firmware.
  • Modified battery algorithm to prolong lifespan and minimize risk of swelling.
  • Enhanced device protection in Windows Security.
  • Updated the Intel Management Engine firmware.
  • Enhanced the High Dynamic Range (HDR) feature.
Best regards
doezel
Posts: 7
Joined: 12. Dec 2019, 21:42

Re: Windows 10 update causing boot failure. wrmsr(0x122,0x0`00000003)

Post by doezel »

Same conclusion here as 'mga':

Version 6.0.6 is the latest version where the win10 image boots. Every later version just hangs (seamingly) the same way.
sgadsby
Posts: 10
Joined: 28. Nov 2019, 09:55

Re: Windows 10 update causing boot failure. wrmsr(0x122,0x0`00000003)

Post by sgadsby »

FYI the December cumulative update (KB4530684) triggers the same hang symptom as the November cumulative update (KB4524570).

My machine is an HP Elitebook x360 and the CPU is affected by the TSX issue. Microsoft provide a mechanism for disabling TSX:
> https://support.microsoft.com/en-us/hel ... extensions

I have tried setting DisableTsx on the host to no effect. I also tried setting it in the guest VM before attempting the update but no effect. I applied a November BIOS update from HP (R90, SP100156) that includes new microcode however no effect.
Locked