Error (hardening error?) when running vboxheadless as scheduled task

Discussions related to using VirtualBox on Windows hosts.
Post Reply
adron
Posts: 2
Joined: 19. Sep 2017, 17:35

Error (hardening error?) when running vboxheadless as scheduled task

Post by adron »

Hi,
I have a problem running a virtual machine as a scheduled task. It works when I start the machine from the gui, and it works when i start the machine using vboxheadless from an interactive command prompt, but it fails when run from the scheduler. I think it is a hardening error. The log file says (full file attached):
09:41:31.366929 00:00:03.225213 00000000000001d8 EMT-0    supR3HardenedErrorV: supR3HardenedScreenImage/LdrLoadDll: rc=VERR_LDRVI_NOT_SIGNED fImage=1 fProtect=0x0 fAccess=0x0 \Device\HarddiskVolume2\Windows\System32\NetSetupShim.dll: Not signed.
09:41:31.367906 00:00:03.225569 00000000000001d8 EMT-0    supR3HardenedErrorV: supR3HardenedMonitor_LdrLoadDll: rejecting 'C:\Windows\System32\NetSetupShim.dll' (C:\Windows\System32\NetSetupShim.dll): rcNt=0xc0000190
09:41:31.367906 00:00:03.225702 00000000000001d8 EMT-0    NetworkAttachmentType_Bridged: Failed to get NetCfg, hrc=ERROR_TRUST_FAILURE 0x800706FE (0x800706fe)
09:41:31.367906 00:00:03.226112 00000000000001d8 EMT-0    AssertLogRel F:\tinderbox\win-5.1\src\VBox\Main\src-client\ConsoleImpl2.cpp(5063) int __cdecl Console::i_configNetwork(const char *,unsigned int,unsigned int,struct INetworkAdapter *,struct CFGMNODE *,struct CFGMNODE *,struct CFGMNODE *,bool,bool): !FAILED(hrc)
09:41:31.367906 00:00:03.226117 00000000000001d8 EMT-0    hrc=ERROR_TRUST_FAILURE 0x800706FE
09:41:31.494548 00:00:03.352343 00000000000012ec VMPwrUp  VMSetError: F:\tinderbox\win-5.1\src\VBox\VMM\VMMR3\VM.cpp(363) int __cdecl VMR3Create(unsigned int,const struct VMM2USERMETHODS *,void (__cdecl *)(struct UVM *,void *,int,const char *,unsigned int,const char *,const char *,char *),void *,int (__cdecl *)(struct UVM *,struct VM *,void *),void *,struct VM **,struct UVM **); rc=VERR_MAIN_CONFIG_CONSTRUCTOR_COM_ERROR
09:41:31.494548 00:00:03.352350 00000000000012ec VMPwrUp  VMSetError: The configuration constructor in main failed due to a COM error. Check the release log of the VM for further details.
There is no VBoxHardening.log created in the directory of the virtual machine when I start it using vboxheadless from the task scheduler. However, when I run the virtual machine from the gui (which is successful) I do get a VBoxHardening.log. I am attaching that log file. But since it starts successfully, I am guessing that it is not truly relevant. How can I make vboxheadless write a VBoxHardening.log ? What should I do to troubleshoot?
Attachments
VBoxHardening.zip
(32.67 KiB) Downloaded 12 times
VBox.log
(6.48 KiB) Downloaded 10 times
adron
Posts: 2
Joined: 19. Sep 2017, 17:35

Re: Error (hardening error?) when running vboxheadless as scheduled task

Post by adron »

After studying the source codes some, I discovered the --sup-hardening-log command line argument. Attached, find the hardening log file from a failed launch, and the part about NetSetupShim.dll inline:
12f0.3e6c: supR3HardNtViCallWinVerifyTrustCatFile: hFile=0000000000000934 pwszName=\Device\HarddiskVolume2\Windows\System32\NetSetupShim.dll
12f0.3e6c: supR3HardNtViCallWinVerifyTrustCatFile: Cached context 00000000010c8990
12f0.3e6c: supR3HardNtViCallWinVerifyTrustCatFile: hCatAdmin=00000000010c8990
12f0.3e6c: supR3HardNtViCallWinVerifyTrustCatFile: cbHash=20 wszDigest=1E5A9ACAE97AEA2587277AEA0A8C325D8569A5A4
12f0.3e6c: supR3HardNtViCallWinVerifyTrustCatFile: Retrying with fresh context (CryptCATAdminEnumCatalogFromHash -> 1062; iCat=0x0)
12f0.3e6c: supR3HardNtViCallWinVerifyTrustCatFile: New context 00000000010c94d0
12f0.3e6c: supR3HardNtViCallWinVerifyTrustCatFile: hCatAdmin=00000000010c94d0
12f0.3e6c: supR3HardNtViCallWinVerifyTrustCatFile: cbHash=20 wszDigest=1E5A9ACAE97AEA2587277AEA0A8C325D8569A5A4
12f0.3e6c: supR3HardNtViCallWinVerifyTrustCatFile: CryptCATAdminEnumCatalogFromHash failed ERRROR_NOT_FOUND (1062)
12f0.3e6c: supR3HardNtViCallWinVerifyTrustCatFile: Cached context 00000000010c9590
12f0.3e6c: supR3HardNtViCallWinVerifyTrustCatFile: hCatAdmin=00000000010c9590
12f0.3e6c: supR3HardNtViCallWinVerifyTrustCatFile: cbHash=32 wszDigest=FF796AF91D296D8B355235EED4F0C35900918A0D18CBD714CF333209B44374FF
12f0.3e6c: supR3HardNtViCallWinVerifyTrustCatFile: Retrying with fresh context (CryptCATAdminEnumCatalogFromHash -> 1062; iCat=0x0)
12f0.3e6c: supR3HardNtViCallWinVerifyTrustCatFile: New context 00000000010c8b10
12f0.3e6c: supR3HardNtViCallWinVerifyTrustCatFile: hCatAdmin=00000000010c8b10
12f0.3e6c: supR3HardNtViCallWinVerifyTrustCatFile: cbHash=32 wszDigest=FF796AF91D296D8B355235EED4F0C35900918A0D18CBD714CF333209B44374FF
12f0.3e6c: supR3HardNtViCallWinVerifyTrustCatFile: CryptCATAdminEnumCatalogFromHash failed ERRROR_NOT_FOUND (1062)
12f0.3e6c: supR3HardNtViCallWinVerifyTrustCatFile -> -22900 (org 22900)
12f0.3e6c: supHardenedWinVerifyImageByHandle: -> -22900 (\Device\HarddiskVolume2\Windows\System32\NetSetupShim.dll) WinVerifyTrust
12f0.3e6c: Error (rc=0):
12f0.3e6c: supR3HardenedScreenImage/LdrLoadDll: rc=Unknown Status -22900 (0xffffa68c) fImage=1 fProtect=0x0 fAccess=0x0 \Device\HarddiskVolume2\Windows\System32\NetSetupShim.dll: Not signed.
12f0.3e6c: supR3HardenedWinVerifyCacheInsert: \Device\HarddiskVolume2\Windows\System32\NetSetupShim.dll
12f0.3e6c: Error (rc=0):
12f0.3e6c: supR3HardenedMonitor_LdrLoadDll: rejecting 'C:\Windows\System32\NetSetupShim.dll' (C:\Windows\System32\NetSetupShim.dll): rcNt=0xc0000190
12f0.3e6c: supR3HardenedMonitor_LdrLoadDll: returns rcNt=0xc0000190 'C:\Windows\System32\NetSetupShim.dll'
Attachments
hardening.zip
(21.64 KiB) Downloaded 11 times
ams_tschoening
Posts: 24
Joined: 24. May 2017, 16:30

Re: Error (hardening error?) when running vboxheadless as scheduled task

Post by ams_tschoening »

Did you ever find a solution/workaround for this problem? I'm running into the same issue when trying to run a headless VM using a default restricted user account. Making the same user be a member of the ADMINISTRATORS group or even without that, but logging it on interactively or using a shell, things succeed. Did you have a similar setup?

It seems that for some reason the enumeration of hashes/digests fails for "NetSetupShim.dll" when using a restricted, non-interactive user, even though the logs of a succeeding execution clearly show the same hash and digest in both cases. So the the file itself is not the problem and Windows catalogs contain signatures for that, otherwise execution would always fail. The following is simply how a succeeding execution looks like for that file:

Code: Select all

supR3HardNtViCallWinVerifyTrustCatFile: hFile=0000000000000930 pwszName=\Device\HarddiskVolume4\Windows\System32\NetSetupShim.dll
supR3HardNtViCallWinVerifyTrustCatFile: Cached context 0000000001433810
supR3HardNtViCallWinVerifyTrustCatFile: hCatAdmin=0000000001433810
supR3HardNtViCallWinVerifyTrustCatFile: cbHash=20 wszDigest=592E7D18568150098B2F131AD72F2156D1CA3A58
Compared to a failing one, while the important thing is that the first attempted hash/digest is exactly the same like before:

Code: Select all

supR3HardNtViCallWinVerifyTrustCatFile: hFile=0000000000000808 pwszName=\Device\HarddiskVolume4\Windows\System32\NetSetupShim.dll
supR3HardNtViCallWinVerifyTrustCatFile: Cached context 00000000019efab0
supR3HardNtViCallWinVerifyTrustCatFile: hCatAdmin=00000000019efab0
supR3HardNtViCallWinVerifyTrustCatFile: cbHash=20 wszDigest=592E7D18568150098B2F131AD72F2156D1CA3A58
supR3HardNtViCallWinVerifyTrustCatFile: Retrying with fresh context (CryptCATAdminEnumCatalogFromHash -> 1062; iCat=0x0)
supR3HardNtViCallWinVerifyTrustCatFile: New context 00000000019ef030
supR3HardNtViCallWinVerifyTrustCatFile: hCatAdmin=00000000019ef030
supR3HardNtViCallWinVerifyTrustCatFile: cbHash=20 wszDigest=592E7D18568150098B2F131AD72F2156D1CA3A58
supR3HardNtViCallWinVerifyTrustCatFile: CryptCATAdminEnumCatalogFromHash failed ERROR_NOT_FOUND (1062)
supR3HardNtViCallWinVerifyTrustCatFile: Cached context 00000000019eef70
supR3HardNtViCallWinVerifyTrustCatFile: hCatAdmin=00000000019eef70
supR3HardNtViCallWinVerifyTrustCatFile: cbHash=32 wszDigest=668C2310EFB19B6732352E1B4C6B047E3037FC14D9878DA0CC690CFA6D37CE20
supR3HardNtViCallWinVerifyTrustCatFile: Retrying with fresh context (CryptCATAdminEnumCatalogFromHash -> 1062; iCat=0x0)
supR3HardNtViCallWinVerifyTrustCatFile: New context 00000000019efab0
supR3HardNtViCallWinVerifyTrustCatFile: hCatAdmin=00000000019efab0
supR3HardNtViCallWinVerifyTrustCatFile: cbHash=32 wszDigest=668C2310EFB19B6732352E1B4C6B047E3037FC14D9878DA0CC690CFA6D37CE20
supR3HardNtViCallWinVerifyTrustCatFile: CryptCATAdminEnumCatalogFromHash failed ERROR_NOT_FOUND (1062)
supR3HardNtViCallWinVerifyTrustCatFile -> -22900 (org 22900)
So I guess the problem is hidden in the error message:

Code: Select all

supR3HardNtViCallWinVerifyTrustCatFile: Retrying with fresh context (CryptCATAdminEnumCatalogFromHash -> 1062; iCat=0x0)
supR3HardNtViCallWinVerifyTrustCatFile: CryptCATAdminEnumCatalogFromHash failed ERROR_NOT_FOUND (1062)
According to MSDN, "1062" might be the following:

Code: Select all

ERROR_SERVICE_NOT_ACTIVE
1062 (0x426)
The service has not been started.
Windows starts a lot of services, components etc. on demand for interactive users, so that might make sense at all. Though, I don't know yet which service it is missing, so that one might check permissions or alike.
fth0
Volunteer
Posts: 5668
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: Error (hardening error?) when running vboxheadless as scheduled task

Post by fth0 »

To differentiate between a timing problem and other possible causes, could you let your scheduled task wait for a minute before starting the VM?
ams_tschoening
Posts: 24
Joined: 24. May 2017, 16:30

Re: Error (hardening error?) when running vboxheadless as scheduled task

Post by ams_tschoening »

That's already the case by default when scheduling tasks at startup in Windows, it's actually even 5 minutes. Things as well simply work how they are configured depending on if the user is part of the ADMINISTRATORS group or not, so any timing problem is pretty unlikely. It's about permissions, which services are started automatically for which users/groups under which circumstances and stuff like that. I reported a similar problem with COM components of VirtualBox already, but event viewer at least told about which components are the problem. Those components were only allowed to be started for/by SYSTEM, ADMINISTRATORS and interactive users by default, which didn't apply for my special non-admin user and task scheduler. After adding my user, I made another step forward up until the problem I have now. It's most likely a comparable issue somewhere, I only need to find where... :-/
fth0
Volunteer
Posts: 5668
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: Error (hardening error?) when running vboxheadless as scheduled task

Post by fth0 »

Ok, simply forget what I wrote. I thought I had read a timing related question from you, but cannot find it any more. Additionally, I had not read your mailing list posts before answering. With all those posts in no clear order (perhaps because of your posting problem on the mailing list), it's a little bit difficult to see what problem is left ...
ams_tschoening
Posts: 24
Joined: 24. May 2017, 16:30

Re: Error (hardening error?) when running vboxheadless as scheduled task

Post by ams_tschoening »

The posts on the mailing list have the correct order, which is the one in which I triggered the described problems. Though, there are somewhat independent from each other as well and the last mail is covering the same topic like what this thread is about.
fth0
Volunteer
Posts: 5668
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: Error (hardening error?) when running vboxheadless as scheduled task

Post by fth0 »

Regarding the Windows services, perhaps you could use the sc command to query a list of running services in the non-interactive and the interactive setup, make good guesses and disable services in the interactive setup. Alternatively, you could try to debug with WinDbg. Admittedly, both are not the preferred methods of choice.

Regarding the disappearing of the running VM, you could read about the role of VBoxSVC in 10.2. Oracle VM VirtualBox Executables and Components. There are corresponding log files in the global configuration directory C:\Users\<username>\.VirtualBox. Perhaps they indicate what happens.
ams_tschoening
Posts: 24
Joined: 24. May 2017, 16:30

Re: Error (hardening error?) when running vboxheadless as scheduled task

Post by ams_tschoening »

I've decided to file a bug about this problem and here is what I think is going on:

The signature verification used by HARDENING relies on some COM-component and default security settings of those only allow SYSTEM, ADMINISTRATORS and INTERACTIVE to activate those. I have documented that in the related ticket #20340 already. When using task scheduler with my restricted user it's no member of any of these groups, therefore necessary components are not activated and signatures can't be checked. This changes instantly when using the same command line etc. with the same user with an interactive shell.

While that could be argued as a limitation of Windows default settings, the problem in my opinion in this case is that HARDENING is requiring that signature verification, not Windows, while without it things would simply work. So to make HARDENING succeed in this case, I would need to make my restricted user being a member of ADMINISTRATORS, which obviously doesn't make too much sense from a security perspective: Providing far more permissions than absolutely necessary to make an additional security(!) check succeed first. :-)

Like discussed in #20340, this might be worked around by creating an additional group and providing necessary permissions on the COM component of interest to that group. The problem is that I couldn't find the exact component yet... :-/

https://stackoverflow.com/questions/673 ... 1062-for-n
https://docs.microsoft.com/en-us/answer ... -retu.html

Another workaround would be to make HARDENING optionally being disabled, it harms in this case, or at least make it more tolerant and ignore some of those errors under some circumstances. As said, the mentioned DLLs are all Windows default DLLs, unchanged, which otherwise easily pass signature verification. This is especially worth to be considered because the error about concrete "NetSetupShim.dll" seems to only occur if the VM uses bridged networking, with e.g. NAT that DLL doesn't seem to be used at all and doesn't occur in the logs of HARDENING. That made the VM start in my tests, even though the root cause of not being able to verify some DLLs was still the same. So even though the VM started, a lot of HARDENING errors about failed verification have still been logged but seemed to be ignored or no actual code was triggered of those DLLs or whatever.

Of course one can't configure the VM to work around limitations of HARDENING, it's more likely to use ADMIN-users instead, which makes this whole security check a bit pointless. :-)
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: Error (hardening error?) when running vboxheadless as scheduled task

Post by scottgus1 »

I cannot personally suggest a fix to the autostart issues. But if you need to get the host running I can suggest a work-around:

Auto-logon the host to the regular account that will run the VMs, then have that account run a script under the account's normal settings which starts the VMs. Have another script that locks the desktop, leaving it logged in but secured from access.

FWIW the devs have been asked for no-hardening Virtualbox, and they have said no they won't provide a switch or pre-built installer with hardening turned off. Hardening actually fills up a pretty big security hole on Windows. Like all security solutions there are bugs, which it looks like you may have found one.
There is apparently a way to build Virtualbox from source code with the hardening disabled, I don't know anything about doing that.

FWIW2 it seems to me that this sort of thing would have got caught early on, since Virtualbox also is the base for Oracle's pay-for hypervisors, and there has to be a paying customer that wants auto-start VMs. Could be a new bug. But, Windows being what it is, it could be a local condition on your host too. For a full troubleshooting test, see if you can switch in a different host drive and fresh-install Windows and Virtualbox, then try auto-start on the fresh OS.
ams_tschoening
Posts: 24
Joined: 24. May 2017, 16:30

Re: Error (hardening error?) when running vboxheadless as scheduled task

Post by ams_tschoening »

scottgus1 wrote: FWIW the devs have been asked for no-hardening Virtualbox, and they have said no they won't provide a switch or pre-built installer with hardening turned off.[...]
That's why people need to collect arguments over and over again to reconsider that decision. It doesn't make any sense to argument with security when things simply don't work in a secured context.
scottgus1 wrote: FWIW2 it seems to me that this sort of thing would have got caught early on, since Virtualbox also is the base for Oracle's pay-for hypervisors, and there has to be a paying customer that wants auto-start VMs.
This problem is Windows only and if at all Windows+VBox are most likely used client side only, which makes it less necessary to have a setup like mine. Instead, my setup is pretty common on non-Windows and simply works there, for myself with Ubuntu 16.04 as well. Which makes sense because non-Windows doesn't have COM, its security restrictions and HARDENING simply works a bit different there. So it's very likely that Windows-users like me simply didn't trigger that problem or didn't investigate it too deeply yet. The size of Oracle doesn't mean anything, VBox has a lot of bugs and even customers of Oracle have a lot of problems with that company and their products in general. Even this forum is full of reported problems regarding to HARDENING in general.
scottgus1 wrote: Could be a new bug. But, Windows being what it is, it could be a local condition on your host too. For a full troubleshooting test, see if you can switch in a different host drive and fresh-install Windows and Virtualbox, then try auto-start on the fresh OS.
That doesn't make too much sense, because 1. the thread wasn't started by me, but someone else years ago running into the exact same problem. And 2. I already tested on multiple different Windows, even different versions with one being Windows 10 Prof and some Windows Server 2019. Looking at what I've found especially regarding COM security, there isn't much room for a broken Windows itself anyway.
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: Error (hardening error?) when running vboxheadless as scheduled task

Post by scottgus1 »

ams_tschoening wrote:1. the thread wasn't started by me, but someone else years ago running into the exact same problem.
I'm not sure why both of you having the same problem on your host OS's is not possible. A Windows update or the same 3rd-party software may have caused it? A recent post with multiple contributors had issues with Host-Only arising from Commodo being installed.
ams_tschoening wrote:And 2. I already tested on multiple different Windows, even different versions with one being Windows 10 Prof and some Windows Server 2019
Ah, yes, the good ol' "I already did that", that was NOT mentioned in an earlier post, so that the unpaid volunteer helpers wouldn't waste their time suggesting something the poster had already tried...

Please tell us the full story next time?
ams_tschoening wrote:That's why people need to collect arguments over and over again to reconsider that decision.
Feel free to post an Enhancement on the Bugtracker.

Building Windows Virtualbox without hardening:
https://www.virtualbox.org/wiki/Windows ... structions
ams_tschoening wrote:this forum is full of reported problems regarding to HARDENING in general.
Context required.
When a 3rd-party DLL injector doesn't sign their files according to industry-standard security practices, hardening will trigger on it, because it could be malware. Or the 3rd-party antivirus hacks core system files in the exact same way malware does. Thus the vastly vast majority of the hardening topics. Get the file signed properly, or make the AV stop hacking, and the problem goes away. Or a reinstall of both items allows coexistence. Forum gurus have helped a lot of these hardening problems get fixed.

I'm not denying you have discovered a problem. I offered a suggestion that has turned out to have been the correct suggestion for many other posters: their host OS was damaged and a fresh install fixed the problem. You did not tell the entire story, thus my suggestion to you.

You've done a lot of deep digging! I hope the bug you found will get fixed! :)
Post Reply