Diagnosing VirtualBox Hardening Issues

[mpack]

Back in 4.3.14 VirtualBox introduced a new security feature on Windows Hosts: HARDENING.

This was introduced to guard against the possibility that malware running on the HOST could inject a DLL¹ into a VM which could then run at elevated privileges (hiding behind Oracle certification) which would be denied to it if it were to run stand alone. The technology press wrote scare stores about this possibility, and with the reputation of Oracle and VirtualBox at stake the problem could not be ignored. Note that the concern is that this injected DLL could use elevated privileges to attack the HOST, we aren't merely talking guest OS malware here.

¹ DLL == Dynamic Link Library, an executable file which is loaded and linked at runtime and runs in the context of a parent program. The Windows OS makes extensive use of DLLs, just look at the \Windows\System32 folder some time! Injecting a DLL means to trick a process into loading a "foreign" library module that it wouldn't ordinarily have loaded.

[mpack]

4.3.14 made a first stab at dealing with the problem by forbidding all non-VirtualBox non-System DLLs from running as part of the VirtualBox program. That might sound reasonable, after all injecting a DLL sounds like a shady thing to do... but in fact it turns out that LOTs of legitimate Windows apps need to install DLLs that run in other processes, including :-

• Graphics drivers (e.g. from NVidia) may inject a DLL which filters graphics function calls to the host OS, redirecting them from software to hardware accelerated versions. VirtualBox VMs use OpenGL, which is precisely the main API that most benefits from this.
  
• Unofficial Windows Theme providers often use hacked versions of the Windows theme DLL (uxtheme.dll).
  
• Accessibility tools designed to help partially sighted people sometimes inject themselves into standard apps in order to speak the text, force high contrast colors etc.
  
• Antivirus software is invasive by its very nature, with behaviour hard to distinguish from malware.
  
• Many others.

Later VirtualBox versions relaxed the rules:

(RULE1) "Foreign" DLLs can now run inside the VirtualBox process space, provided they are "signed", i.e. have a trusted provider certificate embedded in the software. Malware writers don't want to be identified, and bona fide signature owners don't want to have their certificates revoked, so this should be effective at excluding malware and spyware.

(RULE2) VirtualBox no longer "errors out" when an unsigned DLL is found. Instead the rule is that VirtualBox now merely discards the errant DLL and continues.

[mpack]

BEGINNING A DIAGNOSIS:

The first thing to do is to view the hardening log, stored as <VM folder>\Logs\VBoxHardening.log. This file can be viewed in any standard text editor, such as Notepad - or better yet Notepad++.

The very last line in the log reports an error code. If this error code is 0 then you don't have a hardening problem at all.

WHAT CAN CAUSE A HARDENING RELATED FAILURE?: The root cause of most hardening failures (see note) is no valid certificate found in a DLL which has been loaded into the VirtualBox execution space. No valid certificate can be due to one of four things (1) the DLL never had a valid certificate - e.g. malware, free software, or the vendor made a mistake, (2) the DLL has been hacked, invalidating the signature, (3) the Windows host certificates database has been damaged by host corruption or a buggy Windows update, (4) something, most likely your antivirus, is blocking VirtualBox's attempts to access the executable to check its certificate.

In the first two cases the fix is to exclude the DLL or replace it with one with a valid certificate.

If you suspect the third case then you need to ask Microsoft about how to check and repair the certificates database. For some reason Windows 7 hosts seem especially prone.

In the fourth error case the fix is to configure a host antivirus exclusion to allow VirtualBox to make its checks, or choose a different antivirus product (VirtualBox works best with the "Windows Defender" AV which is now built into Windows 10 - there is considerable doubt that Windows 10 gets any benefit at all from third party apps).

NOTE Recent (5.2.x ?) versions of VirtualBox may be producing errors with increased frequency for a new class of hardening problems (though perhaps it is user behaviour which has changed). This is signified by an error message "Thread not alone" being shown to the user, and "More than one thread in process" being recorded in the log. This message implies that a foreign program has successfully injected itself into the VirtualBox executable space and is now running, this success indicates that the associated DLLs presumably must have been correctly signed. Even so VirtualBox cannot allow this because a foreign program could conceivably be faking results to get past VirtualBox's hardening checks. The usual culprits behind this error are certain kinds of "web safe" internet browser filters or remote desktop applications. Those applications must be uninstalled, or VirtualBox must be added as an exception to their activities.

[mpack]

Error Symptom 1: Exit with error code 1

This seems to have several causes, but one of the main ones seems to be that some DLL that VirtualBox requires has failed to load. That probably indicates corruption somewhere. Sometimes you find an error related to USER32.DLL in the hardening log. This may be fixed by running "sfc /scannow" in an administrator command prompt window.

See also the "More than one thread in process" note in the previous post.

[mpack]

Error Symptom 2: Error code 0xC0000005

Error 0xC0000005 is what's known as an general protection fault or memory access error: the application tried to access an invalid memory address or jump to code which doesn't exist. Hence this is most likely a consequence of RULE2: a bad DLL got unloaded, but some other DLL is still trying to use it.

You can often narrow the cause if you look at which DLL faulted. Try disabling audio as a test, likewise try disabling 3D acceleration. Quite often you find that the bad memory access came from a graphics related DLL, which implies that the graphics provider omitted to sign one of their DLLs. You should check for updates for your graphics card drivers.

¹ If you find that the VM will not start in normal mode, but will start when run in headless mode, then that makes it pretty much certain that your problem is the graphics drivers. This test could be tried if disabling 3D acceleration had no effect.

[mpack]

Other things to try:

• Disable your real-time antivirus, at least temporarily, to test whether this is conflicting with VirtualBox. On Windows 10 hosts the Windows Defender AV is designed by people who know how to write low level code for Windows without fighting it or breaking it. It does not conflict with VirtualBox. You would perhaps be wise to stick to that.
  .
• Expanding on the previous point, some antivirus don't provide an effective "disable" function. To be absolutely sure it won't interfere you have to uninstall it completely and perform a full reboot of your host PC.
  .
• Make sure Hyper-v is disabled. Modern VirtualBox VMs usually can't run if Hyper-v is hogging the hardware resources required for virtualization. This may present itself first as a hardening error, so it's worth checking. Note that this advice may soon change, as VirtualBox v6.x adds experimental support for Windows 10 Hypervisor Platform.
  .
• Power down your VM (not suspended state) and restart.
  .
• Fully power down and then restart your host. Sometimes this shakes something loose.
  .
• Fully uninstall and then reinstall VirtualBox using "Run as administrator" to run the installer. Sometimes this replaces a corrupted DLL that was causing a problem, or changes the order that Windows calls on device drivers.

[mpack]

The following information is courtesy of user "Socratis". He has made a list of new features that Microsoft have added to Windows 10 which implicitly enable Hyper-v and therefore cause problems for VirtualBox. In an ideal world these should all be disabled, and please understand that Microsoft seems to be adding to this list all the time, and sometimes the same feature is given different names.

Here's the list that I have so far:

• Application Guard
• Credential Guard
• Device Guard
• <any> * Guard
• Containers
• Core Isolation
• Memory Integrity
• Virtualization Based Security
• Hyper-V
• Virtual Machine Platform
• Windows Hypervisor Platform
• Windows Sandbox
• Windows Subsystem for Linux 2 (WSL2) (WSL1 does not enable Hyper-v)

If Hyper-v cannot be disabled, e.g. due to company policy, then VirtualBox v6.x has a new ability to run via the Windows Hypervisor Platform, however Windows 10 (1809) or later is required, and it currently causes a significant drop in VM performance.

[mpack]

And finally...

If the above didn't help you solve the problem then ask a question about it in the Windows Hosts forum. Remember to attach a zipped copy of your VBoxHardening.log file.