[Solved] Debug Build without hardening successful but runtime error

Discussions related to using the OSE version of VirtualBox.
Post Reply
bkth
Posts: 2
Joined: 8. Sep 2017, 20:35

[Solved] Debug Build without hardening successful but runtime error

Post by bkth »

Hi all,

I am trying to have a Windows debug build of VirtualBox, after following the guide and troubleshooting my way to a non-failing build, I am presented with the following error when trying to run any commands:

Code: Select all

E:\VirtualBox-5.1.26\out\win.amd64\debug\bin>VBoxManage.exe list vms
VBoxManage.exe: error: Failed to create the VirtualBox object!
VBoxManage.exe: error: Completely failed to instantiate CLSID_VirtualBox: CO_E_SERVER_EXEC_FAILURE
VBoxManage.exe: error: Details: code CO_E_SERVER_EXEC_FAILURE (0x80080005), component VirtualBoxClientWrap, interface IVirtualBoxClient
Edit: I run this command from a normal prompt, not an admin one.

Here are the steps I used to build

From a normal prompt:

Code: Select all

E:\VirtualBox-5.1.26>kmk BUILD_TYPE=debug
E:/VirtualBox-5.1.26/src/VBox/Main/webservice/Makefile.kmk:105: VBOX_PATH_GSOAP not found...
kBuild: Pass - Build Programs
kBuild: Pass - Libraries
kBuild: Pass - DLLs
kBuild: Pass - Programs
kBuild: Pass - Other Stuff
kBuild: Pass - Staging
From an admin prompt:

Code: Select all

E:\VirtualBox-5.1.26\out\win.amd64\debug\bin>comregister.cmd
s64BitProxyStub=VBoxProxyStub.dll

E:\VirtualBox-5.1.26\out\win.amd64\debug\bin>"E:\VirtualBox-5.1.26\out\win.amd64\debug\bin\VBoxSVC.exe" /UnregServer

E:\VirtualBox-5.1.26\out\win.amd64\debug\bin>C:\WINDOWS\system32\regsvr32 /s /u "E:\VirtualBox-5.1.26\out\win.amd64\debug\bin\VBoxC.dll"

E:\VirtualBox-5.1.26\out\win.amd64\debug\bin>C:\WINDOWS\syswow64\regsvr32 /s /u "E:\VirtualBox-5.1.26\out\win.amd64\debug\bin\x86\VBoxClient-x86.dll"

E:\VirtualBox-5.1.26\out\win.amd64\debug\bin>C:\WINDOWS\system32\regsvr32 /s /u "E:\VirtualBox-5.1.26\out\win.amd64\debug\bin\VBoxProxyStub.dll"

E:\VirtualBox-5.1.26\out\win.amd64\debug\bin>C:\WINDOWS\syswow64\regsvr32 /s /u "E:\VirtualBox-5.1.26\out\win.amd64\debug\bin\x86\VBoxProxyStub-x86.dll"

E:\VirtualBox-5.1.26\out\win.amd64\debug\bin>if 0 == 1 goto end

E:\VirtualBox-5.1.26\out\win.amd64\debug\bin>"E:\VirtualBox-5.1.26\out\win.amd64\debug\bin\VBoxSVC.exe" /RegServer

E:\VirtualBox-5.1.26\out\win.amd64\debug\bin>C:\WINDOWS\system32\regsvr32 /s    "E:\VirtualBox-5.1.26\out\win.amd64\debug\bin\VBoxC.dll"

E:\VirtualBox-5.1.26\out\win.amd64\debug\bin>C:\WINDOWS\syswow64\regsvr32 /s    "E:\VirtualBox-5.1.26\out\win.amd64\debug\bin\x86\VBoxClient-x86.dll"

E:\VirtualBox-5.1.26\out\win.amd64\debug\bin>if 0 == 1 goto end

E:\VirtualBox-5.1.26\out\win.amd64\debug\bin>if exist "E:\VirtualBox-5.1.26\out\win.amd64\debug\bin\VBoxProxyStub.dll" C:\WINDOWS\system32\regsvr32 /s "E:\VirtualBox-5.1.26\out\win.amd64\debug\bin\VBoxProxyStub.dll"

E:\VirtualBox-5.1.26\out\win.amd64\debug\bin>if exist "E:\VirtualBox-5.1.26\out\win.amd64\debug\bin\x86\VBoxProxyStub-x86.dll" C:\WINDOWS\syswow64\regsvr32 /s "E:\VirtualBox-5.1.26\out\win.amd64\debug\bin\x86\VBoxProxyStub-x86.dll"

Code: Select all

E:\VirtualBox-5.1.26\out\win.amd64\debug\bin>loadall.cmd
[...]
installed successfully
+ echo loadall.sh: Successfully installed all drivers
loadall.sh: Successfully installed all drivers
+ exit 0
full dump at pastebin com Z8d6tAc4 (I am not allowed to post URLs)

For good measure here is how I run cscript configure.vbs

Code: Select all

cscript configure.vbs --with-MinGW-w64=E:\mingw64 --with-libSDL=E:\SDL-1.2.15 --with-openssl=C:\OpenSSL-Win64 --with-openssl32=C:\OpenSSL-Win32 --with-libcurl=E:\libcurl-vc10-x64-release-dll-ipv6-sspi-winssl --with-libcurl32=E:\libcurl-vc10-x86-release-dll-ipv6-sspi-winssl --with-python=C:\Python27
then the content of my LocalConfig.kmk

Code: Select all

VBOX_WITHOUT_HARDENING=1
VBOX_SIGNING_MODE=test
VBOX_PATH_SIGN_TOOLS=C:\Program Files (x86)\Windows Kits\10\bin\x64
VBOX_INF2CAT="C:\Program Files (x86)\Windows Kits\10\bin\x86\Inf2cat.exe"
The problem persists across restarts, i.e. I tried to do a clean build by removing the debug dir, rebuilding, then after executing comregister.cmd and loadall.cmd, reboot but that did not change anything.

I also experience weird situation where at one point I was having the error, then I just kept retrying the command (I usually use a pretty harmless one like VBoxManage.exe list vms) and it worked once all of a sudden before failing to work anymore.

Interestingly enough most of the times when I encounter this error I cannot delete the debug directory, Windows complains that it is in use but I cannot find any process using it using resmon.exe.

Any help is appreciated, let me know if you need more details.
Last edited by socratis on 11. Sep 2017, 19:43, edited 1 time in total.
Reason: Marked as [Solved].
socratis
Site Moderator
Posts: 27330
Joined: 22. Oct 2010, 11:03
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Win(*>98), Linux*, OSX>10.5
Location: Greece

Re: Debug Build without hardening successful but runtime error

Post by socratis »

Moving to "OSE" from "Windows Hosts".
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
bkth
Posts: 2
Joined: 8. Sep 2017, 20:35

Re: Debug Build without hardening successful but runtime error

Post by bkth »

Hi All, I managed to make it work in the end. I just nuked my output directory to re-do a clean build from scratch after troubleshooting my way to make it work in the first place.

Cheers!
socratis
Site Moderator
Posts: 27330
Joined: 22. Oct 2010, 11:03
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Win(*>98), Linux*, OSX>10.5
Location: Greece

Re: Debug Build without hardening successful but runtime error

Post by socratis »

I'm not on Windows myself, I'm on OSX, so I couldn't have helped you. I'm also brand new to the OSE build process, but another moderator that has been building and using his own version on Linux, told me (when he was guiding me) that he always clears the output directory. He was also bitten by a similar bug a while back. Ever since, my build process starts with "rm -r vbox/out". ;)

Thanks for the feedback, glad you got it solved. Marking as such.
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Post Reply