We’re having some odd problems running our testing server. I’ll do my best to explain the setup and the problem:
We have an automated testing server set up with Windows Server 2012 R2 host OS. On this server, we have an instance of Oracle VirtualBox installed (version 5.1.2.8956). Finally, on the tester’s computer, we are running HP Quality Center (version 9.2.0 – this is a very old version, but we are in the process of upgrading). All users are running Windows 7.
The tester starts up an automated test in Quality Center, which starts a PowerShell script on the server. This PowerShell script starts a virtual machine, using a VirtualBox COM object, by cloning an existing machine from a list of approved machines. Once the machine is cloned and started up, the specified test will then run on that virtual machine. Results are copied from the guest OS to the host OS, and the VM is removed.
The server handles multiple automated test requests at once. Each test request will run against a different VM. These tests are also not resource-bound (we have dual 6-core Xeon processors with 128GB RAM).
We are seeing issues that pop up with some regularity. These errors appear in the Windows Event Viewer as “Application Errors”, with Event ID 1000 and Task Category (100). An example event log looks like this:
Code: Select all
Faulting application name: VirtualBox.exe, version: 5.1.2.8956, time stamp: 0x5790f053
Faulting module name: VBoxC.dll, version: 5.1.2.8956, time stamp: 0x5790f057
Exception code: 0xc0000005
Fault offset: 0x00000000000a6a07
Faulting process id: 0xe34
Faulting application start time: 0x01d3410a03f156c8
Faulting application path: E:\VirtualBox\VirtualBox.exe
Faulting module path: E:\VirtualBox\VBoxC.dll
Report Id: 9a36be58-ad04-11e7-80f7-a0369f65611f
Faulting package full name:
Faulting package-relative application ID:
However, there is another behavior that we have observed. It could be related, or it could be a red herring, and I haven’t figured out which yet. Anyway, The VirtualBox manager running on the server seems to slowly accumulate “inaccessible” virtual machines. For now, we simply clear these out periodically, but it doesn’t seem like this should happen at all.
Any advice for what to do here would be greatly appreciated.