15 to 18 Machine Limit on Version 5 and above

Discussions related to using VirtualBox on Linux hosts.
FitchT
Posts: 17
Joined: 15. Nov 2017, 23:18

15 to 18 Machine Limit on Version 5 and above

Post by FitchT »

I have 22 Redhat 6.9 servers running several hundred machines in Virtualbox.

I've posted before and scoured the internet for an answer.

On any 5.x server I cannot power up more than 15 machines at time, 18 on 5.2. Going back to 4.3 solves the issue. I can launch 30+ easily.

Yes I've yum update everything, I've been working on this from time to time for months now.

The error received is the usual NS_ERROR_FAILURE 0x80004005 Once I close Any other machine and retry I can launch one more. The guest machine doesn't matter, Win 32 64 XP 7 8 10 FreeBSD, no difference.

Would appreciate the help please.
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: 15 to 18 Machine Limit on Version 5 and above

Post by mpack »

Pick a failing VM and provide a VM log. With the VM fully shut down, right click it in the GUI. Select "Show Log" and save "VBox.log" (ONLY) to a zip file. Attach the zip here.
klaus
Oracle Corporation
Posts: 1115
Joined: 10. May 2007, 14:57

Re: 15 to 18 Machine Limit on Version 5 and above

Post by klaus »

And VBoxSVC.log, too. It's important, because the cause of the limit (which is totally unintentional, and I can't think of a reason) might be in the "central" part of the API.
FitchT
Posts: 17
Joined: 15. Nov 2017, 23:18

Re: 15 to 18 Machine Limit on Version 5 and above

Post by FitchT »

Thank you. All attached.
Attachments
vbox.zip
(80.63 KiB) Downloaded 11 times
klaus
Oracle Corporation
Posts: 1115
Joined: 10. May 2007, 14:57

Re: 15 to 18 Machine Limit on Version 5 and above

Post by klaus »

Woot? You attached super short log files (the 2nd and 3rd one in the zip), and they're debug log files. They are not supposed to exist at all. Their content gives us some hint for an incorrectly used log statement, but that's not what I expect to be a real issue. They're not what I'm after, I need VBoxSVC.log (usually lives in ~/.config/VirtualBox/ or ~/.VirtualBox).

The first one in the zip is making some sense - but doesn't say much about the cause: A VirtualBox VM process can't create a new thread, our runtime function returns VERR_TRY_AGAIN, which probably originates in an EAGAIN from the underlying syscall. Which shouldn't happen, as libc should repeat the syscall automatically. It matches the crash you showed in the 2nd screenshot.
FitchT
Posts: 17
Joined: 15. Nov 2017, 23:18

Re: 15 to 18 Machine Limit on Version 5 and above

Post by FitchT »

Thank you. Here is another set with the VBoxSVC.log from ./config
Attachments
vboxlogs2.zip
(10.25 KiB) Downloaded 8 times
klaus
Oracle Corporation
Posts: 1115
Joined: 10. May 2007, 14:57

Re: 15 to 18 Machine Limit on Version 5 and above

Post by klaus »

It' unclear what happened in the time covered by VBoxSVC.log - from the looks I would say absolutely nothing. That's again not the log I'm after - I need a VBoxSVC.log file which shows the successful start of X VMs and afterwards failures to start more.
FitchT
Posts: 17
Joined: 15. Nov 2017, 23:18

Re: 15 to 18 Machine Limit on Version 5 and above

Post by FitchT »

ok I'll find it. Thank you so much for your assistance.
Martin
Volunteer
Posts: 2560
Joined: 30. May 2007, 18:05
Primary OS: Fedora other
VBox Version: PUEL
Guest OSses: XP, Win7, Win10, Linux, OS/2

Re: 15 to 18 Machine Limit on Version 5 and above

Post by Martin »

00:00:00.131994 nspr-2 Home directory: '/root/.config/VirtualBox'
Are all VMs running under the user 'root'?
FitchT
Posts: 17
Joined: 15. Nov 2017, 23:18

Re: 15 to 18 Machine Limit on Version 5 and above

Post by FitchT »

They are running under a user named "script"
I'm attaching two files I found:
I did:
[root@HITKVIRT21 /]# find . -name VBoxSVC.log
./home/script/.config/VirtualBox/VBoxSVC.log
./root/.config/VirtualBox/VBoxSVC.log
[root@HITKVIRT21 /]#
to find them.
Attachments
vbox3.zip
(3.59 KiB) Downloaded 9 times
FitchT
Posts: 17
Joined: 15. Nov 2017, 23:18

Re: 15 to 18 Machine Limit on Version 5 and above

Post by FitchT »

Were yo able to download the file, did I screw it up?
klaus
Oracle Corporation
Posts: 1115
Joined: 10. May 2007, 14:57

Re: 15 to 18 Machine Limit on Version 5 and above

Post by klaus »

Downloading is never the problem - finding time to look at the logs is. Your "OtherVBoxSVC.log" contains rather strange things. It seems that the VMs which are just started are crashing instantly, before it's possible to establish an API session. The signal number appears to be 5, i.e. SIGTRAP. Scratching my head currently, but don't have time to think about ways to track this down further. It must be something rather specific to your setup (but I'm not ruling out that it might be something which is default with some distros which we're not testing).

Try starting a VM manually, with "VirtualBox --startvm <vmname>" once you're running into the limit. I wonder if this creates a VBox.log with useful content or produces some message in the terminal window.
FitchT
Posts: 17
Joined: 15. Nov 2017, 23:18

Re: 15 to 18 Machine Limit on Version 5 and above

Post by FitchT »

Additional development. This went away for awhile, but is now back, when starting that last VM, the GUI crashes and this is displayed for each machine. Though the GUI now says all the VMs are powered down, they are in fact working perfectly and you can RDP to them etc.
Attachments
blowup.jpg
blowup.jpg (127.01 KiB) Viewed 3027 times
FitchT
Posts: 17
Joined: 15. Nov 2017, 23:18

Re: 15 to 18 Machine Limit on Version 5 and above

Post by FitchT »

Thank you so much for your time. I can't believe nobody else has reported this. I've had this issue on 5.x since April.

I will test the manual start now and report back.
klaus
Oracle Corporation
Posts: 1115
Joined: 10. May 2007, 14:57

Re: 15 to 18 Machine Limit on Version 5 and above

Post by klaus »

If you have running VMs but the manager UI shows them as powered off then very likely you have several VBoxSVC processes running for one user (in the past I would've attributed it to a VBoxSVC crash, but since 4.x the VMs monitor if VBoxSVC is alive, and if not they terminate within 30 seconds - possibly with some spurious error messages caused by the VBoxSVC crash).
Post Reply