Increasing the number of Virtual Processors makes Windows Server 2019 guest hung

[Dimitrisv]

Hi!

I just got my Threadripper kit and went to do the obvious thing: To increase the number of cores in my VMs. My host runs Windows 10 Pro.

In Ubuntu guests, I managed to increase the number of cores to 10 and everything runs smoothly.

However, in Windows Server 2019 guests is another story. I increased from two to 10 and it just wouldn't boot - it will go to an installation/update cycle (crashing along the way). I then opted to go for incremental increases.
Going to 4 cores everything is fine and the kit load in less than 10 seconds.
Going to 8 cores, the machine still boots but it behaves erratically:
Sometimes after pressing the ctl+alt+del to provide the password it may hung for minutes. Other times it boots and then some of the services (i.e. SQL Server) do not auto start.

Please note that I am running the DataCentre Edition with unlimited CPU licenses.

Has anyone here been familiar with such issues? Is there a solution please?

Thanking you in anticipation!

Keep safe!

[mpack]

Please provide a VM log file. Make sure the VM is fully shut down, then right click it in the manager UI. Select "Show Log" and save "VBox.log" (no other file) to a zip file. Attach the zip here.

I just got my Threadripper kit and went to do the obvious thing: To increase the number of cores in my VMs.

Why? Increasing the core count should not be an end in itself: more cores will not by itself make anything go faster. 2 cores is usually optimal for VM on a host that has 4 or more cores available.

Practically speaking (i.e. unless you rewrite the OS): multiple cores let you do more things in parallel (assuming that it was previously bogged down for lack of CPU), it will not do any one thing faster. It's like imagining that your own speed would double if you owned a second car.

[Dimitrisv]

Thank you!

Here you go.

It looks too small. Are you sure it contains all the information since the generation of the machine? If not just let me know to increase again the count. It is rather stable now with 6 cores. But it will be great to figure out why it misbehaves (only on Windows guests)

Point taken about the number of cores.

But I am running Neo4j and MS SQL in my host, then two Ubuntus and one WinSrv. So I need the cores

[Dimitrisv]

The previous one I think only has the logs for the latest boot (that works fine on 6 cores)

Here is the log for an attempted boot with 10 cores... and then auto restart etc... then it gives me log in screen insert the password I press contr+alt+del and wait for a few minutes - goes to blue screen as if it wants to updates the windows, then gives me again the screen with the background to get the input box, but never gets there.. crashes...

[Dimitrisv]

Last but not least and before you tell me off for using LSI Logic SAS controller, here is the dump with 10 cores, when using SATA drive.

With NVMe controller it just crashes (even though I have two Samsungs 980 Pro NVMe on RAID 0 using AMDs own raid controller and latest drivers in my physical host).

[mpack]

I don't need the logs for your various different attempts. I only need to know the physical capabilities of your host, and I get that from one log.

For reference, your host has 64GB RAM, 53GB available, and 24 cores - at least according to AMD's definition of a core, which I'm always doubtful about since the "fx 8-core" fiasco. So we can rule out your problems being caused by over-committing CPU, at least until you go above 12 cores.

So I'm back to my original comment: why does your VM need so many cores? Because performance dropping due to management overhead for unnecessary cores is known behaviour. Unlike physical hardware a VM can't simply ignore additional cores.

[Dimitrisv]

OK. I am quite happy with the 6 core performance. It boots the server in about 5 seconds.

It is just a matter of academic curiosity why it gets messed up with over 8 cores.

I get your point. I noticed say in Ubuntu that after a certain point, adding cores is not speeding up anything. To the contrary.

So to wrap it up: Do you think that its far better say to create more VMs for specific roles rather than beefing up a VM and adding more services to it? The architecture of the solution that I am working with requires a lot of resources/subsystems. See: https://egeria.odpi.org/

I also noticed that Neo4j which is Java Powered even in its unlimited enterprise version, it seems to support only four cores per query! I mean even though say you have a licence for 24 cores. Executing a complex query in a huge dataset - (say 150 million records) it will not utilise more than 4 cores.

In any event thank you so much for your support.

As you see, I started by examining going for an Apple Silicon M1 and ended up, happily, with this beast for the same price!

[mpack]

As you see, I started by examining going for an Apple Silicon M1 and ended up, happily, with this beast for the same price!

It's certainly quite a beast. 64GB and 24 cores (48 threads) is server class (assuming the I/O is equally good). I have to make do with 6th gen I7: 16GB RAM, 4 cores (8 threads).

I wouldn't necessarily say to create multiple smaller VMs. Just keep an eye on whether more of everything automatically helps. For some things, e.g. SSD disk space, it certainly does (or certainly doesn't hurt anyway). For CPU cores it depends on what you needed the cores for. Extra cores do hurt a VM because each core has a management overhead that has to be repaid in order for the investment to be worthwhile. If the core sits idle then it's costing you.

[Dimitrisv]

Quad channel memory gives me an aggregate bandwidth of over 82GB/s
Samsung's 980 PRO NVMe on RAID 0 gives approximately 11GB/s throughput - even with the built in controller of ASUS ROG Zenith II Extreme.

For the past four months I had to work with a 2010 HP, I3 with 16GB of Memory and just an SSD and used to pay about $300 per month at Amazon to test my work.
I am telling you, the upside was that I had to write good code in order to make it work in such a slow system.

Is it please possible to provide an estimate of the overhead per pair. if say for the second processor is x, what will it be for the fourth, eighth, sixteenth processor? What are we talking about here?

[mpack]

No, I can't provide that estimate. It will depend on the exact scenario and the hardware (e.g. I would not expect Intel Skylake to be the same as AMD Ryzen).

If you really care about the numbers then do what you did: test your own realistic scenario.

Personally I just stick to 2 cores per VM. I'm never doing a lot in parallel in a VM, so that's all I need. Also, it's the only fair allocation on a 4 core host IMO.

[fth0]

In the VM configuration, set System > Acceleration > Paravirtualization Interface to Hyper-V, and see if that makes a difference. This could at least solve the time synchronization problems that can also be seen in the 6-core log file. Please post a new VBox.log file.

[Dimitrisv]

Thank you both so much!

It worked! For starters the respective logs are 20% "thinner".

No I couldn't spot the difference in speed on the 6 core configuration - as you know it loads in about 10 seconds. But now it loads with 10 cores with no hussle. First time boot (from 6 to 10 took a while longer (this is the log I am attaching)). Second time boot with 10 cores crashed, third time worked fine.

Then kept on working in various scenarios.

My final test was to test it with the unlucky 13 number of cores. It worked fine. My MSSQL Server application performed superbly in the 10,13 core cases.

It will be great if you can add any other adjustments. I can see that in the initial stages of the windows booting (i.e. first 2-3 seconds) the indicator of the VM usage is deep red, then it returns to its mid range.

[mpack]

I confess I hadn't realized that syncing clocks (presumably to solve the time disparity) added so much overhead.

[Dimitrisv]

Yet when I chose the Hardware Clock in UTC Time it crashes.

I am looking forward to fth0's take and further suggestions.

[mpack]

That is not the kind of time sync I was talking about. I was talking about measuring passage of time by counting clock ticks, not setting of calendar time.

In any case the subject matter seems to be drifting. Calendar time problems does not seem like a good fit to the topic title, we like to keep to one subject per topic.