Utilizing Threadripper w/ Affinity

[mcc85]

Greetings everyone,

My name is Michael and I work for an IT company that serves residential and commercial based IT services. I recently had a client who was interested in running as many computers in his house as he could, he had 3 desktops and 2 laptops all doing a similar thing with some currency market software. I advised him that he could just build a single computer with a lot of CPU cores and then he would have the same thing, and would probably save a lot of money on his power bill by doing so. Anyway, he wasn't all too concerned with the savings on that but he was interested in the build and so he swiped away and I built him a computer that uses the new AMD 1950X Threadripper cpu. I built it with about 64GB of memory, a very fast PCIe SSD, and even a 4TB backup drive should he need it.

My build had a lot of unexpected issues, but I was able to completely deliver and the customer is absolutely pleased with the results. However, I have noticed that no matter how much tinkering I do with this thing or the operating system, I can't seem to get the individual VM's to use specific CPU cores. I have his computer running 6 individual VM's that are set up with 8GB of ram, and 100GB HDD space. The execution of these was quite a ride, but I think we're at a point now where they're operating at full capacity and wanted to know if there's a way that I can dedicate individual cores to each specific VM. For instance, I have each VM using a total of 4 'cores' (they're actually threads), and they all seem to operate pretty well. I was even able to add in a secondary GBE Ethernet card just in case there were any bandwidth limitations on a single card.

Anyway, I'm having an issue dedicating specific cores to these machines, as I'm able to find the process in task manager but when I actually select the number of cores to use, I get an error message that says "unable to set process affinity access is denied". At first I thought it was a user account issue, but that doesn't seem to be the case. I can't actually get any of the cores to specifically use a set number of these cores, it's all being done automatically. I'm not sure if that's a Ryzen thing, or if it has to do with the software or operating system itself.

Anyone able to shed some light on this subject? Considering this is a new hardware platform, I doubt that there are many answers out there yet and all the articles and links I do find happen to be years old, or totally irrelevant.

Thanks
- MC

[mpack]

Your arithmetic doesn't seem to be adding up for me. The Threadripper is a 16 core device, right? But you said you were running 6 VMs each with 4 cores. That means you have assigned 6*4 = 24 cores, and I guess forgot the host OS entirely.

This is assuming that AMD have cleaned up their act, and cores now really does mean core, and not just ALU.

I'd also search the forum for recent discussion of Ryzen, since I know there was problems originally.

[mcc85]

Yeah i put next to the 4 cores in parenthesis that they're actually threads, the program sees each thread as it's own core. so yes they are running 2 cores each but the program is set to 4 each. the 1950x has 16 cores and 32 threads... so my math is just fine dude.

[mpack]

so yes they are running 2 cores each but the program is set to 4 each.

I'm afraid that makes little sense to me. Cores are cores, threads are threads. The two terms are not interchangeable. Cores versus Threads. Let's park that for now.

I assume that the program running inside the guest actually cares about threads, not cores, since I doubt that it's doing its own virtual CPU management a la VirtualBox (in fact it can't, since it has no access to VT-x). So, I guess that a great deal will depend on how efficiently the application marshals the use of its threads. And of course memory and network bandwidth is not infinite, so you can't expect a linear scaling of performance like you would with several independant PCs (at least up to the network limit).

[mcc85]

"You must be a member for 1 days and have 1 posts before you can post urls."

Virtualbox is treating each thread as its own core, so does task manager. I'm not sure if that's just the way AM4/TR4 is built, but that's what it's telling me.

I was trying to add images earlier but it won't let me add a link to a bigger screenshot that I took to try and make light of the confusion. I understand perfectly well that a thread and a core are not the same thing, but apparently task manager and virtualbox are handling each thread as it's own 'core' - unless i'm wrong here, and that I should set each one to 2? either way, all I want to do is dedicate specific CPU cores to each VM and the affinity settings just give me that error message I posted earlier.

[mcc85]

I'm surprised that no one has responded to this inquiry yet...

I'm posting a higher resolution picture of the snapshot I took showing the virtualbox settings and whatnot.


https://drive.google.com/open?id=0B8sPa ... 1hSWlZxUk0

[Martin]

This is a forum from users for users. You probably didn't get further responses because nobody of us users ever saw a report which showed a benefit of CPU affinity for a VM.
What advantage do you think to get from this?

[BillG]

I think that you are overthinking this. As far as I know, VirtualBox does not (and cannot) allocate anything to specific cores or threads in the host machine.

The number of CPUs you allocate to a vm determines how many processors the guest OS thinks it has. If you set it to four, the guest OS thinks it is running in a 4 core machine and device manager will show four CPU instances. (In that respect, it is similar to threads, not cores). That is about it.

Any request for CPU time from any vm will be submitted by VirtualBox to the task scheduler in the host OS just like a CPU request from any other application program and handled in the same way. It will not go to any particular core of the physical machine. You can fiddle with the vm settings to prevent one vm from robbing other vms if it has a CPU intensive thread but I just leave it to VirtualBox and the host OS to manage. I don't try to outthink the task scheduler.

[mcc85]

This is a forum from users for users. You probably didn't get further responses because nobody of us users ever saw a report which showed a benefit of CPU affinity for a VM.
What advantage do you think to get from this?

The advantage I was hoping to get by using affinity was so that I could dedicate CPU cores to each specific VM. I built a machine for someone specifically to crunch some heavy numbers and isolating the workloads to specific VM's seemed to be an ideal scenario where he would get a maximum possible efficiency. Seeing that I can't necessarily do that because the Task Manager gives me an error "Access is denied", it's basically Windows automatically assigning the work throughout all available cores.

[mcc85]

I think that you are overthinking this. As far as I know, VirtualBox does not (and cannot) allocate anything to specific cores or threads in the host machine.

The number of CPUs you allocate to a vm determines how many processors the guest OS thinks it has. If you set it to four, the guest OS thinks it is running in a 4 core machine and device manager will show four CPU instances. (In that respect, it is similar to threads, not cores). That is about it.

Any request for CPU time from any vm will be submitted by VirtualBox to the task scheduler in the host OS just like a CPU request from any other application program and handled in the same way. It will not go to any particular core of the physical machine. You can fiddle with the vm settings to prevent one vm from robbing other vms if it has a CPU intensive thread but I just leave it to VirtualBox and the host OS to manage. I don't try to outthink the task scheduler.

I understand all of this but it's rubbish. I'm not saying what you tell me is rubbish, I just think that it's rubbish that you can't dedicate a VM to specific CPU cores as the VM's I am running are quite CPU intensive. It's not being used for cryptomining or anything of that nature, but what each VM is doing is essentially replacing the many physical machines that this guy has running.

I suppose trying to argue with the task scheduler is gonna get me nowhere, but perhaps the very thing that I am trying to do will pave way for innovation on the software devs since the CPU manufacturers are trying to deliver products which use more and more cores... being able to use them any which way you want to should be an option in the future.