SMP & network performance

Discussions related to using VirtualBox on Linux hosts.

SMP & network performance

Postby liquid_man » 5. Jan 2012, 21:16

I decided to run some network throughput benchmarks today (mainly interested in max packets per second), and have noticed a large discrepancy when assigning one CPU to a VM vs 2 or more CPUs. My setup (all 64-bit):
Host: Ubuntu 11.04 2.6.38-13-generic #53-Ubuntu SMP, 3 GHz Intel i7 950 (4 phys cores + HT), 12GB RAM, Ethernet: Intel 82574L Gigabit
Guest: Ubuntu 11.10 3.0.0-12-server kernel, 1GB Ram assigned
VirtualBox version: 4.1.8

Following are the results of the basic testing I performed using netserver/netperf against a bridged network interface (bridged to above Intel device). The following commands were run on the Guest against its local interface (not the loopback):
netserver -4 (starts an ipv4 tcp/udp server).
netperf -H <IP_address_of_eth0> -t TCP_CRR (runs a TCP connect/request/response transaction benchmark)

Single-CPU VM
~17-18k transactions per second

2-CPU VM
~5k TPS

2-CPU VM with eth0 interrupts and netserver/netperf all pinned to the same core
~7k TPS
Confirmed v. low scheduling interrupts during benchmark (watching /proc/interrupts)

2-CPU VM with 2nd core disabled via hotplugging
Disabled the second cpu with: echo '0' > /sys/devices/system/cpu/cpu1/online and confirmed via /proc/interrupts and other system tools.
~8.5k TPS

Also worth noting that on the host system, the same test yields around 26k TPS. netfilter/conntrack is disabled on both host and guest.

So even with the second cpu disabled I'm seeing around a 50% performance degradation vs the single-cpu VM. The results with more than 2 CPUs were very similar to the 2-CPU scenario.

I would like to understand why this is the case (I'm certainly no virtualization expert); are additional extensions/emulations loaded when starting a multicore guest? I did have a quick look at VBox.log and the main thing I noticed was that HwVirtExtForced
is enabled when >1 CPU is configured. Could this be the cause of degradation, and if so, where can I read more about these extensions?

Any insight greatly appreciated.

Update
I repeated the same test on an OS X 10.6 host on similar architecture (quad core intel i7) and the results were the same, also on VBox 4.1.8.

I decided to extend the test to something CPU bound and ran a Linpack benchmark (single thread), but the results are unaffected by number of vCPUs (which is good). And so I also ran a a disk read benchmark using hdparm, and this was also unaffected, so this seems to be confined to network performance for now.
Last edited by liquid_man on 8. Jan 2012, 00:07, edited 2 times in total.
liquid_man
 
Posts: 2
Joined: 5. Jan 2012, 20:28
Primary OS: Ubuntu other
VBox Version: OSE other
Guest OSses: Ubuntu, CentOS, WinXP

Re: SMP performance

Postby Martin » 6. Jan 2012, 17:59

Looks like your CPU is only a dual core version. The host and the Vbox processes also need some CPU time, which isn't available when the guest needs two cores.
HT doesn't really help here. Rule of thumb: Don't assign all available CPU cores to a single guest, leave one core available for the host/Vbox.
Martin
Volunteer
 
Posts: 805
Joined: 30. May 2007, 18:05
Primary OS: Fedora 18
VBox Version: PUEL
Guest OSses: XP, Win7, Linux, OS/2

Re: SMP performance

Postby liquid_man » 6. Jan 2012, 18:57

No it's got 4 physical cores, it's the i7 950 (have edited the original post). I even pinned the VirtualBox process to 2 physical cores (3rd and 4th) and monitored cpu usage on the host to make sure they were the ones being used. The rest of my system was predominantly using cores 1 and 2.

Also, I disabled and enabled hyperthreading via the BIOS and no difference (HT adds another 4 virtual cores), so it doesn't seem to be any hyperthreading interference.
liquid_man
 
Posts: 2
Joined: 5. Jan 2012, 20:28
Primary OS: Ubuntu other
VBox Version: OSE other
Guest OSses: Ubuntu, CentOS, WinXP


Return to VirtualBox on Linux Hosts

Who is online

Users browsing this forum: No registered users and 20 guests