VBox 5.0.24r108355 disk performance on a ram drive

[Tenuki]

let's not...
=========
... talk about the risks, the usage or the sense of all this. RAM drives are volatile; I know, and I've got a solution and true use for this. : )
If you want to reply "get yourself an ssd", please be told that an ssd produces the same results in VirtualBox as my ram drive: Something appears to step on the brake when storing bigger files in a vdi.


issue:
======
drive access for > ~100 MB files quite slow, even if the VDI is on a ram drive


given environment:
=================
VBox version: 5.0.24r108355
OS: Windows 7 up to 10 & Linux Gentoo & CentOS
CPU: 12 core (6 true + 6 hyper threading) Intel CPU for the Windows machine, 12 core Opteron for the Linux machine
RAM: 64 GB


description:
===========
Dear forum,

I couldn't find anything existing, so I thought I ask. Maybe somebody has already found a solution for himself.

Operating a guest on a VM using a VDI file appears to have very fast access for small files and quick accesses, but when writing big files, the whole transfer process becomes quite slow, like on a physical disk drive.
To give you an idea about what the relations are, I've ran a couple of tests and benchmarks:
small files some few kbyte up to ~100 MB --> transfer rates of 2330 MB/s
big files ~100 MB and above --> transfer rates of 104 MB/s (like a good old physical SATA1 drive)

I've tested this on two machines with the same type of RAM, a Windows host, and a Linux host.
For Windows I'm using the free tool "OSFMount" (tested others as well, but same outcome).
My Linux box is running the current CentOS using tmpfs to realise the RAM disk. (VMs are in headless operation)
Both RAM drives are set to 50 GB. Smaller drive sizes produce equal results.


test scenarios:
=============
Windows host (tested Win7 up to win10) all drives using ntfs
1) copied files from one vdi to another vdi (both on the same physical ram drive)
2) copied files from one vdi to another vdi (both on a different physical ram drive)
3) copied files from one vdi to the same vdi (both on the same physical ram drive)
4) copied files from one vdi to the same vdi (both on a different physical ram drive)
5) ran a disk benchmark utility
6) defragmented the disk (defragmenting the physical ram drive is MUCH MUCH faster than when defragmenting a VDI running that same VDI file which is ON a ram disk)
7) OS boot speed is excellent with a VDI on a ram drive, much faster than on the physical HDD. (any windows version) - up to 15 seconds faster on the desktop!

Linux host (tested with the current Gentoo Linux and the current CentOS) all drives using ext4
1) copied files from one vdi to another vdi (both on the same physical ram drive)
2) copied files from one vdi to another vdi (both on a different physical ram drive)
3) copied files from one vdi to the same vdi (both on the same physical ram drive)
4) copied files from one vdi to the same vdi (both on a different physical ram drive)
5) (does only apply for Gentoo Linux) running through emerge --sync (many maaany small files being deleted and written) within seconds when the vdi is on a ram drive. Using a vdi on a physical disk drive, the same procedure takes minutes.

So.... for me it looks like something that is not hardware dependant is stepping on the brake when storing big files.
Could this be the emulated CPU power used to calculate the changes for the file system? Or is it the write process itself? Thinking it could have something to do with CPU calculations as those are emulated, not handed over to the "real" CPU?

Thanks for posting your ideas!!

[scottgus1]

I wish I had 64GB to even think of testing a 50GB ram drive! I have also thought about putting a whole guest on a ram drive before, for speed purposes. A good idea if one maintains the recovery system.

That said, I have read many times from the real gurus in the forum that benchmarks don't work well in a guest. The timing systems the benchmarks use are all simulated, so the data can be misleading. The best way to do a benchmark test in a guest is with a pencil, paper, and a stopwatch.

You could try transferring around a known group of small files, measuring the time with the stopwatch (take several passes and average the result). Then try one big file, copied several times. Scale the results to see how performance goes.

(One possibility is that the OS might start doing something else while the transfer is taking place? Are automatic updates turned off?)

[Tenuki]

scottgus1, thanks for sharing your ideas! Very much appreciated!

Yep, thought something like this. @ times
Automatic updates turned off, automatic defrag turned off, etcetc.
Following the "busy CPU and I/O" theory, this would also mean, that linux-from-scratch, where you can really build an operating system as thin and efficient only for your needs, would show much better transfer speeds, as there is nothing going on, but the copy process.
But same goes for Windows; I've checked the ressource manager before, while and after the transfer processes. This ressource manager was always telling me that there is no unnecessary CPU, RAM or file system activity going on. (CPU idling ~0-1%, RAM idling on the same level, I/O: 0-1% - inside the VM)

Already from the sheer feeling of time it takes MUCH MUCH longer to transfer bigger files inside the VM from and to the VDI file on a ram drive. - Like in the tests 1 to 4. Doing the same stuff outside the VM the transfer rates are BY FAR quicker and the time used to get the copy done, is almost too short to see the copy progress dialogue. (need really big files outside the VM to even see this!)
Small files in huge masses are almost equally quickly copied outside the vm than huge big files.

But now the question is, why is there such a difference inside the vm: Small files are still fast, big files feel slow and heavy.
I'll run a test with a (physical) stop watch and dedicated files asap and report back with the results. Maybe this helps the developers to improve general performance in VMs on SSDs also, as there is the same transfer speed issue! (Which then is a realworld problem, as SSDs are more common than ram drives.. hehe)

[Tenuki]

I remembered, there is an open topic regarding disk io speed on VBox. So I ran a benchmark with an interesting outcome. Maybe somebody knows the reason for VBox IO being so slow!

Edit: Oh right.. of course I did upgrade my VirtualBox installation since back then to 5.1.14 r112924 (Qt5.6.2)

Scenario host PC: same as in post #1
Scenario host OS: Windows 7 Pro64
Scenario guest OS: Windows 2012 R2 Core Edition (which comes for free and without all the IO overhead)


Got a screeny attached, which shows quite nicely what I mean. Somehow the disk on the ram disk hosted virtualbox guest is not what it could be.

It tends to be best at 2 mb blocks, but the speed is far from anywhere close what the RAM transfer rate is.. and therefore it is no wonder why the box feels slow, even when the vdi file is on fast media.
On my ram disk, I measured transfer rates of 3,x GB/s and up, while VBOX (according to the benchmark) can deliver (at best) 350 MB/s. That is cheap ssd speed territory.
Question is though, where is the bottleneck......

CPU is idling according to host_PC_task_manager..

[Tenuki]

one more screeny of a ram disk benchmark on the host system itself

Mind the speed peak looks different here. best read speed is a 1 MB block here, best block write size is 64 KByte.


top IO speeds compared:
READ outside VBOX: 7,9 GB/s
READ inside VBOX: 1,65 GB/s
WRITE inside VBOX: 360 MB/s
WRITE outside VBOX: 7,5 GB/s