I suspect something's wrong with the 'use host I/O cache' option (#17573)
I suspect something's wrong with the 'use host I/O cache' option (#17573)
ModEdit; related ticket: #17573: Regression: "Use Host I/O cache" setting seems to be inverted in 5.2
I'm not sure when it started, maybe 1 or to releases back, but I now have a strong suspicion host cache isn't used anymore. I'm using VBox periodically to test operating systems for certain tasks, so it often goes install/update/install software. As I don't care about possible data loss I always enable host I/O cache, and previously it made noticeable speedup for most operations. Recently though everything is painfully slow. Debian install slows to a crawl. Apt dist-upgrade takes many minutes instead of seconds or a couple minutes (host data rate during guest apt upgrade is in hundreds of KB/s with 100% disk load, which never happened before). I'm not even 100% sure it's only caching problem anymore. I've tried moving .vdis to different disks and even to software raid. The only noticeable improvement was when I moved it to an SSD. But I can't use SSD for VMs, I need quick access to a number of different VMs, I simply can't afford that much SSD space just for occasional testing.
VirtualBox 5.2.8. Host is Win10 x64 1709 (4 cores, 16GB), guests are various linuxes and Windows 7-8-10 (mostly 1-2 cores, 1-4GB, only 1 guest is running at a time). Guest additions could be installed or not - I see this pattern of very slow disk access with 100% disk load on the host in both cases. Windows guests seem to also be affected.
I've checked the guest logs and VBox logs and I'm not seeing anything obvious. As it affects all or at least most VMs it's either the change in recent Vbox releases or a Windows 10 update that broke something.
Has anyone noticed anything similar?
Last edited by socratis on 29. Jul 2018, 18:15, edited 1 time in total.
Reason: Added ticket related information.
Reason: Added ticket related information.
Re: I suspect something's wrong with the 'use host I/O cache' option
This issue is still bothering me. VirtualBox updated to 5.2.12, Windows 10 to 1803.
I did a little benchmarking today.
Debian 9.3 vm
Installing a dozen package updates with apt.
Source disk (Snapshot 1) is on HDD, differencing disk (Current state) is on another disk (I've tried this before with all other kinds of storage placement, it doesn't really matter).
cache off 5:58
cache on 15:39
cache off, diff disk on ssd 2:18
cache on, diff disk on ssd 3:35
(cache="use host I/O cache")
That is so not right. Basically turning "use host i/o cache" option makes writes slow as hell, it's even quite noticeable writing to SSD. Just a few months ago "use cache" option instead gave a huge performance boost.
I'm very surprised I'm not seeing other reports of the problem. The only one I could find that looks similar: https://superuser.com/questions/1296927 ... -cache-off
(The poster isn't even on a Windows host! But the problem appears to be the same.)
At least I know I'm not alone. Maybe this affects only specific systems, and maybe most people don't turn host cache on. But how can we diagnose the problem then?
I did a little benchmarking today.
Debian 9.3 vm
Installing a dozen package updates with apt.
Source disk (Snapshot 1) is on HDD, differencing disk (Current state) is on another disk (I've tried this before with all other kinds of storage placement, it doesn't really matter).
cache off 5:58
cache on 15:39
cache off, diff disk on ssd 2:18
cache on, diff disk on ssd 3:35
(cache="use host I/O cache")
That is so not right. Basically turning "use host i/o cache" option makes writes slow as hell, it's even quite noticeable writing to SSD. Just a few months ago "use cache" option instead gave a huge performance boost.
I'm very surprised I'm not seeing other reports of the problem. The only one I could find that looks similar: https://superuser.com/questions/1296927 ... -cache-off
(The poster isn't even on a Windows host! But the problem appears to be the same.)
At least I know I'm not alone. Maybe this affects only specific systems, and maybe most people don't turn host cache on. But how can we diagnose the problem then?
-
- Site Moderator
- Posts: 27329
- Joined: 22. Oct 2010, 11:03
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Win(*>98), Linux*, OSX>10.5
- Location: Greece
Re: I suspect something's wrong with the 'use host I/O cache' option
Because maybe that's not the default?pupertest wrote:maybe most people don't turn host cache on
You know what you could try? Downgrading VirtualBox to a version that was working differently. You don't remember when you started seeing the changes, so you may have to experiment a little bit further. I suggest using the bisection method, so start with 5.2.12 and 5.1.0, then go to 5.2.0 and 5.1.18, and so on...
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Re: I suspect something's wrong with the 'use host I/O cache' option
Possibly a duplicate of the following ticket
https://www.virtualbox.org/ticket/17573
https://www.virtualbox.org/ticket/17573
Re: I suspect something's wrong with the 'use host I/O cache' option
My understanding was that most people who use VirtualBox for testing/demo purposes always enable it (like I do), because the performance gain is so significant (see below).Because maybe that's not the default?
I was hesitant to downgrade my install for testing previous versions with a bunch of vm configs in place, but I do want to resolve the problem for myself and others.
Here's the results.
5.2.0
cache on 16:30
5.1.10 - wouldn't start the vm with a hardening error (incompatibility with the latest Win10 version, I assume)
5.1.38
cache on 2:39
And here's the expected performance.
I've looked through the 5.2.0 changelog and there's the obvious "Storage: first milestone of the I/O stack redesign landed". So I assume in that redesign "use host I/O cache" option was somehow very negatively affected.
And presumably in 5.1 maintenance releases that redesign isn't implemented.
-
- Site Moderator
- Posts: 27329
- Joined: 22. Oct 2010, 11:03
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Win(*>98), Linux*, OSX>10.5
- Location: Greece
Re: I suspect something's wrong with the 'use host I/O cache' option
I do it on average between 3 and 5 times a day, no issues (that I don't expect) mostly...pupertest wrote:I was hesitant to downgrade my install for testing previous versions with a bunch of vm configs in place
I don't even know what these numbers mean, but when I talked about bisection, this is what I mean:pupertest wrote:5.2.0: cache on 16:30
5.1.10 - wouldn't start the vm with a hardening error (incompatibility with the latest Win10 version, I assume)
5.1.38: cache on 2:39
Test with 5.2.0 OK, 5.2.12 not OK. Then test with 5.2.6 (the middle). If OK, then move to 5.2.8 or 5.2.10. See what's working and what's not, by halving the "distance"/versions between what's working vs. what's not.
It's a faster way compared to testing 5.2.0, 5.2.2, 5.2.4, ..., 5.2.12.
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Re: I suspect something's wrong with the 'use host I/O cache' option
The numbers are a continuation of a benchmarking an OS update operation in a VM.
Basically all 5.2 versions have the problem (latest, as well as several previous, and I confirmed it for 5.2.0).
Latest 5.1 release (5.1.38) does not.
So the logical conclusion is that the mentioned I/O stack redesign in 5.2 branch is the reason, "use host I/O cache" option is broken in there.
Basically all 5.2 versions have the problem (latest, as well as several previous, and I confirmed it for 5.2.0).
Latest 5.1 release (5.1.38) does not.
So the logical conclusion is that the mentioned I/O stack redesign in 5.2 branch is the reason, "use host I/O cache" option is broken in there.
-
- Site Moderator
- Posts: 27329
- Joined: 22. Oct 2010, 11:03
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Win(*>98), Linux*, OSX>10.5
- Location: Greece
Re: I suspect something's wrong with the 'use host I/O cache' option
OK, excellent. Let's see what the developers have to say on the subject...
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.