repetitive Disk errors in Guest resulting in I/O-Error

Discussions related to using VirtualBox on Solaris hosts.

repetitive Disk errors in Guest resulting in I/O-Error

Postby bauer40 » 3. Jan 2018, 12:27

Configuration: VBox 5.2.0 on Solaris 11, local SATA Storage on an SVM Volume; Guest is Debian 8.10.

I encounter repetitive disk errors on the guest, resulting in filesystem corruptions, and I need an advice where I might look at. The /var/adm/messages of the physical machine does not log anything of interest.

My Debian Machine logs the following:

Code: Select all   Expand viewCollapse view
Jan  3 11:20:25 exchange64 kernel: [161564.875823] ata4: hard resetting link
Jan  3 11:20:25 exchange64 kernel: [161565.195657] ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
Jan  3 11:20:25 exchange64 kernel: [161565.196329] ata4.00: configured for UDMA/33
Jan  3 11:20:25 exchange64 kernel: [161565.196341] ata4.00: device reported invalid CHS sector 0
Jan  3 11:20:25 exchange64 kernel: [161565.196359] ata4: EH complete
Jan  3 11:20:56 exchange64 kernel: [161595.908145] ata4: hard resetting link
Jan  3 11:20:56 exchange64 kernel: [161596.226784] ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
Jan  3 11:20:56 exchange64 kernel: [161596.227395] ata4.00: configured for UDMA/33
Jan  3 11:20:56 exchange64 kernel: [161596.227407] ata4.00: device reported invalid CHS sector 0
Jan  3 11:20:56 exchange64 kernel: [161596.227425] ata4: EH complete


in parallel, the VBox.log says

Code: Select all   Expand viewCollapse view
45:00:54.021840 AssertLogRel /export/home/vbox/tinderbox/sol-rel/src/VBox/Devices/Storage/DevAHCI.cpp(4520) bool ahciR3CmdPrepare(AHCIPort*, AHCIREQ*): ASMAtomicReadU32(&pAhciPort->cTasksActive) <= AHCI_NR_COMMAND_SLOTS
45:00:54.021872 AHCI#0P1: There are more than 32 (+1) requests activeAssertLogRel /export/home/vbox/tinderbox/sol-rel/src/VBox/Devices/Storage/DevAHCI.cpp(4520) bool ahciR3CmdPrepare(AHCIPort*, AHCIREQ*): ASMAtomicReadU32(&pAhciPort->cTasksActive) <= AHCI_NR_COMMAND_SLOTS
45:00:54.022105 AHCI#0P1: There are more than 32 (+1) requests activeAssertLogRel /export/home/vbox/tinderbox/sol-rel/src/VBox/Devices/Storage/DevAHCI.cpp(4520) bool ahciR3CmdPrepare(AHCIPort*, AHCIREQ*): ASMAtomicReadU32(&pAhciPort->cTasksActive) <= AHCI_NR_COMMAND_SLOTS
45:00:54.022256 AHCI#0P1: There are more than 32 (+1) requests activeAssertLogRel /export/home/vbox/tinderbox/sol-rel/src/VBox/Devices/Storage/DevAHCI.cpp(4520) bool ahciR3CmdPrepare(AHCIPort*, AHCIREQ*): ASMAtomicReadU32(&pAhciPort->cTasksActive) <= AHCI_NR_COMMAND_SLOTS
45:00:54.022464 AHCI#0P1: There are more than 32 (+1) requests activeAHCI#0: Port 1 reset


Over the time, my filesystem becomes corrupted and falls to read-only.

My (truncated) VBox.Log is attached.

Any ideas anybody? Thank you!
Attachments
VBox.log.gz
truncated VBox Logfile
(51.44 KiB) Downloaded 42 times
bauer40
 
Posts: 37
Joined: 14. Sep 2008, 09:16

Re: repetitive Disk errors in Guest resulting in I/O-Error

Postby mpack » 3. Jan 2018, 14:08

You seem to have allocated 100% of available cores to the guest. This will not help system stability. In particular the host background disk writes seem to be getting stalled. I don't know if this is your problem, but it certainly will not help. You have 4 cores, so ideally give 2 to the guest. Certainly no more than 3.
mpack
Site Moderator
 
Posts: 29820
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: repetitive Disk errors in Guest resulting in I/O-Error

Postby bauer40 » 3. Jan 2018, 14:23

I have allocated four virtual cores to the VM, and the machine has four cores and eight threads:

root@infra1:/root# psrinfo -t
socket: 0
core: 0
cpus: 0,4
core: 1
cpus: 1,5
core: 2
cpus: 2,6
core: 3
cpus: 3,7

So, theoretically, I use only half the threads I have available ... but sure, I can reduce the number of cores to the VM and see if this helps.
bauer40
 
Posts: 37
Joined: 14. Sep 2008, 09:16

Re: repetitive Disk errors in Guest resulting in I/O-Error

Postby socratis » 3. Jan 2018, 15:12

VirtualBox cares about physical processors, not logical ones. See: CPU Cores versus threads and Why is it a bad idea to allocate as many VCPUs as there are physical CPUs?
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
socratis
Site Moderator
 
Posts: 25844
Joined: 22. Oct 2010, 11:03
Location: Greece
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Win(*>98), Linux*, OSX>10.5

Re: repetitive Disk errors in Guest resulting in I/O-Error

Postby bauer40 » 4. Jan 2018, 08:10

Downgraded my VM to two CPUs. We'll see what's the result.

I have another idea: the disk that starts to fail had a snapshot. As it always failed when I wrote to that disk, I have a feeling it might relate. I'll set up a similar configuration and see what happens when I overwrite the snapshotted disk over and over again - that's what I did with the disk in trouble.
bauer40
 
Posts: 37
Joined: 14. Sep 2008, 09:16

Re: repetitive Disk errors in Guest resulting in I/O-Error

Postby Steffen M. » 28. Mar 2018, 14:12

Hi,

we've exactly the same problem with VirtualBox 5.2.6 on Solaris 11.3 SRU 29 (as host) and Ubuntu 16.04.4 LTS (Kernel 4.4.0-116-generic) as guest. Our host machine consists of 12 physical and 24 logical cores (2 x Intel E5-2667) with 6 physical cores each and 256 GB RAM. The Ubuntu guest has got 6 cores and 24 GB of RAM allocated.

When the problem occurred the last time, no snapshot was involved. High I/O load (installation of a large tool) was going on in the guest. The host didn't have any further load.

The flag "Use Host I/O Cache" is not active, the virtual disk is dynamically allocated and stored on a ZFS.

Does anybody have an idea what we can do to tackle it down? Thank you very much in advance!

Kind regards,
Steffen
Steffen M.
 
Posts: 1
Joined: 12. Sep 2013, 16:56

Re: repetitive Disk errors in Guest resulting in I/O-Error

Postby Mechanicjay » 20. Jul 2018, 20:38

I'm seeing the same exact thing on my system

FreeBSD11 with virutalbox 5.2.8 -- virtualdisks on a ZFS volume with dynamic allocation.
Mechanicjay
 
Posts: 2
Joined: 20. Jul 2018, 20:31

Re: repetitive Disk errors in Guest resulting in I/O-Error

Postby Mechanicjay » 25. Jul 2018, 21:03

I've turned on HOST I/O Cache and I've not had this issue pop up again even with high IO loads. The VM's are also blazingly fast now....

Just a datapoint.
Mechanicjay
 
Posts: 2
Joined: 20. Jul 2018, 20:31


Return to VirtualBox on Solaris Hosts

Who is online

Users browsing this forum: No registered users and 1 guest