Virtual SATA/IDE controller errors

[mutface]

Thanks fixedwheel.

the 420Gb virtual machine is on top of a software RAID1 partition... but the 1 Tb one is not...

I don't know what divider=10 means - so I have the divider setting to the default install.

[jjc]

I'm having the same errors (ATA, ext, filesystem journal, etc.) leading to a filesystem read-only remount that effectively disables my VBox VM guests. Here's an example batch that I was only able to get because we log syslog on a remote box (since drive write failed, these errors generally don't show up in the VM's syslog after killing the process and restarting the VM):

kernel Remounting filesystem read-only
kernel ata1.00: cmd ca/00:10:dd:37:05/00:00:00:00:00/e0 tag 0 dma 8192 out

VBox Version 3.2.2 and 3.2.4 so far (I just upgraded to 3.2.6 and will see) - PUE license (we're a non-profit).

Both my VM guests and host are CentOS 5.4 x64. It's happened on two different hosts and numerous guests. Both hosts were new very beefy servers with dual quad-core Xeons, 24 and 32 GB RAM, RAID1 and RAID5 SATA disks. Guests were configured with 2-6 virtual cores, 2-12 GB RAM, and SATA virtual controllers with vdisk size ~40GB. Guests had low disk I/O usage.

I really hope the new version or someone's advice will fix this. My boss is talking about switching to VMWare and I'm trying to avoid it if at all possible.

Any advice would be MUCH appreciated. This has been hounding me for a couple months already. Luckily, these are not mission-critical VMs.

[alitokmen]

Hello

I use VirtualBox 3.2.6 under MacOS X and have the same issue with Debian Linux 5.0.5 x64 guest.

On the other hand, when booting, the guest OS shows a message like:

Code: Select all

SMB base address uninitialized - upgrade BIOS or use force_addr=0xaddr

I don't know if it has a link with the disk issues, but what I know is that the same message doesn't appear on a Debian Linux 5.0.5 x86 guest, and disk operations do not fail.

Cheers

[mutface]

Yes - we tested Virtual Box 3 on Opensuse 11.1 and Virtual Box ran smoothly. (32-bit) - single core processor.

It's only in CentOS 5.4 with Dual Quad Core Xeon (64-bit) and Intel S5000 Server Board with Virtualization technology that we have gotten the ATA errors.

We've just setup a 30Gb Virtual Box running CentOS 5.4 and after 4 days the filesystem became read-only; so it seems it's nothing to do with the size of the Guest machine...

Strange.

Any help would be appreciated...

[AndrewL733]

I am having a similar problem when I set up my Virtual Machines to use EITHER Virtual SATA or SAS controllers. With SATA, I'm getting the same errors posted above I tried disabling NCQ on my Linux guest and that did not help. The problem occurrs when I try to READ data over the network connection. Writing isn't a problem up to 20 MB/sec or more. But reading 4 MB/sec from a video editing program (I test these) kills my Linux VM every time. And, of course, OS X crashes when anything goes wrong on a network connection!

When I reconfigured my VM to use SAS (I installed the LSI mptsas drivers on my Linux guest), I am getting a similar issue. The VM disk system just dies with a different message:

Aug 19 10:09:20 es-vboxdemo klogd: mptscsih: ioc0: task abort: FAILED (sc=ffff880003db3900)
Aug 19 10:09:20 es-vboxdemo smbd[5054]: [2010/08/19 10:09:20, 0] smbd/process.c:62(srv_send_smb)
Aug 19 10:09:20 es-vboxdemo klogd: mptscsih: ioc0: attempting task abort! (sc=ffff880003db3f00)
Aug 19 10:09:20 es-vboxdemo smbd[5054]: Error writing 39 bytes to client. -1. (Transport endpoint is not connected)
Aug 19 10:09:20 es-vboxdemo klogd: sd 0:0:1:0: [sdb] CDB: Read(10): 28 00 00 00 37 50 00 00 c8 00
Aug 19 10:09:20 es-vboxdemo klogd: mptscsih: ioc0: task abort: FAILED (sc=ffff880003db3f00)
Aug 19 10:09:20 es-vboxdemo klogd: mptscsih: ioc0: attempting target reset! (sc=ffff880003db3700)
Aug 19 10:09:20 es-vboxdemo klogd: sd 0:0:1:0: [sdb] CDB: Read(10): 28 00 00 00 34 18 00 02 30 00
Aug 19 10:09:20 es-vboxdemo klogd: mptscsih: ioc0: target reset: SUCCESS (sc=ffff880003db3700)
Aug 19 10:09:20 es-vboxdemo klogd: mptscsih: ioc0: attempting task abort! (sc=ffff88003ee17100)
Aug 19 10:09:20 es-vboxdemo klogd: sd 0:0:0:0: [sda] CDB: Read(10): 28 00 01 38 53 39 00 00 40 00
Aug 19 10:09:20 es-vboxdemo klogd: mptscsih: ioc0: task abort: FAILED (sc=ffff88003ee17100)
Aug 19 10:09:20 es-vboxdemo klogd: mptscsih: ioc0: attempting task abort! (sc=ffff880003db3f00)

I don't have these problems with the exact same Linux guest and VMware, whether I emulate IDE or SAS disks (SAS with the same LSI mptsas driver)

Guest: My Linux distro is Mandriva 2010 64-bit but I'm running my own 2.6.32.11 kernel.
Host: OS X 10.6.4 or Windows 7 64-bit, 4 GB RAM, 7200 RPM OS Drive on a laptop
VirtualBox Version 3.2.8

Very discouraging!!!

[mutface]

Just tried installing Zimbra on the 30Gb virtual host and because Zimbra is quite resource demanding - even the 30Gb virtual host experienced the same data I/O problems and I had to reboot..... and within a day the same thing happened....

Any ideas why this problem is happening?

[operativo]

Hi all, same problems here:

Host: Ubuntu 9.04, Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz
Guest: Ubuntu 10.04 and 8.04 (6 machines in total)
Virtualbox: 3.2.8

Every night I perform a backup of the host machine and during this backup (which involve a certain IO load on host's disks) I noticed bad errors in guests, like:

Code: Select all

[48349.041447] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
[48349.042080] ata3.00: failed command: FLUSH CACHE
[48349.042487] ata3.00: cmd e7/00:00:00:00:00/00:00:00:00:00/a0 tag 0
[48349.042488]          res 40/00:00:00:00:00/00:00:00:00:00/40 Emask 0x4 (timeout)
[48349.043682] ata3.00: status: { DRDY }
[48349.044012] ata3: hard resetting link
[48349.390350] ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[48349.390687] ata3.00: configured for UDMA/133
[48349.390693] ata3.00: device reported invalid CHS sector 0
[48349.390709] ata3: EH complete

or also:

Code: Select all

[28127.795852]          res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[28128.124660] ata1: soft resetting link
[28128.303695] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[28128.304139] ata1.00: configured for UDMA/133
[28128.304150] ata1: EH complete
[28128.304246] sd 0:0:0:0: [sda] 8388608 512-byte hardware sectors (4295 MB)
[28128.304254] sd 0:0:0:0: [sda] Write Protect is off
[28128.304267] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

Sometime nothing happens and I'm happy but some other times the ext3 filesystem goes mad and will switch in read-only mode, so my machines became unuseful.

I also tried suggestions on Chapter 12 of virtualbox manual, no success.

Have you ever tried to use ionice to increase virtual machines IO priority or to decrease host machines IO-heavy processes?

Using ionice I put my virtual machines processes into "Real time" priority so that their IO is always served before any other process. Tonight I will now if it worked and I'll let you know.

PS: I also noticed that when heavy IO is performed on host the time in virtual machines slow down, loosing also 30s per minute, even if my host has frequency scaling disabled. Since I use heartbeat on some virtual machines this is a reeeally bad problem.

Regards

Dario

[jjc]

Hi Dario,

I have tried renice (which also improves the IO niceness) of the VM process. From what I remember, this didn't help. But, maybe you'll have better luck.

As for time drift issues, I've found them to be hopeless no matter what the setting I tried (including numerous tricks with NTP) until I installed the Guest Additions and set this setting:

Code: Select all

VBoxManage guestproperty set MY_VM_MACHINE_NAME_HERE "/VirtualBox/GuestAdd/VBoxService/--timesync-set-start" true 

Everything with time issues has been fine since I installed Guest Additions and made the above change on the host for the guest named "MY_VM_MACHINE_NAME_HERE" (the guest VM will have to be shutdown to make this change).

As for my original SATA/IDE errors, I haven't had any in a while, but my host disks have also been under less usage recently, so that could be part of the reason. Note, however, that the errors I was seeing were somewhat different then those you reported, so it could be a slightly different cause.

Good Luck!
Jake

[operativo]

Hi Dario,

Hi Jake

As for time drift issues, I've found them to be hopeless no matter what the setting I tried (including numerous tricks with NTP) until I installed the Guest Additions and set this setting:

Code: Select all

VBoxManage guestproperty set MY_VM_MACHINE_NAME_HERE "/VirtualBox/GuestAdd/VBoxService/--timesync-set-start" true 

Thank you for the hint, do you know what does exactly this option do?

As for my original SATA/IDE errors, I haven't had any in a while, but my host disks have also been under less usage recently, so that could be part of the reason. Note, however, that the errors I was seeing were somewhat different then those you reported, so it could be a slightly different cause.

With ionice i noticed that less errors occured but during the night (while performing a backup with high IO load) I still have "Lost interrupts" error. I have to find another way to reduce IO load, I think I will shape the network traffic causing this high IO load (I perform vdi backups using a gigabit ethernet).

Good Luck!
Jake

Thank you! I hope this issue will be fixed (or mitigated) soon.

[Vorlon]

I have a similar problem. Moreover, the problem also occurs when running the same VMs under VMWare Server 2.0.2, and I have seen posts that says it happens under other version of VMWare.

The problem also hits me when I have intensive I/o, so that often several VMs are hit with error messages relating to mptscsih.

I've tried changing the kernel module mptscsih by loading the Debian 486 kernel instead of the 686 version, but it did not help.