I have 4 VM boxes in a datacenter which host a number of VM's. The hosts are all Ubuntu server 10.04 machines. The virtualbox versions tried were 4.0 and 4.1 (the latest of each). The guests were: fedora 13 32&64, centos 5.5 32&64, ubuntu 10.04 32&64.
A few days ago a few of the guests starting getting ext4 inode errors and all remounted read-only. I searched around for the issue, but nothing obvious stuck out. I tried upgrading to virtualbox 4.1, but I ran into bug where snapshots didn't complete and when that happens you are in a world of hurt recovering those (lots of .xml hand-editing). So I reverted back down to 4.0 and moved or rebuilt the vm's on a different partition. The partition they were all on were raid 5's and the raid wasn't reporting errors. Everything was rebuilt and worked yesterday and today, 5 vm's are back to being read-only. Now searching around i seem to be the only one hitting this problem so hard. I'm either really unlucky or I'm missing something in my searches. At first it was easy for me to blame the hardware, but now it is so unlikely that 4 separate machines went bad all around the same time.
So, is there anything that is known with Linux hosts + Linux guests that would cause such horrible disk errors?
Thanks for any help you can give, I'm about to the end of my rope on this issue!