That is what confuses me. This particular VM has three VDI files: 1) WinXP boot disk with OS and Programs (C:\); 2) data disk (E:\); 3) swap disk (S:\). 1 and 2 are stored on an ext4 USB 2.0 drive; 3 is on the ext3 formatted boot drive of the laptop (so that the swap file is theoretically not competing for USB bandwidth).
When running with VirtualBox 3.0.10, I've never seen this issue. With 3.0.12 I had the host freeze several times - so I reverted to 3.0.10. With 3.1.0_BETA 1 & 2, I've received the Guru Meditation notification multiple times. Nothing out of the ordinary was going on with storage or disk access at the host level - all drives were mounted. I do have an rsync backup script that runs at ~18 min after the hour, which hits the internal drive pretty hard as it tries to determine which files have changed for backup. I suppose this may be related - but from what I remember, the VM hung when this script was not running. Again - with 3.0.x I've never had VDI access issues (not sure if the codebase was modified as relates to File I/O).
Ubuntu 9.04 32 bit > Dell Latitude D820 > T2300@1.66GHz > 4 GB RAM