Problems copying data from USB drive to Linux Guest

[Ravi Joshi]

Environment : Windows 7 Ultimate hosting Ubuntu Server 12.
I have an external USB disk that has two partitions, first is NTFS and second is EXT3. This second Linux partition holds some data (20gb) that I need to copy to Ubuntu VM. I have extension pack installed, and am able to share USB drive & mount the ext partition in Ubuntu and see the contents. So, this part work fine.

However, when I start copying directory from usb to local using 'rsync -a --progress ' command, the copying starts and continues for some time (about 3 min) when the problem begins. The file copy seems to stop with cursor not blinking or progress not displayed on screen. The VM becomes unresponsive, and only way to get out this situation is by actually killing virtualbox processes. I have tried this multiple times, and the problem occurs when copying different folders, so it's not like it's happening when only a certain file is being copied.
I am not sure what I can do to solve this problem. Any help is much appreciated.

PS: At this point, copying using shared folder feature is not an option.