VBox Video Driver seemingly having trouble (poor resolution)

[cwru53]

Information:
Fedora 14 x86_64 Host, Windows XP SP2 Guest
Vbox Version = 3.2.12 r68302
AMD 5200+ Dual Core, 8GB RAM (assigned both processors and 3.5 GB to Windoze)
3D and 2D acceleration enabled in settings
No sata controller (only IDE) in settings
128MB Video RAM selected in settings.

Problem:
After guest addition installation, I only get 640x480 4 bit display resolution. I cannot change this, for reason it is stuck on this


What I have tried:
1. I always install guest additions in safe mode (I do select direct3D capability)
2. Once I remove the VBox video driver, I get 32bit resolution. If I reinstall the video driver manually, same problem appears.
3. VBoxManage controlvm "Windoze XP" setvideomodehint "1024" "768" "32" to better set video hint.
4. Remove VBox Video driver in device manager (This actually kind of fixes the problem, I can adjust screen size a color depth).

Does anyone know a solution or what the problem may really be? I check the services (funny thing is, if I boot in safe mode, I get full color depth... I assume the Vbox video driver isn't loading in safe mode?!). Without the video driver, no direct X for me

Please help, and thanks in advanced.

[Sasquatch]

Dump the VRAM down to 16 MB or so. And try it without 3D/2D enabled.

[cwru53]

Ok, I tried what the hairy man suggested. I did it in incremental steps. Did not correct the problem.

I am beginning to think it is a problem with the windows system (happened after updating windows and reinstalling an anti-virus software -- Comodo). I will install another windows system (hopefully 7) and see if it has the same problem. I have been using this virtual drive since at least the release of Fedora 12 if not earlier.

[cwru53]

So I gave up, had a copy of the virtual drive backed-up prior to having the problem. So I just restored it. If I have the problem again, I will make sure I get more details. One question though, about the time the problem appeared, I did run dxdiag and tested the capabilities. I also tried it with the Vbox OSE version, same problem prior to using the backup virtual drive.

Does dxdiag do some crazy things or replace some files vbox depends on?

Finally, sorry if the hairy comment offended you, but it's a nice name.

[Winipulator]

This problem has plagued me from time to time. One solution I've had was to disable nested paging. Another solution that worked for me was, as Sasquatch mentioned, to reduce video ram. For me, reducing video ram from 64 megs to 62 worked. Anything over 62 megs would cause this problem to come back. (My hardware video memory is 256 megs.) I've had it occur in one XP VM but not another, with the same exact VM settings. Using the program "File Compare" to compare the xml files of both VMs showed no significant differences in configuration, so this indicates that the problem has to do with some aspect of the Windows configuration. But I haven't been able to narrow it down to any specific thing. My ultimate solution was to delete the VMi that had this problem, and copy from the VMI that didn't have the problem. I'm looking forward to the next VB release which may help remedy this issue. Good luck.

[Winipulator]

OK I ran into this problem yet again. And I think I found the solution. (At least a solution that works for me.)

Inside the XP guest, right-click My Computer > Properties > Advanced > Performance Settings > Advanced Tab > Memory Usage

When its set to "System cache" the 4-bit color @ 640x480 problem seems to always resurface sooner or later. If video memory is set above 64 megs on my system, its pretty much guaranteed to always happen.

When its set to "Programs" I can set video memory to the full 128 megs that VB allows on my system, enable nested paging in VB settings, have 2D & 3D acceleration enabled, what have you. Full 32 bit color @ 1680X1040 with no issues.

If this has been addressed in another thread, my apologies. I hope this helps someone out there. Good luck.