Windows XP: In both VM and native (using Linux as host OS)
Please post any comments/suggestions etc in this thread: Howto: Windows XP in both VM and native - DISCUSSION Modedit: [TerryE] I have told VT that we will police this request by deleting any posts other than his from this topic, so please follow this suggestion.
I thought I'd summarise how I got WinXP running in a "dual" boot (running a single Windows installation both in VirtualBox and natively). Many thanks to TerryE for his input. This article covers how to set up such a configuration from scratch. If you're looking to boot an existing Windows installation in VirtualBox please have a look at HOWTO: manage VDIs and import native installations instead. While using an existing installation is less work initially you're very likely to encounter driver related problems if you want to keep booting the installation natively also. This is why I chose to write about a from-scratch method instead.
WARNING: Only follow these instructions if you understand what you're doing, I'm not responsible for you losing your data!
Notes
The methods described below have been tested using VirtualBox v1.6.6 and v2.0.4 with a Windows XP SP2 32-bit guest and Ubuntu 8.04 32-bit host.
SATA support for raw disk access is broken in VirtualBox v2 versions prior to 2.0.4 (but not v1.6.6, see ticket #2182) ** Edit: v2.0.4 needs a workaround, see the ticket
If you find any errors or spelling mistakes (however small) please let me know via PM (unless it's major). If on the other hand you have suggestions reply to this thread.
And finally: Before replying with a question a) read the manual and b) search the forums. Most questions will have been answered before.
General Standard settings (as chosen by the wizard) will be adequate, taking the following into account:
[I/O APIC] This does not have to be turned on. (It would incur a slight performance penalty). Note: ACPI on the other hand has to be turned on, of course.
[SATA Controller] Enable this since it will offer much better I/O performance (see Installation section)
[Network] Enable one adapter and set its MAC address to that of your real network card (see Windows Activation section).
[Devices] Apart from the network adapter no other devices should be selected at this point, i.e. leaving USB, Audio etc. off. (These can be enabled, as required, after a second hardware profile has been created.)
DMI BIOS settings (information taken from this thread)
If you have an OEM version of Windows XP you should set these additional parameters as follows so your copy of Windows does not have to be reactivated. In total there are 13 such parameters (which you can set, see section 9.13 in the manual). To determine them use the below commands.
Note: Make sure you choose partition type 0x07 (instead of 0x17 - Hidden HPFS/NTFS) if you're planning to use NTFS (recommended) or you might end up with the "Autochk program not found" error when trying to boot natively. (See this posting.)
Configurations fall into the following three categories. By this I mean the location of the primary Windows partition. You might of course decide to use multiple partitions.
Using the first partition on a disk or a whole disk
This is the most straightforward scenario.
Using a primary partition (but not the first)
The Windows installer will try by default to install boot files (ntldr, NTDETECT.COM, etc.) onto the first partition of a disk and will fail if it is not allowed write to it. The solution to this is to set the partition you want to install Windows on as active (aka boot flag). The safest way to do it is to boot a LiveCD in VirtualBox with the raw vmdk attached and set it there. Alternatively, and only if you know what you're doing, you can do the following before and after you create the raw vmdk.
Using a logical disk in the extended partition If you've already got one or more operating systems installed, or you only have one disk, chances are you fall into this category. In this case Windows will try to write to the first (or active) partition just like mentioned above. The solution in this case to create a tiny virtual disk (16MB will do) for Windows to put the boot partition on. Set this as the first disk in VirtualBox.
Last edited by vkov_tinsky on 24. Oct 2008, 16:11, edited 5 times in total.
WARNING: You do not want applications on the Host OS to write to the partitions you're using in the guest as this can lead to data corruption. If you're using e.g. Ubuntu which automounts certain partitions it is advisable that you add the Windows partition(s) to fstab with the read-only flag.
Notes
For in-depth information about virtual disk images please see Tutorial: All about VDIs by TerryE.
The following assumes you have already created partitions (not necessarily formatted) to house your new Windows installation (unless you're planning to use the a whole disk).
Normally you won't want not use the -register parameter under Linux since it will register the vmdk with the root account instead of your own.
After you've created the vmdk you might have to change its permissions, owner and group so VirtualBox can access it.
Check which partition you'll be using (if you're not planning to use the whole disk; This illustrates listing partitions on your first disk.)
Create vmdk for one or more partitions. (This illustrates using only the 2nd partition, i.e. /dev/sda2. Using the -relative parameter means only access to your chosen partitions will be required afterwards.)
This is however still somewhat dangerous since any application running under your account has full access to that partition (or disk).[/size]
Use a separate user, just for raw disk access - (suggested by TerryE)
To set up such a user and give it the right privileges and permissions you can do the following (where MYGROUP is the group you belong to which most likely has the same name as your username - you can check by running the id command):
#! /bin/bash
VBUSER=rawvb # name of custom account created (which is a member of disk group)
VM_NAME=xpraw # name of virtual machine
chmod -R g=u /home/$USER/.VirtualBox # make sure the other user will be able to access VirtualBox settings etc.
xauth extract /home/$USER/cookieTmp $DISPLAY
chmod g+r /home/$USER/cookieTmp
sudo -u $VBUSER XAUTHORITY=/tmp/.Xauthority_$VBUSER xauth merge /home/$USER/cookieTmp
rm /home/$USER/cookieTmp
sudo -u $VBUSER XAUTHORITY=/tmp/.Xauthority_$VBUSER VBOX_USER_HOME=/home/$USER/.VirtualBox VBoxManage startvm $VM_NAME
By default you will be prompted for your password when you run this script since you're trying to run xauth and VBoxManage as a different user. You can get around this by adding a rule to (the end of) /etc/sudoers. As an example the following would allow the user vt to run xauth and VBoxManage startvm xpraw as user rawvb without supplying a password and allowing changes to the environment (since we want to set VBOX_USER_HOME to that of the original user).
Note: By default vi is the editor used to make changes to /etc/visudo. To edit it with a different program (in this example nano) run: sudo EDITOR=/usr/bin/nano visudo
[/size]
Add the newly created vmdk to VirtualBox using the Virtual Disk Manager
Assuming you've given yourself (or a dedicated user) enough privileges VirtualBox should not report any errors.
Last edited by vkov_tinsky on 8. Oct 2008, 04:16, edited 14 times in total.
Note: You might get an error message that Windows cannot format your partition (see here). If that is the case a possible workaround is to use IDE for now (and switch to SATA at step 7).
Boot your Windows XP CD in VirtualBox and install to your chosen partition (or disk). Once you get to the Windows desktop do the following:
[Optional] Disable System Restore (This is recommended if you already have another Windows installation present.) However, if you do wish to disable System Restore, then we strongly recommend that you back up the system at key milestones as discussed in the Backup section below.
Now this profile has been prepared for booting natively. (Warning: Do not boot this native profile anymore through VirtualBox.)
Shut down, enable all additional devices you require(e.g. USB and Audio).
Then boot again in VirtualBox using the vm hardware profile and install guest additions.
[Optional] Download and run the Intel Matrix Storage Manager to install SATA drivers.
Shutdown the VM and change your disk(s) from IDE to SATA and boot up again (using the vm hardware profile, of course).
Note: You might also want to disable the Intel Matrix Storage Event Monitor service (IAANTMON) as well as removing IAAnotif from HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Run since neither of these are needed.
[Optional] Make startup programs only run in VM (and vice versa)
This can be done by checking the name of the network card which in most cases will differ from that of the real adapter. Create a .cmd file (e.g. vm_only.cmd) with the following contents and save it to %windir%\system32:
By the time you've also configured drivers and programs natively (in the Booting Natively section) you might end up with something like this.
[/size]
[Optional] Disable services not required in VM (and vice-versa) Wireless Zero Configuration (WZCSVC) and VirtualBox Guest Additions Service (VBoxService) are not needed in both VM and native hardware profiles. To disable them for just one profile select the desired service (Start -> Run -> services.msc) and choose properties. Under the "Log On" tab you can then choose whether to disable/enable the service for each hardware profile separately (like so).
Note: Other services which are only needed in the native hardware profile will appear with certain graphics drivers, e.g. NVIDIA Display Driver Service (NVSvc) and ATI Hotkey Poller.
[Optional] Enable multi-core/processor support (for booting natively, also applies to Hyperthreading)
Extract the following files from your Windows CD (also creating copies of the non-SMP files):
Note: The reason you have to also modify the existing entry (and copy the original files as shown above) is because Windows will overwrite them when it detects SMP support for the first time.
If you installed your guest Windows to a logical drive, copy boot.ini, NTDETECT.COM and ntldr from the boot to the windows partition.
Last edited by vkov_tinsky on 29. Oct 2008, 17:02, edited 18 times in total.
Note: You might also have to add another entry to boot.ini on your Windows partition to point to the correct disk and partition. (See also this posting.)
Boot your new Windows installation natively (using the native hardware profile)
Notes: It can take quite some time (~2mins) before Windows has finished scanning your hardware, i.e. USB keyboards and mice might not work at first, be patient. Be aware this will not work if auto-login is enabled, so keep it off. (See also this posting.) There is also a chance that you need to have your Windows CD inserted when booting or some drivers might not be installed.
[Optional] If you chose to enable multiprocessor support check in Device Manager whether your PC is shown as ACPI Multiprocessor PC.
Install additional drivers
[Optional] Make startup programs/services only run natively
See points 8) and 9) in the Installation section on how this works. For programs, simply create another .cmd file, this time with the opposite rule:
Note: One could alternatively check for the number of processors (if the smp kernel is in use) using the NUMBER_OF_PROCESSORS environment variable.
[Optional] Get rid of the need to choose the correct hardware profile and boot.ini options on startup
Hardware profiles
Set the native profile as default and reduce the profile prompt timeout to zero (HKLM\SYSTEM\CurrentControlSet\Control\IDConfigDB\UserWaitInterval). To now automatically choose the vm profile when booting in VirtualBox create a script as detailed here (under Hardware profiles - Usage). This basically boils down to modifying a registry key before boot and after shutdown.
boot.ini
There are two options: You can either incorporate this into the script which already sets the correct hardware profile. Create a second boot.ini configuration and replace it before boot (vm). As above you obviously need to restore the original afterwards (so it's ready to be be booted natively again).
The other solution is to create a tiny virtual disk to hold the Windows boot files (boot.ini, NTDETECT.COM and ntldr). This way you automatically have two separate boot.ini configurations.
Last edited by vkov_tinsky on 28. Oct 2008, 22:07, edited 15 times in total.
Instead of using backup software within the guest Windows you can simply save the whole partition (as suggested by TerryE).
Compact and Zero free space on the partition by defragmenting the drive then running scdelete on the guest (where X is the raw partition which you want to back up, as it appears under the guest OS).
You may of course choose a different compression method (e.g. bzip2 or 7zip) depending on space/time trade-offs. Gzip is the fastest and 7zip gives the best compression, with bzip2 lying between the two.[/size]
To subsequently restore your backup you perform the reverse operation:
WARNING: Be extremely careful which partition you specify to restore to. A single digit mistype and you could lose all information stored on another partition![/size]
Last edited by vkov_tinsky on 18. Sep 2008, 21:36, edited 5 times in total.
Switching between VirtualBox and native boots normally would result in Windows XP deciding that reactivation is required due to the sheer number of hardware changes (e.g. chipset, graphics, audio, hdd, NIC - see this article). I have found a remedy for this (which works with my hardware setup at least).
Activate Windows natively
Create a backup copy of %WINDIR%\system32\wpa.dbl after activation.
Set the virtual network adapter's MAC address to that of your real network adapter.
You can determine your MAC address(es) under Linux by running ifconfig.
If you have multiple network cards in your computer you will have to try each in turn (see next step):
http://www.aumha.org/win5/a/wpa.php wrote:[...] It then calculates and records a number based on the first device of each type that was found during setup, [...]
Boot Windows in VirtualBox
If you set the correct MAC address Windows should not ask for reactivation. If it does:
Shut down the guest.
Temporarily mount the Windows partition and replace %WINDIR%\system32\wpa.dbl with your backup copy.
Repeat step 2. using the MAC address of your next network adapter.
Last edited by vkov_tinsky on 18. Sep 2008, 15:53, edited 6 times in total.
A large portion of this howto applies to Windows Vista too but there are some key differences.
SATA
There is no need to install the driver for this anymore (as it comes with Vista).
Multi-processor/core support (untested)
Since Vista has a new boot manager (BCD) I do not know whether it is possible (and whether there is any need) to specify an alternate kernel to have multi-core support when booting natively.
Activation (untested)
This might work as described in the Activation section above.
Hardware profiles (partially tested)
Although hardware profiles are still used in Vista they are hidden from the user, i.e. there is neither an option to create a new one nor the possibility to choose a hardware profile when booting. It is however possible to manually create a second profile by editing the registry.
Creation
Export the active hardware profile: HKLM\SYSTEM\CurrentControlSet\Hardware Profiles\Current
Open the exported .reg file with an editor and replace all occurences of \Current\ with \000X\ (where X is the next available number - check HKLM\SYSTEM\CurrentControlSet\Hardware Profiles.) Then import the modified .reg file.
Export the following key for the active hardware profile: HKLM\SYSTEM\CurrentControlSet\Control\IDConfigDB\AcpiAlias\000a (where a denotes the active profile - HKLM\SYSTEM\CurrentControlSet\Control\IDConfigDB\CurrentConfig will tell you which one is active.)
Edit the exported .reg file, replacing 000a with 000X (just as above) and import the changes back.
Export the profile summary: HKLM\SYSTEM\CurrentControlSet\Control\IDConfigDB\000a
Again edit and replace all occurences of 000a with 000X. Subsequently modify the last digit of HwProfileGuid in your exported .reg.
Warning: The new HwProfileGuid has to be unique. To check this do a search for it in the registry first.
Note: You can also change the FriendlyName value to e.g. "Native Profile".
Once this is done re-import the modified profile summary.
After rebooting Vista should be aware of your new profile: You can check via Services->[Any Service]->Properties->Log On which should now also show your new profile.
Usage
Since it not possible to select a profile upon boot the only way (as far as I know) to achieve profile switching is to edit the registry before running Vista in the VM as the below script illustrates. In this example the normally read-only Vista partition is temporarily mounted writeable and the required registry key is set. Afterwards (when VirtualBox is closed) it is returned back to its original value (so Vista can be booted natively again).
The regsetdword utility source is attached. (It is based on Petter Nordahl-Hagen's ntreg library.)
Note: It is not possible to substitute VirtualBox with VBoxmanage since that will exit as soon as the VM has been initialised (unless of course a loop monitors the vm state for example).
Replaced grep in favour of find for selective startup
Added comment to highlight ACPI option requirement.
Removed GParted Live reference from Partitioning section (since it doesn't seem to work when creating VMDKs with the -relative parameter)
Instruction changed from uninstall to disable in Installation section
18/09/08 (vkov_tinsky)
Corrected commands (added sudo) in Backup section
Expanded script (running VirtualBox as different user) under VMDK usage section
Added DMI BIOS information to configuration section
Changed Activation solution
Changed selective startup example to use name of network device instead of MAC (since this will be need to be the same for activation purposes).
18/09/08 (terrye)
Added defrag before sdelete in backup section
Reformatted DMI script
BBCode tweaks to remove extraneous blank lines.
19/09/08 (vkov_tinsky)
Corrected multiprocessor enabling steps (now instructs to create copies of the non-SMP files first)
Completed VBox startup script how-to (for using with different user)
Replaced .bat with .cmd (selective startup)
Mentioned possibility for selective startup check to use env. var. detailing number of processors
20/09/08 (vkov_tinsky)
Corrected mistake in VBox startup script, added permission change
Corrected several spelling mistakes
08/10/08 (vkov_tinsky)
Added section detailing challenges with Vista
21/10/08 (vkov_tinsky)
Added point about selective service startup
Added a couple of screenshots to illustrate both types of selective startup
24/10/08 (vkov_tinsky)
Added note about possible need to modify boot.ini before booting natively
Added note about auto-login preventing device driver installation on first native boot
Added command to disable System Restore
Added comment about wrong NTFS partition type possibly causing boot problems
Added comment about needing to have Windows CD inserted in certain cases to have drivers installed correctly (when booting natively for the first time)
Modified note about SATA now that v2.0.4 is out
25/10/08 (vkov_tinsky)
Modified note about SATA: Still broken in 2.0.4 (but works with workaround)
Added notes about removing need for boot.ini & hardware profile choice on boot (in Booting natively section)
28/10/08 (vkov_tinsky)
Added comment about possible disadvantage of using an existing installation
29/10/08 (vkov_tinsky)
Added note about SATA possibly causing problems when Windows is trying to format the partition
11/12/09 (vkov_tinsky)
Attached regsetdword source to thread
Last edited by vkov_tinsky on 11. Dec 2009, 22:59, edited 2 times in total.