Attaching a physical 5Tb external HD to a Windows 2003 Server Guest

[Birdual]

Hi, I am trying to attach a physical LaCie 5Tb USB hard disk (in mode -rawdisk) to a guest correctly running a Windows 2003 Server 64bits.
(My host is Windows 8.1 x64 16Gb RAM, running VirtualBox 5.0.20 r106931)

The problem is that when I attach the hard disk, following the standard procedure (see below), the guest gets frozen in the welcome screen (the black one with the small progress bar, just after the bios POST, sorry I cant upload images yet) probably when detecting the new disk.

I have tried to attach it under the current IDE controller, and also under SCSI and SATA controllers, with no success, all get frozen in the same point.
I have tried to format the hard disk as GPT, also as MBR, with NTFS and FAT32, and with big and small partitions (for example 512Gb)
I have also tried turning the disk offline in the host with the sequence of commands:

Code: Select all

diskpart
list disk
select disk 1
offline disk
attributes disk clear readonly
exit

(disk is #1 in Disk Manager)

BTW, booting the same guest from a Linux Rescue Boot CD (GParted) it sees the 5Tb disk perfectly, no freeze, suggesting I have a problem with my Windows 2003 Server boot process.

When diagnosing the problem, my first surprise is the result of the next command (run as admin):

Code: Select all

vboxmanage internalcommands listpartitions -rawdisk \\.\PhysicalDrive1

Number  Type   StartCHS       EndCHS      Size (MiB)  Start (Sect)

Note the result is empty!

Even despite of this error, I carry on and create my .vmdk with this command:

Code: Select all

vboxmanage internalcommands createrawvmdk -filename "disk1.vmdk" -rawdisk "\\.\PhysicalDrive1"

Resulting in this .vmdk file:

Code: Select all

# Disk DescriptorFile
version=1
CID=c1e6785e
parentCID=ffffffff
createType="fullDevice"

# Extent description
RW 9767541168 FLAT "\\.\PhysicalDrive1" 0

# The disk Data Base 
#DDB

ddb.virtualHWVersion = "4"
ddb.adapterType="ide"
ddb.geometry.cylinders="16383"
ddb.geometry.heads="16"
ddb.geometry.sectors="63"
ddb.uuid.image="f00d66e5-f475-46c8-9d73-d8a7673cc2a9"
ddb.uuid.parent="00000000-0000-0000-0000-000000000000"
ddb.uuid.modification="fa879390-204d-4b85-abc1-be3f90306ed2"
ddb.uuid.parentmodification="00000000-0000-0000-0000-000000000000"
ddb.geometry.biosCylinders="1024"
ddb.geometry.biosHeads="255"
ddb.geometry.biosSectors="63" 

As said, this .vmdk works perfectly when booting with a Linux Rescue Boot CD but fails with Windows 2003 Server.

Any clue?

Thank you!

[mpack]

Unless you have Win2K3 drivers for this device then I doubt you'll get it to work properly. Win2K3 (i.e. XP) has no EFI BIOS support hence no support for drives >2TB. Consider how long ago 2003 was. It's likely to be USB3 as will (if plugged into a USB3 port), which Win2K3 also doesn't support.

OTOH, you can attach anything the host supports, if you attach it as a shared folder.

[Birdual]

Unless you have Win2K3 drivers for this device then I doubt you'll get it to work properly. Win2K3 (i.e. XP) has no EFI BIOS support hence no support for drives >2TB. Consider how long ago 2003 was. It's likely to be USB3 as will (if plugged into a USB3 port), which Win2K3 also doesn't support.

OTOH, you can attach anything the host supports, if you attach it as a shared folder.

Thank you mpack for your answer.

I can add some facts, Win2K3 successfully detects the hard disk when attached through USB 2.0 (not 3.0 afaik, and as you pointed out). The 2Tb limit can be overcome using Win2K3 Disk Manager and turning the hard disk into "GPT" format, then you can create a whole 5Tb partition that Win2K3 will use without problems.

The only objection is that you'll have to fill a whole 5Tb space with a USB 2.0 speed...

Now my question gets different but essentially equal... Why can Win2K3 access my 5Tb hard disk and see all its capacity as USB 2.0 attachment, but gets stuck at boot when attaching it as rawdisk? This one is preferable, as will theoretically transfer at host usb 3.0 speed.

Thanks

[mpack]

The BIOS doesn't recognize disks >2TB (*), because that requires an EFI BIOS (i.e. BIOS level support for GPT). VirtualBox does have an EFI BIOS option, but Win2K3 would have no idea how to interact with one.

(*) Not technically correct. The limitation isn't on size, it's on number of sectors that can be part of one partition. Attempts have been made in the industry to get around the size limit by having larger sectors, but IMHO that's a bad road to go down. The 512 byte sector size is heavily ingrained in computing. 2TB is the limit because it is (2^32)*512 bytes.

Did you ignore my suggestion to use a shared folder? AFAIK that's the only way you'll get full compatibility, full speed.

[Rootman]

I'm sorta confused. Is the issue the fact of the lack of EFI support in 2003? I thought it was the spotty GPT support in 2003, two different animals. Even lowly XP - ONLY the 64 bit version - can support GPT. Windows 2003 is the same level as XP. So unless if the guest is 64 bit you are SOL.

The disk MUST be setup as GPT to show the entire thing (barring the e512 idea Mpack mentions - not a good idea). Since you already tried MBR (limited to 2TB) without any luck, and it still failed, I would say there are other issues at hand. Your USB experiments prove that as well.

And XP / 2003 only support GPT as a SECONDARY data disk, not a boot disk - and this is where EFI support lacking in XP / 2003 comes to play.

Here is about XP - 2003 should be the same: https://www.microsoft.com/resources/doc ... x?mfr=true

[mpack]

Obviously, what Windows supports only comes into play if Windows is running - regardless of whether you mean XP or later, or whether 32bit or 64bit. Prior to the early boot process completing it is the BIOS which is running. If that BIOS doesn't support GPT partitions then you certainly have hurdles to overcome before you can boot from a GPT partitioned drive. You should have less problems with a secondary drive, so long as the BIOS doesn't see anything that confuses it.

[Rootman]

Sorry to post again. If I am reading between the lines then the user is putting the new disk on as a secondary disk and not a boot disk, becasue they mention that the OS will at least see the disk in USB 2.0 mode, and obviously the user is not booting from the USB disk to the OS.

The argument about the BIOS is is specious, the BIOS does not care if it's GPT or not, it sees it as SATA, it's the OS that determines if it can read the partitions and contents from a GPT disk or not. And again, only 64 bit versions of Win 2003 / XP can even understand GPT.

To the OP. You have a real issue, it may or may not be VB that is "causing" the issue. What you want to do is pretty odd, at least for that size of a disk. How about mounting the disk to the HOST and sharing it to the guest over the network? It would not be as fast but should beat out USB 2.0 speeds.

EDIT: OP: One thing you may want to try to see if you can narrow down the issue. Setup a quick Windows 7 / server 2008 or Windows 10 / 2012 guest, if the disks will attach to any of those then the issue is the OS. If it does not, you are no closer to an answer but have eliminated the OS as the problem.

Also as another test, try setting up a virtual disk, make it any size but switch it over to GPT. Again, if the guest 2003 OS is 32 bit - this will not be an option. In the event you are running a 64 bit 2003 guest then you should be able to setup the disk and change it to GPT in the Windows disk management tool. If Windows 2003 can see and function with this disk then the issue is either with VB or just the sheer size of the 5TB drive array. You have proved that your guest can manage a GPT disk.

I see that VB itself has a limit of 2TB on at least my system when setting up a disk. I am not sure if that is coded into VB or a reflection of the fact that I have my guests on a MBR based disk (thus the 2TB limit is the function of the MBR disk).

[mpack]

Hmm. If the BIOS doesn't care ("it sees it as SATA") then I'm left to wonder why the industry saw fit to invent the EFI BIOS, or why the VirtualBox non-EFI BIOS could possibly have a problem with this disk.

[Rootman]

EFI and GPT are two totally different things, yes, they are joined at the hip, but they are TWO different standards, NORMALLY one requires the other, but they are two different things.

EFI is a boot type, instead of the system providing the BIOS to bootstrap the system, it's a "soft BIOS" on the disk, instead of relying on the age old method of CSM / "BIOS" and booting up in 8bit (essentially DOS) until the Windows kernel is loaded, EFI throws the system into protected mode much sooner. It's also not plagued with all the CSM / "BIOS" hacks that have been passed down through the years. It's a little faster and actually cleaner. If the system is booted EFI, it simply looks for the EFI files on the first FAT32 partition on the boot disk in the proper folders and passes the boot process to them.

The EFI standards LINKED the process to the GPT disk partition standard, but the two are not the one in the same thing. In fact there has been effort to boot EFI on a standard MBR disk, I've seen posts on people doing it. WHY exactly they want to do this I don't know but it can work. It does NOT work that way "out of the box".

GPT is a disk partitioning scheme to replace MBR. It provides more room for storing block information - which allows a system to see a disk greater than 2TB. It also allows more than 4 primary partitions (the infamous 3 primaries and 1 extended). It is cleaner and more efficient than the old MBR partition table layout. And, believe it or not, there are people who have figured out how to boot a CSM system on a GPT disk. Why? I don't know that either. It's a different solution than the infamous 4K sectors hack thing.

So, out of the stock box (barring the bizarre tricks people have done mentioned above) EFI MUST have GPT, and this is important to understand - FOR THE BOOT DISK.

You can easily hang an MBR data disk off a EFI booted PC with a GPT compliant booted OS (Vista and newer or XP / 2003 64 bit) just fine. It has nothing to do with the BIOS, it's the OS that determines if the GPT disk can be read or not. The BIOS (technically the system SETUP firmware) must SUPPORT EFI booting in order to use an EFI / GPT boot pair.

Conversely, you can hang a GPT DATA disk off a CSM booted GPT compliant OS just fine too. I have done it many times. If a disk is greater than 2TB, it MUST be GPT (unless using 4k sectors, I won't do that).

So, again, I am not sure why the OPs situation is not working. I think he is using 32 bit 2003 - which simply will NOT read GPT. As a data disk (non boot disk) , XP / 2003 64 bit or Vista and newer of 32 or 64 bit should see a GPT data disk hung off it, regardless of how it's booted, CSM or EFI. If it doesn't on Virtualbox then there are other things going on. The suggestions I tossed out are ways of testing it, especially trying a known GPT readable OS like 7 or 10.

[Birdual]

Sorry for the delay...

Did you ignore my suggestion to use a shared folder?

How about mounting the disk to the HOST and sharing it to the guest over the network?

The point to avoid it is that the guest is a file server, so it will want to share (again) the folders over the network. Perhaps I'm wrong but I think the guest can't re-share a folder that actually comes as shared folder from the host.

Setup a quick Windows 7 / server 2008 or Windows 10 / 2012 guest, if the disks will attach to any of those then the issue is the OS

Also as another test, try setting up a virtual disk, make it any size but switch it over to GPT

Good tests, I'll try and report.

I am not sure why the OPs situation is not working. I think he is using 32 bit 2003

Confirmed that it is a Windows 2003 Server 64bit (see attachment). I think its capability to read GPT is also confirmed because it accesses GPT disk successfully when attached to VB through USB 2.0; it only fails when attaching as host physical disk to VB.
A variant of your option could be that this Windows 2003 system is corrupted, or flawed for any reason...

[Birdual]

Also as another test, try setting up a virtual disk, make it any size but switch it over to GPT.

Results were interesting. The process was:

• - Through command line (GUI limits to 2Tb) I created a 5000000 Mb (not exactly 5 Tb, but suitable for test) virtual hard disk (.VDI):
  
  • vboxmanage createhd --filename BigDisk.vdi --size 5000000
  - Added it as Secondary in the IDE chain (same result for SCSI). W2003booted without getting stuck (when physical it does!).
  
  - Got into guest Disk Administrator and surprise: The disk capacity appeared truncated to 786.69 Gb. Yoy may say: Windows fault, or BIOS fault. But keep in mind: if disk is USB, W2003 reports capacity correctly! If it is physical W2003 gets stuck; and now, if it is a .VHD, W2003 reports wrong capacity.

5Tb truncated.png (10.3 KiB) Viewed 4025 times

BTW, why that strange number, 786.69 Gb? It turns out to be almost exactly the remainder when dividing disk capacity in 2Tb chunks:

5000000 Mb = 2 Tb + 2 Tb + 786.81 Gb

So Windows seems to only detect the last "2 Tb chunk", in this case 786.69 Gb.

Now Windows turns guilty again, but... sure?

I think the critical proof is the USB one: the whole 5Tb are visible for the OS, and it is able to format as GPT, read and write, etc. IMHO, this invalidates any suspicion with regard to Windows or BIOS, and place it on the way that VB exposes big capacity disks to OS's like this (pending to test in newer OS).

Thank you for your attention, even more when being paid to an old Windows 2003!

[scottgus1]

Long thread, may have ben mentioned already but...

I have been able to connect 3 and 4 TB drives through USB 2 and 3 to an XP-32bit physical PC and was able to access the whole drive. It's a combination of the USB and special wizardry of the drive & enclosure manufacturers.

I also had a 3TB drive specially formatted to use on XP-32, was able to use all of it as a secondary drive attached through SATA. It died on me one day, and the warranty replacement was not formatted in the same way, and only the first two TB could be used.

So the 5TB will be usable through USB but no other way. Just a limitation of the old pre-multi-TB OS.

[scottgus1]

Here's another idea, not tried so ymmv, no USB:
5TB drive connects to host via SATA.
Make three vdi's for the guest on the drive, 2TB, 2TB, 1TB approx. (may have to be fixed, because of the later steps, so try fixed if you get errors trying dynamic vdi's.)
If you need more guest drive spaces on the controller, try SATA controller in the guest, like XP can do.
In the guest, set the three new "drives" as dynamic in the guest OS, then span them: https://technet.microsoft.com/en-us/lib ... 2147217396

Another thought, since server 2003 and XP are cousins, see "USB3.0 in XP" viewtopic.php?f=28&t=74575