PCI passthrough

[ivarad]

Hi all,

I'm trying to set up a virtualbox-5.1 with PCI passthrough on
host Ubuntu 17.04 (4.10.0-28-generic #32-Ubuntu SMP Fri Jun 30 05:32:18 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux)
installed on PC with
CPU Intel® Core™2 Quad Processor Q9400
RAM 4Gb
integrated video
PCI card trying to passtrough is a National Instruments DAC card
guest OS is Windows 2000.

Unfortunately, for now I was not able to make guest OS to see the PCI card, that is why I'd like to ask for some help. Let start from the very beginning

1. VT-d and IOMMU are enabled from BIOS. Kernel log contains the following

Code: Select all

Jul 18 14:21:14 epitt-PC kernel: [    0.000000] DMAR: IOMMU enabled
Jul 18 14:21:14 epitt-PC kernel: [    0.024112] DMAR: Host address width 36
Jul 18 14:21:14 epitt-PC kernel: [    0.024113] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
Jul 18 14:21:14 epitt-PC kernel: [    0.024122] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap c9008020e30272 ecap 1000
Jul 18 14:21:14 epitt-PC kernel: [    0.024123] DMAR: DRHD base: 0x000000fed91000 flags: 0x0
Jul 18 14:21:14 epitt-PC kernel: [    0.024126] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap c0000020630272 ecap 1000
Jul 18 14:21:14 epitt-PC kernel: [    0.024127] DMAR: DRHD base: 0x000000fed92000 flags: 0x0
Jul 18 14:21:14 epitt-PC kernel: [    0.024130] DMAR: dmar2: reg_base_addr fed92000 ver 1:0 cap c0000020630272 ecap 1000
Jul 18 14:21:14 epitt-PC kernel: [    0.024130] DMAR: DRHD base: 0x000000fed93000 flags: 0x1
Jul 18 14:21:14 epitt-PC kernel: [    0.024133] DMAR: dmar3: reg_base_addr fed93000 ver 1:0 cap c9008020630272 ecap 1000
Jul 18 14:21:14 epitt-PC kernel: [    0.024134] DMAR: RMRR base: 0x000000d7c00000 end: 0x000000dfffffff
Jul 18 14:21:14 epitt-PC kernel: [    0.024135] DMAR: RMRR base: 0x000000cffbc000 end: 0x000000cfffffff
Jul 18 14:21:14 epitt-PC kernel: [    0.123762] DMAR: Forcing write-buffer flush capability
Jul 18 14:21:14 epitt-PC kernel: [    0.123763] DMAR: Disabling IOMMU for graphics on this chipset
Jul 18 14:21:14 epitt-PC kernel: [    0.809176] DMAR: No ATSR found
Jul 18 14:21:14 epitt-PC kernel: [    0.810237] DMAR: dmar2: Using Register based invalidation
Jul 18 14:21:14 epitt-PC kernel: [    0.810245] DMAR: dmar0: Using Register based invalidation
Jul 18 14:21:14 epitt-PC kernel: [    0.810250] DMAR: dmar3: Using Register based invalidation
Jul 18 14:21:14 epitt-PC kernel: [    0.810264] DMAR: Setting RMRR:
Jul 18 14:21:14 epitt-PC kernel: [    0.810322] DMAR: Setting identity map for device 0000:00:1a.0 [0xcffbc000 - 0xcfffffff]
Jul 18 14:21:14 epitt-PC kernel: [    0.810372] DMAR: Setting identity map for device 0000:00:1a.1 [0xcffbc000 - 0xcfffffff]
Jul 18 14:21:14 epitt-PC kernel: [    0.810422] DMAR: Setting identity map for device 0000:00:1a.2 [0xcffbc000 - 0xcfffffff]
Jul 18 14:21:14 epitt-PC kernel: [    0.810469] DMAR: Setting identity map for device 0000:00:1a.7 [0xcffbc000 - 0xcfffffff]
Jul 18 14:21:14 epitt-PC kernel: [    0.810517] DMAR: Setting identity map for device 0000:00:1d.0 [0xcffbc000 - 0xcfffffff]
Jul 18 14:21:14 epitt-PC kernel: [    0.810564] DMAR: Setting identity map for device 0000:00:1d.1 [0xcffbc000 - 0xcfffffff]
Jul 18 14:21:14 epitt-PC kernel: [    0.810614] DMAR: Setting identity map for device 0000:00:1d.2 [0xcffbc000 - 0xcfffffff]
Jul 18 14:21:14 epitt-PC kernel: [    0.810664] DMAR: Setting identity map for device 0000:00:1d.7 [0xcffbc000 - 0xcfffffff]
Jul 18 14:21:14 epitt-PC kernel: [    0.810675] DMAR: Prepare 0-16MiB unity mapping for LPC
Jul 18 14:21:14 epitt-PC kernel: [    0.810716] DMAR: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
Jul 18 14:21:14 epitt-PC kernel: [    0.810857] DMAR: Intel(R) Virtualization Technology for Directed I/O

however, nothing about DMA-PCI. This may mean that CONFIG_DMAR compilation option is not set, but

Code: Select all

grep CONFIG_DMAR /boot/config-`uname -r` 
CONFIG_DMAR_TABLE=y
CONFIG_DMARD09=m
CONFIG_DMARD10=m

So first question: does this allow to consider that host OS is configured properly? Or I should try to recompile kernel with CONFIG_DMAR=y?

2. Installation of virtualbox was very smooth, all modules seems to be added to the kernel

Code: Select all

lsmod | grep vbox*
vboxpci                24576  0
vboxnetadp             28672  0
vboxnetflt             28672  0
vboxdrv               471040  3 vboxnetadp,vboxnetflt,vboxpci

however, extension packs for the virtualbox-5.1 did not contain "Experimental support for PCI passthrough on Linux hosts" as suggested by the manual.

Does it mean that I need to install another extension pack, or PCI passthrough is already integrated in virtualbox-5.1 main installer?

3. I start virtual box as

Code: Select all

virtualbox "EpiTT-Windows 2000" --pciattach 11:01.0@01:02.0

where 11:01.0 is PCI card address on host and 01:02.0 is suggested address on guest.

Is it enough to start virtualbox as a simple user, or I should do it as root to get PCI passtrough working?

4. I start VM with "Enable VT-x/AMD-V" and "Enable Nested paging" options in Settings -> System -> Acceleration, but in VM status bar window Additional feature status shows "VT-x/AMD-V: Active"; "Nested paging:Inactive". A similar situation was already discussed on this forum at

Code: Select all

viewtopic.php?p=64833

the main conclusion from there is that core2quad CPU does not support nested paging.

Are my troubles with PCI passthrough caused by the fact that core2quad just doesn't support nested paging, so I have to stop trying to get it working on this hardware?

Sorry, for a very long post, but I tried to be as specific as possible.
Thank you in advance for any help and/or suggestions

[WFV]

You are asking for help with a Linux guest on a Windows host in the Linux Host forum...

A couple questions come to mind, the first might be entirely irrelevant:
1. does the Windows 2000 host see the Nat'l Instr DAC pci (is it accessible via Windows)?
2. Have you looked for possible drivers for that for Ubuntu?
3. Do you have the NI install disk and does it have instructions for installing in Linux?
else Googled it?

As for my machine, Linux host, none of the 5 guests show any pci info in the Machine settings however, all of them access my CD and DVD drives that are connected to a pci board without any issues.

[ivarad]

Thank you for you reply

You are asking for help with a Linux guest on a Windows host in the Linux Host forum...

I have a Linux host and Windows 2000 guest, I mentioned it explicitly in the first post

host Ubuntu 17.04 (4.10.0-28-generic #32-Ubuntu SMP Fri Jun 30 05:32:18 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux)
...
guest OS is Windows 2000.

As regard to the questions

1. does the Windows 2000 host see the Nat'l Instr DAC pci (is it accessible via Windows)?

Linux host does see the card, probably I even can find drivers for it, but I have no use for it from Linux. In fact all this not too elegant setup appeared just because one of the old control computer died. That PC used proprietary software for Windows, which requires this PCI card, and does not want to run on any modern Windows OS. From the other hand I don't have now any sufficiently old PC to install at least Windows 2000 on in. Sounds really stupid, but as it is

2. Have you looked for possible drivers for that for Ubuntu?

Not, as described I don't need it, however, if I'll be desperate enough to try it on wine, I may try to do it

3. Do you have the NI install disk and does it have instructions for installing in Linux? else Googled it?

I have all software I need for the guest system, but it does not see the PCI card

As for my machine, Linux host, none of the 5 guests show any pci info in the Machine settings however, all of them access my CD and DVD drives that are connected to a pci board without any issues.

Thank you for this info, I was wondering if the in VM settings there should be something about PCI's if they are attached.

I have some ideas still to try. I'll post any results I'll get here too.

[ivarad]

An update

I have figured out that PCI passthrough will work only with ICH9 chipset (unfortunately it is not clearly stated in the PCI passthrough section of the manual, but it can be understood from the chipset description). Now I'm trying to create a new VM with ICH9 chipset.

Attaching PCI card with

Code: Select all

 VBoxManage modifyvm "EpiTT-Windows 2000" --pciattach 11:01.0@2:01.0

leads to the fail of the VM start with the message

The VM session was aborted. Result Code: NS_ERROR_FAILURE (0x80004005) Component: SessionMachine Interface: ISession {7844aa05-b02e-4cdd-a04f-ade4a762e6b7}

Trying

Code: Select all

 VBoxManage modifyvm "EpiTT-Windows 2000" --pciattach 11:01.0

gives

No PCI Bus is available to register the device with. This is usually a misconfiguration or in rare cases a buggy pci device. (VERR_PDM_NO_PCI_BUS).

When PCI card is detached with

Code: Select all

 VBoxManage modifyvm "EpiTT-Windows 2000" --pcidetach 11:01.0

VM starts as usual

The first error looks like an issue with the virtual box kernel module, I tried to restart it, but it gave no result. In any case it should not be just the module itself, because after detaching the card error disapears.

Any suggestions how this can be solved?

[socratis]

Disclaimer: I haven't tried it, I can't try it (no hardware/software), everything is from memory and from 3rd party posts.

In order to have PCI passthrough you have to have a CPU/motherboard/BIOS that provides VT-d. Note the "-d" part, it's not the same as the more common VT-x. On top of that you have to have a Linux host with a kernel that has support for IOMMU.

Other than that, Google is your friend. You are embarking on a journey that very few have successfully traversed...

PS: Off-topic, which NI-DAQ card are you trying to pass through? Are you using a LabVIEW-based program?

[ivarad]

Yes, it is VT-d double checked it right now and it is enabled. IOMMU also seems to be enabled in the core (I put kernel log in the first post where it says it). So, I'll have a lot of fun with it

The card is NI PCI-6025E. Software is not LabView based, it was designed by some old company that doesn't exist any more (or was bought by some other company), in any case they provide no support anymore for neither there software nor hardware

[rockandsalt]

Hi Ivarad,

Just reviving this old post, did you manage to pull it off? I too have same issue with old labview software on windows xp and I need to set up a pci passthrough with a NI card.

Cheers