PCIe passthrough of video card(s)

Discussions related to using VirtualBox on Linux hosts.

PCIe passthrough of video card(s)

Postby erhardm » 1. Dec 2011, 19:10

Hi,

I have a box with a 990FX chipset which supports IOMMU running linux.

I would like to know if this scenario is possible:

1 x low-end Nvidia card used for the host
1 x AMD 6950 using pci-passthrough on a linux VM
1 x Nvidia CUDA capable card using pci-passthrough on another linux VM.

The cards on the VMs would be used for GPGPU.
I DON'T need VGA passthrough, I would only use them for GPGPU. Also I could plug in each card a separate monitor.

If it's not possible, could you give some technical reasons?
Theoretical is it possible, but not yet implemented?

I would test it but unfortunately I have just the AMD 6950 card and before doing some buying mistakes(?) and trying to get the cards makes sense to hear from you guys.

Regards
erhardm
 
Posts: 11
Joined: 18. Nov 2010, 01:55
Primary OS: Linux other
VBox Version: PUEL
Guest OSses: Win32 OS's/*UNIX OS's

Re: PCIe passthrough of video card(s)

Postby sbesch » 8. Dec 2011, 21:00

I also have been trying to get PCI passthrough working and have learned a few things:

1) Motherboard and BIOS must support VT-d (intel) /IOMMU (AMD). Usually this info is available online before you buy: read manuals and processor spec-sheets.
2) Kernel must be quite recent (I believe newer than 2.6.35) and must have been compiled with IOMMU support enabled. V11 of ubuntu uses version 3 of the kernel and this works as shipped.
3) Here is the worst bit: The PCI slot must NOT use a shared interrupt. This is a VirtualBox limitation, and is a real problem because almost every motherboard that supports IOMMU/VTd does not have any PCI slots that are on a non-shared interrupt. Intel won't tell me if any of their boards meet this requirement (without signing an NDA, believe it or not!) and the information is not available in the manuals. Using lspci, I can confirm that the DQ67SW (which I have) has the only PCI slot on a shared interrupt. The only unshared interrupt appears to be unused. I did find an ASUS motherboard which the manual claims to have a PCI slot on an unshared interrupt, but I can't recommend it because I have not actually evaluated it myself. Note that the same restrictions apply for PCIe as for PCI. I have seen no boards at all that have unshared PCIe slots.

The upshot is that until VirtualBox supports shared interrupts on PCI passthrough, the feature will remain essentially useless. I believe that they will eventually support shared interrupts, since some of their competition already does (XEN for example) and in order to really be a GP virtualization product they will have no choice.

P.S. I thought that I should at least identify the ASUS board mentioned above. It's the ASUS MSA99X EVO, with AMD3+ socket. It supports IOMMU and has an unshared interrupt on the (only) PCI slot (see page 32 of the manual). Caveat Emptor---I have NOT yet tested this MoBo/CPU combination.
sbesch
 
Posts: 2
Joined: 8. Dec 2011, 20:19
Primary OS: Ubuntu other
VBox Version: PUEL
Guest OSses: Win2K, XP, 7

Re: PCIe passthrough of video card(s)

Postby Sasquatch » 21. Dec 2011, 00:48

As noted in the User Manual, there is a limitation to the passthrough. Cards that depend on the agp_gart module cannot be send to the VM. Since all video cards depend on that module to date, it won't happen. Odd enough, my Host (Debian Sid), doesn't show that module as loaded. I was certain it used to be.
Read the Forum Posting Guide before opening a topic.
VirtualBox FAQ: Check this before asking questions.
Online User Manual: A must read if you want to know what we're talking about.
Howto: Install Linux Guest Additions
Howto: Use Shared Folders on Linux Guest
See the Tutorials and FAQ section at the top of the Forum for more guides.
Try searching the forums first with Google and add the site filter for this forum.
E.g. install guest additions site:forums.virtualbox.org

Retired from this Forum since OSSO introduction.
Sasquatch
Volunteer
 
Posts: 17800
Joined: 17. Mar 2008, 13:41
Location: /dev/random
Primary OS: Debian other
VBox Version: PUEL
Guest OSses: Windows XP, Windows 7, Linux

Re: PCIe passthrough of video card(s)

Postby erhardm » 21. Dec 2011, 20:55

Code: Select all   Expand viewCollapse view
td@td-DT:~$ modprobe -l | grep agp
kernel/drivers/char/agp/sis-agp.ko

td@td-DT:~$ modprobe -l | grep gart
td@td-DT:~$

It seems neither my Host(Ubuntu 11.04) has agp_gart loaded.

Is there a way to identify if a MB uses shared pci(e) interrupts? Or the opposite?
erhardm
 
Posts: 11
Joined: 18. Nov 2010, 01:55
Primary OS: Linux other
VBox Version: PUEL
Guest OSses: Win32 OS's/*UNIX OS's


Return to VirtualBox on Linux Hosts

Who is online

Users browsing this forum: No registered users and 21 guests