KVM - graphics card passthrough not available

by vigilian   Last Updated October 26, 2017 10:00 AM

I have a problem to passthrough the graphic card nvidia of my system into a vm with kvm-qemu.

my system:

z170 chipset based motherboard
nvidia 970 gtx

According to the wikipedia page this is totally supported by iommu.

My OS is ubuntu 16.04.3 server edition. Everything is up-to-date and as a kvm-qemu manager I'm using the kimchi release.

I've followed this tutorial. I didn't patch the kernel with the acs patch because I'm using the 4.10 kernel but maybe I should?

Anyway I've got this result:

  • IOMMU seems to be activated with the gub command

    Oct 24 22:45:03 47kdocker kernel: [ 0.000000] DMAR: IOMMU enabled

  • there seems to be 2 dmar activated, I don't know if it's normal. One with the intel integrated graphic card. The other one with everything else.

    @47kdocker:/sys/devices/virtual/iommu/dmar1/devices$ ls
    0000:00:00.0 0000:00:14.2 0000:00:1b.0 0000:00:1c.0 0000:00:1d.0 0000:00:1f.3 0000:01:00.1
    0000:00:01.0 0000:00:16.0 0000:00:1b.2 0000:00:1c.2 0000:00:1f.0 0000:00:1f.4 0000:06:00.0
    0000:00:14.0 0000:00:17.0 0000:00:1b.3 0000:00:1c.3 0000:00:1f.2 0000:01:00.0 0000:07:00.0
    @47kdocker:/sys/devices/virtual/iommu/dmar1/devices$ ls ../../dmar0
    devices intel-iommu power subsystem uevent
    @47kdocker:/sys/devices/virtual/iommu/dmar1/devices$ ls ../../dmar0/devices

  • The IOMMU group 1 is populated with 3 devices: gpu, audio controller from the gpu and the pci-express hub. Is it the problem?

    find /sys/kernel/iommu_groups/ -type l /sys/kernel/iommu_groups/1/devices/0000:01:00.1

    00:01.0 PCI bridge [0604]: Intel Corporation Sky Lake PCIe Controller (x16) [8086:1901] (rev 07)

  • But apparently I've loaded the graphic card with the good modules so normally it should be ready to be sent to the vm.

    01:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 970] (rev a1)
    Subsystem: eVga.com. Corp. GM204 [GeForce GTX 970]
    Kernel driver in use: vfio-pci
    Kernel modules: nvidiafb, nouveau
    01:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller (rev a1) │ Subsystem: eVga.com. Corp. GM204 High Definition Audio Controller
    Kernel driver in use: vfio-pci
    Kernel modules: snd_hda_intel

  • According to the creator of kimchi, the device should be detected, but only the audio controller is detected.

  • I've tried to add that to my xml VM from there with the needed modifications :

    qemu:commandline> qemu:env name="QEMU_PA_SAMPLES" value="4096" /> qemu:env name="QEMU_AUDIO_DRV" value="pa" /> qemu:arg value="-device" /> qemu:arg value="ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1" /> qemu:arg value="-device" /> qemu:arg value="vfio-pci,host=[GPU],bus=root.1,addr=00.0,multifunction=on,x-vga=on" /> qemu:arg value="-device" /> qemu:arg value="vfio-pci,host=[Audio],bus=root.1,addr=00.1" /> qemu:arg value="-cpu" /> qemu:arg value="host" /> /qemu:commandline>

I really don't know what to check next... I would like to not install a graphic interface on this server. So do I need to add the igfx option from there ? I' m really out of ideas here.Is it because of the IOMMU group? So Is it because of the ACS patch ? It seems that if the card was correctly detected as available, kimchi interface and so libvirtd would have detected as that... But since it is not present in the list maybe the problem is more upstream than libvirtd?

Thanks for the help.

Related Questions

Splitting an IOMMU group for PCIe Passthrough

Updated February 25, 2017 05:00 AM

How do I enable KVM device passthrough in Linux?

Updated July 19, 2018 21:00 PM

kvm vendor_id state='on' causes hypervisor freeze

Updated October 19, 2017 17:00 PM