Ask Your Question
1

vfio-pci vs pci-assign for sr-iov "direct" ports

asked 2015-03-11 10:48:44 -0500

etlv gravatar image

updated 2015-03-11 10:49:47 -0500

Hi All,

I am using Ubuntu 14.04 on my compute node which is running "juno" release. After updating my kernel lately from 3.13 to 3.16, I noticed I cannot create anymore VMs with sr-iov ports. Further investigation of nova-compute logs showed that since the upgrade, during VM spawning qemu is using "-device vfio-pci" instead of "-device pci-assign" of the VM vNICs.

I would like to stick with pci-assign devices.

Any idea how to make nova\qemu use it instead of vfio-pci?

Thanks in advance,

Itai

edit retag flag offensive close merge delete

Comments

I think it will use vfio if the vfio_pci kernel module is loaded.

darragh-oreilly gravatar imagedarragh-oreilly ( 2015-03-11 13:46:24 -0500 )edit

Thanks, I will try it.

etlv gravatar imageetlv ( 2015-03-12 08:21:20 -0500 )edit

It seems like even if the vfio_pci kernel module is UNloaded, every time nova is booting the VM, the module is re-loaded... any other ideas? I would like to force nova use -device pci-assign...

etlv gravatar imageetlv ( 2015-03-16 10:44:26 -0500 )edit

try blacklisting that module

darragh-oreilly gravatar imagedarragh-oreilly ( 2015-03-19 13:41:26 -0500 )edit

2 answers

Sort by ยป oldest newest most voted
0

answered 2015-03-19 09:06:20 -0500

etlv gravatar image

It seems like even if the vfio_pci kernel module is UNloaded, every time nova is booting the VM, the module is re-loaded... any other ideas? I would like to force nova use -device pci-assign...

edit flag offensive delete link more
1

answered 2015-04-10 19:32:59 -0500

Alex Williamson gravatar image

libvirt has a driver option that can specify "vfio" or "kvm". I suspect the issue you're seeing is due to the IOMMU group configuration of the system, which is a function of the hardware. These IOMMU groups protect you from peer-to-peer DMA, which can allow the owner of one VF to read or write to the MMIO space of another VF, either maliciously or accidentally. A write-up of IOMMU groups and what they're trying to do can be found http://vfio.blogspot.com/2014/08/iommu-groups-inside-and-out.html (here). Newer kernels are introducing quirks to configure and enable isolation of various interconnect components as we're able to work with hardware vendors to supplement missing PCIe ACS support in the hardware.

edit flag offensive delete link more

Comments

Thanks. Could you please provide more information about kernel changes ? Are you referring to the ACS to the ACS override patch ? Is there plan to merge it in the kernel source tree ?

ludovic-beliveau gravatar imageludovic-beliveau ( 2015-09-22 17:22:29 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

2 followers

Stats

Asked: 2015-03-11 10:48:44 -0500

Seen: 1,837 times

Last updated: Mar 19 '15