Getting Devstack GPU Instance on XenServer 6.5 to run
I am attempting to get a PV domU guest (Ubuntu Trusty Tahr 14.04) running on XenServer 6.5 running DevStack Kilo to passthrough a GPU to an instance using the pci_alias assignment.
I can see the GPU on the DevStack VM with lspci -nn:
00:00.0 3D controller [0302]: NVIDIA Corporation GK110BGL [Tesla K40c]
IOMMU is also enabled on the DevStack VM: dmesg | grep -iE "dmar|iommu"
[ 0.000000] Command line: root=/dev/mapper/stack--vg-root ro console=hvc0 splash quiet intel_iommu=on $vt_handoff -- quiet console=hvc0 [366537.439789] Kernel command line: root=/dev/mapper/stack--vg-root ro console=hvc0 splash quiet intel_iommu=on $vt_handoff -- quiet console=hvc0 [366537.439849] Intel-IOMMU: enabled
I used the following for the nova.conf configuration:
pci_passthrough_whitelist = [{"vendor_id":"10de","product_id":"1024"}] pci_alias = {"vendor_id":"10de","product_id":"1024","name":"K40c"} scheduler_available_filters = nova.scheduler.filters.all_filters scheduler_available_filters = nova.scheduler.filters.pci_passthrough_filter.PciPassthroughFilter scheduler_default_filters = PciPassthroughFilter
Using the Flavor type m1.small I was able to create an instance. Then I terminated it and added the pci_alias to m1.small with "nova flavor-key m1.small set pci_passthrough:alias=K40c:1" and attempted to create it again and get the following error in the n-cpu log:
File "/usr/local/lib/python2.7/dist-packages/libvirt.py", line 3263, in defineXML libvirtError: internal error: Insufficient specification for PCI address
The only difference in the two XMLs passed to defineXML seem to be the addition of:
<hostdev mode="subsystem" type="pci" managed="yes"> <source> <address bus="0x00" domain="0x0000" function="0x0" slot="0x00"/> </source> </hostdev>