Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

sriov vif binds to wrong device

Hi,

When I launch a VM with an SR-IOV port, the vif binds to the wrong PCI device (it seems to ignore the whitelist rules)

I'm using RDO (Kilo) and have enabled SR-IOV as per https://wiki.openstack.org/wiki/SR-IOV-Passthrough-For-Networking. The hardware is HP DL360G8 with Intel 82599 10G cards.

Specifically I did the following:

Controller

edited /etc/neutron/plugins/ml2/ml2_conf.ini and set the following

  • type_drivers =local,flat,vlan,gre,vxlan
  • tenant_network_types = flat,vlan,gre,vxlan
  • mechanism_drivers =openvswitch,sriovnicswitch

edited /etc/neutron/plugins/ml2/ml2_conf_sriov.ini: - supported_pci_vendor_devs = 8086:10fb, 8086:10ed

Compute

edited /etc/nova/nova.conf and set the pci white list - pci_passthrough_whitelist = {"address":":07:10.","physical_network":"physnet1"}

N.B I got the address from lspci. My cards has two ports, and only one is connected (the first). when I execute lspci -nn | grep 82599 I see the following

lspci -nn | grep 82599
  07:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
  07:00.1 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
  07:10.0 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
  07:10.1 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
  07:10.2 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
  ..... 
  07:11.0 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
  07:11.1 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
  ....
  07:11.7 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)

So when I create a direct port, I would expect it to bind to a PCI device that has the address 07:10.* and not the 07:11.* When I use neutron port show the binding profile looks correct:

neutron port-show 030575a1-b3f8-461e-ac06-18371de87cca

| binding:profile | {"pci_slot": "0000:07:10.7", "physical_network": "physnet1", "pci_vendor_info": "8086:10ed"} | | binding:vif_details | {"port_filter": false, "vlan": "522"} | | binding:vif_type | hw_veb | | binding:vnic_type | direct |

Based on this, I believed it would bind to pci_slot": "0000:07:10.7",

On the compute node, when I do ip link show, it shows that it is actually connecting to a different device. N.B my device has two porrts ens2f0 (connected) and ens2f1 (not connected)

ip link show ens2f0
6: ens2f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP mode DEFAULT qlen 1000
link/ether 90:e2:ba:49:b2:28 brd ff:ff:ff:ff:ff:ff
  vf 0 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
  vf 1 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
  vf 2 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
  vf 3 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
  vf 4 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
  vf 5 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
  vf 6 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
  vf 7 MAC 00:00:00:00:00:00, spoof checking on, link-state auto


ip link show ens2f1
7: ens2f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
  link/ether 90:e2:ba:49:b2:29 brd ff:ff:ff:ff:ff:ff
  vf 0 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
  vf 1 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
  vf 2 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
  vf 3 MAC fa:16:3e:fd:96:2c, vlan 522, spoof checking on, link-state auto
  vf 4 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
  vf 5 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
  vf 6 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
  vf 7 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

Is this a bug, or a mis-configuration?