Ask Your Question
0

sriov vif binds to wrong device

asked 2015-06-04 07:40:10 -0600

DaveJ gravatar image

Hi,

When I launch a VM with an SR-IOV port, the vif binds to the wrong PCI device (it seems to ignore the whitelist rules)

I'm using RDO (Kilo) and have enabled SR-IOV as per https://wiki.openstack.org/wiki/SR-IO.... The hardware is HP DL360G8 with Intel 82599 10G cards.

Specifically I did the following:

Controller

edited /etc/neutron/plugins/ml2/ml2_conf.ini and set the following

  • type_drivers =local,flat,vlan,gre,vxlan
  • tenant_network_types = flat,vlan,gre,vxlan
  • mechanism_drivers =openvswitch,sriovnicswitch

edited /etc/neutron/plugins/ml2/ml2_conf_sriov.ini: - supported_pci_vendor_devs = 8086:10fb, 8086:10ed

Compute

edited /etc/nova/nova.conf and set the pci white list - pci_passthrough_whitelist = {"address":":07:10.","physical_network":"physnet1"}

N.B I got the address from lspci. My cards has two ports, and only one is connected (the first). when I execute lspci -nn | grep 82599 I see the following

lspci -nn | grep 82599
  07:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
  07:00.1 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
  07:10.0 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
  07:10.1 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
  07:10.2 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
  ..... 
  07:11.0 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
  07:11.1 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
  ....
  07:11.7 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)

So when I create a direct port, I would expect it to bind to a PCI device that has the address 07:10.* and not the 07:11.* When I use neutron port show the binding profile looks correct:

neutron port-show 030575a1-b3f8-461e-ac06-18371de87cca

| binding:profile | {"pci_slot": "0000:07:10.7", "physical_network": "physnet1", "pci_vendor_info": "8086:10ed"} | | binding:vif_details | {"port_filter": false, "vlan": "522"} | | binding:vif_type | hw_veb | | binding:vnic_type | direct |

Based on this, I believed it would bind to pci_slot": "0000:07:10.7",

On the compute node, when I do ip link show, it shows that it is actually connecting to a different device. N.B my device has two porrts ens2f0 (connected) and ens2f1 (not connected)

ip link show ens2f0
6: ens2f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP mode DEFAULT qlen 1000
link/ether 90:e2:ba:49:b2:28 brd ff:ff:ff:ff:ff:ff
  vf 0 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
  vf 1 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
  vf 2 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
  vf 3 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
  vf 4 MAC 00:00:00:00 ...
(more)
edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted
0

answered 2015-06-04 08:50:11 -0600

DaveJ gravatar image

I'm not sure if the issue is related to this: https://bugs.launchpad.net/nova/+bug/...

I was able to work around it by changing my whitelist configuration to specify the device name:

pci_passthrough_whitelist = {"devname":"ens2f0","physical_network":"physnet1"}

Where devname = port 0 of my card.

Interestingly now the port seems to get attached to a PCI address | binding:profile | {"pci_slot": "0000:07:11.6", "physical_network": "physnet1", "pci_vendor_info": "8086:10ed"} |

When I do ip link show I can see it attached to the last vf of port 0

ip link show ens2f0
5: ens2f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP mode     DEFAULT qlen 1000
link/ether 90:e2:ba:40:8c:ec brd ff:ff:ff:ff:ff:ff
vf 0 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
vf 1 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
...
...
vf 7 MAC fa:16:3e:25:29:eb, vlan 522, spoof checking on, link-state auto
edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2015-06-04 07:40:10 -0600

Seen: 614 times

Last updated: Jun 04 '15