Instance error - Error launching a defined domain with XML - SRIOV

asked 2015-05-18 17:33:09 -0500

GLaupre gravatar image

updated 2015-05-19 16:47:59 -0500


I am trying to use a 2 nodes setup. I want to use SR-IOV support with an Intel card on my compute node. I followed the openstack wiki to make my configuration. My compute node is a Centos 7.1.1503, kernel 3.10.0-229.

When I lauched an instance using the command

nova boot --flavor m1.small --image archlinux-2014.10.01 --nic port-id=2397c544-b12b-4048-9431-6a6725508e19  test

I have this error on the nova-compute screen: (may not be the exact port id)

I created a port using the following command:

[stack@peryn5 devstack]$ neutron port-create private100 --binding:vnic-type direct --name sriov
Created a new port:
| Field                 | Value                                          
| admin_state_up        | True                                              
| binding:vif_type      | unbound                                                           
| binding:vnic_type     | direct                         
| fixed_ips             | {"subnet_id": "991e7366-xxxx", "ip_address": ""} 
| id                    | 2397c544-b12b-4048-9431-6a6725508e19                     
| mac_address           | fa:16:3e:02:56:12                                                  
| name                  | sriov                                                               
| network_id            | 1d6092a1-c656-41c1-bd66-7144582b3237                           
| security_groups       | 1bd422cd-9b38-4b9b-bb7f-c3761ca9882f                 
| status                | DOWN                                                           
| tenant_id             | a5161f5f5abb45379e5a0ad6bba0818d                     

In my nova.conf

pci_passthrough_whitelist = {"device":"enp4s0f0","physical_network":"mynetwork"}

my ml2_conf.ini (compute node)

tenant_network_types = vlan
type_drivers = vlan
mechanism_drivers = openvswitch,sriovnicswitch
network_vlan_ranges = mynetwork:100:199
firewall_driver = neutron.agent.firewall.NoopFirewallDriver
local_ip =
tunnel_types = 
root_helper_daemon = sudo /usr/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf
root_helper = sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf

my ml2_conf_sriov.ini (compute node) [ml2_sriov]

supported_pci_vendor_devs = 8086:10ed
agent_required = True
physical_device_mappings = mynetwork:enp4s0f0

ip link show

11: enp4s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether 00:1b:21:b5:40:0c brd ff:ff:ff:ff:ff:ff
    vf 0 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
    vf 1 MAC fa:16:3e:b1:35:a9, vlan 100, spoof checking on, link-state auto
    vf 2 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

The VF seems to be configured correctly, the instance stops to be launched. The instance log are not even started. I have try several forum but I have no more idea for now.

===============EDIT 1===============

I can boot an instance if I use the vnic-type=macvtap but the vnic-type=direct doesn't work. What I want to do is to attach the pci device into the guest using virt-manager and following the guide from redhat.

Using the virt-manager I stop the instance. I edit the configuration XML file using

virsh edit instance-00000034

under <devices> I found my virtual function

<interface type='direct'>
  <mac address='fa:16:3e:d2:40:53'/>
  <source dev='enp4s16f2' mode='passthrough'/>
  <model type='virtio'/>
  <driver name='vhost'/>
  <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>

Two things are strange: first why the type is direct as I created my port using the vnic-type=macvtap. Secondly, the pci address seems to be wrong because this VF is under 0000:04:10:2 as shown bellow:

11: enp4s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether 00:1b:21:b5:40:0c brd ff:ff:ff:ff:ff ...
edit retag flag offensive close merge delete


Interestingly, if I created the port with the type macvtap, the instance spawned, but with very very bad network performance. (a lot of retransmission packets)

GLaupre gravatar imageGLaupre ( 2015-05-18 18:46:09 -0500 )edit

what is the value of /sys/module/vfio_iommu_type1/parameters/allow_unsafe_interrupts. if it is 'N' , then make it yes

echo 1 > /sys/module/vfio_iommu_type1/parameters/allow_unsafe_interrupts. Then try to launch the VM.

Ranjit gravatar imageRanjit ( 2015-05-19 01:00:16 -0500 )edit

I tried to enable it. It didn't change, thank for the suggestion though

GLaupre gravatar imageGLaupre ( 2015-05-19 11:24:46 -0500 )edit