VirtualInterfaceCreateException: Virtual Interface creation failed
Hello, I've followed the ubuntu icehouse guide using 14.04 beta on virtualbox. I've made it through the networking section without issues. I can ping my external gateway and seem to be all ok. Then i try to launch an instance and it does with errors about timing out for "vif plugging callback." In the code i can see where this exception is thrown and there is a comment about "# We never heard from Neutron." I'm curious how i can test what it's trying to do to see what might be causing us not to hear back from neutron. All nodes have passed all the tests in the document.
I can't for the life of me figure out what i've got wrong to cause this. Anyone have any idea it's greatly appreciated.
August
error from compute log:
2014-04-10 09:16:39.796 3372 WARNING nova.virt.libvirt.driver [req-bd6b9994-6d14-46a6-bc8d-d5e357761035 f90878d3b5054a948dba93e9fad7a864 93341443727b4a38ab2d38a7fcbfab97] Timeout waiting for vif plugging callback for instance 2f406c15-d686-44af-bc63-056b86365e6e
2014-04-10 09:16:40.504 3372 INFO nova.virt.libvirt.driver [req-bd6b9994-6d14-46a6-bc8d-d5e357761035 f90878d3b5054a948dba93e9fad7a864 93341443727b4a38ab2d38a7fcbfab97] [instance: 2f406c15-d686-44af-bc63-056b86365e6e] Deleting instance files /var/lib/nova/instances/2f406c15-d686-44af-bc63-056b86365e6e
2014-04-10 09:16:40.506 3372 INFO nova.virt.libvirt.driver [req-bd6b9994-6d14-46a6-bc8d-d5e357761035 f90878d3b5054a948dba93e9fad7a864 93341443727b4a38ab2d38a7fcbfab97] [instance: 2f406c15-d686-44af-bc63-056b86365e6e] Deletion of /var/lib/nova/instances/2f406c15-d686-44af-bc63-056b86365e6e complete
2014-04-10 09:16:40.578 3372 ERROR nova.compute.manager [req-bd6b9994-6d14-46a6-bc8d-d5e357761035 f90878d3b5054a948dba93e9fad7a864 93341443727b4a38ab2d38a7fcbfab97] [instance: 2f406c15-d686-44af-bc63-056b86365e6e] Instance failed to spawn
2014-04-10 09:16:40.578 3372 TRACE nova.compute.manager [instance: 2f406c15-d686-44af-bc63-056b86365e6e] Traceback (most recent call last):
2014-04-10 09:16:40.578 3372 TRACE nova.compute.manager [instance: 2f406c15-d686-44af-bc63-056b86365e6e] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1718, in _spawn
2014-04-10 09:16:40.578 3372 TRACE nova.compute.manager [instance: 2f406c15-d686-44af-bc63-056b86365e6e] block_device_info)
2014-04-10 09:16:40.578 3372 TRACE nova.compute.manager [instance: 2f406c15-d686-44af-bc63-056b86365e6e] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2251, in spawn
2014-04-10 09:16:40.578 3372 TRACE nova.compute.manager [instance: 2f406c15-d686-44af-bc63-056b86365e6e] block_device_info)
2014-04-10 09:16:40.578 3372 TRACE nova.compute.manager [instance: 2f406c15-d686-44af-bc63-056b86365e6e] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3654, in _create_domain_and_network
2014-04-10 09:16:40.578 3372 TRACE nova.compute.manager [instance: 2f406c15-d686-44af-bc63-056b86365e6e] raise exception.VirtualInterfaceCreateException()
2014-04-10 09:16:40.578 3372 TRACE nova.compute.manager [instance: 2f406c15-d686-44af-bc63-056b86365e6e] VirtualInterfaceCreateException: Virtual Interface creation failed
2014-04-10 09:16:40.578 3372 TRACE nova.compute.manager [instance: 2f406c15-d686-44af-bc63-056b86365e6e]
more details:
Same issue here. On the network node it seems that neutron wants to talk back to nova but connects to localhost instead of the controller ip:
Same problem here on RDO/Fedora 20/Icehouse. Started seeing this just since yesterday since I had the same automated deployment done two days back without any issues.
interesting. on my compute node the ovs-vswitchd.log registers this when i launch an instance (i have vif_plugging_timeout set to 30 to speed the failure up):
but nothing on the network node.
more logs:
when i launch an instance, the controller immediately reports in the neutron server.log:
(more)