Instance launch fails with nova/oslo errors
Since a few days ago every attempt to launch an instance fails on my devstack VM (same attempts worked fine before). I sync my local with repository updates regularly.
Here are the errors I see in Nova sceduler logs:
2014-02-11 11:58:28.076 ERROR nova.scheduler.filter_scheduler [req-71ab0a13-ead3-4e60-90c4-322a0fb96646 admin demo] [instance: 290ca314-c2c6-42e4-b8ab-0bda2bfa0780] Error from last host: devstackready02 (node devstackready02): [u'Traceback (most recent call last):\n', u' File "/opt/stack/nova/nova/compute/manager.py", line 1072, in _build_instance\n set_access_ip=set_access_ip)\n', u' File "/opt/stack/nova/nova/compute/manager.py", line 357, in decorated_function\n return function(self, context, *args, **kwargs)\n', u' File "/opt/stack/nova/nova/compute/manager.py", line 1480, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', u' File "/opt/stack/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/opt/stack/nova/nova/compute/manager.py", line 1477, in _spawn\n block_device_info)\n', u' File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2205, in spawn\n admin_pass=admin_password)\n', u' File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2570, in _create_image\n content=files, extra_md=extra_md, network_info=network_info)\n', u' File "/opt/stack/nova/nova/api/metadata/base.py", line 163, in __init__\n ec2utils.get_ip_info_for_instance_from_nw_info(network_info)\n', u' File "/opt/stack/nova/nova/api/ec2/ec2utils.py", line 147, in get_ip_info_for_instance_from_nw_info\n fixed_ips = nw_info.fixed_ips()\n', u' File "/opt/stack/nova/nova/network/model.py", line 366, in _sync_wrapper\n self.wait()\n', u' File "/opt/stack/nova/nova/network/model.py", line 398, in wait\n self[:] = self._gt.wait()\n', u' File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 168, in wait\n return self._exit_event.wait()\n', u' File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 120, in wait\n current.throw(*self._exc)\n', u' File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 194, in main\n result = function(*args, **kwargs)\n', u' File "/opt/stack/nova/nova/compute/manager.py", line 1258, in _allocate_network_async\n dhcp_options=dhcp_options)\n', u' File "/opt/stack/nova/nova/network/api.py", line 94, in wrapped\n return func(self, context, *args, **kwargs)\n', u' File "/opt/stack/nova/nova/network/api.py", line 48, in wrapper\n res = f(self, context, *args, **kwargs)\n', u' File "/opt/stack/nova/nova/network/api.py", line 302, in allocate_for_instance\n nw_info = self.network_rpcapi.allocate_for_instance(context, **args)\n', u' File "/opt/stack/nova/nova/network/rpcapi.py", line 167, in allocate_for_instance\n macs=jsonutils.to_primitive(macs))\n', u' File "/opt/stack/oslo.messaging/oslo/messaging/rpc/client.py", line 150, in call\n wait_for_reply=True, timeout=timeout)\n', u' File "/opt/stack/oslo.messaging/oslo/messaging/transport.py", line 87, in _send\n timeout=timeout)\n', u' File "/opt/stack/oslo.messaging/oslo/messaging/_drivers/amqpdriver.py", line 390, in send\n ...