Revision history [back]

click to hide/show revision 1
initial version

Launched instance failed with error - Liberty release

Dear Openstack expert I am using only private network - (Self service network ) and did the configuration as per liberty guide also used kvm libr_type. I created the private network and subnet and trying to launch a instance with command line option nova boot... while using the command, I am able to see scheduling , spawning .. but after that it throws me an error.

follow are the logs Nova-compute.log on Compute node

2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s) 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager Traceback (most recent call last): 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager dhcp_options=dhcp_options) 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager self._delete_ports(neutron, instance, created_port_ids) 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__ 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager six.reraise(self.type_, self.value, self.tb) 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager security_group_ids, available_macs, dhcp_opts) 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager raise exception.PortBindingFailed(port_id=port_id) 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager PortBindingFailed: Binding failed for port edd06d07-addf-4132-8310-1ab7c2300e0d, please check neutron logs for more information. 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [req-9830fcd3-d807-416d-8ab9-74bb8b7eb7cd 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] Instance failed to spawn 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] Traceback (most recent call last): 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2155, in _build_resources 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] yield resources 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2009, in _build_and_run_instance 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] block_device_info=block_device_info) 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2512, in spawn 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] write_to_disk=True) 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4383, in _get_guest_xml 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] network_info_str = str(network_info) 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 515, in __str__ 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] return self._sync_wrapper(fn, args, *kwargs) 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 498, in _sync_wrapper 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] self.wait() 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 530, in wait 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] self[:] = self._gt.wait() 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 175, in wait 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] return self._exit_event.wait() 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] return hubs.get_hub().switch() 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 294, in switch 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] return self.greenlet.switch() 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 214, in main 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] result = function(args, *kwargs) 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1581, in _allocate_network_async 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] six.reraise(exc_info) 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] dhcp_options=dhcp_options) 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] self._delete_ports(neutron, instance, created_port_ids) 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__ 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] six.reraise(self.type_, self.value, self.tb) 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] security_group_ids, available_macs, dhcp_opts) 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] raise exception.PortBindingFailed(port_id=port_id) 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] PortBindingFailed: Binding failed for port edd06d07-addf-4132-8310-1ab7c2300e0d, please check neutron logs for more information. 2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] 2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s) 2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager Traceback (most recent call last): 2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async 2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager dhcp_options=dhcp_options) 2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance 2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager self._delete_ports(neutron, instance, created_port_ids) 2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__ 2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager six.reraise(self.type_, self.value, self.tb) 2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance 2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager security_group_ids, available_macs, dhcp_opts) 2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port 2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager raise exception.PortBindingFailed(port_id=port_id) 2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager PortBindingFailed: Binding failed for port 8fc86e57-88c5-40e8-b338-95df46188c7b, please check neutron logs for more information. 2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [req-ea27c7c2-17ef-44d2-80e5-2e4f1571f178 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] Instance failed to spawn 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] Traceback (most recent call last): 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2155, in _build_resources 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] yield resources 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2009, in _build_and_run_instance 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] block_device_info=block_device_info) 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2512, in spawn 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] write_to_disk=True) 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4383, in _get_guest_xml 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] network_info_str = str(network_info) 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 515, in __str__ 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] return self._sync_wrapper(fn, *args, *kwargs) 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 498, in _sync_wrapper 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] self.wait() 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 530, in wait 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] self[:] = self._gt.wait() 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 175, in wait 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] return self._exit_event.wait() 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] return hubs.get_hub().switch() 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 294, in switch 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] return self.greenlet.switch() 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 214, in main 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] result = function(args, *kwargs) 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1581, in _allocate_network_async 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] six.reraise(exc_info) 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] dhcp_options=dhcp_options) 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] self._delete_ports(neutron, instance, created_port_ids) 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__ 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] six.reraise(self.type_, self.value, self.tb) 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] security_group_ids, available_macs, dhcp_opts) 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] raise exception.PortBindingFailed(port_id=port_id) 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] PortBindingFailed: Binding failed for port 8fc86e57-88c5-40e8-b338-95df46188c7b, please check neutron logs for more information. 2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] 2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s) 2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager Traceback (most recent call last): 2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async 2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager dhcp_options=dhcp_options) 2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance 2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager self._delete_ports(neutron, instance, created_port_ids) 2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__ 2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager six.reraise(self.type_, self.value, self.tb) 2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance 2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager security_group_ids, available_macs, dhcp_opts) 2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port 2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager raise exception.PortBindingFailed(port_id=port_id) 2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager PortBindingFailed: Binding failed for port 0ca9b9e7-15cb-45d5-b231-678632dae244, please check neutron logs for more information. 2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [req-3f4491a1-318f-4a80-b116-f7bd2db6a235 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] Instance failed to spawn 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] Traceback (most recent call last): 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2155, in _build_resources 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] yield resources 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2009, in _build_and_run_instance 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] block_device_info=block_device_info) 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2512, in spawn 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] write_to_disk=True) 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4383, in _get_guest_xml 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] network_info_str = str(network_info) 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 515, in __str__ 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] return self._sync_wrapper(fn, *args, *kwargs) 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 498, in _sync_wrapper 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] self.wait() 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 530, in wait 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] self[:] = self._gt.wait() 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 175, in wait 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] return self._exit_event.wait() 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] return hubs.get_hub().switch() 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 294, in switch 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] return self.greenlet.switch() 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 214, in main 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] result = function(args, *kwargs) 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1581, in _allocate_network_async 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] six.reraise(exc_info) 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] dhcp_options=dhcp_options) 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] self._delete_ports(neutron, instance, created_port_ids) 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__ 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] six.reraise(self.type_, self.value, self.tb) 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] security_group_ids, available_macs, dhcp_opts) 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] raise exception.PortBindingFailed(port_id=port_id) 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] PortBindingFailed: Binding failed for port 0ca9b9e7-15cb-45d5-b231-678632dae244, please check neutron logs for more information. 2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] 2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s) 2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager Traceback (most recent call last): 2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async 2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager dhcp_options=dhcp_options) 2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance 2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager self._delete_ports(neutron, instance, created_port_ids) 2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__ 2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager six.reraise(self.type_, self.value, self.tb) 2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance 2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager security_group_ids, available_macs, dhcp_opts) 2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port 2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager raise exception.PortBindingFailed(port_id=port_id) 2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager PortBindingFailed: Binding failed for port 094bb490-0290-448a-95d5-87e9e9fa00ea, please check neutron logs for more information. 2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [req-2437a193-8fed-4e23-932f-9ea3f70097f7 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] Instance failed to spawn 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] Traceback (most recent call last): 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2155, in _build_resources 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] yield resources 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2009, in _build_and_run_instance 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] block_device_info=block_device_info) 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2512, in spawn 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] write_to_disk=True) 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4383, in _get_guest_xml 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] network_info_str = str(network_info) 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 515, in __str__ 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] return self._sync_wrapper(fn, *args, *kwargs) 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 498, in _sync_wrapper 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] self.wait() 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 530, in wait 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] self[:] = self._gt.wait() 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 175, in wait 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] return self._exit_event.wait() 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] return hubs.get_hub().switch() 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 294, in switch 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] return self.greenlet.switch() 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 214, in main 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] result = function(args, *kwargs) 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1581, in _allocate_network_async 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] six.reraise(exc_info) 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] dhcp_options=dhcp_options) 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] self._delete_ports(neutron, instance, created_port_ids) 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__ 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] six.reraise(self.type_, self.value, self.tb) 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] security_group_ids, available_macs, dhcp_opts) 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] raise exception.PortBindingFailed(port_id=port_id) 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] PortBindingFailed: Binding failed for port 094bb490-0290-448a-95d5-87e9e9fa00ea, please check neutron logs for more information. 2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] 2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s) 2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager Traceback (most recent call last): 2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async 2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager dhcp_options=dhcp_options) 2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance 2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager self._delete_ports(neutron, instance, created_port_ids) 2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__ 2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager six.reraise(self.type_, self.value, self.tb) 2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance 2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager security_group_ids, available_macs, dhcp_opts) 2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port 2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager raise exception.PortBindingFailed(port_id=port_id) 2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager PortBindingFailed: Binding failed for port 6d34dcd0-94d7-4cee-8a8c-f9824afbf6c8, please check neutron logs for more information. 2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [req-f72dcca5-7555-443c-835f-30b27ff52f95 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: 896a8573-210e-483b-b1db-764005d109db] Instance failed to spawn 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] Traceback (most recent call last): 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2155, in _build_resources 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] yield resources 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2009, in _build_and_run_instance 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] block_device_info=block_device_info) 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2512, in spawn 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] write_to_disk=True) 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4383, in _get_guest_xml 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] network_info_str = str(network_info) 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 515, in __str__ 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] return self._sync_wrapper(fn, *args, *kwargs) 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 498, in _sync_wrapper 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] self.wait() 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 530, in wait 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] self[:] = self._gt.wait() 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 175, in wait 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] return self._exit_event.wait() 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 125, in wait 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] current.throw(self._exc) 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 214, in main 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] result = function(args, *kwargs) 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1581, in _allocate_network_async 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] six.reraise(exc_info) 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] dhcp_options=dhcp_options) 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] self._delete_ports(neutron, instance, created_port_ids) 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__ 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] six.reraise(self.type_, self.value, self.tb) 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] security_group_ids, available_macs, dhcp_opts) 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] raise exception.PortBindingFailed(port_id=port_id) 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] PortBindingFailed: Binding failed for port 6d34dcd0-94d7-4cee-8a8c-f9824afbf6c8, please check neutron logs for more information. 2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] 2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s) 2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager Traceback (most recent call last): 2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async 2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager dhcp_options=dhcp_options) 2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance 2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager self._delete_ports(neutron, instance, created_port_ids) 2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__ 2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager six.reraise(self.type_, self.value, self.tb) 2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance 2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager security_group_ids, available_macs, dhcp_opts) 2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port 2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager raise exception.PortBindingFailed(port_id=port_id) 2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager PortBindingFailed: Binding failed for port 6f858109-3fd2-4e66-802b-5579aa3c93c4, please check neutron logs for more information. 2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [req-4eb4e5a4-7c55-4510-a021-d5b221699893 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] Instance failed to spawn 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] Traceback (most recent call last): 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2155, in _build_resources 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] yield resources 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2009, in _build_and_run_instance 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] block_device_info=block_device_info) 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2512, in spawn 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] write_to_disk=True) 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4383, in _get_guest_xml 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] network_info_str = str(network_info) 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 515, in __str__ 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] return self._sync_wrapper(fn, args, *kwargs) 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 498, in _sync_wrapper 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] self.wait() 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 530, in wait 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] self[:] = self._gt.wait() 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 175, in wait 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] return self._exit_event.wait() 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 125, in wait 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] current.throw(self._exc) 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 214, in main 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] result = function(args, *kwargs) 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1581, in _allocate_network_async 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] six.reraise(exc_info) 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] dhcp_options=dhcp_options) 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] self._delete_ports(neutron, instance, created_port_ids) 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__ 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] six.reraise(self.type_, self.value, self.tb) 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] security_group_ids, available_macs, dhcp_opts) 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] raise exception.PortBindingFailed(port_id=port_id) 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] PortBindingFailed: Binding failed for port 6f858109-3fd2-4e66-802b-5579aa3c93c4, please check neutron logs for more information. 2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s) 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager Traceback (most recent call last): 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager dhcp_options=dhcp_options) 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager self._delete_ports(neutron, instance, created_port_ids) 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__ 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager six.reraise(self.type_, self.value, self.tb) 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager security_group_ids, available_macs, dhcp_opts) 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager raise exception.PortBindingFailed(port_id=port_id) 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager PortBindingFailed: Binding failed for port 6e9d9dcb-7dec-4560-a227-67bd293e66da, please check neutron logs for more information. 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [req-219c8310-c0f7-4b47-b3ec-8d36f0c44c31 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] Instance failed to spawn 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] Traceback (most recent call last): 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2155, in _build_resources 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] yield resources 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2009, in _build_and_run_instance 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] block_device_info=block_device_info) 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2512, in spawn 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] write_to_disk=True) 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4383, in _get_guest_xml 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] network_info_str = str(network_info) 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 515, in __str__ 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] return self._sync_wrapper(fn, args, *kwargs) 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 498, in _sync_wrapper 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] self.wait() 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 530, in wait 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] self[:] = self._gt.wait() 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 175, in wait 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] return self._exit_event.wait() 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] return hubs.get_hub().switch() 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 294, in switch 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] return self.greenlet.switch() 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 214, in main 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] result = function(args, *kwargs) 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1581, in _allocate_network_async 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] six.reraise(exc_info) 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] dhcp_options=dhcp_options) 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] self._delete_ports(neutron, instance, created_port_ids) 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__ 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] six.reraise(self.type_, self.value, self.tb) 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] security_group_ids, available_macs, dhcp_opts) 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] raise exception.PortBindingFailed(port_id=port_id) 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] PortBindingFailed: Binding failed for port 6e9d9dcb-7dec-4560-a227-67bd293e66da, please check neutron logs for more information. 2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] 2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s) 2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager Traceback (most recent call last): 2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async 2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager dhcp_options=dhcp_options) 2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance 2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager self._delete_ports(neutron, instance, created_port_ids) 2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__ 2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager six.reraise(self.type_, self.value, self.tb) 2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance 2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager security_group_ids, available_macs, dhcp_opts) 2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port 2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager raise exception.PortBindingFailed(port_id=port_id) 2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager PortBindingFailed: Binding failed for port 41151c02-9e02-49f7-8101-226c2bb3c1a6, please check neutron logs for more information. 2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [req-0c4932bb-e055-40a7-a48e-2d8042823f9a 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] Instance failed to spawn 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] Traceback (most recent call last): 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2155, in _build_resources 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] yield resources 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2009, in _build_and_run_instance 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] block_device_info=block_device_info) 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2512, in spawn 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] write_to_disk=True) 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4383, in _get_guest_xml 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] network_info_str = str(network_info) 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 515, in __str__ 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] return self._sync_wrapper(fn, *args, *kwargs) 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 498, in _sync_wrapper 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] self.wait() 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 530, in wait 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] self[:] = self._gt.wait() 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 175, in wait 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] return self._exit_event.wait() 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] return hubs.get_hub().switch() 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 294, in switch 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] return self.greenlet.switch() 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 214, in main 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] result = function(args, *kwargs) 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1581, in _allocate_network_async 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] six.reraise(*exc_info) 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] dhcp_options=dhcp_options) 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] self._delete_ports(neutron, instance, created_port_ids) 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__ 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] six.reraise(self.type_, self.value, self.tb) 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] security_group_ids, available_macs, dhcp_opts) 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] raise exception.PortBindingFailed(port_id=port_id) 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] PortBindingFailed: Binding failed for port 41151c02-9e02-49f7-8101-226c2bb3c1a6, please check neutron logs for more information. 2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] [root@pun01cloud02 nova]#


nova-conductor 2016-06-14 10:39:34.305 13323 WARNING nova.scheduler.utils [req-4eb4e5a4-7c55-4510-a021-d5b221699893 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. Traceback (most recent call last):

File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 142, in inner return func(args, *kwargs)

File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 84, in select_destinations filter_properties)

File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 90, in select_destinations raise exception.NoValidHost(reason=reason)

NoValidHost: No valid host was found. There are not enough hosts available.

2016-06-14 10:39:34.306 13323 WARNING nova.scheduler.utils [req-4eb4e5a4-7c55-4510-a021-d5b221699893 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] Setting instance to ERROR state. 2016-06-14 11:21:27.004 13357 INFO oslo.messaging._drivers.impl_rabbit [req-219c8310-c0f7-4b47-b3ec-8d36f0c44c31 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb 7f681545c00 - - -] Connecting to AMQP server on 192.168.170.30:5672 2016-06-14 11:21:35.096 13357 INFO oslo.messaging._drivers.impl_rabbit [req-219c8310-c0f7-4b47-b3ec-8d36f0c44c31 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb 7f681545c00 - - -] Connected to AMQP server on 192.168.170.30:5672 2016-06-14 11:21:44.220 13304 ERROR nova.scheduler.utils [req-219c8310-c0f7-4b47-b3ec-8d36f0c44c31 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] Error from last host: pun01cloud02.ad.infosys.com (node pun01cloud02.ad.infosys.com): [u'Traceback (most recent c all last):\n', u' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1905, in _do_build_and_run_instance\n filter_properties)\n', u' File "/usr /lib/python2.7/site-packages/nova/compute/manager.py", line 2057, in _build_and_run_instance\n instance_uuid=instance.uuid, reason=six.text_type(e))\n', u'Reschedul edException: Build of instance cb9bf36a-271e-4412-8c76-ec7df942699a was re-scheduled: Binding failed for port 6e9d9dcb-7dec-4560-a227-67bd293e66da, please check neutro n logs for more information.\n'] 2016-06-14 11:21:44.222 13304 INFO oslo.messaging._drivers.impl_rabbit [req-219c8310-c0f7-4b47-b3ec-8d36f0c44c31 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb 7f681545c00 - - -] Connecting to AMQP server on 192.168.170.30:5672 2016-06-14 11:21:52.311 13304 INFO oslo.messaging._drivers.impl_rabbit [req-219c8310-c0f7-4b47-b3ec-8d36f0c44c31 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb 7f681545c00 - - -] Connected to AMQP server on 192.168.170.30:5672 2016-06-14 11:21:52.329 13304 WARNING nova.scheduler.utils [req-219c8310-c0f7-4b47-b3ec-8d36f0c44c31 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. Traceback (most recent call last):

File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 142, in inner return func(args, *kwargs)

File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 84, in select_destinations filter_properties)

File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 90, in select_destinations raise exception.NoValidHost(reason=reason)

NoValidHost: No valid host was found. There are not enough hosts available.

2016-06-14 11:21:52.330 13304 WARNING nova.scheduler.utils [req-219c8310-c0f7-4b47-b3ec-8d36f0c44c31 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] Setting instance to ERROR state. 2016-06-14 13:32:21.258 13218 INFO oslo.messaging._drivers.impl_rabbit [req-0c4932bb-e055-40a7-a48e-2d8042823f9a 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb 7f681545c00 - - -] Connecting to AMQP server on 192.168.170.30:5672 2016-06-14 13:32:29.346 13218 INFO oslo.messaging._drivers.impl_rabbit [req-0c4932bb-e055-40a7-a48e-2d8042823f9a 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb 7f681545c00 - - -] Connected to AMQP server on 192.168.170.30:5672 2016-06-14 13:32:38.472 13304 ERROR nova.scheduler.utils [req-0c4932bb-e055-40a7-a48e-2d8042823f9a 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] Error from last host: pun01cloud02.ad.infosys.com (node pun01cloud02.ad.infosys.com): [u'Traceback (most recent c all last):\n', u' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1905, in _do_build_and_run_instance\n filter_properties)\n', u' File "/usr /lib/python2.7/site-packages/nova/compute/manager.py", line 2057, in _build_and_run_instance\n instance_uuid=instance.uuid, reason=six.text_type(e))\n', u'Reschedul edException: Build of instance c616fb8f-9e6c-40f7-bf78-9cbf4db70633 was re-scheduled: Binding failed for port 41151c02-9e02-49f7-8101-226c2bb3c1a6, please check neutro n logs for more information.\n'] 2016-06-14 13:32:38.496 13304 WARNING nova.scheduler.utils [req-0c4932bb-e055-40a7-a48e-2d8042823f9a 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. Traceback (most recent call last):

File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 142, in inner return func(args, *kwargs)

File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 84, in select_destinations filter_properties)

File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 90, in select_destinations raise exception.NoValidHost(reason=reason)

NoValidHost: No valid host was found. There are not enough hosts available.

2016-06-14 13:32:38.496 13304 WARNING nova.scheduler.utils [req-0c4932bb-e055-40a7-a48e-2d8042823f9a 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] Setting instance to ERROR state.

neutron log

2016-06-14 13:32:38.022 13718 ERROR neutron.plugins.ml2.managers [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Failed to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 on host pun01cloud02.ad.infosys.com 2016-06-14 13:32:38.022 13718 INFO neutron.plugins.ml2.plugin [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Attempt 6 to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 2016-06-14 13:32:38.026 13718 ERROR neutron.plugins.ml2.managers [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Failed to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 on host pun01cloud02.ad.infosys.com 2016-06-14 13:32:38.026 13718 ERROR neutron.plugins.ml2.managers [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Failed to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 on host pun01cloud02.ad.infosys.com 2016-06-14 13:32:38.026 13718 INFO neutron.plugins.ml2.plugin [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Attempt 7 to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 2016-06-14 13:32:38.030 13718 ERROR neutron.plugins.ml2.managers [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Failed to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 on host pun01cloud02.ad.infosys.com 2016-06-14 13:32:38.030 13718 ERROR neutron.plugins.ml2.managers [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Failed to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 on host pun01cloud02.ad.infosys.com 2016-06-14 13:32:38.030 13718 INFO neutron.plugins.ml2.plugin [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Attempt 8 to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 2016-06-14 13:32:38.034 13718 ERROR neutron.plugins.ml2.managers [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Failed to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 on host pun01cloud02.ad.infosys.com 2016-06-14 13:32:38.034 13718 ERROR neutron.plugins.ml2.managers [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Failed to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 on host pun01cloud02.ad.infosys.com 2016-06-14 13:32:38.034 13718 INFO neutron.plugins.ml2.plugin [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Attempt 9 to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 2016-06-14 13:32:38.038 13718 ERROR neutron.plugins.ml2.managers [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Failed to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 on host pun01cloud02.ad.infosys.com 2016-06-14 13:32:38.038 13718 ERROR neutron.plugins.ml2.managers [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Failed to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 on host pun01cloud02.ad.infosys.com 2016-06-14 13:32:38.038 13718 INFO neutron.plugins.ml2.plugin [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Attempt 10 to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 2016-06-14 13:32:38.042 13718 ERROR neutron.plugins.ml2.managers [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Failed to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 on host pun01cloud02.ad.infosys.com 2016-06-14 13:32:38.042 13718 ERROR neutron.plugins.ml2.managers [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Failed to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 on host pun01cloud02.ad.infosys.com 2016-06-14 13:32:38.144 13718 INFO neutron.wsgi [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] 192.168.170.31 - - [14/Jun/2016 13:32:38] "POST /v2.0/ports.json HTTP/1.1" 201 984 8.349892 2016-06-14 13:32:38.173 13712 INFO neutron.wsgi [-] (13712) accepted ('192.168.170.30', 56235) 2016-06-14 13:32:38.252 13718 WARNING neutron.plugins.ml2.drivers.l2pop.mech_driver [req-dde5693a-320d-46a2-bd57-8b07c3827600 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Unable to retrieve active L2 agent on host pun01cloud02.ad.infosys.com 2016-06-14 13:32:38.278 13718 INFO neutron.wsgi [req-dde5693a-320d-46a2-bd57-8b07c3827600 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] 192.168.170.31 - - [14/Jun/2016 13:32:38] "DELETE /v2.0/ports/41151c02-9e02-49f7-8101-226c2bb3c1a6.json HTTP/1.1" 204 173 0.132086 2016-06-14 13:32:38.362 13712 INFO neutron.wsgi [req-ae5f1066-fd17-44b7-b94c-dfaf72a7a9c8 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] 192.168.170.30 - - [14/Jun/2016 13:32:38] "GET /v2.0/ports.json?device_id=c616fb8f-9e6c-40f7-bf78-9cbf4db70633&device_id=cb9bf36a-271e-4412-8c76-ec7df942699a&device_id=896a8573-210e-483b-b1db-764005d109db&device_id=7bbacafd-ca1e-43ba-937b-48e0efa15c5a&device_id=20422504-9c20-470a-acc1-01cb19aa18e9&device_id=7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63&device_id=e6832245-d2ad-4ddc-b0e7-404290b20a83&device_id=5cdad546-9e44-4e85-bfba-38a7f2b4db39&device_id=664baa6f-0861-45b7-8727-90b91fb414e6&device_id=11268a67-5d10-4a34-b8cf-72031da17ed6&device_id=8a642bba-d5ed-406f-8a70-c421b95d79f7 HTTP/1.1" 200 226 0.187757 2016-06-14 13:32:38.379 13718 INFO neutron.wsgi [req-175aebd2-9610-4dc2-81c1-dd7757314d36 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] 192.168.170.31 - - [14/Jun/2016 13:32:38] "GET /v2.0/ports.json?device_id=c616fb8f-9e6c-40f7-bf78-9cbf4db70633 HTTP/1.1" 200 226 0.022293 2016-06-14 13:32:40.480 13714 INFO neutron.wsgi [-] (13714) accepted ('192.168.170.30', 56248) 2016-06-14 13:32:40.563 13714 INFO neutron.wsgi [req-0334722f-a1b1-4267-bcf9-b3ccafb43d65 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] 192.168.170.30 - - [14/Jun/2016 13:32:40] "GET /v2.0/ports.json?device_id=c616fb8f-9e6c-40f7-bf78-9cbf4db70633&device_id=cb9bf36a-271e-4412-8c76-ec7df942699a&device_id=896a8573-210e-483b-b1db-764005d109db&device_id=7bbacafd-ca1e-43ba-937b-48e0efa15c5a&device_id=20422504-9c20-470a-acc1-01cb19aa18e9&device_id=7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63&device_id=e6832245-d2ad-4ddc-b0e7-404290b20a83&device_id=5cdad546-9e44-4e85-bfba-38a7f2b4db39&device_id=664baa6f-0861-45b7-8727-90b91fb414e6&device_id=11268a67-5d10-4a34-b8cf-72031da17ed6&device_id=8a642bba-d5ed-406f-8a70-c421b95d79f7 HTTP/1.1" 200 226 0.082877 2016-06-14 13:32:40.940 13718 WARNING neutron.notifiers.nova [-] Nova returned NotFound for event: [{'tag': u'41151c02-9e02-49f7-8101-226c2bb3c1a6', 'name': 'network-vif-deleted', 'server_uuid': u'c616fb8f-9e6c-40f7-bf78-9cbf4db70633'}] 2016-06-14 13:32:42.441 13712 INFO neutron.wsgi [-] (13712) accepted ('192.168.170.30', 56255) 2016-06-14 13:32:42.469 13712 INFO neutron.wsgi [req-632cd300-c55b-4512-a64b-bf20c29065f6 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] 192.168.170.30 - - [14/Jun/2016 13:32:42] "GET /v2.0/ports.json?device_id=c616fb8f-9e6c-40f7-bf78-9cbf4db70633&device_id=cb9bf36a-271e-4412-8c76-ec7df942699a&device_id=896a8573-210e-483b-b1db-764005d109db&device_id=7bbacafd-ca1e-43ba-937b-48e0efa15c5a&device_id=20422504-9c20-470a-acc1-01cb19aa18e9&device_id=7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63&device_id=e6832245-d2ad-4ddc-b0e7-404290b20a83&device_id=5cdad546-9e44-4e85-bfba-38a7f2b4db39&device_id=664baa6f-0861-45b7-8727-90b91fb414e6&device_id=11268a67-5d10-4a34-b8cf-72031da17ed6&device_id=8a642bba-d5ed-406f-8a70-c421b95d79f7 HTTP/1.1" 200 226 0.027571 2016-06-14 13:34:32.968 13724 INFO neutron.wsgi [-] (13724) accepted ('192.168.170.30', 56267) 2016-06-14 13:34:33.038 13724 INFO neutron.wsgi [req-44bc7ad2-216e-41f1-9527-d188620cdf88 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] 192.168.170.30 - - [14/Jun/2016 13:34:33] "GET /v2.0/agents.json HTTP/1.1" 200 227 0.069254 2016-06-14 13:34:42.212 13719 INFO neutron.wsgi [-] (13719) accepted ('192.168.170.30', 56273) 2016-06-14 13:34:42.235 13719 INFO neutron.wsgi [req-e520122a-becd-4022-b8fd-200be14c59a3 6d1c9262d6704ca89f14283f9b23b516 65ca6620737246d794d005693b4c0a94 - - -] 192.168.170.30 - - [14/Jun/2016 13:34:42] "GET /v2.0/agents.json HTTP/1.1" 200 2009 0.022329 2016-06-14 13:47:21.319 13711 INFO neutron.wsgi [req-b945f561-d689-445a-bc61-2694f5c9415d 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] (13711) timed out ('192.168.170.30', 56180) 2016-06-14 13:47:28.696 13719 INFO neutron.wsgi [req-06549a88-baa2-4be3-b4b5-52782b41febe 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] (13719) timed out ('192.168.170.30', 56211) 2016-06-14 13:47:35.640 13717 INFO neutron.wsgi [req-aa8c475e-68b9-4745-8350-206c7fed9476 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] (13717) timed out ('192.168.170.30', 56225) 2016-06-14 13:47:38.379 13718 INFO neutron.wsgi [req-175aebd2-9610-4dc2-81c1-dd7757314d36 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] (13718) timed out ('192.168.170.31', 40956) 2016-06-14 13:47:38.462 13712 INFO neutron.wsgi [req-ae5f1066-fd17-44b7-b94c-dfaf72a7a9c8 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] (13712) timed out ('192.168.170.30', 56235) 2016-06-14 13:47:40.564 13714 INFO neutron.wsgi [req-0334722f-a1b1-4267-bcf9-b3ccafb43d65 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] (13714) timed out ('192.168.170.30', 56248) 2016-06-14 13:47:42.473 13712 INFO neutron.wsgi [req-632cd300-c55b-4512-a64b-bf20c29065f6 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] (13712) timed out ('192.168.170.30', 56255) 2016-06-14 15:24:56.523 13681 INFO neutron.wsgi [-] (13681) accepted ('192.168.170.30', 56365) 2016-06-14 15:24:56.609 13681 INFO neutron.wsgi [req-30ae842f-924f-432e-b64b-6825260af7e0 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] 192.168.170.30 - - [14/Jun/2016 15:24:56] "GET /v2.0/ports.json?device_id=c616fb8f-9e6c-40f7-bf78-9cbf4db70633&device_id=cb9bf36a-271e-4412-8c76-ec7df942699a&device_id=896a8573-210e-483b-b1db-764005d109db&device_id=7bbacafd-ca1e-43ba-937b-48e0efa15c5a&device_id=20422504-9c20-470a-acc1-01cb19aa18e9&device_id=7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63&device_id=e6832245-d2ad-4ddc-b0e7-404290b20a83&device_id=5cdad546-9e44-4e85-bfba-38a7f2b4db39&device_id=664baa6f-0861-45b7-8727-90b91fb414e6&device_id=11268a67-5d10-4a34-b8cf-72031da17ed6&device_id=8a642bba-d5ed-406f-8a70-c421b95d79f7 HTTP/1.1" 200 226 0.085084 2016-06-14 15:39:56.705 13681 INFO neutron.wsgi [req-30ae842f-924f-432e-b64b-6825260af7e0 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] (13681) timed out ('192.168.170.30', 56365) 2016-06-14 15:44:46.334 13696 INFO neutron.wsgi [-] (13696) accepted ('192.168.170.30', 56443) 2016-06-14 15:44:46.404 13696 INFO neutron.wsgi [req-623d8154-2eff-4836-94c9-b9227294eae8 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] 192.168.170.30 - - [14/Jun/2016 15:44:46] "GET /v2.0/agents.json HTTP/1.1" 200 227 0.069245 2016-06-14 15:44:49.949 13701 INFO neutron.wsgi [-] (13701) accepted ('192.168.170.30', 56450) 2016-06-14 15:44:50.018 13701 INFO neutron.wsgi [req-85a0bbcf-df81-475b-a6c1-a17a1b97237a 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] 192.168.170.30 - - [14/Jun/2016 15:44:50] "GET /v2.0/agents.json HTTP/1.1" 200 227 0.068902 2016-06-14 15:44:55.192 13722 INFO neutron.wsgi [-] (13722) accepted ('192.168.170.30', 56456) 2016-06-14 15:44:55.261 13722 INFO neutron.wsgi [req-97ead42e-dff4-4fbe-bc26-01cc625b779e 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] 192.168.170.30 - - [14/Jun/2016 15:44:55] "GET /v2.0/agents.json HTTP/1.1" 200 227 0.068442

Please advise and let me know if you need any additional information

thanks Sac

Launched instance failed with error - Liberty release

Dear Openstack expert I am using only private network - (Self service network ) and did the configuration as per liberty guide also used kvm libr_type. I created the private network and subnet and trying to launch a instance with command line option nova boot... while using the command, I am able to see scheduling , spawning .. but after that it throws me an error.

follow are the logs Nova-compute.log on Compute node

2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s)
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager Traceback (most recent call last):
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager     dhcp_options=dhcp_options)
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager     self._delete_ports(neutron, instance, created_port_ids)
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager     six.reraise(self.type_, self.value, self.tb)
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager     security_group_ids, available_macs, dhcp_opts)
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager     raise exception.PortBindingFailed(port_id=port_id)
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager PortBindingFailed: Binding failed for port edd06d07-addf-4132-8310-1ab7c2300e0d, please check neutron logs for more information.
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [req-9830fcd3-d807-416d-8ab9-74bb8b7eb7cd 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] Instance failed to spawn
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] Traceback (most recent call last):
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2155, in _build_resources
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]     yield resources
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2009, in _build_and_run_instance
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]     block_device_info=block_device_info)
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2512, in spawn
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]     write_to_disk=True)
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4383, in _get_guest_xml
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]     network_info_str = str(network_info)
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 515, in __str__
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]     return self._sync_wrapper(fn, args, *kwargs)
*args, **kwargs)
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 498, in _sync_wrapper
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]     self.wait()
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 530, in wait
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]     self[:] = self._gt.wait()
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]   File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 175, in wait
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]     return self._exit_event.wait()
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]   File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]     return hubs.get_hub().switch()
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]   File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 294, in switch
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]     return self.greenlet.switch()
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]   File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 214, in main
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]     result = function(args, *kwargs)
function(*args, **kwargs)
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1581, in _allocate_network_async
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]     six.reraise(exc_info)
six.reraise(*exc_info)
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]     dhcp_options=dhcp_options)
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]     self._delete_ports(neutron, instance, created_port_ids)
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]     six.reraise(self.type_, self.value, self.tb)
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]     security_group_ids, available_macs, dhcp_opts)
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]     raise exception.PortBindingFailed(port_id=port_id)
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83] PortBindingFailed: Binding failed for port edd06d07-addf-4132-8310-1ab7c2300e0d, please check neutron logs for more information.
2016-06-14 09:54:45.406 122194 ERROR nova.compute.manager [instance: e6832245-d2ad-4ddc-b0e7-404290b20a83]
2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s)
2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager Traceback (most recent call last):
2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async
2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager     dhcp_options=dhcp_options)
2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance
2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager     self._delete_ports(neutron, instance, created_port_ids)
2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager     six.reraise(self.type_, self.value, self.tb)
2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance
2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager     security_group_ids, available_macs, dhcp_opts)
2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port
2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager     raise exception.PortBindingFailed(port_id=port_id)
2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager PortBindingFailed: Binding failed for port 8fc86e57-88c5-40e8-b338-95df46188c7b, please check neutron logs for more information.
2016-06-14 10:11:50.994 122194 ERROR nova.compute.manager
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [req-ea27c7c2-17ef-44d2-80e5-2e4f1571f178 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] Instance failed to spawn
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] Traceback (most recent call last):
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2155, in _build_resources
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]     yield resources
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2009, in _build_and_run_instance
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]     block_device_info=block_device_info)
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2512, in spawn
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]     write_to_disk=True)
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4383, in _get_guest_xml
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]     network_info_str = str(network_info)
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 515, in __str__
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]     return self._sync_wrapper(fn, *args, *kwargs)
**kwargs)
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 498, in _sync_wrapper
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]     self.wait()
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 530, in wait
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]     self[:] = self._gt.wait()
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]   File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 175, in wait
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]     return self._exit_event.wait()
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]   File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]     return hubs.get_hub().switch()
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]   File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 294, in switch
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]     return self.greenlet.switch()
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]   File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 214, in main
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]     result = function(args, *kwargs)
function(*args, **kwargs)
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1581, in _allocate_network_async
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]     six.reraise(exc_info)
six.reraise(*exc_info)
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]     dhcp_options=dhcp_options)
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]     self._delete_ports(neutron, instance, created_port_ids)
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]     six.reraise(self.type_, self.value, self.tb)
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]     security_group_ids, available_macs, dhcp_opts)
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]     raise exception.PortBindingFailed(port_id=port_id)
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63] PortBindingFailed: Binding failed for port 8fc86e57-88c5-40e8-b338-95df46188c7b, please check neutron logs for more information.
2016-06-14 10:11:50.995 122194 ERROR nova.compute.manager [instance: 7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63]
2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s)
2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager Traceback (most recent call last):
2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async
2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager     dhcp_options=dhcp_options)
2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance
2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager     self._delete_ports(neutron, instance, created_port_ids)
2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager     six.reraise(self.type_, self.value, self.tb)
2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance
2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager     security_group_ids, available_macs, dhcp_opts)
2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port
2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager     raise exception.PortBindingFailed(port_id=port_id)
2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager PortBindingFailed: Binding failed for port 0ca9b9e7-15cb-45d5-b231-678632dae244, please check neutron logs for more information.
2016-06-14 10:14:02.055 122194 ERROR nova.compute.manager
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [req-3f4491a1-318f-4a80-b116-f7bd2db6a235 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] Instance failed to spawn
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] Traceback (most recent call last):
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2155, in _build_resources
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]     yield resources
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2009, in _build_and_run_instance
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]     block_device_info=block_device_info)
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2512, in spawn
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]     write_to_disk=True)
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4383, in _get_guest_xml
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]     network_info_str = str(network_info)
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 515, in __str__
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]     return self._sync_wrapper(fn, *args, *kwargs)
**kwargs)
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 498, in _sync_wrapper
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]     self.wait()
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 530, in wait
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]     self[:] = self._gt.wait()
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]   File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 175, in wait
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]     return self._exit_event.wait()
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]   File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]     return hubs.get_hub().switch()
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]   File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 294, in switch
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]     return self.greenlet.switch()
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]   File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 214, in main
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]     result = function(args, *kwargs)
function(*args, **kwargs)
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1581, in _allocate_network_async
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]     six.reraise(exc_info)
six.reraise(*exc_info)
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]     dhcp_options=dhcp_options)
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]     self._delete_ports(neutron, instance, created_port_ids)
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]     six.reraise(self.type_, self.value, self.tb)
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]     security_group_ids, available_macs, dhcp_opts)
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]     raise exception.PortBindingFailed(port_id=port_id)
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9] PortBindingFailed: Binding failed for port 0ca9b9e7-15cb-45d5-b231-678632dae244, please check neutron logs for more information.
2016-06-14 10:14:02.056 122194 ERROR nova.compute.manager [instance: 20422504-9c20-470a-acc1-01cb19aa18e9]
2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s)
2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager Traceback (most recent call last):
2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async
2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager     dhcp_options=dhcp_options)
2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance
2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager     self._delete_ports(neutron, instance, created_port_ids)
2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager     six.reraise(self.type_, self.value, self.tb)
2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance
2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager     security_group_ids, available_macs, dhcp_opts)
2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port
2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager     raise exception.PortBindingFailed(port_id=port_id)
2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager PortBindingFailed: Binding failed for port 094bb490-0290-448a-95d5-87e9e9fa00ea, please check neutron logs for more information.
2016-06-14 10:22:43.573 124526 ERROR nova.compute.manager
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [req-2437a193-8fed-4e23-932f-9ea3f70097f7 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] Instance failed to spawn
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] Traceback (most recent call last):
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2155, in _build_resources
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]     yield resources
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2009, in _build_and_run_instance
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]     block_device_info=block_device_info)
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2512, in spawn
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]     write_to_disk=True)
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4383, in _get_guest_xml
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]     network_info_str = str(network_info)
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 515, in __str__
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]     return self._sync_wrapper(fn, *args, *kwargs)
**kwargs)
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 498, in _sync_wrapper
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]     self.wait()
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 530, in wait
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]     self[:] = self._gt.wait()
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]   File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 175, in wait
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]     return self._exit_event.wait()
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]   File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]     return hubs.get_hub().switch()
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]   File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 294, in switch
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]     return self.greenlet.switch()
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]   File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 214, in main
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]     result = function(args, *kwargs)
function(*args, **kwargs)
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1581, in _allocate_network_async
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]     six.reraise(exc_info)
six.reraise(*exc_info)
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]     dhcp_options=dhcp_options)
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]     self._delete_ports(neutron, instance, created_port_ids)
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]     six.reraise(self.type_, self.value, self.tb)
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]     security_group_ids, available_macs, dhcp_opts)
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]     raise exception.PortBindingFailed(port_id=port_id)
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a] PortBindingFailed: Binding failed for port 094bb490-0290-448a-95d5-87e9e9fa00ea, please check neutron logs for more information.
2016-06-14 10:22:43.574 124526 ERROR nova.compute.manager [instance: 7bbacafd-ca1e-43ba-937b-48e0efa15c5a]
2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s)
2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager Traceback (most recent call last):
2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async
2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager     dhcp_options=dhcp_options)
2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance
2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager     self._delete_ports(neutron, instance, created_port_ids)
2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager     six.reraise(self.type_, self.value, self.tb)
2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance
2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager     security_group_ids, available_macs, dhcp_opts)
2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port
2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager     raise exception.PortBindingFailed(port_id=port_id)
2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager PortBindingFailed: Binding failed for port 6d34dcd0-94d7-4cee-8a8c-f9824afbf6c8, please check neutron logs for more information.
2016-06-14 10:30:35.532 124526 ERROR nova.compute.manager
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [req-f72dcca5-7555-443c-835f-30b27ff52f95 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: 896a8573-210e-483b-b1db-764005d109db] Instance failed to spawn
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] Traceback (most recent call last):
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2155, in _build_resources
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]     yield resources
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2009, in _build_and_run_instance
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]     block_device_info=block_device_info)
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2512, in spawn
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]     write_to_disk=True)
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4383, in _get_guest_xml
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]     network_info_str = str(network_info)
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 515, in __str__
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]     return self._sync_wrapper(fn, *args, *kwargs)
**kwargs)
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 498, in _sync_wrapper
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]     self.wait()
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 530, in wait
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]     self[:] = self._gt.wait()
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]   File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 175, in wait
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]     return self._exit_event.wait()
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]   File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 125, in wait
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]     current.throw(self._exc)
current.throw(*self._exc)
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]   File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 214, in main
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]     result = function(args, *kwargs)
function(*args, **kwargs)
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1581, in _allocate_network_async
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]     six.reraise(exc_info)
six.reraise(*exc_info)
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]     dhcp_options=dhcp_options)
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]     self._delete_ports(neutron, instance, created_port_ids)
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]     six.reraise(self.type_, self.value, self.tb)
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]     security_group_ids, available_macs, dhcp_opts)
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]     raise exception.PortBindingFailed(port_id=port_id)
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db] PortBindingFailed: Binding failed for port 6d34dcd0-94d7-4cee-8a8c-f9824afbf6c8, please check neutron logs for more information.
2016-06-14 10:30:59.106 124526 ERROR nova.compute.manager [instance: 896a8573-210e-483b-b1db-764005d109db]
2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s)
2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager Traceback (most recent call last):
2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async
2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager     dhcp_options=dhcp_options)
2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance
2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager     self._delete_ports(neutron, instance, created_port_ids)
2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager     six.reraise(self.type_, self.value, self.tb)
2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance
2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager     security_group_ids, available_macs, dhcp_opts)
2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port
2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager     raise exception.PortBindingFailed(port_id=port_id)
2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager PortBindingFailed: Binding failed for port 6f858109-3fd2-4e66-802b-5579aa3c93c4, please check neutron logs for more information.
2016-06-14 10:39:25.863 124526 ERROR nova.compute.manager
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [req-4eb4e5a4-7c55-4510-a021-d5b221699893 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] Instance failed to spawn
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] Traceback (most recent call last):
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2155, in _build_resources
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]     yield resources
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2009, in _build_and_run_instance
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]     block_device_info=block_device_info)
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2512, in spawn
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]     write_to_disk=True)
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4383, in _get_guest_xml
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]     network_info_str = str(network_info)
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 515, in __str__
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]     return self._sync_wrapper(fn, args, *kwargs)
*args, **kwargs)
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 498, in _sync_wrapper
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]     self.wait()
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 530, in wait
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]     self[:] = self._gt.wait()
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]   File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 175, in wait
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]     return self._exit_event.wait()
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]   File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 125, in wait
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]     current.throw(self._exc)
current.throw(*self._exc)
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]   File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 214, in main
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]     result = function(args, *kwargs)
function(*args, **kwargs)
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1581, in _allocate_network_async
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]     six.reraise(exc_info)
six.reraise(*exc_info)
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]     dhcp_options=dhcp_options)
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]     self._delete_ports(neutron, instance, created_port_ids)
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]     six.reraise(self.type_, self.value, self.tb)
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]     security_group_ids, available_macs, dhcp_opts)
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]     raise exception.PortBindingFailed(port_id=port_id)
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] PortBindingFailed: Binding failed for port 6f858109-3fd2-4e66-802b-5579aa3c93c4, please check neutron logs for more information.
2016-06-14 10:39:25.912 124526 ERROR nova.compute.manager [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de]
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s)
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager Traceback (most recent call last):
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager     dhcp_options=dhcp_options)
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager     self._delete_ports(neutron, instance, created_port_ids)
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager     six.reraise(self.type_, self.value, self.tb)
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager     security_group_ids, available_macs, dhcp_opts)
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager     raise exception.PortBindingFailed(port_id=port_id)
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager PortBindingFailed: Binding failed for port 6e9d9dcb-7dec-4560-a227-67bd293e66da, please check neutron logs for more information.
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [req-219c8310-c0f7-4b47-b3ec-8d36f0c44c31 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] Instance failed to spawn
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] Traceback (most recent call last):
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2155, in _build_resources
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]     yield resources
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2009, in _build_and_run_instance
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]     block_device_info=block_device_info)
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2512, in spawn
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]     write_to_disk=True)
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4383, in _get_guest_xml
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]     network_info_str = str(network_info)
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 515, in __str__
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]     return self._sync_wrapper(fn, args, *kwargs)
*args, **kwargs)
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 498, in _sync_wrapper
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]     self.wait()
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 530, in wait
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]     self[:] = self._gt.wait()
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]   File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 175, in wait
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]     return self._exit_event.wait()
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]   File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]     return hubs.get_hub().switch()
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]   File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 294, in switch
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]     return self.greenlet.switch()
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]   File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 214, in main
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]     result = function(args, *kwargs)
function(*args, **kwargs)
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1581, in _allocate_network_async
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]     six.reraise(exc_info)
six.reraise(*exc_info)
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]     dhcp_options=dhcp_options)
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]     self._delete_ports(neutron, instance, created_port_ids)
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]     six.reraise(self.type_, self.value, self.tb)
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]     security_group_ids, available_macs, dhcp_opts)
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]     raise exception.PortBindingFailed(port_id=port_id)
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] PortBindingFailed: Binding failed for port 6e9d9dcb-7dec-4560-a227-67bd293e66da, please check neutron logs for more information.
2016-06-14 11:21:43.973 124526 ERROR nova.compute.manager [instance: cb9bf36a-271e-4412-8c76-ec7df942699a]
2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s)
2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager Traceback (most recent call last):
2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async
2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager     dhcp_options=dhcp_options)
2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance
2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager     self._delete_ports(neutron, instance, created_port_ids)
2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager     six.reraise(self.type_, self.value, self.tb)
2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance
2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager     security_group_ids, available_macs, dhcp_opts)
2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port
2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager     raise exception.PortBindingFailed(port_id=port_id)
2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager PortBindingFailed: Binding failed for port 41151c02-9e02-49f7-8101-226c2bb3c1a6, please check neutron logs for more information.
2016-06-14 13:32:38.278 134468 ERROR nova.compute.manager
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [req-0c4932bb-e055-40a7-a48e-2d8042823f9a 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] Instance failed to spawn
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] Traceback (most recent call last):
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2155, in _build_resources
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]     yield resources
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2009, in _build_and_run_instance
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]     block_device_info=block_device_info)
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2512, in spawn
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]     write_to_disk=True)
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4383, in _get_guest_xml
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]     network_info_str = str(network_info)
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 515, in __str__
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]     return self._sync_wrapper(fn, *args, *kwargs)
**kwargs)
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 498, in _sync_wrapper
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]     self.wait()
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 530, in wait
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]     self[:] = self._gt.wait()
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]   File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 175, in wait
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]     return self._exit_event.wait()
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]   File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]     return hubs.get_hub().switch()
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]   File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 294, in switch
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]     return self.greenlet.switch()
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]   File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 214, in main
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]     result = function(args, *kwargs)
function(*args, **kwargs)
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1581, in _allocate_network_async
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]     six.reraise(*exc_info)
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1564, in _allocate_network_async
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]     dhcp_options=dhcp_options)
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 727, in allocate_for_instance
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]     self._delete_ports(neutron, instance, created_port_ids)
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]     six.reraise(self.type_, self.value, self.tb)
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 719, in allocate_for_instance
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]     security_group_ids, available_macs, dhcp_opts)
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 342, in _create_port
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]     raise exception.PortBindingFailed(port_id=port_id)
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] PortBindingFailed: Binding failed for port 41151c02-9e02-49f7-8101-226c2bb3c1a6, please check neutron logs for more information.
2016-06-14 13:32:38.279 134468 ERROR nova.compute.manager [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633]
[root@pun01cloud02 nova]#

nova]#

------------------------------------------------------------------------ nova-conductor 2016-06-14 10:39:34.305 13323 WARNING nova.scheduler.utils [req-4eb4e5a4-7c55-4510-a021-d5b221699893 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. Traceback (most recent call last):

last): File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 142, in inner return func(*args, **kwargs)

File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 142, in inner return func(args, *kwargs)

File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 84, in select_destinations filter_properties) File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 90, in select_destinations raise exception.NoValidHost(reason=reason)

File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 84, in select_destinations filter_properties)

File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 90, in select_destinations raise exception.NoValidHost(reason=reason)

NoValidHost: No valid host was found. There are not enough hosts available.

available.

2016-06-14 10:39:34.306 13323 WARNING nova.scheduler.utils [req-4eb4e5a4-7c55-4510-a021-d5b221699893 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: 745faff2-a781-4d3a-8ac1-a4d6388874de] Setting instance to ERROR state. 2016-06-14 11:21:27.004 13357 INFO oslo.messaging._drivers.impl_rabbit [req-219c8310-c0f7-4b47-b3ec-8d36f0c44c31 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb 7f681545c00 - - -] Connecting to AMQP server on 192.168.170.30:5672 2016-06-14 11:21:35.096 13357 INFO oslo.messaging._drivers.impl_rabbit [req-219c8310-c0f7-4b47-b3ec-8d36f0c44c31 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb 7f681545c00 - - -] Connected to AMQP server on 192.168.170.30:5672 2016-06-14 11:21:44.220 13304 ERROR nova.scheduler.utils [req-219c8310-c0f7-4b47-b3ec-8d36f0c44c31 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] Error from last host: pun01cloud02.ad.infosys.com (node pun01cloud02.ad.infosys.com): [u'Traceback (most recent c all last):\n', u' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1905, in _do_build_and_run_instance\n filter_properties)\n', u' File "/usr /lib/python2.7/site-packages/nova/compute/manager.py", line 2057, in _build_and_run_instance\n instance_uuid=instance.uuid, reason=six.text_type(e))\n', u'Reschedul edException: Build of instance cb9bf36a-271e-4412-8c76-ec7df942699a was re-scheduled: Binding failed for port 6e9d9dcb-7dec-4560-a227-67bd293e66da, please check neutro n logs for more information.\n'] 2016-06-14 11:21:44.222 13304 INFO oslo.messaging._drivers.impl_rabbit [req-219c8310-c0f7-4b47-b3ec-8d36f0c44c31 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb 7f681545c00 - - -] Connecting to AMQP server on 192.168.170.30:5672 2016-06-14 11:21:52.311 13304 INFO oslo.messaging._drivers.impl_rabbit [req-219c8310-c0f7-4b47-b3ec-8d36f0c44c31 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb 7f681545c00 - - -] Connected to AMQP server on 192.168.170.30:5672 2016-06-14 11:21:52.329 13304 WARNING nova.scheduler.utils [req-219c8310-c0f7-4b47-b3ec-8d36f0c44c31 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. Traceback (most recent call last):

last): File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 142, in inner return func(*args, **kwargs)

File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 142, in inner return func(args, *kwargs)

File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 84, in select_destinations filter_properties) File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 90, in select_destinations raise exception.NoValidHost(reason=reason)

File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 84, in select_destinations filter_properties)

File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 90, in select_destinations raise exception.NoValidHost(reason=reason)

NoValidHost: No valid host was found. There are not enough hosts available.

available.

2016-06-14 11:21:52.330 13304 WARNING nova.scheduler.utils [req-219c8310-c0f7-4b47-b3ec-8d36f0c44c31 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: cb9bf36a-271e-4412-8c76-ec7df942699a] Setting instance to ERROR state. 2016-06-14 13:32:21.258 13218 INFO oslo.messaging._drivers.impl_rabbit [req-0c4932bb-e055-40a7-a48e-2d8042823f9a 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb 7f681545c00 - - -] Connecting to AMQP server on 192.168.170.30:5672 2016-06-14 13:32:29.346 13218 INFO oslo.messaging._drivers.impl_rabbit [req-0c4932bb-e055-40a7-a48e-2d8042823f9a 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb 7f681545c00 - - -] Connected to AMQP server on 192.168.170.30:5672 2016-06-14 13:32:38.472 13304 ERROR nova.scheduler.utils [req-0c4932bb-e055-40a7-a48e-2d8042823f9a 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] Error from last host: pun01cloud02.ad.infosys.com (node pun01cloud02.ad.infosys.com): [u'Traceback (most recent c all last):\n', u' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1905, in _do_build_and_run_instance\n filter_properties)\n', u' File "/usr /lib/python2.7/site-packages/nova/compute/manager.py", line 2057, in _build_and_run_instance\n instance_uuid=instance.uuid, reason=six.text_type(e))\n', u'Reschedul edException: Build of instance c616fb8f-9e6c-40f7-bf78-9cbf4db70633 was re-scheduled: Binding failed for port 41151c02-9e02-49f7-8101-226c2bb3c1a6, please check neutro n logs for more information.\n'] 2016-06-14 13:32:38.496 13304 WARNING nova.scheduler.utils [req-0c4932bb-e055-40a7-a48e-2d8042823f9a 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. Traceback (most recent call last):

last): File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 142, in inner return func(*args, **kwargs)

File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 142, in inner return func(args, *kwargs)

File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 84, in select_destinations filter_properties) File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 90, in select_destinations raise exception.NoValidHost(reason=reason)

File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 84, in select_destinations filter_properties)

File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 90, in select_destinations raise exception.NoValidHost(reason=reason)

NoValidHost: No valid host was found. There are not enough hosts available.

available.

2016-06-14 13:32:38.496 13304 WARNING nova.scheduler.utils [req-0c4932bb-e055-40a7-a48e-2d8042823f9a 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] [instance: c616fb8f-9e6c-40f7-bf78-9cbf4db70633] Setting instance to ERROR state.

state.

neutron log

2016-06-14 13:32:38.022 13718 ERROR neutron.plugins.ml2.managers [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Failed to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 on host pun01cloud02.ad.infosys.com
2016-06-14 13:32:38.022 13718 INFO neutron.plugins.ml2.plugin [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Attempt 6 to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6
2016-06-14 13:32:38.026 13718 ERROR neutron.plugins.ml2.managers [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Failed to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 on host pun01cloud02.ad.infosys.com
2016-06-14 13:32:38.026 13718 ERROR neutron.plugins.ml2.managers [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Failed to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 on host pun01cloud02.ad.infosys.com
2016-06-14 13:32:38.026 13718 INFO neutron.plugins.ml2.plugin [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Attempt 7 to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6
2016-06-14 13:32:38.030 13718 ERROR neutron.plugins.ml2.managers [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Failed to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 on host pun01cloud02.ad.infosys.com
2016-06-14 13:32:38.030 13718 ERROR neutron.plugins.ml2.managers [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Failed to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 on host pun01cloud02.ad.infosys.com
2016-06-14 13:32:38.030 13718 INFO neutron.plugins.ml2.plugin [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Attempt 8 to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6
2016-06-14 13:32:38.034 13718 ERROR neutron.plugins.ml2.managers [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Failed to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 on host pun01cloud02.ad.infosys.com
2016-06-14 13:32:38.034 13718 ERROR neutron.plugins.ml2.managers [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Failed to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 on host pun01cloud02.ad.infosys.com
2016-06-14 13:32:38.034 13718 INFO neutron.plugins.ml2.plugin [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Attempt 9 to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6
2016-06-14 13:32:38.038 13718 ERROR neutron.plugins.ml2.managers [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Failed to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 on host pun01cloud02.ad.infosys.com
2016-06-14 13:32:38.038 13718 ERROR neutron.plugins.ml2.managers [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Failed to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 on host pun01cloud02.ad.infosys.com
2016-06-14 13:32:38.038 13718 INFO neutron.plugins.ml2.plugin [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Attempt 10 to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6
2016-06-14 13:32:38.042 13718 ERROR neutron.plugins.ml2.managers [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Failed to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 on host pun01cloud02.ad.infosys.com
2016-06-14 13:32:38.042 13718 ERROR neutron.plugins.ml2.managers [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Failed to bind port 41151c02-9e02-49f7-8101-226c2bb3c1a6 on host pun01cloud02.ad.infosys.com
2016-06-14 13:32:38.144 13718 INFO neutron.wsgi [req-2c21638a-5e47-4875-8433-46165bc6dc24 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] 192.168.170.31 - - [14/Jun/2016 13:32:38] "POST /v2.0/ports.json HTTP/1.1" 201 984 8.349892
2016-06-14 13:32:38.173 13712 INFO neutron.wsgi [-] (13712) accepted ('192.168.170.30', 56235)
2016-06-14 13:32:38.252 13718 WARNING neutron.plugins.ml2.drivers.l2pop.mech_driver [req-dde5693a-320d-46a2-bd57-8b07c3827600 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] Unable to retrieve active L2 agent on host pun01cloud02.ad.infosys.com
2016-06-14 13:32:38.278 13718 INFO neutron.wsgi [req-dde5693a-320d-46a2-bd57-8b07c3827600 041240a0cd9a49629f88bc587ac7876a 693e487029a04138abc75f4a8127117f - - -] 192.168.170.31 - - [14/Jun/2016 13:32:38] "DELETE /v2.0/ports/41151c02-9e02-49f7-8101-226c2bb3c1a6.json HTTP/1.1" 204 173 0.132086
2016-06-14 13:32:38.362 13712 INFO neutron.wsgi [req-ae5f1066-fd17-44b7-b94c-dfaf72a7a9c8 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] 192.168.170.30 - - [14/Jun/2016 13:32:38] "GET /v2.0/ports.json?device_id=c616fb8f-9e6c-40f7-bf78-9cbf4db70633&device_id=cb9bf36a-271e-4412-8c76-ec7df942699a&device_id=896a8573-210e-483b-b1db-764005d109db&device_id=7bbacafd-ca1e-43ba-937b-48e0efa15c5a&device_id=20422504-9c20-470a-acc1-01cb19aa18e9&device_id=7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63&device_id=e6832245-d2ad-4ddc-b0e7-404290b20a83&device_id=5cdad546-9e44-4e85-bfba-38a7f2b4db39&device_id=664baa6f-0861-45b7-8727-90b91fb414e6&device_id=11268a67-5d10-4a34-b8cf-72031da17ed6&device_id=8a642bba-d5ed-406f-8a70-c421b95d79f7 HTTP/1.1" 200 226 0.187757
2016-06-14 13:32:38.379 13718 INFO neutron.wsgi [req-175aebd2-9610-4dc2-81c1-dd7757314d36 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] 192.168.170.31 - - [14/Jun/2016 13:32:38] "GET /v2.0/ports.json?device_id=c616fb8f-9e6c-40f7-bf78-9cbf4db70633 HTTP/1.1" 200 226 0.022293
2016-06-14 13:32:40.480 13714 INFO neutron.wsgi [-] (13714) accepted ('192.168.170.30', 56248)
2016-06-14 13:32:40.563 13714 INFO neutron.wsgi [req-0334722f-a1b1-4267-bcf9-b3ccafb43d65 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] 192.168.170.30 - - [14/Jun/2016 13:32:40] "GET /v2.0/ports.json?device_id=c616fb8f-9e6c-40f7-bf78-9cbf4db70633&device_id=cb9bf36a-271e-4412-8c76-ec7df942699a&device_id=896a8573-210e-483b-b1db-764005d109db&device_id=7bbacafd-ca1e-43ba-937b-48e0efa15c5a&device_id=20422504-9c20-470a-acc1-01cb19aa18e9&device_id=7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63&device_id=e6832245-d2ad-4ddc-b0e7-404290b20a83&device_id=5cdad546-9e44-4e85-bfba-38a7f2b4db39&device_id=664baa6f-0861-45b7-8727-90b91fb414e6&device_id=11268a67-5d10-4a34-b8cf-72031da17ed6&device_id=8a642bba-d5ed-406f-8a70-c421b95d79f7 HTTP/1.1" 200 226 0.082877
2016-06-14 13:32:40.940 13718 WARNING neutron.notifiers.nova [-] Nova returned NotFound for event: [{'tag': u'41151c02-9e02-49f7-8101-226c2bb3c1a6', 'name': 'network-vif-deleted', 'server_uuid': u'c616fb8f-9e6c-40f7-bf78-9cbf4db70633'}]
2016-06-14 13:32:42.441 13712 INFO neutron.wsgi [-] (13712) accepted ('192.168.170.30', 56255)
2016-06-14 13:32:42.469 13712 INFO neutron.wsgi [req-632cd300-c55b-4512-a64b-bf20c29065f6 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] 192.168.170.30 - - [14/Jun/2016 13:32:42] "GET /v2.0/ports.json?device_id=c616fb8f-9e6c-40f7-bf78-9cbf4db70633&device_id=cb9bf36a-271e-4412-8c76-ec7df942699a&device_id=896a8573-210e-483b-b1db-764005d109db&device_id=7bbacafd-ca1e-43ba-937b-48e0efa15c5a&device_id=20422504-9c20-470a-acc1-01cb19aa18e9&device_id=7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63&device_id=e6832245-d2ad-4ddc-b0e7-404290b20a83&device_id=5cdad546-9e44-4e85-bfba-38a7f2b4db39&device_id=664baa6f-0861-45b7-8727-90b91fb414e6&device_id=11268a67-5d10-4a34-b8cf-72031da17ed6&device_id=8a642bba-d5ed-406f-8a70-c421b95d79f7 HTTP/1.1" 200 226 0.027571
2016-06-14 13:34:32.968 13724 INFO neutron.wsgi [-] (13724) accepted ('192.168.170.30', 56267)
2016-06-14 13:34:33.038 13724 INFO neutron.wsgi [req-44bc7ad2-216e-41f1-9527-d188620cdf88 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] 192.168.170.30 - - [14/Jun/2016 13:34:33] "GET /v2.0/agents.json HTTP/1.1" 200 227 0.069254
2016-06-14 13:34:42.212 13719 INFO neutron.wsgi [-] (13719) accepted ('192.168.170.30', 56273)
2016-06-14 13:34:42.235 13719 INFO neutron.wsgi [req-e520122a-becd-4022-b8fd-200be14c59a3 6d1c9262d6704ca89f14283f9b23b516 65ca6620737246d794d005693b4c0a94 - - -] 192.168.170.30 - - [14/Jun/2016 13:34:42] "GET /v2.0/agents.json HTTP/1.1" 200 2009 0.022329
2016-06-14 13:47:21.319 13711 INFO neutron.wsgi [req-b945f561-d689-445a-bc61-2694f5c9415d 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] (13711) timed out ('192.168.170.30', 56180)
2016-06-14 13:47:28.696 13719 INFO neutron.wsgi [req-06549a88-baa2-4be3-b4b5-52782b41febe 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] (13719) timed out ('192.168.170.30', 56211)
2016-06-14 13:47:35.640 13717 INFO neutron.wsgi [req-aa8c475e-68b9-4745-8350-206c7fed9476 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] (13717) timed out ('192.168.170.30', 56225)
2016-06-14 13:47:38.379 13718 INFO neutron.wsgi [req-175aebd2-9610-4dc2-81c1-dd7757314d36 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] (13718) timed out ('192.168.170.31', 40956)
2016-06-14 13:47:38.462 13712 INFO neutron.wsgi [req-ae5f1066-fd17-44b7-b94c-dfaf72a7a9c8 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] (13712) timed out ('192.168.170.30', 56235)
2016-06-14 13:47:40.564 13714 INFO neutron.wsgi [req-0334722f-a1b1-4267-bcf9-b3ccafb43d65 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] (13714) timed out ('192.168.170.30', 56248)
2016-06-14 13:47:42.473 13712 INFO neutron.wsgi [req-632cd300-c55b-4512-a64b-bf20c29065f6 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] (13712) timed out ('192.168.170.30', 56255)
2016-06-14 15:24:56.523 13681 INFO neutron.wsgi [-] (13681) accepted ('192.168.170.30', 56365)
2016-06-14 15:24:56.609 13681 INFO neutron.wsgi [req-30ae842f-924f-432e-b64b-6825260af7e0 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] 192.168.170.30 - - [14/Jun/2016 15:24:56] "GET /v2.0/ports.json?device_id=c616fb8f-9e6c-40f7-bf78-9cbf4db70633&device_id=cb9bf36a-271e-4412-8c76-ec7df942699a&device_id=896a8573-210e-483b-b1db-764005d109db&device_id=7bbacafd-ca1e-43ba-937b-48e0efa15c5a&device_id=20422504-9c20-470a-acc1-01cb19aa18e9&device_id=7dd1fcdb-5dc5-4953-a5de-c9cf38bebc63&device_id=e6832245-d2ad-4ddc-b0e7-404290b20a83&device_id=5cdad546-9e44-4e85-bfba-38a7f2b4db39&device_id=664baa6f-0861-45b7-8727-90b91fb414e6&device_id=11268a67-5d10-4a34-b8cf-72031da17ed6&device_id=8a642bba-d5ed-406f-8a70-c421b95d79f7 HTTP/1.1" 200 226 0.085084
2016-06-14 15:39:56.705 13681 INFO neutron.wsgi [req-30ae842f-924f-432e-b64b-6825260af7e0 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] (13681) timed out ('192.168.170.30', 56365)
2016-06-14 15:44:46.334 13696 INFO neutron.wsgi [-] (13696) accepted ('192.168.170.30', 56443)
2016-06-14 15:44:46.404 13696 INFO neutron.wsgi [req-623d8154-2eff-4836-94c9-b9227294eae8 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] 192.168.170.30 - - [14/Jun/2016 15:44:46] "GET /v2.0/agents.json HTTP/1.1" 200 227 0.069245
2016-06-14 15:44:49.949 13701 INFO neutron.wsgi [-] (13701) accepted ('192.168.170.30', 56450)
2016-06-14 15:44:50.018 13701 INFO neutron.wsgi [req-85a0bbcf-df81-475b-a6c1-a17a1b97237a 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] 192.168.170.30 - - [14/Jun/2016 15:44:50] "GET /v2.0/agents.json HTTP/1.1" 200 227 0.068902
2016-06-14 15:44:55.192 13722 INFO neutron.wsgi [-] (13722) accepted ('192.168.170.30', 56456)
2016-06-14 15:44:55.261 13722 INFO neutron.wsgi [req-97ead42e-dff4-4fbe-bc26-01cc625b779e 22ec63ad76594b32a982ab32c6c26816 94513aaa320943d2b9bdb7f681545c00 - - -] 192.168.170.30 - - [14/Jun/2016 15:44:55] "GET /v2.0/agents.json HTTP/1.1" 200 227 0.068442

0.068442

Please advise and let me know if you need any additional information

thanks Sac