Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Nova compute node network connectivity error.

I have a single compute node in a 4 compute node cluster, that when I have vms on that node, I have no network connectivity to the nodes. As soon as I live migrate them off that host to any other host, network connectivty comes right up. Any ideas what I should look for? I've looked through all of the logs and I see this in the neutron logs.

<166>Aug 27 16:40:27 node-5 neutron-openvswitch-agent 2015-08-27 16:40:27.263 5677 INFO oslo.messaging._drivers.impl_rabbit [-] Delaying reconnect for 5.0 seconds ... <163>Aug 27 16:40:27 node-5 neutron-openvswitch-agent 2015-08-27 16:40:27.264 5677 ERROR oslo.messaging._drivers.impl_rabbit [-] Failed to consume message from queue: (0, 0): (320) CONNECTION_FORCED - broker forced connection closure with reason 'shutdown' 2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit Traceback (most recent call last): 2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit File "/usr/lib/python2.7/dist-packages/oslo/messaging/_drivers/impl_rabbit.py", line 701, in ensure 2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit return method() 2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit File "/usr/lib/python2.7/dist-packages/oslo/messaging/_drivers/impl_rabbit.py", line 797, in _consume 2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit return self.connection.drain_events(timeout=poll_timeout) 2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit File "/usr/lib/python2.7/dist-packages/kombu/connection.py", line 279, in drain_events 2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit return self.transport.drain_events(self.connection, kwargs) 2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit File "/usr/lib/python2.7/dist-packages/kombu/transport/pyamqp.py", line 91, in drain_events 2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit return connection.drain_events(kwargs) 2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit File "/usr/lib/python2.7/dist-packages/amqp/connection.py", line 320, in drain_events 2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit return amqp_method(channel, args) 2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit File "/usr/lib/python2.7/dist-packages/amqp/connection.py", line 526, in _close 2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit (class_id, method_id), ConnectionError) 2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit ConnectionForced: (0, 0): (320) CONNECTION_FORCED - broker forced connection closure with reason 'shutdown' 2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit

Nova compute node network connectivity error.

I have a single compute node in a 4 compute node cluster, that when I have vms on that node, I have no network connectivity to the nodes. As soon as I live migrate them off that host to any other host, network connectivty comes right up. Any ideas what I should look for? I've looked through all of the logs and I see this in the neutron logs.

<166>Aug 27 16:40:27 node-5 neutron-openvswitch-agent 2015-08-27 16:40:27.263 5677 INFO oslo.messaging._drivers.impl_rabbit [-] Delaying reconnect for 5.0 seconds ...
<163>Aug 27 16:40:27 node-5 neutron-openvswitch-agent 2015-08-27 16:40:27.264 5677 ERROR oslo.messaging._drivers.impl_rabbit [-] Failed to consume message from queue: (0, 0): (320) CONNECTION_FORCED - broker forced connection closure with reason 'shutdown'
2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit Traceback (most recent call last):
2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit   File "/usr/lib/python2.7/dist-packages/oslo/messaging/_drivers/impl_rabbit.py", line 701, in ensure
2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit     return method()
2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit   File "/usr/lib/python2.7/dist-packages/oslo/messaging/_drivers/impl_rabbit.py", line 797, in _consume
2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit     return self.connection.drain_events(timeout=poll_timeout)
2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit   File "/usr/lib/python2.7/dist-packages/kombu/connection.py", line 279, in drain_events
2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit     return self.transport.drain_events(self.connection, kwargs)
**kwargs)
2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit   File "/usr/lib/python2.7/dist-packages/kombu/transport/pyamqp.py", line 91, in drain_events
2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit     return connection.drain_events(kwargs)
connection.drain_events(**kwargs)
2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit   File "/usr/lib/python2.7/dist-packages/amqp/connection.py", line 320, in drain_events
2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit     return amqp_method(channel, args)
2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit   File "/usr/lib/python2.7/dist-packages/amqp/connection.py", line 526, in _close
2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit     (class_id, method_id), ConnectionError)
2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit ConnectionForced: (0, 0): (320) CONNECTION_FORCED - broker forced connection closure with reason 'shutdown'
2015-08-27 16:40:27.264 5677 TRACE oslo.messaging._drivers.impl_rabbit 

oslo.messaging._drivers.impl_rabbit