nova service-list shows that all compute node are "down".

asked 2016-05-19 03:38:04 -0500

sosogh@gmail.com gravatar image

We have run openstack(icehouse on ubt 12.04/14.04) for a year. We setup RabbitMQ in master/slave mode . Openstack component connect to MQ via its vip. Some days ago ,the MQ master node went down , so the vip floated to slave node, after master node rebooted , vip floated back to master node.

And we have a problem now: nova service-list shows that all compute node are "down".

But in fact , there exists connection between compute node and MQ root@computenode:~# sudo netstat -antp |grep 5672 tcp 0 0 10.0.0.10:28846 10.0.2.7:5672 ESTABLISHED 53206/python
tcp 0 0 10.0.0.10:42368 10.0.2.7:5672 ESTABLISHED 2519/python
tcp 0 0 10.0.0.10:28847 10.0.2.7:5672 ESTABLISHED 53206/python
tcp 0 0 10.0.0.10:42367 10.0.2.7:5672 ESTABLISHED 2519/python
tcp 0 0 10.0.0.10:28845 10.0.2.7:5672 ESTABLISHED 53206/python

pid 2519 is neutron-openvswitch-agent pid 53206 is nova-compute

I know that if I restart nova-compute on the compute node, it will fix the problem , but it will cause a short network outage(some seconds) for the VM on that compute node .

My question is that : Is there any way to fix this problem without causing a short network outage for the VM ? Thank you .

edit retag flag offensive close merge delete