nova_compute docker restarting continuously

asked 2019-08-10 13:10:13 -0500

snowmoon gravatar image

Hello , I have installed latest queens setup in centos 7 with 3 controller and 4 HCI node . In 2 HCI node nova_compute container is keep restarting and nova service showing down in horizon dash board .logged into nova_container and checked the logs .

/var/log/nova/nova-compute.log

2019-08-10 14:34:22.170 8 WARNING oslo_config.cfg [req-469b3c5d-f764-4531-9cfc-6b584cffa966 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. 2019-08-10 14:34:22.170 8 WARNING oslo_config.cfg [req-469b3c5d-f764-4531-9cfc-6b584cffa966 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. 2019-08-10 14:34:22.170 8 WARNING oslo_config.cfg [req-469b3c5d-f764-4531-9cfc-6b584cffa966 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url). Its value may be silently ignored in the future. 2019-08-10 14:34:32.201 8 WARNING nova.conductor.api [req-469b3c5d-f764-4531-9cfc-6b584cffa966 - - - - -] Timed out waiting for nova-conductor. Is it running? Or did this service start before nova-conductor? Reattempting establishment of nova-conductor connection...: MessagingTimeout: Timed out waiting for a reply to message ID ab6f667415c441ea8dc75662a1c5b68c

While restarting its giving the below error message

2019-08-10 18:08:15.048 8 ERROR nova.virt.libvirt.driver retry=retry) 2019-08-10 18:08:15.048 8 ERROR nova.virt.libvirt.driver File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 548, in _send 2019-08-10 18:08:15.048 8 ERROR nova.virt.libvirt.driver result = self._waiter.wait(msg_id, timeout) 2019-08-10 18:08:15.048 8 ERROR nova.virt.libvirt.driver File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 440, in wait 2019-08-10 18:08:15.048 8 ERROR nova.virt.libvirt.driver message = self.waiters.get(msg_id, timeout=timeout) 2019-08-10 18:08:15.048 8 ERROR nova.virt.libvirt.driver File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 328, in get 2019-08-10 18:08:15.048 8 ERROR nova.virt.libvirt.driver 'to message ID %s' % msg_id) 2019-08-10 18:08:15.048 8 ERROR nova.virt.libvirt.driver MessagingTimeout: Timed out waiting for a reply to message ID a5fe2cc26a52419d9fb4801175c9ec27 2019-08-10 18:08:15.048 8 ERROR nova.virt.libvirt.driver 2019-08-10 18:08:15.116 8 ERROR oslo_service.service [req-6fa5033c-5596-499b-af40-55958304dba0 - - - - -] Error starting thread.: MessagingTimeout: Timed out waiting for a reply to message ID ef2a1d8a15aa43059947b139f8ca6d2f 2019-08-10 18:08:15.116 8 ERROR oslo_service.service Traceback (most recent call last): 2019-08-10 18:08:15.116 8 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/oslo_service/service.py", line 731, in run_service 2019-08-10 18:08:15.116 8 ERROR oslo_service.service service.start() 2019-08-10 18:08:15.116 8 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/nova/service.py", line 161, in start 2019-08-10 18:08:15.116 8 ERROR oslo_service.service self.manager.init_host() 2019-08-10 18:08:15.116 8 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1183, in init_host 2019-08-10 18:08:15.116 8 ERROR ... (more)

edit retag flag offensive close merge delete

Comments

The error is "Timed out waiting for nova-conductor". Since only two nodes have the problem, check for message-queue related problems. A typical cause: Time is not synchronized among nodes.

Bernd Bausch gravatar imageBernd Bausch ( 2019-08-11 03:57:11 -0500 )edit