Changed host name on controller; all compute services are now "down"
Hi,
I'm starting to install OpenStack on a controller and compute node (both running CentOS 7) with the official installation guide. I've just changed the hostname of my controller server from "localhost.localdomain" to "controller"; I had forgotten to do so earlier. Everything has just broken. When I run the command openstack compute service list
, all the services are "down."
+----+------------------+------------+----------+---------+-------+----------------------------+
| ID | Binary | Host | Zone | Status | State | Updated At |
+----+------------------+------------+----------+---------+-------+----------------------------+
| 11 | nova-compute | compute1 | nova | enabled | down | 2017-07-06T15:52:01.000000 |
| 12 | nova-consoleauth | controller | internal | enabled | down | None |
| 13 | nova-conductor | controller | internal | enabled | down | None |
| 14 | nova-scheduler | controller | internal | enabled | down | None |
+----+------------------+------------+----------+---------+-------+----------------------------+
The error messages on the controller are all very similar.
tail -n 50 /var/log/nova/nova-scheduler.log
…
2017-07-06 10:29:14.306 28323 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 273, in _wait_method
2017-07-06 10:29:14.306 28323 ERROR oslo_service.service self.wait()
2017-07-06 10:29:14.306 28323 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 69, in wait
2017-07-06 10:29:14.306 28323 ERROR oslo_service.service return self.dispatch_method(method_sig, args, content)
2017-07-06 10:29:14.306 28323 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 87, in dispatch_method
2017-07-06 10:29:14.306 28323 ERROR oslo_service.service return amqp_method(self, args)
2017-07-06 10:29:14.306 28323 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 529, in _close
2017-07-06 10:29:14.306 28323 ERROR oslo_service.service (class_id, method_id), ConnectionError)
2017-07-06 10:29:14.306 28323 ERROR oslo_service.service AccessRefused: (0, 0): (403) ACCESS_REFUSED - Login was refused using authentication mechanism AMQPLAIN. For details see the broker logfile.
2017-07-06 10:29:14.306 28323 ERROR oslo_service.service
The compute node cannot start the service openstack-nova-compute.service
. This was working properly before the controller hostname was changed and was "up" on openstack compute service list
.
systemctl start openstack-nova-compute.service
Job for openstack-nova-compute.service failed because the control process exited with error code. See "systemctl status openstack-nova-compute.service" and "journalctl -xe" for details.
The commands it recommends are not too helpful.
journalctl -xe
-- Unit openstack-nova-compute.service has begun starting up.
Jul 06 10:35:38 compute1 systemd[1]: openstack-nova-compute.service: main proces
Jul 06 10:35:38 compute1 systemd[1]: Failed to start OpenStack Nova Compute Serv
-- Subject: Unit openstack-nova-compute.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit openstack-nova-compute.service has failed.
--
-- The result is failed.
The error messages are generic.
systemctl status openstack-nova-compute.service
● openstack-nova-compute.service - OpenStack Nova Compute Server
Loaded: loaded (/usr/lib/systemd/system/openstack-nova-compute.service; enabled; vendor preset: disabled)
Active: activating (start) since Thu 2017-07-06 10:39:25 MDT; 767ms ago
Main PID: 8461 (nova-compute)
CGroup: /system.slice/openstack-nova-compute.service
└─8461 /usr/bin/python2 /usr/bin/nova-compute
Jul 06 10:39:25 compute1 systemd[1]: Starting OpenStack Nova Compute Server...
The services stay in the "starting" state, which leads me to believe they are not ...
Most likely your new hostname doesn't match the AMQP configuration in nova.conf. The config variable to check is transport_url. You can also check the AMQP log file (probably Rabbitmq).