Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

RabbitMQ: "vhost '/nova' is down" ("handshake_error")

After a complete system failure due to a broken switch, RabbitMQ no longer works. Nova and Neutron both report errors on Rabbit:

Nova Scheduler: ERROR oslo.messaging._drivers.impl_rabbit [req-df5ed105-8ace-4b46-8338-bef2703b53ba - - - - -] Unable to connect to AMQP server on 172.29.236.254:5671 after None tries: Connection.open: (541) INTERNAL_ERROR - access to vhost '/nova' refused for user 'nova': vhost '/nova' is down: InternalError: Connection.open: (541) INTERNAL_ERROR - access to vhost '/nova' refused for user 'nova': vhost '/nova' is down

Nova Conductor: ERROR oslo_service.service [req-966eb127-b744-4aff-8cef-d99c88180494 - - - - -] Error starting thread.: MessageDeliveryFailure: Unable to connect to AMQP server on 172.29.236.254:5671 after None tries: Connection.open: (541) INTERNAL_ERROR - access to vhost '/nova' refused for user 'nova': vhost '/nova' is down

Nova Auth: ERROR oslo_service.service MessageDeliveryFailure: Unable to connect to AMQP server on 172.29.236.254:5671 after None tries: Connection.open: (541) INTERNAL_ERROR - access to vhost '/nova' refused for user 'nova': vhost '/nova' is down

Neutron DHCP: ERROR oslo_messaging._drivers.amqpdriver [-] Failed to process incoming message, retrying...: MessageDeliveryFailure: Unable to connect to AMQP server on 172.29.236.254:5671 after None tries: Connection.open: (541) INTERNAL_ERROR - access to vhost '/neutron' refused for user 'neutron': vhost '/neutron' is down

Obviously these errors all have in common that the "vhost is down". Rabbit MQ reports similar errors: 2020-03-02 22:45:14.003 [error] <0.9336.110> Error on AMQP connection <0.9336.110> (172.29.238.25:41662 -> 172.29.236.254:5671 - neutron-l3-agent:19718:d44d78bb-3d13-45aa-8956-2c820feea111, vhost: 'none', user: 'neutron', state: opening), channel 0: {handshake_error,opening, {amqp_error,internal_error, "access to vhost '/neutron' refused for user 'neutron': vhost '/neutron' is down", 'connection.open'}}

The "handshake_error" makes me think there is a problem with SSL. I tried deactivating SSL in nova.conf but it did not solve the errors. What can I do to solve the problem?