Ask Your Question
4

Openstack services cannot talk to RabbitMQ: Connection refused but log says unreachable?

asked 2013-07-12 12:38:54 -0500

filpano gravatar image

I've got single-node Openstack Folsom deployment on a Debian 7.1 server. I've followed the Folsom guide so far and have got everything set up, but when I try to log in through the dashboard, I get the following message:

ConnectionError at /admin/

[Errno 111] Connection refused

Request Method:     GET
Request URL:    https://172.23.18.174/admin/
Django Version:     1.4.5
Exception Type:     ConnectionError
Exception Value:    

[Errno 111] Connection refused

Exception Location:     /usr/lib/python2.7/dist-packages/requests/adapters.py in send, line 243
Python Executable:  /usr/bin/python
Python Version:     2.7.3
Python Path:    

['/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../..',
 '/usr/lib/python2.7',
 '/usr/lib/python2.7/plat-linux2',
 '/usr/lib/python2.7/lib-tk',
 '/usr/lib/python2.7/lib-old',
 '/usr/lib/python2.7/lib-dynload',
 '/usr/local/lib/python2.7/dist-packages',
 '/usr/lib/python2.7/dist-packages',
 '/usr/lib/pymodules/python2.7',
 '/usr/share/openstack-dashboard/',
 '/usr/share/openstack-dashboard/openstack_dashboard']

Server time:    Fri, 12 Jul 2013 16:04:49 +0000

Now I've researched this issue somewhat and have found that connection refused stems from a misconfigured quantum.conf. So I double checked, and my quantum.conf is the following (relevant settings only):

# Address to bind the API server
bind_host = 0.0.0.0

# Port the (sic) bind the API server to
bind_port = 9696

...

# IP address of the RabbitMQ server
rabbit_password = password
# Port where RabbitMQ server is running/listening
rabbit_port = 5672
...
# User ID used for RabbitMQ connections
rabbit_userid = guest

... (everything else is unchanged)

So the login credentials should be OK (nova.conf credentials match too). Other credentials are, of course, sourced upon login.

However, my quantum-server.log shows the following:

2013-07-12 12:00:42 ERROR [quantum.openstack.common.rpc.common] AMQP server on localhost:5672 is unreachable: Socket closed. Trying again in 29 seconds.

Anyone know what exactly is going on here? One thing that I noticed was that I lack a /etc/rabbitmq/rabbitmq.conf file, although I saw it mentioned somewhere. Might this have anything to do with this?

Upon further inspection, it would seem that the nova services I have installed so far cannot talk to RabbitMQ at all (as nova-scheduler, nova-consoleauth, nova-cert and nova-conductor all show XXX as their state), as I've found the same error message from quantum logs in the nova-cert.log:

2013-07-12 12:26:47.526 ERROR nova.openstack.common.rpc.common [req-705a947d-28b8-4c1e-8475-2afb19ad96a6 None None] AMQP server on 172.23.18.174:5672 is unreachable: Socket closed. Trying again in 30 seconds.

It should be noted that the machine running all these services is 172.23.18.174. However, I've tried setting both IPs in quantum.conf and nova.conf to either one or the other, but this did not alleviate the issue. Anyone know what is going on here?

edit retag flag offensive close merge delete

Comments

what does 'netstat -an | grep 5672' give?

darragh-oreilly gravatar imagedarragh-oreilly ( 2013-07-12 15:41:59 -0500 )edit

@filpano have you found an answer to this question?

smaffulli gravatar imagesmaffulli ( 2013-07-26 16:54:52 -0500 )edit

3 answers

Sort by ยป oldest newest most voted
2

answered 2013-08-22 12:37:41 -0500

nko321 gravatar image

I had this problem. darragh-oreilly got me on the right track. rabbitmq-server wasn't running. I did a "service rabbitmq-server restart", waited about 15 seconds, and I'm back to getting successful connections!

edit flag offensive delete link more
1

answered 2014-05-07 07:34:51 -0500

dazzpj gravatar image

I had the very same problem and went round in circles checking and re-checking all config pieces, eventually i went back to the start with rabbitmq and changed the password on the guest user, although I had changed it during my original configuration and was sure I had not made a mistake, after changing the password everything started working as expected.

edit flag offensive delete link more
0

answered 2014-07-19 03:25:35 -0500

Vanathu chinnappan gravatar image

i am facing same issues

root@compute2:/var/log/nova# tail -f nova-compute.log 2014-07-19 13:50:33.129 2704 INFO nova.openstack.common.rpc.common [req-276f428d-2ce4-40da-ad99-ea7397777432 None None] Reconnecting to AMQP server on 172.23.228.150:5672 2014-07-19 13:50:36.147 2704 ERROR nova.openstack.common.rpc.common [req-276f428d-2ce4-40da-ad99-ea7397777432 None None] AMQP server on 172.23.228.150:5672 is unreachable: Socket closed. Trying again in 30 seconds. 2014-07-19 13:51:06.176 2704 INFO nova.openstack.common.rpc.common [req-276f428d-2ce4-40da-ad99-ea7397777432 None None] Reconnecting to AMQP server on 172.23.228.150:5672 2014-07-19 13:51:09.190 2704 ERROR nova.openstack.common.rpc.common [req-276f428d-2ce4-40da-ad99-ea7397777432 None None] AMQP server on 172.23.228.150:5672 is unreachable: Socket closed. Trying again in 30 seconds. 2014-07-19 13:51:39.219 2704 INFO nova.openstack.common.rpc.common [req-276f428d-2ce4-40da-ad99-ea7397777432 None None] Reconnecting to AMQP server on 172.23.228.150:5672 2014-07-19 13:51:42.231 2704 ERROR nova.openstack.common.rpc.common [req-276f428d-2ce4-40da-ad99-ea7397777432 None None] AMQP server on 172.23.228.150:5672 is unreachable: Socket closed. Trying again in 30 seconds. 2014-07-19 13:52:12.245 2704 INFO nova.openstack.common.rpc.common [req-276f428d-2ce4-40da-ad99-ea7397777432 None None] Reconnecting to AMQP server on 172.23.228.150:5672 2014-07-19 13:52:15.260 2704 ERROR nova.openstack.common.rpc.common [req-276f428d-2ce4-40da-ad99-ea7397777432 None None] AMQP server on 172.23.228.150:5672 is unreachable: Socket closed. Trying again in 30 seconds. 2014-07-19 13:52:45.289 2704 INFO nova.openstack.common.rpc.common [req-276f428d-2ce4-40da-ad99-ea7397777432 None None] Reconnecting to AMQP server on 172.23.228.150:5672 2014-07-19 13:52:48.305 2704 ERROR nova.openstack.common.rpc.common [req-276f428d-2ce4-40da-ad99-ea7397777432 None None] AMQP server on 172.23.228.150:5672 is unreachable: Socket closed. Trying again in 30 seconds. 2014-07-19 13:53:18.334 2704 INFO nova.openstack.common.rpc.common [req-276f428d-2ce4-40da-ad99-ea7397777432 None None] Reconnecting to AMQP server on 172.23.228.150:5672 2014-07-19 13:53:21.350 2704 ERROR nova.openstack.common.rpc.common [req-276f428d-2ce4-40da-ad99-ea7397777432 None None] AMQP server on 172.23.228.150:5672 is unreachable: Socket closed. Trying again in 30 seconds. 2014-07-19 13:53:51.379 2704 INFO nova.openstack.common.rpc.common [req-276f428d-2ce4-40da-ad99-ea7397777432 None None] Reconnecting to AMQP server on 172.23.228.150:5672 2014-07-19 13:53:54.389 2704 ERROR nova.openstack.common.rpc.common [req-276f428d-2ce4-40da-ad99-ea7397777432 None None] AMQP server on 172.23.228.150:5672 is unreachable: Socket closed. Trying again in 30 seconds. 2014-07-19 13:54:24.418 2704 INFO nova.openstack.common.rpc.common [req-276f428d-2ce4-40da-ad99-ea7397777432 None None] Reconnecting to AMQP server on 172.23.228.150:5672 2014-07-19 13:54:24.428 2704 ERROR nova.openstack.common.rpc.common [req-276f428d-2ce4-40da-ad99-ea7397777432 None None] AMQP server on 172.23.228.150:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 30 seconds. 2014-07-19 13:54:54.454 2704 INFO nova.openstack.common.rpc.common [req-276f428d-2ce4-40da-ad99-ea7397777432 None None] Reconnecting to AMQP server on 172.23.228.150:5672 2014-07-19 13:54:57 ... (more)

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

Stats

Asked: 2013-07-12 12:38:54 -0500

Seen: 7,006 times

Last updated: Jul 19 '14