nova list - ERROR: HTTPConnectionPool(host='openstackcloud1', port=35357): Max retries exceeded with url: /v2.0/tokens (Caused by <class 'socket.error'>: [Errno 111] Connection refused)
Not sure how to troubleshoot this.... Nova was working...then it blew up.... Its been a couple weeks since I've looked at the problem, but I think this happend after I installed the nova-api-metadata package....I have since removed it, but nothing with nova is working.
root@openstackcloud1:~# nova list
ERROR: HTTPConnectionPool(host='openstackcloud1', port=35357): Max retries exceeded with url: /v2.0/tokens (Caused by <class 'socket.error'>: [Errno 111] Connection refused)
Any pointers would be extremely helpful... I'm using Ubuntu 12.04.03 and OpenStack Havana. My keystone, cinder, heat, and horizon services are working fine.
+----------------------------------+----------+----------------+---------------------------+
| id | name | type | description |
+----------------------------------+----------+----------------+---------------------------+
| 1d8c950f65a342c8b400529a531e03ed | cinder | volumev2 | Cinder Volume Service V2 |
| 4cf047e43a834388b1e6fc5e4292cd29 | cinder | volume | Cinder Volume Service |
| f7c043e5b305456384d6002a191e7aed | glance | image | Glance Image Service |
| f17ff36eded547a799e5a71c1d95ec9f | heat | orchestration | Heat Orchestration API |
| c81aede2173e4e71b82618e04d351331 | heat-cfn | cloudformation | Heat CloudFormation API |
| 1e9dc6ce38404fb290a2e6d75c6b4b9d | keystone | identity | Keystone Identity Service |
| 3a44e89c99d54af18d8741e6e1e2c652 | nova | compute | Nova Compute Service |
+----------------------------------+----------+----------------+---------------------------+
root@openstackcloud1:~# keystone endpoint-list
+----------------------------------+-----------+----------------------------------------------+----------------------------------------------+----------------------------------------------+----------------------------------+
| id | region | publicurl | internalurl | adminurl | service_id |
+----------------------------------+-----------+----------------------------------------------+----------------------------------------------+----------------------------------------------+----------------------------------+
| 1f40acb0c58249dabdb07ad1674b1d53 | regionOne | http://openstackcloud1:8776/v2/%(tenant_id)s | http://openstackcloud1:8776/v2/%(tenant_id)s | http://openstackcloud1:8776/v2/%(tenant_id)s | 1d8c950f65a342c8b400529a531e03ed |
| 488535d7ab72474ca8310bb3684e024d | regionOne | http://openstackcloud1:8774/v2/%(tenant_id)s | http://openstackcloud1:8774/v2/%(tenant_id)s | http://openstackcloud1:8774/v2/%(tenant_id)s | 3a44e89c99d54af18d8741e6e1e2c652 |
| 5c96f322cc994c37b22ff7e8f10b0b07 | regionOne | http://openstackcloud1:5000/v2.0 | http://openstackcloud1:5000/v2.0 | http://openstackcloud1:35357/v2.0 | 1e9dc6ce38404fb290a2e6d75c6b4b9d |
| 6c6f23b9c64a4c71abc875f3f4b4d494 | regionOne | http://openstackcloud1:9292 | http://openstackcloud1:9292 | http://openstackcloud1:9292 | f7c043e5b305456384d6002a191e7aed |
| d99c5ab966cd4850baea5cfb3d4b15c4 | regionOne | http://openstackcloud1:8000/v1 | http://openstackcloud1:8000/v1 | http://openstackcloud1:8000/v1 | c81aede2173e4e71b82618e04d351331 |
| dd252e79ea8b47c08f86943343b3f0b4 | regionOne | http://openstackcloud1:8004/v1/%(tenant_id)s | http://openstackcloud1:8004/v1/%(tenant_id)s | http://openstackcloud1:8004/v1/%(tenant_id)s | f17ff36eded547a799e5a71c1d95ec9f |
| f649dd9c49aa4bdd90838b5ee596a8f5 | regionOne | http://openstackcloud1:8776/v1/%(tenant_id)s | http://openstackcloud1:8776/v1/%(tenant_id)s | http://openstackcloud1:8776/v1/%(tenant_id)s | 4cf047e43a834388b1e6fc5e4292cd29 |
+----------------------------------+-----------+----------------------------------------------+----------------------------------------------+----------------------------------------------+----------------------------------+
Debug for nova list
INFO (connectionpool:202) Starting new HTTP connection (1): openstackcloud1
DEBUG (shell:740) HTTPConnectionPool(host='openstackcloud1', port=8774): Max retries exceeded with url: /v2/728de3699e104a9b8a9f2ef8b15c84c7/servers/detail (Caused by <class 'socket.error'>: [Errno 111] Connection refused)
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/novaclient/shell.py", line 737, in main
OpenStackComputeShell().main(map(strutils.safe_decode, sys.argv[1:]))
File "/usr/lib/python2.7/dist-packages/novaclient/shell.py", line 673, in main
args.func(self.cs, args)
File "/usr/lib/python2.7/dist-packages/novaclient/v1_1/shell.py", line 1129, in do_list
servers = cs.servers.list(search_opts=search_opts)
File "/usr/lib/python2.7/dist-packages/novaclient/v1_1/servers.py", line 404, in list
return self._list("/servers%s%s" % (detail, query_string), "servers")
File "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 61, in _list
_resp, body = self.api.client.get(url)
File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 229, in get
return self._cs_request(url, 'GET', **kwargs)
File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 213, in _cs_request
**kwargs)
File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 195, in _time_request
resp, body = self.request(url, method, **kwargs)
File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 166, in request
**kwargs)
File ...
Check the logs to see what causes the socket.error exception. You may want to search also on this site for the error messages as I suspect similar issues have been solved before.
1. Check your keystone process is running 2. Did you make any configuration changes in nova.conf ? Please check. Revert back if any. If not do let me know. I can provide you working nova.conf with your changes. 3. Hope you host name is resolvable. 4. telnet openstackcloud1 35357 5. Try nova --debug list I suspect the 2nd point as issue in your case. Please check all these and revert.
1. Keystone is running 2. (This I will need to verify when I get to my desk) 3. Yes the hostname is resolvable on my network 4. I am able to telnet into 35357 5. Debug results now in my original question
I didn't copy the entire curl string from my --debug because it was quite long
Posted my nova.conf on my question.