401 errors on glance/nova image list [closed]
seemingly over night none of my users or my admin account can log in to the horizon dashboard. I tailed the logs of nova-api and got basically a 401 error. I then issued the command
nova --debug image-list
and recieved the following output:
REQ: curl -i http://10.10.1.10:8774/v2/f7b27e8db30645dda873d4c772a528e6/images/detail -X GET -H "X-Auth-Project-Id: admin" -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: [token code]"
send: u'GET /v2/f7b27e8db30645dda873d4c772a528e6/images/detail HTTP/1.1\r\nHost: 10.10.1.10:8774\r\nx-auth-project-id: admin\r\nx-auth-token: [tokencode]=\r\naccept-encoding: gzip, deflate\r\naccept: application/json\r\nuser-agent: python-novaclient\r\n\r\n'
reply: 'HTTP/1.1 401 Unauthorized\r\n'
header: Www-Authenticate: Keystone uri='http://127.0.0.1:5000/'
header: Content-Length: 276
header: Content-Type: text/plain; charset=UTF-8
header: Date: Fri, 24 Jan 2014 20:34:20 GMT
RESP:{'date': 'Fri, 24 Jan 2014 20:34:20 GMT', 'status': '401', 'content-length': '276', 'content-type': 'text/plain; charset=UTF-8', 'www-authenticate': "Keystone uri='http://127.0.0.1:5000/'"} 401 Unauthorized
This server could not verify that you are authorized to access the document you requested. Either you supplied the wrong credentials (e.g., bad password), or your browser does not understand how to supply the credentials required.
Authentication required
DEBUG (shell:543) n/a (HTTP 401)
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/novaclient/shell.py", line 540, in main
OpenStackComputeShell().main(sys.argv[1:])
File "/usr/local/lib/python2.7/dist-packages/novaclient/shell.py", line 476, in main
args.func(self.cs, args)
File "/usr/local/lib/python2.7/dist-packages/novaclient/v1_1/shell.py", line 517, in do_image_list
image_list = cs.images.list()
File "/usr/local/lib/python2.7/dist-packages/novaclient/v1_1/images.py", line 47, in list
return self._list("/images/detail", "images")
File "/usr/local/lib/python2.7/dist-packages/novaclient/base.py", line 62, in _list
_resp, body = self.api.client.get(url)
File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 241, in get
return self._cs_request(url, 'GET', **kwargs)
File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 238, in _cs_request
raise ex
Unauthorized: n/a (HTTP 401)
ERROR: n/a (HTTP 401)
Both glance and nova output similar things. However I can still issue keystone commands. Issuing a keystone endpoint-list command returns:
+----------------------------------+-----------+-----------------------------------------+-----------------------------------------+-----------------------------------------+----------------------------------+
| id | region | publicurl | internalurl | adminurl | service_id |
+----------------------------------+-----------+-----------------------------------------+-----------------------------------------+-----------------------------------------+----------------------------------+
| 170adbc342774125bbb3501ff75a77b6 | RegionOne | http://10.10.1.10:9292/v1 | http://10.10.1.10:9292/v1 | http://10.10.1.10:9292/v1 | 9eb322e90a894dbcb6ccb2273df7b080 |
| 3c59ced5b1f542b6a66f83fc0cda7feb | RegionOne | http://10.10.1.10:5000/v2.0 | http://10.10.1.10:5000/v2.0 | http://10.10.1.10:35357/v2.0 | a3f470c0e4dc4926bb5456354182e922 |
| 8a79e7b7331e44058d1f51deaaad87e9 | RegionOne | http://10.10.1.10:8774/v2/%(tenant_id)s | http://10.10.1.10:8774/v2/%(tenant_id)s | http://10.10.1.10:8774/v2/%(tenant_id)s | cf35bb2df8324710b9fcd1a91901f353 |
+----------------------------------+-----------+-----------------------------------------+-----------------------------------------+-----------------------------------------+----------------------------------+
What has gone wrong? Where can I start looking?
Did you manage to resolve this issue, if so can you post the fix.
Thanks
The version of OpenStack that this is opened against has been declared EOL as per http://releases.openstack.org/ Please open a new issue if you are still seeing this problem with a more recent, supported version of OpenStack. Thanks.