seemingly over night none of my users or my admin account can log in to the horizon dashboard. I tailed the logs of nova-api and got basically a 401 error. I then issued the command

   nova --debug image-list

and recieved the following output:

REQ: curl -i http://10.10.1.10:8774/v2/f7b27e8db30645dda873d4c772a528e6/images/detail -X GET -H "X-Auth-Project-Id: admin" -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: [token code]"

send: u'GET /v2/f7b27e8db30645dda873d4c772a528e6/images/detail HTTP/1.1\r\nHost: 10.10.1.10:8774\r\nx-auth-project-id: admin\r\nx-auth-token: [tokencode]=\r\naccept-encoding: gzip, deflate\r\naccept: application/json\r\nuser-agent: python-novaclient\r\n\r\n'
reply: 'HTTP/1.1 401 Unauthorized\r\n'
header: Www-Authenticate: Keystone uri='http://127.0.0.1:5000/'
header: Content-Length: 276
header: Content-Type: text/plain; charset=UTF-8
header: Date: Fri, 24 Jan 2014 20:34:20 GMT
RESP:{'date': 'Fri, 24 Jan 2014 20:34:20 GMT', 'status': '401', 'content-length': '276', 'content-type': 'text/plain; charset=UTF-8', 'www-authenticate': "Keystone uri='http://127.0.0.1:5000/'"} 401 Unauthorized

This server could not verify that you are authorized to access the document you requested. Either you supplied the wrong credentials (e.g., bad password), or your browser does not understand how to supply the credentials required.

 Authentication required

DEBUG (shell:543) n/a (HTTP 401)
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/novaclient/shell.py", line 540, in main
    OpenStackComputeShell().main(sys.argv[1:])
  File "/usr/local/lib/python2.7/dist-packages/novaclient/shell.py", line 476, in main
    args.func(self.cs, args)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/v1_1/shell.py", line 517, in do_image_list
    image_list = cs.images.list()
  File "/usr/local/lib/python2.7/dist-packages/novaclient/v1_1/images.py", line 47, in list
    return self._list("/images/detail", "images")
  File "/usr/local/lib/python2.7/dist-packages/novaclient/base.py", line 62, in _list
    _resp, body = self.api.client.get(url)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 241, in get
    return self._cs_request(url, 'GET', **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 238, in _cs_request
    raise ex
Unauthorized: n/a (HTTP 401)
ERROR: n/a (HTTP 401)

Both glance and nova output similar things. However I can still issue keystone commands. Issuing a keystone endpoint-list command returns:

+----------------------------------+-----------+-----------------------------------------+-----------------------------------------+-----------------------------------------+----------------------------------+
|                id                |   region  |                publicurl                |               internalurl               |                 adminurl                |            service_id            |
+----------------------------------+-----------+-----------------------------------------+-----------------------------------------+-----------------------------------------+----------------------------------+
| 170adbc342774125bbb3501ff75a77b6 | RegionOne |        http://10.10.1.10:9292/v1        |        http://10.10.1.10:9292/v1        |        http://10.10.1.10:9292/v1        | 9eb322e90a894dbcb6ccb2273df7b080 |
| 3c59ced5b1f542b6a66f83fc0cda7feb | RegionOne |       http://10.10.1.10:5000/v2.0       |       http://10.10.1.10:5000/v2.0       |       http://10.10.1.10:35357/v2.0      | a3f470c0e4dc4926bb5456354182e922 |
| 8a79e7b7331e44058d1f51deaaad87e9 | RegionOne | http://10.10.1.10:8774/v2/%(tenant_id)s | http://10.10.1.10:8774/v2/%(tenant_id)s | http://10.10.1.10:8774/v2/%(tenant_id)s | cf35bb2df8324710b9fcd1a91901f353 |
+----------------------------------+-----------+-----------------------------------------+-----------------------------------------+-----------------------------------------+----------------------------------+

What has gone wrong? Where can I start looking?