Keystone Connection Failure After Running For Sometime

asked 2016-08-03 17:32:40 -0600

teeyen gravatar image

updated 2016-08-04 05:54:36 -0600

After running successfully for few weeks,

$ systemctl status -l  openstack-keystone

Active: active (running) since Fri 2016-07-08 16:16:17 PDT; 3 weeks 4 days ago

the connection to keystone, via my Python program using keystone v2 client APIs, failed.

2016-08-03T12:46:58-0700 Auth success status,0 
2016-08-03T13:10:57-0700 AuthorizationFailure(u'Authorization Failed: Unable to establish connection to http://localhost:35357/v2.0/tokens',) 
2016-08-03T13:10:57-0700 Auth failure status,1

Here is the thing.. I can access the keystone other ways

 ~> curl -s -X POST http://localhost:35357/v2.0/tokens   -H "Content-Type: application/json"   -d '{"auth": {"tenantName": "'"admin"'", "passwordCredentials": {"username": "'"admin"'", "password": "'"foo"'"}}}'   | python -m json.tool
{
    "access": {
.
.
.
        "user": {
            "id": "03c45e2cc3f345ec8e1d8c17f04f25cd",
            "name": "admin",
            "roles": [
                {
                    "name": "admin"
                },
                {
                    "name": "_member_"
                }
            ],
            "roles_links": [],
            "username": "admin"
        }
    }
}

openstack --os-auth-url http://localhost:35357 --os-project-name admin --os-username admin --os-auth-type password user list
Password: 
+----------------------------------+---------+
| ID                               | Name    |
+----------------------------------+---------+
| 03c45e2cc3f345ec8e1d8c17f04f25cd | admin   |
.
.
.
+----------------------------------+---------+

From the keystone.log, I see

2016-08-03 12:46:58.028 14834 INFO eventlet.wsgi.server [-] 127.0.0.1 - - [03/Aug/2016 12:46:58] "POST /v2.0/tokens HTTP/1.1" 200 1093 0.112887
2016-08-03 12:46:58.144 14833 INFO keystone.common.wsgi [-] POST http://localhost:35357/v2.0/tokens
2016-08-03 12:46:58.249 14833 INFO eventlet.wsgi.server [-] 127.0.0.1 - - [03/Aug/2016 12:46:58] "POST /v2.0/tokens HTTP/1.1" 200 1093 0.107923
2016-08-03 13:00:02.145 6225 INFO keystone.common.kvs.core [-] Using default dogpile sha1_mangle_key as KVS region token-driver key_mangler
2016-08-03 13:00:02.145 6225 CRITICAL keystone [-] NotImplemented: The action you have requested has not been implemented.
2016-08-03 13:00:02.145 6225 TRACE keystone Traceback (most recent call last):
2016-08-03 13:00:02.145 6225 TRACE keystone   File "/usr/bin/keystone-manage", line 44, in <module>
2016-08-03 13:00:02.145 6225 TRACE keystone     cli.main(argv=sys.argv, config_files=config_files)
2016-08-03 13:00:02.145 6225 TRACE keystone   File "/usr/lib/python2.7/site-packages/keystone/cli.py", line 585, in main
2016-08-03 13:00:02.145 6225 TRACE keystone     CONF.command.cmd_class.main()
2016-08-03 13:00:02.145 6225 TRACE keystone   File "/usr/lib/python2.7/site-packages/keystone/cli.py", line 243, in main
2016-08-03 13:00:02.145 6225 TRACE keystone     token_manager.flush_expired_tokens()
2016-08-03 13:00:02.145 6225 TRACE keystone   File "/usr/lib/python2.7/site-packages/keystone/token/persistence/backends/kvs.py", line 357, in flush_expired_tokens
2016-08-03 13:00:02.145 6225 TRACE keystone     raise exception.NotImplemented()
2016-08-03 13:00:02.145 6225 TRACE keystone NotImplemented: The action you have requested has not been implemented.
2016-08-03 13:00:02.145 6225 TRACE keystone
2016-08-03 13:14:01.873 14836 INFO keystone.common.wsgi [-] GET http://localhost:35357/
2016-08-03 13:14:01.875 14836 INFO eventlet.wsgi.server [-] 127.0.0.1 - - [03/Aug/2016 13:14:01] "GET / HTTP/1.1" 300 756 0.003248

Why am I getting connection failure?

edit retag flag offensive close merge delete