Ask Your Question
1

grizzly and 401 Unauthorized

asked 2015-03-09 08:26:08 -0500

abuzzi gravatar image

updated 2015-03-20 17:14:59 -0500

smaffulli gravatar image

We had a working grizzly environment then suddendly it stopped working. Currently we see any command (ie: nova list) get denied by 401 unauthorized. If we issue "nova --debug list” we see a token properly generated:

RESP BODY: {"access": {"token": {"issued_at": "2015-03-09T12:34:06.944224", "expires": "2015-03-10T12:34:06Z”,

Then nova try to authenticate against keystone:

REQ: curl -i http://10.58.10.119:8774/v2/c2c033b070dd402580a90e5fdc538520/servers/detail -X GET -H "X-Auth-Project-Id:

But it get back a:

RESP BODY: 401 Unauthorized

If we look at mysql db we see no entries into keystone-token table:

mysql> use keystone
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select * from token;
Empty set (0.00 sec)

mysql>

Apparently when the token was first generated it was not put into the database, hence when the nova service is trying to get authorization it get denied because the token is not valid / not existent (?). Changed keystone's backend from kvs (memory storage) to sql and now token are available to mysql:

root@mic-openstack-control:/etc# mysql -u root -p -e 'USE keystone; SELECT * FROM token;' | wc -l
Enter password: 
3595
root@mic-openstack-control:/etc#

Unfortunately HTTP 401 is still there:

root@mic-openstack-control:/home/cisco# nova list        
ERROR: Unauthorized (HTTP 401)
root@mic-openstack-control:/home/cisco# glance image-list
Request returned failure status.
Invalid OpenStack Identity credentials.
root@mic-openstack-control:/home/cisco# cinder list        
ERROR: Unauthorized
root@mic-openstack-control:/home/cisco#

Any hint is appreciated.

root@mic-openstack-control:/home/cisco# source openrc_admin
root@mic-openstack-control:/home/cisco# nova --debug list

REQ: curl -i http://10.58.10.119:5000/v2.0/tokens -X POST -H "Content-Type: application/json" -H "Accept: application/json" -H "User-Agent: python-novaclient" -d '{"auth": {"tenantName": "admin", "passwordCredentials": {"username": "admin", "password": "cisco"}}}'

INFO (connectionpool:191) Starting new HTTP connection (1): 10.58.10.119
DEBUG (connectionpool:283) "POST /v2.0/tokens HTTP/1.1" 200 5922
RESP: [200] {'date': 'Mon, 09 Mar 2015 13:24:35 GMT', 'content-type': 'application/json', 'content-length': '5922', 'vary': 'X-Auth-Token'}
RESP BODY: {"access": {"token": {"issued_at": "2015-03-09T13:24:35.299621", "expires": "2015-03-10T13:24:35Z", "id": "MIIKcwYJKoZIhvcNAQcCoIIKZDCCCmACAQExCTAHBgUrDgMCGjCCCUwGCSqGSIb3DQEHAaCCCT0Eggk5eyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMjAxNS0wMy0wOVQxMzoyNDozNS4yOTk2MjEiLCAiZXhwaXJlcyI6ICIyMDE1LTAzLTEwVDEzOjI0OjM1WiIsICJpZCI6ICJwbGFjZWhvbGRlciIsICJ0ZW5hbnQiOiB7ImRlc2NyaXB0aW9uIjogImFkbWluIHRlbmFudCIsICJlbmFibGVkIjogdHJ1ZSwgImlkIjogImMyYzAzM2IwNzBkZDQwMjU4MGE5MGU1ZmRjNTM4NTIwIiwgIm5hbWUiOiAiYWRtaW4ifX0sICJzZXJ2aWNlQ2F0YWxvZyI6IFt7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xMC41OC4xMC4xMTk6ODc3NC92Mi9jMmMwMzNiMDcwZGQ0MDI1ODBhOTBlNWZkYzUzODUyMCIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMC41OC4xMC4xMTk6ODc3NC92Mi9jMmMwMzNiMDcwZGQ0MDI1ODBhOTBlNWZkYzUzODUyMCIsICJpZCI6ICIwZjc3ZjI0MGRlNGM0YmMxODFkMWVjNmJiYjRjYTcyMyIsICJwdWJsaWNVUkwiOiAiaHR0cDovLzEwLjU4LjEwLjExOTo4Nzc0L3YyL2MyYzAzM2IwNzBkZDQwMjU4MGE5MGU1ZmRjNTM4NTIwIn1dLCAiZW5kcG9pbnRzX2xpbmtzIjogW10sICJ0eXBlIjogImNvbXB1dGUiLCAibmFtZSI6ICJub3ZhIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzEwLjU4LjEwLjExOTo5Njk2LyIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMC41OC4xMC4xMTk6OTY5Ni8iLCAiaWQiOiAiMjczMzQyYTYxMjEyNDRlZTgwN2Y5MDU0YzExODU1OGQiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMC41OC4xMC4xMTk6OTY5Ni8ifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAibmV0d29yayIsICJuYW1lIjogInF1YW50dW0ifSwgeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTAuNTguMTAuMTE5OjkyOTIiLCAicmVnaW9uIjogIlJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTAuNTguMTAuMTE5OjkyOTIiLCAiaWQiOiAiM2RlNGQ0Y2UzMzFmNGQzOTgyYTc1YzA0NWNkODg1NGMiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMC41OC4xMC4xMTk6OTI5MiJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJpbWFnZSIsICJuYW1lIjogImdsYW5jZSJ9LCB7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xMC41OC4xMC4xMTk6ODc3Ni92MS9jMmMwMzNiMDcwZGQ0MDI1ODBhOTBlNWZkYzUzODUyMCIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMC41OC4xMC4xMTk6ODc3Ni92MS9jMmMwMzNiMDcwZGQ0MDI1ODBhOTBlNWZkYzUzODUyMCIsICJpZCI6ICI1NmU2NjY1MjI1YWQ0M2E1OWJkNDdkYjFkYWM3YTk3MCIsICJwdWJsaWNVUkwiOiAiaHR0cDovLzEwLjU4LjEwLjExOTo4Nzc2L3YxL2MyYzAzM2IwNzBkZDQwMjU4MGE5MGU1ZmRjNTM4NTIwIn1dLCAiZW5kcG9pbnRzX2xpbmtzIjogW10sICJ0eXBlIjogInZvbHVtZSIsICJuYW1lIjogImNpbmRlciJ9LCB7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xMC41OC4xMC4xMTk6ODc3My9zZXJ2aWNlcy9BZG1pbiIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMC41OC4xMC4xMTk6ODc3My9zZXJ2aWNlcy9DbG91ZCIsICJpZCI6ICI0NDI4ZTZjNjI2NmE0M2Y2OTQ0ODY2MGQwMTgxYTZiYyIsICJwdWJsaWNVUkwiOiAiaHR0cDovLzEwLjU4LjEwLjExOTo4NzczL3NlcnZpY2VzL0Nsb3VkIn1dLCAiZW5kcG9pbnRzX2xpbmtzIjogW10sICJ0eXBlIjogImVjMiIsICJuYW1lIjogIm5vdmFfZWMyIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzEwLjU4LjEwLjExOTozNTM1Ny92Mi4wIiwgInJlZ2lvbiI6ICJSZWdpb25PbmUiLCAiaW50ZXJuYWxVUkwiOiAiaHR0cDovLzEwLjU4LjEwLjExOTo1MDAwL3YyLjAiLCAiaWQiOiAiODEzZGY1MzRhNGU3NGM3NGJkZTkwNDI4NDAzZGE1M2IiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMC41OC4xMC4xMTk6NTAwMC92Mi4wIn1dLCAiZW5kcG9pbnRzX2xpbmtzIjogW10sICJ0eXBlIjogImlkZW50aXR5IiwgIm5hbWUiOiAia2V5c3RvbmUifV0sICJ1c2VyIjogeyJ1c2VybmFtZSI6ICJhZG1pbiIsICJyb2xlc19saW5rcyI6IFtdLCAiaWQiOiAiMDM3NjZlMGM3M2MwNGNjZmIwZTc1NDZiNjljM2EwY2YiLCAicm9sZXMiOiBbeyJuYW1lIjogImFkbWluIn1dLCAibmFtZSI6ICJhZG1pbiJ9LCAibWV0YWRhdGEiOiB7ImlzX2FkbWluIjogMCwgInJvbGVzIjogWyIxZmVjMTNkMDIzZTE0ODA2YWFiZmQwZjA1Y2I3MGUzYiJdfX19MYH-MIH8AgEBMFwwVzELMAkGA1UEBhMCVVMxDjAMBgNVBAgTBVVuc2V0MQ4wDAYDVQQHEwVVbnNldDEOMAwGA1UEChMFVW5zZXQxGDAWBgNVBAMTD3d3dy5leGFtcGxlLmNvbQIBATAHBgUrDgMCGjANBgkqhkiG9w0BAQEFAASBgLUZ6lyFd-PuGx5c4hVQZN4IkQfdaAioxPMFscOZCjNpUG63vdag4OQsiKus7neUwoOQKVUJY-e-44hFg+hWQ2-6qrNbl6d4QGcIqb3HXGxjovaybx86DJJVEeblKpw2yMbO+9IPLk0bIJO7EymXlS1n-ApYTrXbcU30lZ9VuS2W", "tenant": {"description": "admin tenant", "enabled": true, "id": "c2c033b070dd402580a90e5fdc538520", "name": "admin"}}, "serviceCatalog": [{"endpoints": [{"adminURL": "http://10.58.10.119:8774/v2/c2c033b070dd402580a90e5fdc538520", "region": "RegionOne", "internalURL": "http://10.58.10.119:8774/v2/c2c033b070dd402580a90e5fdc538520", "id": "0f77f240de4c4bc181d1ec6bbb4ca723", "publicURL": "http://10.58.10.119:8774/v2/c2c033b070dd402580a90e5fdc538520"}], "endpoints_links": [], "type": "compute", "name": "nova"}, {"endpoints": [{"adminURL": "http://10.58.10.119:9696/", "region": "RegionOne", "internalURL": "http://10.58.10.119:9696/", "id": "273342a6121244ee807f9054c118558d", "publicURL": "http://10.58.10.119:9696/"}], "endpoints_links": [], "type": "network", "name": "quantum"}, {"endpoints": [{"adminURL": "http://10.58.10.119:9292", "region": "RegionOne", "internalURL": "http://10.58.10.119:9292", "id": "3de4d4ce331f4d3982a75c045cd8854c", "publicURL": "http://10.58.10.119:9292"}], "endpoints_links": [], "type": "image", "name": "glance"}, {"endpoints": [{"adminURL": "http://10.58.10.119:8776/v1/c2c033b070dd402580a90e5fdc538520", "region": "RegionOne", "internalURL": "http://10.58.10.119:8776/v1/c2c033b070dd402580a90e5fdc538520", "id": "56e6665225ad43a59bd47db1dac7a970", "publicURL": "http://10.58.10.119:8776/v1/c2c033b070dd402580a90e5fdc538520 ...
(more)
edit retag flag offensive close merge delete

1 answer

Sort by » oldest newest most voted
0

answered 2015-03-11 15:01:42 -0500

abuzzi gravatar image

updated 2015-03-20 17:13:42 -0500

smaffulli gravatar image

Following command revealed a certificate issue:

root@mic-openstack-control:/home/cisco# curl http://localhost:35357/v2.0/certificates/signing
Certificate:
            Validity
        Not Before: Mar  3 16:35:55 2014 GMT
        Not After : Mar  3 16:35:55 2015 GMT

We renewed keystone certificates with:

keystone-manage pki_setup --keystone-user keystone --keystone-group keystone
edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2015-03-09 08:18:41 -0500

Seen: 289 times

Last updated: Mar 20 '15