Ask Your Question
0

Icehouse + RedHat Deployment Bug?

asked 2014-04-29 15:30:07 -0500

jkary gravatar image

Hello,

I'm wondering if I've hit a bug or have a configuration error on my 3 node deployment. All services start up and the message queue appears to be working. When I launch an image I get the following:

2014-04-29 16:16:49.723 2252 ERROR nova.api.openstack [req-a5503226-ad62-47fd-9ce2-9bd0b3c79d54 ca8cf7879f304f328420a023aa47d821 cc55e4c69e404fc4b89e2983a36efd80] Caught error: Timed out waiting for a reply to message ID 1c8a38a6a601410981cc630ddefa6346
2014-04-29 16:16:49.723 2252 TRACE nova.api.openstack Traceback (most recent call last):
2014-04-29 16:16:49.723 2252 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/api/openstack/__init__.py", line 125, in __call__
2014-04-29 16:16:49.723 2252 TRACE nova.api.openstack     return req.get_response(self.application)
2014-04-29 16:16:49.723 2252 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/webob/request.py", line 1296, in send
2014-04-29 16:16:49.723 2252 TRACE nova.api.openstack     application, catch_exc_info=False)
2014-04-29 16:16:49.723 2252 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/webob/request.py", line 1260, in call_application
2014-04-29 16:16:49.723 2252 TRACE nova.api.openstack     app_iter = application(self.environ, start_response)
2014-04-29 16:16:49.723 2252 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/webob/dec.py", line 144, in __call__
2014-04-29 16:16:49.723 2252 TRACE nova.api.openstack     return resp(environ, start_response)
2014-04-29 16:16:49.723 2252 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", line 615, in __call__
2014-04-29 16:16:49.723 2252 TRACE nova.api.openstack     return self.app(env, start_response)
2014-04-29 16:16:49.723 2252 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/webob/dec.py", line 144, in __call__
2014-04-29 16:16:49.723 2252 TRACE nova.api.openstack     return resp(environ, start_response)
2014-04-29 16:16:49.723 2252 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/webob/dec.py", line 144, in __call__
2014-04-29 16:16:49.723 2252 TRACE nova.api.openstack     return resp(environ, start_response)
2014-04-29 16:16:49.723 2252 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/Routes-1.12.3-py2.6.egg/routes/middleware.py", line 131, in __call__
2014-04-29 16:16:49.723 2252 TRACE nova.api.openstack     response = self.app(environ, start_response)
2014-04-29 16:16:49.723 2252 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/webob/dec.py", line 144, in __call__
2014-04-29 16:16:49.723 2252 TRACE nova.api.openstack     return resp(environ, start_response)
2014-04-29 16:16:49.723 2252 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/webob/dec.py", line 130, in __call__
2014-04-29 16:16:49.723 2252 TRACE nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
2014-04-29 16:16:49.723 2252 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/webob/dec.py", line 195, in call_func
2014-04-29 16:16:49.723 2252 TRACE nova.api.openstack     return self ...
(more)
edit retag flag offensive close merge delete
add a comment

2 answers

Sort by ยป oldest newest most voted
0

answered 2014-04-30 08:54:02 -0500

jkary gravatar image

Turning on DEBUG reveals some more information. Can anyone tell me if this is the result of the compute node not being registered properly?

2014-04-30 09:45:06.403 2449 INFO nova.api.openstack [req-98a87c05-5b69-4a78-975c-a1870db5979b ca8cf7879f304f328420a023aa47d821 cc55e4c69e404fc4b89e2983a36efd80] http://nmtg-ctrl001:8774/v2/cc55e4c69e404fc4b89e2983a36efd80/servers returned with HTTP 500
2014-04-30 09:45:06.404 2449 DEBUG nova.api.openstack.wsgi [req-98a87c05-5b69-4a78-975c-a1870db5979b ca8cf7879f304f328420a023aa47d821 cc55e4c69e404fc4b89e2983a36efd80] Returning 500 to user: The server has either erred or is incapable of performing the requested operation. __call__ /usr/lib/python2.6/site-packages/nova/api/openstack/wsgi.py:1215
2014-04-30 09:45:06.404 2449 INFO nova.osapi_compute.wsgi.server [req-98a87c05-5b69-4a78-975c-a1870db5979b ca8cf7879f304f328420a023aa47d821 cc55e4c69e404fc4b89e2983a36efd80] 10.81.76.105 "POST /v2/cc55e4c69e404fc4b89e2983a36efd80/servers HTTP/1.1" status: 500 len: 335 time: 60.1515419
2014-04-30 09:45:06.462 2449 DEBUG keystoneclient.middleware.auth_token [-] Authenticating user token __call__ /usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py:602
2014-04-30 09:45:06.462 2449 DEBUG keystoneclient.middleware.auth_token [-] Removing headers from request environment: X-Identity-Status,X-Domain-Id,X-Domain-Name,X-Project-Id,X-Project-Name,X-Project-Domain-Id,X-Project-Domain-Name,X-User-Id,X-User-Name,X-User-Domain-Id,X-User-Domain-Name,X-Roles,X-Service-Catalog,X-User,X-Tenant-Id,X-Tenant-Name,X-Tenant,X-Role _remove_auth_headers /usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py:661
2014-04-30 09:45:06.463 2449 DEBUG keystoneclient.middleware.auth_token [-] Returning cached token _cache_get /usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py:1010
2014-04-30 09:45:06.463 2449 DEBUG keystoneclient.middleware.auth_token [-] Received request from user: ca8cf7879f304f328420a023aa47d821 with project_id : cc55e4c69e404fc4b89e2983a36efd80 and roles: admin,_member_  _build_user_headers /usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py:909
2014-04-30 09:45:06.465 2449 DEBUG routes.middleware [-] Matched GET /cc55e4c69e404fc4b89e2983a36efd80/servers/detail __call__ /usr/lib/python2.6/site-packages/Routes-1.12.3-py2.6.egg/routes/middleware.py:100
2014-04-30 09:45:06.465 2449 DEBUG routes.middleware [-] Route path: '/{project_id}/servers/detail', defaults: {'action': u'detail', 'controller': <nova.api.openstack.wsgi.Resource object at 0x4471610>} __call__ /usr/lib/python2.6/site-packages/Routes-1.12.3-py2.6.egg/routes/middleware.py:102
2014-04-30 09:45:06.465 2449 DEBUG routes.middleware [-] Match dict: {'action': u'detail', 'controller': <nova.api.openstack.wsgi.Resource object at 0x4471610>, 'project_id': u'cc55e4c69e404fc4b89e2983a36efd80'} __call__ /usr/lib/python2.6/site-packages/Routes-1.12.3-py2.6.egg/routes/middleware.py:103
2014-04-30 09:45:06.466 2449 DEBUG nova.api.openstack.wsgi [req-8cad30be-8cbd-4dd6-8439-3b87ba41066e ca8cf7879f304f328420a023aa47d821 cc55e4c69e404fc4b89e2983a36efd80] Calling method '<bound method Controller.detail of <nova.api.openstack.compute.servers.Controller object at 0x446da10>>' (Content-type='None', Accept='application/json') _process_stack /usr/lib/python2.6/site-packages/nova/api/openstack/wsgi.py:945
2014-04-30 09:45:06.466 2449 DEBUG nova.compute.api [req-8cad30be-8cbd-4dd6-8439-3b87ba41066e ca8cf7879f304f328420a023aa47d821 cc55e4c69e404fc4b89e2983a36efd80] Searching by: {'deleted': False, u'project_id': u'cc55e4c69e404fc4b89e2983a36efd80', u'limit': u'21'} get_all /usr/lib/python2.6/site-packages/nova/compute/api.py:1839
2014-04-30 09:45:06.489 2449 INFO nova.osapi_compute.wsgi.server [req-8cad30be-8cbd-4dd6-8439-3b87ba41066e ca8cf7879f304f328420a023aa47d821 cc55e4c69e404fc4b89e2983a36efd80] 10.81.76.105 "GET /v2/cc55e4c69e404fc4b89e2983a36efd80/servers/detail?limit=21&project_id=cc55e4c69e404fc4b89e2983a36efd80 HTTP/1.1" status: 200 len: 187 time: 0.0271010
2014-04-30 09:45:06.628 2449 DEBUG keystoneclient.middleware.auth_token [-] Authenticating user token __call__ /usr/lib/python2.6/site-packages/keystoneclient/middleware ...
(more)
edit flag offensive delete link more
add a comment
0

answered 2014-04-29 23:21:35 -0500

Vikrant Pawar gravatar image

updated 2014-04-29 23:21:52 -0500

Hi,

Did you updated from havana ?

I faces similar issue, it was related to qpid_topology_version, by default some were using 1 while others were using 2.

~Regards, Vikrant Pawar

edit flag offensive delete link more

Comments

I installed this from scratch using the RedHat Deployment document located on the http://OpenStack.org site.

I checked my topology versions and found the compute and controller node were both running ver. 1 while the network node did not have a version property found in the neutron configuration files.

jkary gravatar imagejkary ( 2014-04-30 07:46:20 -0500 )edit

That's correct topology is mentioned only in controller and compute node.

To check whether compute is registered or not you could issue nova hypervisor-list from controller.

Did you confirm that you do not have any error message in compute side.

Also check if any service is not dead on either server.

Vikrant Pawar gravatar imageVikrant Pawar ( 2014-05-02 00:33:18 -0500 )edit
add a comment

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

subscribe to rss feed

Stats

Asked: 2014-04-29 15:30:07 -0500

Seen: 409 times

Last updated: Apr 30 '14

Related questions

How exactly do swift-bench timeouts work?

Swift Replicator Timeouts

What causes the meta-data service timing out on some nodes?

Unable to delete instances in 'ERROR' state

lbaas ssh timeout

Cinder RabbitMQ timeout problem with multiple rabbit hosts

can't ssh to new created instances any more

FreeBSD Guest VM DHCP Timeout

Juno MessagingTimeout launching an instance

Tacker Installation Failure