Ask Your Question
0

nova list on one tenant does not work.

asked 2014-11-18 10:39:07 -0500

bgyako gravatar image

updated 2014-11-18 12:30:34 -0500

When I try nova list on one specific tenant receive below error. Nothing shows up in nova.logs , other tenants work fine and other commands work fine on all tenants including this one. Have no idea where to start.

Additional Information: I have 2 controllers that are clustered and haproxy in front of controllers. Found error on the other controllerr in nova-api.logs. Listed error below.

Using Icehouse and ubuntu 14.04

nova --debug list :

New session created for: (http://10.1.0.2:8774)
INFO (connectionpool:258) Starting new HTTP connection (1): 10.1.0.2
DEBUG (connectionpool:375) Setting read timeout to 600.0
DEBUG (connectionpool:415) "GET /v2/3bc42bbacbd341ccaaadf718f7484fe2/servers/detail HTTP/1.1" 500 128
RESP: [500] CaseInsensitiveDict({'date': 'Tue, 18 Nov 2014 16:36:55 GMT', 'content-length': '128', 'content-type': 'application/json; charset=UTF-8', 'x-compute-request-id': 'req-980a4a03-36e2-4212-b20c-8b05bda355f2'})
RESP BODY: {"computeFault": {"message": "The server has either erred or is incapable of performing the requested operation.", "code": 500}}

DEBUG (shell:777) The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-980a4a03-36e2-4212-b20c-8b05bda355f2)
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/novaclient/shell.py", line 774, in main
    OpenStackComputeShell().main(map(strutils.safe_decode, sys.argv[1:]))
  File "/usr/lib/python2.7/dist-packages/novaclient/shell.py", line 710, in main
    args.func(self.cs, args)
  File "/usr/lib/python2.7/dist-packages/novaclient/v1_1/shell.py", line 1138, in do_list
    search_opts=search_opts)
  File "/usr/lib/python2.7/dist-packages/novaclient/v1_1/servers.py", line 583, in list
    return self._list("/servers%s%s" % (detail, query_string), "servers")
  File "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 64, in _list
    _resp, body = self.api.client.get(url)
  File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 283, in get
    return self._cs_request(url, 'GET', **kwargs)
  File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 260, in _cs_request
    **kwargs)
  File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 242, in _time_request
    resp, body = self.request(url, method, **kwargs)
  File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 236, in request
    raise exceptions.from_response(resp, body, url, method)
ClientException: The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-980a4a03-36e2-4212-b20c-8b05bda355f2)
ERROR: The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-980a4a03-36e2-4212-b20c-8b05bda355f2)

Nova-api.log:

2014-11-18 13:14:27.400 12147 ERROR nova.api.openstack [req-d9bb8b91-7405-4618-8b15-0db91b8afe7b f07d2a17b4324da6a6b13d470bcc84dd 3bc42bbacbd341ccaaadf718f7484fe2] Caught error: <html><body><h1>400 Bad request</h1>
Your browser sent an invalid request.
</body></html>

2014-11-18 13:14:27.400 12147 TRACE nova.api.openstack Traceback (most recent call last):
2014-11-18 13:14:27.400 12147 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/__init__.py", line 125, in __call__
2014-11-18 13:14:27.400 12147 TRACE nova.api.openstack     return req.get_response(self.application)
2014-11-18 13:14:27.400 12147 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1320, in ...
(more)
edit retag flag offensive close merge delete

Comments

have you checked nova-api.log, please paste the content of nova-api.log file.

Syed Awais Ali gravatar imageSyed Awais Ali ( 2014-11-18 11:27:43 -0500 )edit

Added info above. Found error on second controller.

bgyako gravatar imagebgyako ( 2014-11-18 12:15:36 -0500 )edit

try running this command Nova-manage db sync for database tables being created.

Syed Awais Ali gravatar imageSyed Awais Ali ( 2014-11-18 12:21:54 -0500 )edit

that works fine, does not help with nova list. I also have HAproxy in front of the clustered controllers, wondering if something got messed up.

bgyako gravatar imagebgyako ( 2014-11-18 12:30:00 -0500 )edit

2 answers

Sort by ยป oldest newest most voted
0

answered 2014-11-18 12:37:20 -0500

bgyako gravatar image

updated 2014-11-18 12:50:47 -0500

Issue resolved:

Since all requests are going through HAproxy needed to add tune.bufsize parameter of haproxy. I set the value to 32K not sure if it should be smaller or larger will look into it.

edit flag offensive delete link more

Comments

Yes, that was the problem, since you said that you are using haproxy so that thing came into my mind !!

Syed Awais Ali gravatar imageSyed Awais Ali ( 2014-11-18 12:40:19 -0500 )edit

Not to familiar with all the haproxy settings, weird thing is it worked fine until one project added a bunch of instances. Need to get more familiar with haproxy. What is the rule of thumb on how many global maxconn and tune.bufsize be set to?

bgyako gravatar imagebgyako ( 2014-11-18 12:47:55 -0500 )edit
1

answered 2014-11-18 12:33:39 -0500

exactly, I guess you need to increase tune.bufsize parameter of haproxy to 32k, and then see what it does.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2014-11-18 10:39:07 -0500

Seen: 381 times

Last updated: Nov 18 '14