Ask Your Question

bdastur-s's profile - activity

2016-04-08 11:28:39 -0500 received badge  Famous Question (source)
2015-07-27 03:46:45 -0500 received badge  Notable Question (source)
2015-07-20 12:26:29 -0500 received badge  Popular Question (source)
2015-07-14 10:59:44 -0500 asked a question ImportError: No module name middleware

I am seeing the below traceback from nova-api and failure in Juno release. I searched for similar threads, and suggestion was a missing python module python-keystonemiddleware, but I have that module installed

[root@]# rpm -qa | grep key keyutils-libs-1.5.8-3.el7.x86_64 python-keystonemiddleware-1.2.0-2.el7ost.noarch gpg-pubkey-fd431d51-4ae0493b python-keystoneclient-0.11.1-1.el7ost.noarch gpg-pubkey-2fa658e0-45700c69 python-keyring-4.0-1.el7ost.noarch

Any suggestions on how to resolve this would be appreciated.

2015-07-14 08:51:19.914 1 CRITICAL nova [-] ImportError: No module named middleware
2015-07-14 08:51:19.914 1 TRACE nova Traceback (most recent call last):
2015-07-14 08:51:19.914 1 TRACE nova   File "/usr/bin/nova-api", line 10, in <module>
2015-07-14 08:51:19.914 1 TRACE nova     sys.exit(main())
2015-07-14 08:51:19.914 1 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/cmd/api.py", line 55, in main 
2015-07-14 08:51:19.914 1 TRACE nova     server = service.WSGIService(api, use_ssl=should_use_ssl)
2015-07-14 08:51:19.914 1 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/service.py", line 333, in __init__
2015-07-14 08:51:19.914 1 TRACE nova     self.app = self.loader.load_app(name)
2015-07-14 08:51:19.914 1 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/wsgi.py", line 521, in   load_app
2015-07-14 08:51:19.914 1 TRACE nova     return deploy.loadapp("config:%s" % self.config_path, name=name)
2015-07-14 08:51:19.914 1 TRACE nova   File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
2015-07-14 08:51:19.914 1 TRACE nova     return loadobj(APP, uri, name=name, **kw)
2015-07-14 08:51:19.914 1 TRACE nova   File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 272, in loadobj
2015-07-14 08:51:19.914 1 TRACE nova     return context.create()
2015-07-14 08:51:19.914 1 TRACE nova   File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 710, in create
2 015-07-14 08:51:19.914 1 TRACE nova     return self.object_type.invoke(self)
2015-07-14 08:51:19.914 1 TRACE nova   File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 144, in invoke
2015-07-14 08:51:19.914 1 TRACE nova     **context.local_conf)
2015-07-14 08:51:19.914 1 TRACE nova   File "/usr/lib/python2.7/site-packages/paste/deploy/util.py", line 56, in fix_call
2015-07-14 08:51:19.914 1 TRACE nova     val = callable(*args, **kw)
2015-07-14 08:51:19.914 1 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/api/openstack/urlmap.py", line 159, in urlmap_factory
2015-07-14 08:51:19.914 1 TRACE nova     app = loader.get_app(app_name, global_conf=global_conf)
2015-07-14 08:51:19.914 1 TRACE nova   File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 350, in get_app
2015-07-14 08:51:19.914 1 TRACE nova     name=name, global_conf=global_conf).create()
2015-07-14 08:51:19.914 1 TRACE nova   File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 710, in create
2015-07-14 08:51:19 ...
(more)
2014-12-08 00:22:48 -0500 received badge  Famous Question (source)
2014-11-13 04:24:14 -0500 received badge  Notable Question (source)
2014-10-29 21:04:18 -0500 answered a question Unable to ping the tenant router gateway from external network

Can you give more details on what troubleshooting you already did.

Can you see if you can reach out from your tenant router namespace.

ip netns will give you the qrouter and qdhcp namespaces. Now you can execute any command from within that namespace: eg ip netns exec <qrouter-namespace> ping <gw ip="">

2014-10-27 08:36:30 -0500 received badge  Popular Question (source)
2014-10-27 04:17:02 -0500 commented question nova scheduler scheduling instance on invalid host

In this case, someone accidentally re-imaged the compute node.

2014-10-27 03:10:33 -0500 answered a question how can i trace a request from Horizon to Nova

I have used pdb (Python Debugger) to trace similar code path. Here is what I would do (considering this is a test system, where you can start services in debugger):

  1. See what nova processes are running on your controller node. "systemctl | grep nova"
  2. From nova-api.log, I grep'ed certain keywords to find the function in compute/api.py.
  3. Add breakpoints in the code "/usr/lib/python2.7/site-packages/nova/<your path="" file="">", using import pdb; pdb.set_trace()
  4. Start the process manually: eg: "systemctl stop openstack-nova-api.service"; /usr/bin/python /usr/bin/nova-api

Once you hit the break point you will be in pdb mode, it is very convenient to step through the code. You can do anything here, get backtrace, check variables, arguments and a lot more.

2014-10-27 02:05:49 -0500 received badge  Editor (source)
2014-10-26 19:23:18 -0500 asked a question nova scheduler scheduling instance on invalid host

I have an openstack setup with 1 controller and 2 computes. One of the compute nodes is "out-of-service", meaning there is not nova-compute running on the host. The host is still up and accessible.

Now, I have an automated script which exercises the stack by trying to create instances on the hosts. It uses the python nova client APIs to get the available list of of hosts, which still returns the bad compute, and when trying to create an instance on the compute, it hangs.

I tried following the nova-scheduler and nova-api code paths, and I see that in the scheduler, the filter_properties are set with forced_host set to the bad compute node, which is used to start the instance.

I have verified that all my filters on the controller are turned on, and I can see that it is able to get the host details like available RAM, disk etc, but the compute is not running on the host.

From my client script, I have no way of figuring out if the host is bad or not. So what is the correct behavior here. The current behavior is definitely not correct, as there is no indication of error except for going to the controller and going through the logs.

Ideally I would like to see that in the hypervisor list, the bad host is not shown up so I don't pick it. Also is there a way manually to remove a compute from the stack in this case.

regards, Behzad