Ask Your Question
0

How to clean nova DB

asked 2012-05-10 21:13:12 -0500

warik gravatar image

Hi stackers,

I have a question regarding the nova DB. I have currently a cluster of 3 Openstack Essex servers (2012.1-0ubuntu2.1) on Precise. Earlier today, one of the nova node crashed (the service nova-compute from the host too), without noticed it, i tried to terminated an instance on that host. It tried to terminated it.. usually it takes 5 seconds to delete an instances but not this time...because the host was down. I connected the node, restart nova-api, nova-network and finally nova-compute. But nova-compute doesn't start because the instance is not on kvm anymore:

////////// root@nova-compute:~# virsh list --all

Id Name State

root@nova-compute:~# //////////

When i am trying to launch nova-compute here is what I get from the logs:

root@nova-compute:~# service nova-compute start

LOG file:

May 10 14:07:20 nova-compute 2012-05-10 14:07:20 WARNING nova.compute.manager [req-738f927d-ab0a-4da1-b795-6723df600067 None None] trying to reboot a non-running instance: 5ed0222e-3220-4be8-849a-d92916728ef9 (state: 8 expected: 1)

May 10 14:07:20 nova-compute 2012-05-10 14:07:20 CRITICAL nova [-] Instance instance-00000004 could not be found.#0122012-05-10 14:07:20 TRACE nova Traceback (most recent call last):#0122012-05-10 14:07:20 TRACE nova File "/usr/bin/nova-compute", line 49, in <module>#0122012-05-10 14:07:20 TRACE nova service.wait()#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 413, in wait#0122012-05-10 14:07:20 TRACE nova _launcher.wait()#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 131, in wait#0122012-05-10 14:07:20 TRACE nova service.wait()#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 166, in wait#0122012-05-10 14:07:20 TRACE nova return self._exit_event.wait()#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait#0122012-05-10 14:07:20 TRACE nova return hubs.get_hub().switch()#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 177, in switch#0122012-05-10 14:07:20 TRACE nova return self.greenlet.switch()#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 192, in main#0122012-05-10 14:07:20 TRACE nova result = function(args, *kwargs)#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 101, in run_server#0122012-05-10 14:07:20 TRACE nova server.start()#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 162, in start#0122012-05-10 14:07:20 TRACE nova self.manager.init_host()#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 247, in init_host#0122012-05-10 14:07:20 TRACE nova self.reboot_instance(context, instance['uuid'])#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 114, in wrappe

May 10 14:07:20 nova-compute kernel: [ 2745.651930] init: nova-compute main process (4788) terminated ... (more)

edit retag flag offensive close merge delete

3 answers

Sort by ยป oldest newest most voted
0

answered 2012-05-16 23:08:59 -0500

warik gravatar image

Thanks Max Schilling, that solved my question.

edit flag offensive delete link more
0

answered 2012-07-26 07:47:44 -0500

Max, your method was really helpful. Thank you.

edit flag offensive delete link more
0

answered 2012-05-16 22:16:02 -0500

mx-chilly gravatar image

I remember having a similar problem not long ago. You don't need to delete the entry from the "instance" table, just set "deleted=1" in this entry:

update instance set deleted=1 where id=XXX;

With XXX being the id of the entry you want to change.

That is just my "quick&dirty" solution, I'm not sure whether that's all nova changes in the db when it deletes an instance. For example I noticed that if you're instance had a floating ip you'll also need to modify that entry, so that the floating ip is available once more.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2012-05-10 21:13:12 -0500

Seen: 94 times

Last updated: Jul 26 '12