Ask Your Question
0

How to clean nova DB

asked 2012-05-10 21:13:12 -0500

warik gravatar image

Hi stackers,

I have a question regarding the nova DB. I have currently a cluster of 3 Openstack Essex servers (2012.1-0ubuntu2.1) on Precise. Earlier today, one of the nova node crashed (the service nova-compute from the host too), without noticed it, i tried to terminated an instance on that host. It tried to terminated it.. usually it takes 5 seconds to delete an instances but not this time...because the host was down. I connected the node, restart nova-api, nova-network and finally nova-compute. But nova-compute doesn't start because the instance is not on kvm anymore:

////////// root@nova-compute:~# virsh list --all

Id Name State

root@nova-compute:~# //////////

When i am trying to launch nova-compute here is what I get from the logs:

root@nova-compute:~# service nova-compute start

LOG file:

May 10 14:07:20 nova-compute 2012-05-10 14:07:20 WARNING nova.compute.manager [req-738f927d-ab0a-4da1-b795-6723df600067 None None] trying to reboot a non-running instance: 5ed0222e-3220-4be8-849a-d92916728ef9 (state: 8 expected: 1)

May 10 14:07:20 nova-compute 2012-05-10 14:07:20 CRITICAL nova [-] Instance instance-00000004 could not be found.#0122012-05-10 14:07:20 TRACE nova Traceback (most recent call last):#0122012-05-10 14:07:20 TRACE nova File "/usr/bin/nova-compute", line 49, in <module>#0122012-05-10 14:07:20 TRACE nova service.wait()#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 413, in wait#0122012-05-10 14:07:20 TRACE nova _launcher.wait()#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 131, in wait#0122012-05-10 14:07:20 TRACE nova service.wait()#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 166, in wait#0122012-05-10 14:07:20 TRACE nova return self._exit_event.wait()#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait#0122012-05-10 14:07:20 TRACE nova return hubs.get_hub().switch()#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 177, in switch#0122012-05-10 14:07:20 TRACE nova return self.greenlet.switch()#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 192, in main#0122012-05-10 14:07:20 TRACE nova result = function(args, *kwargs)#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 101, in run_server#0122012-05-10 14:07:20 TRACE nova server.start()#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 162, in start#0122012-05-10 14:07:20 TRACE nova self.manager.init_host()#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 247, in init_host#0122012-05-10 14:07:20 TRACE nova self.reboot_instance(context, instance['uuid'])#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 114, in wrappe

May 10 14:07:20 nova-compute kernel: [ 2745.651930] init: nova-compute main process (4788) terminated ... (more)

edit retag flag offensive close merge delete

3 answers

Sort by ยป oldest newest most voted
0

answered 2012-05-16 23:08:59 -0500

warik gravatar image

Thanks Max Schilling, that solved my question.

edit flag offensive delete link more
0

answered 2012-07-26 07:47:44 -0500

Max, your method was really helpful. Thank you.

edit flag offensive delete link more
0

answered 2012-05-16 22:16:02 -0500

mx-chilly gravatar image

I remember having a similar problem not long ago. You don't need to delete the entry from the "instance" table, just set "deleted=1" in this entry:

update instance set deleted=1 where id=XXX;

With XXX being the id of the entry you want to change.

That is just my "quick&dirty" solution, I'm not sure whether that's all nova changes in the db when it deletes an instance. For example I noticed that if you're instance had a floating ip you'll also need to modify that entry, so that the floating ip is available once more.

edit flag offensive delete link more

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2012-05-10 21:13:12 -0500

Seen: 186 times

Last updated: Jul 26 '12