Error to rebuild an instance with volume attached (in Juno)
In the current environment, it is ok to rebuild an instance without any attached cinder volume. While, when there is a volume attached, either boot volume or data volume, error always occurs when an instance is to be rebuilt. Related log shows it failed to destroy that instance.
2015-12-24 18:25:42.633 26486 ERROR oslo.messaging.rpc.dispatcher [req-67fec45f-c7ba-4c4f-9f4c-0a9875d94069 ] Exception during message handling: Failed to terminate process 36082 with SIGKILL: Device or resource busy
2015-12-24 18:25:42.633 26486 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
2015-12-24 18:25:42.633 26486 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply
2015-12-24 18:25:42.633 26486 TRACE oslo.messaging.rpc.dispatcher incoming.message))
2015-12-24 18:25:42.633 26486 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch
2015-12-24 18:25:42.633 26486 TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args)
2015-12-24 18:25:42.633 26486 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch
2015-12-24 18:25:42.633 26486 TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args)
2015-12-24 18:25:42.633 26486 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 414, in decorated_function
2015-12-24 18:25:42.633 26486 TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2015-12-24 18:25:42.633 26486 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/server.py", line 139, in inner
2015-12-24 18:25:42.633 26486 TRACE oslo.messaging.rpc.dispatcher return func(*args, **kwargs)
2015-12-24 18:25:42.633 26486 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/exception.py", line 88, in wrapped
2015-12-24 18:25:42.633 26486 TRACE oslo.messaging.rpc.dispatcher payload)
2015-12-24 18:25:42.633 26486 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line 82, in __exit__
2015-12-24 18:25:42.633 26486 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-12-24 18:25:42.633 26486 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/exception.py", line 71, in wrapped
2015-12-24 18:25:42.633 26486 TRACE oslo.messaging.rpc.dispatcher return f(self, context, *args, **kw)
2015-12-24 18:25:42.633 26486 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 298, in decorated_function
2015-12-24 18:25:42.633 26486 TRACE oslo.messaging.rpc.dispatcher pass
2015-12-24 18:25:42.633 26486 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line 82, in __exit__
2015-12-24 18:25:42.633 26486 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_ ...
May I ask which backend of cinder are you using? lvm,emc,3par or else?
Thanks for your attention. It's LVM.
Have you tried to destroy the instance with cinder volume attached by virsh commandline itself?
It seems not an OpenStack error but an error came from libvirt.
I'm not sure... try "virsh destroy instance_name" on your compute node and then let's see if it can give some further information.
Thanks for your interest and information. I will post information about fix in Openstack.