Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Nova services are reinstalled still it showing the same problem... check the end to end logs while launching new instance. Database was resynced even.

2014-05-23 12:18:05.880 25578 ERROR nova.openstack.common.rpc.amqp [req-3a852934-7d2b-418b-b177-345ff85b7e18 bcf3a6be7a574127b217437c8ed614b1 84acd57e249e4bdcb5e8000e4fa1b818] Exception during message handling 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last): 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 461, in _process_data 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp *args) 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp result = getattr(proxyobj, method)(ctxt, *kwargs) 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/exception.py", line 90, in wrapped 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp payload) 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/exception.py", line 73, in wrapped 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp return f(self, context, args, *kw) 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 243, in decorated_function 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp pass 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 229, in decorated_function 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp return function(self, context, args, *kwargs) 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 294, in decorated_function 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp function(self, context, args, *kwargs) 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 271, in decorated_function 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp e, sys.exc_info()) 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 258, in decorated_function 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp return function(self, context, args, *kwargs) 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1630, in run_instance 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp do_run_instance() 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/openstack/common/lockutils.py", line 246, in inner 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp return f(args, *kwargs) 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1629, in do_run_instance 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp legacy_bdm_in_spec) 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 968, in _run_instance 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp notify("error", msg=unicode(e)) # notify that build failed 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 943, in _run_instance 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp image_meta = self._prebuild_instance(context, instance) 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 971, in _prebuild_instance 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp self._check_instance_exists(context, instance) 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1209, in _check_instance_exists 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp raise exception.InstanceExists(name=instance['name']) 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp InstanceExists: Instance instance-00000002 already exists. 2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp 2014-05-23 12:18:28.855 25578 ERROR nova.virt.libvirt.driver [-] Getting disk size of instance-00000002: [Errno 2] No such file or directory: '/var/lib/nova/instances/ded56ad8-e5a2-46a6-b62e-c6519d6af402/disk' 2014-05-23 12:19:30.565 25578 ERROR nova.virt.libvirt.driver [-] Getting disk size of instance-00000002: [Errno 2] No such file or directory: '/var/lib/nova/instances/ded56ad8-e5a2-46a6-b62e-c6519d6af402/disk' 2014-05-23 12:20:32.255 25578 ERROR nova.virt.libvirt.driver [-] Getting disk size of instance-00000002: [Errno 2] No such file or directory: '/var/lib/nova/instances/ded56ad8-e5a2-46a6-b62e-c6519d6af402/disk' 2014-05-23 12:21:33.983 25578 ERROR nova.virt.libvirt.driver [-] Getting disk size of instance-00000002: [Errno 2] No such file or directory: '/var/lib/nova/instances/ded56ad8-e5a2-46a6-b62e-c6519d6af402/disk' 2014-05-23 12:22:20.331 25578 ERROR nova.virt.libvirt.driver [-] Getting disk size of instance-00000002: [Errno 2] No such file or directory: '/var/lib/nova/instances/ded56ad8-e5a2-46a6-b62e-c6519d6af402/disk' 2014-05-23 12:22:28.801 25578 ERROR nova.virt.libvirt.driver [-] Getting disk size of instance-00000002: [Errno 2] No such file or directory: '/var/lib/nova/instances/ded56ad8-e5a2-46a6-b62e-c6519d6af402/disk'

2014-05-23 12:23:08.108 25578 ERROR nova.virt.libvirt.driver [-] Getting disk size of instance-00000002: [Errno 2] No such file or directory: '/var/lib/nova/instances/ded56ad8-e5a2-46a6-b62e-c6519d6af402/disk'

click to hide/show revision 2
No.2 Revision

Nova services are reinstalled still it showing the same problem... check the end to end logs while launching new instance. Database was resynced even.

==================================================================================================
2014-05-23 12:18:05.880 25578 ERROR nova.openstack.common.rpc.amqp [req-3a852934-7d2b-418b-b177-345ff85b7e18 bcf3a6be7a574127b217437c8ed614b1 84acd57e249e4bdcb5e8000e4fa1b818] Exception during message handling
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 461, in _process_data
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp     *args)
**args)
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp     result = getattr(proxyobj, method)(ctxt, *kwargs)
**kwargs)
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/exception.py", line 90, in wrapped
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp     payload)
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/exception.py", line 73, in wrapped
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp     return f(self, context, args, *kw)
*args, **kw)
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 243, in decorated_function
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp     pass
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 229, in decorated_function
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp     return function(self, context, args, *kwargs)
*args, **kwargs)
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 294, in decorated_function
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp     function(self, context, args, *kwargs)
*args, **kwargs)
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 271, in decorated_function
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp     e, sys.exc_info())
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 258, in decorated_function
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp     return function(self, context, args, *kwargs)
*args, **kwargs)
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1630, in run_instance
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp     do_run_instance()
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/openstack/common/lockutils.py", line 246, in inner
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp     return f(args, *kwargs)
f(*args, **kwargs)
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1629, in do_run_instance
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp     legacy_bdm_in_spec)
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 968, in _run_instance
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp     notify("error", msg=unicode(e))  # notify that build failed
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 943, in _run_instance
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp     image_meta = self._prebuild_instance(context, instance)
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 971, in _prebuild_instance
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp     self._check_instance_exists(context, instance)
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1209, in _check_instance_exists
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp     raise exception.InstanceExists(name=instance['name'])
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp InstanceExists: Instance instance-00000002 already exists.
2014-05-23 12:18:05.880 25578 TRACE nova.openstack.common.rpc.amqp 
2014-05-23 12:18:28.855 25578 ERROR nova.virt.libvirt.driver [-] Getting disk size of instance-00000002: [Errno 2] No such file or directory: '/var/lib/nova/instances/ded56ad8-e5a2-46a6-b62e-c6519d6af402/disk'
2014-05-23 12:19:30.565 25578 ERROR nova.virt.libvirt.driver [-] Getting disk size of instance-00000002: [Errno 2] No such file or directory: '/var/lib/nova/instances/ded56ad8-e5a2-46a6-b62e-c6519d6af402/disk'
2014-05-23 12:20:32.255 25578 ERROR nova.virt.libvirt.driver [-] Getting disk size of instance-00000002: [Errno 2] No such file or directory: '/var/lib/nova/instances/ded56ad8-e5a2-46a6-b62e-c6519d6af402/disk'
2014-05-23 12:21:33.983 25578 ERROR nova.virt.libvirt.driver [-] Getting disk size of instance-00000002: [Errno 2] No such file or directory: '/var/lib/nova/instances/ded56ad8-e5a2-46a6-b62e-c6519d6af402/disk'
2014-05-23 12:22:20.331 25578 ERROR nova.virt.libvirt.driver [-] Getting disk size of instance-00000002: [Errno 2] No such file or directory: '/var/lib/nova/instances/ded56ad8-e5a2-46a6-b62e-c6519d6af402/disk'
2014-05-23 12:22:28.801 25578 ERROR nova.virt.libvirt.driver [-] Getting disk size of instance-00000002: [Errno 2] No such file or directory: '/var/lib/nova/instances/ded56ad8-e5a2-46a6-b62e-c6519d6af402/disk'

'/var/lib/nova/instances/ded56ad8-e5a2-46a6-b62e-c6519d6af402/disk' 2014-05-23 12:23:08.108 25578 ERROR nova.virt.libvirt.driver [-] Getting disk size of instance-00000002: [Errno 2] No such file or directory: '/var/lib/nova/instances/ded56ad8-e5a2-46a6-b62e-c6519d6af402/disk'

'/var/lib/nova/instances/ded56ad8-e5a2-46a6-b62e-c6519d6af402/disk' ================================================================================================