Ask Your Question
0

Instances will not boot

asked 2013-12-18 13:43:39 -0600

OldParrothead gravatar image

I had to move a rack to a new IP structure and now instances will not boot. I terminated all old instances as they were useless and test anyway. Tried to boot new instances and nova keeps telling me it cannot find the file for the instance? The file gets created, the instance get all the information, and it takes 10-15 minutes to fail. Have checked all IP multiple times, including keystone endpoints. This is what I get:

2013-12-18 14:16:27.286 2517 ERROR nova.virt.libvirt.driver [-] Getting disk size of instance-0000000f: [Errno 2] No such file or directory: '/var/lib/nova/instances/420926e7-f1f8-4d82-ad93-29de9aca7f77/disk'
2013-12-18 14:17:33.097 2517 ERROR nova.virt.libvirt.driver [-] Getting disk size of instance-0000000f: [Errno 2] No such file or directory: '/var/lib/nova/instances/420926e7-f1f8-4d82-ad93-29de9aca7f77/disk'
2013-12-18 14:18:38.888 2517 ERROR nova.virt.libvirt.driver [-] Getting disk size of instance-0000000f: [Errno 2] No such file or directory: '/var/lib/nova/instances/420926e7-f1f8-4d82-ad93-29de9aca7f77/disk'
2013-12-18 14:19:44.651 2517 ERROR nova.virt.libvirt.driver [-] Getting disk size of instance-0000000f: [Errno 2] No such file or directory: '/var/lib/nova/instances/420926e7-f1f8-4d82-ad93-29de9aca7f77/disk'
2013-12-18 14:20:50.443 2517 ERROR nova.virt.libvirt.driver [-] Getting disk size of instance-0000000f: [Errno 2] No such file or directory: '/var/lib/nova/instances/420926e7-f1f8-4d82-ad93-29de9aca7f77/disk'
2013-12-18 14:21:56.320 2517 ERROR nova.virt.libvirt.driver [-] Getting disk size of instance-0000000f: [Errno 2] No such file or directory: '/var/lib/nova/instances/420926e7-f1f8-4d82-ad93-29de9aca7f77/disk'

Anybody seen this one before??

Thanks

edit retag flag offensive close merge delete

Comments

I'm not clear: are you saying that /var/lib/nova/instances/420926e7-f1f8-4d82-ad93-29de9aca7f77/disk actually exists on the filesystem? If so, it it (and everything under /var/lib/nova) owned by the "nova" user?

larsks gravatar imagelarsks ( 2013-12-18 15:12:13 -0600 )edit

That file did exist, I am sure after researching other places that one may have been a UUID conflict. So, I tried again and get a more fruitful result...still fails, but more informative errors (to Python folks anyway).. 2013-12-19 11:43:09.980 ERROR nova.compute.manager [req-68352c27-536d-41ac-95ba-496cf199cf26 7f034b093c9540da9887ec2f147108f3 f70b99e9536a472d881470355164e22b] [instanc e: 8f36f1c6-288d-459c-b0ca-74e498561a90] Instance failed to spawn 2013-12-19 11:43:09.980 1525 TRACE nova.compute.manager [instance: 8f36f1c6-288d-459c-b0ca-74e498561a90] Traceback (most recent call last): 2013-12-19 11:43:09.980 1525 TRACE nova.compute.manager [instance: 8f36f1c6-288d-459c-b0ca-74e498561a90] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.p y", line 1119, in _spawn 2013-12-19 11:43:09.980 1525 TRACE nova.compute.manager [instance: 8f36f1c6-288d-459c-b0ca-74e498561a90] block_device_info) 2013-12-19 11:43:09.980 1525 TRACE nova.compute.manager [instance: 8f36f1c6-288

OldParrothead gravatar imageOldParrothead ( 2013-12-19 11:00:34 -0600 )edit

Is `libvirtd` running on your compute node (can you run "virsh list" on the compute node without errors)? Are you certain that your compute node has sufficient memory for what you're trying to do? Lack of memory can often manifest in strange ways with unhelpful error messages.

larsks gravatar imagelarsks ( 2013-12-19 11:26:10 -0600 )edit
add a comment

1 answer

Sort by ยป oldest newest most voted
0

answered 2013-12-19 11:19:32 -0600

OldParrothead gravatar image

This is not an answer, but more information and the code is more readable here then in comments:

2013-12-19 11:43:09.980 ERROR nova.compute.manager [req-68352c27-536d-41ac-95ba-496cf199cf26 7f034b093c9540da9887ec2f147108f3 f70b99e9536a472d881470355164e22b] [instanc
e: 8f36f1c6-288d-459c-b0ca-74e498561a90] Instance failed to spawn
2013-12-19 11:43:09.980 1525 TRACE nova.compute.manager [instance: 8f36f1c6-288d-459c-b0ca-74e498561a90] Traceback (most recent call last):
2013-12-19 11:43:09.980 1525 TRACE nova.compute.manager [instance: 8f36f1c6-288d-459c-b0ca-74e498561a90]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.p
y", line 1119, in _spawn
2013-12-19 11:43:09.980 1525 TRACE nova.compute.manager [instance: 8f36f1c6-288d-459c-b0ca-74e498561a90]     block_device_info)
2013-12-19 11:43:09.980 1525 TRACE nova.compute.manager [instance: 8f36f1c6-288d-459c-b0ca-74e498561a90]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driv
er.py", line 1531, in spawn
2013-12-19 11:43:09.980 1525 TRACE nova.compute.manager [instance: 8f36f1c6-288d-459c-b0ca-74e498561a90]     block_device_info)
2013-12-19 11:43:09.980 1525 TRACE nova.compute.manager [instance: 8f36f1c6-288d-459c-b0ca-74e498561a90]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driv
er.py", line 2447, in _create_domain_and_network
2013-12-19 11:43:09.980 1525 TRACE nova.compute.manager [instance: 8f36f1c6-288d-459c-b0ca-74e498561a90]     domain = self._create_domain(xml, instance=instance)
2013-12-19 11:43:09.980 1525 TRACE nova.compute.manager [instance: 8f36f1c6-288d-459c-b0ca-74e498561a90]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driv
er.py", line 2408, in _create_domain
2013-12-19 11:43:09.980 1525 TRACE nova.compute.manager [instance: 8f36f1c6-288d-459c-b0ca-74e498561a90]     domain.createWithFlags(launch_flags)
2013-12-19 11:43:09.980 1525 TRACE nova.compute.manager [instance: 8f36f1c6-288d-459c-b0ca-74e498561a90]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", li
ne 187, in doit
2013-12-19 11:43:09.980 1525 TRACE nova.compute.manager [instance: 8f36f1c6-288d-459c-b0ca-74e498561a90]     result = proxy_call(self._autowrap, f, *args, **kwargs)
2013-12-19 11:43:09.980 1525 TRACE nova.compute.manager [instance: 8f36f1c6-288d-459c-b0ca-74e498561a90]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", li
ne 147, in proxy_call
2013-12-19 11:43:09.980 1525 TRACE nova.compute.manager [instance: 8f36f1c6-288d-459c-b0ca-74e498561a90]     rv = execute(f,*args,**kwargs)
2013-12-19 11:43:09.980 1525 TRACE nova.compute.manager [instance: 8f36f1c6-288d-459c-b0ca-74e498561a90]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", li
ne 76, in tworker
2013-12-19 11:43:09.980 1525 TRACE nova.compute.manager [instance: 8f36f1c6-288d-459c-b0ca-74e498561a90]     rv = meth(*args,**kwargs)
2013-12-19 11:43:09.980 1525 TRACE nova.compute.manager [instance: 8f36f1c6-288d-459c-b0ca-74e498561a90]   File "/usr/lib/python2.7/dist-packages/libvirt.py", line 711,
 in createWithFlags
2013-12-19 11:43:09.980 1525 TRACE nova.compute.manager [instance: 8f36f1c6-288d-459c-b0ca-74e498561a90]     if ret == -1: raise libvirtError ('virDomainCreateWithFlags
() failed', dom=self)
2013-12-19 11:43:09.980 1525 TRACE nova.compute.manager [instance: 8f36f1c6-288d-459c-b0ca-74e498561a90] libvirtError: Unable to read from monitor: Connection reset by
peer
2013-12-19 11:43:09.980 1525 TRACE nova.compute.manager [instance: 8f36f1c6-288d-459c-b0ca-74e498561a90]
2013-12-19 11:43:12.030 ERROR nova.compute.manager [req-68352c27-536d-41ac-95ba-496cf199cf26 7f034b093c9540da9887ec2f147108f3 f70b99e9536a472d881470355164e22b] [instanc
e: 8f36f1c6-288d-459c-b0ca-74e498561a90] Error: ['Traceback (most recent call last):\n', '  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 864, i
n _run_instance\n    set_access_ip=set_access_ip)\n', '  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1123, in _spawn\n    LOG.exception(_(\'In
stance failed to spawn\'), instance=instance)\n ...
(more)
edit flag offensive delete link more

Comments

In general it's a better idea to update your question rather than to post updates like this as an answer.

larsks gravatar imagelarsks ( 2013-12-19 11:23:53 -0600 )edit

Thanks, and I will do that from now on.

OldParrothead gravatar imageOldParrothead ( 2013-12-19 12:10:44 -0600 )edit
add a comment

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

2 followers

subscribe to rss feed

Stats

Asked: 2013-12-18 13:43:39 -0600

Seen: 1,148 times

Last updated: Dec 19 '13

Related questions

Update instances metadata with cli

instance migration to specific compute-node using API

How to get info about deleted instance?

instance boot halting for few minutes

How to define vCPU=2.0 GHz?

nova-network create on compute node

nova-api did not start [closed]

Instance going to error state while resizing in vmware

nova-manage db sync [closed]

Improving disk performance?