New VM instance stuck in spawning state

asked 2013-11-08 21:15:12 -0600

anonymous user


I used devstack for installing openstack. My newly spawned VM on the controller node is stuck in spawning state on the horizon dashboard. After looking at the running processes I found the culprit:

root     25004  0.0  0.0 295012  1960 ?        Ssl  18:56   0:00 /usr/bin/qemu-nbd -c /dev/nbd5 /opt/stack/data/nova/instances/4aeefbdf-4688-4abd-959e-2fca7f3106f5/disk
root     25016  0.0  0.0  22176   684 pts/16   D+   18:56   0:00 /bin/mount /dev/nbd5 /tmp/openstack-vfs-localfsKn9pTC

ctrl1:/opt/stack/data/nova/instances/4aeefbdf-4688-4abd-959e-2fca7f3106f5$ ls -l
total 8668
-rw-rw---- 1 bruce libvirtd       0 Nov  8 18:56 console.log
-rw-r--r-- 1 bruce libvirtd  262656 Nov  8 18:56 disk
-rw-rw-r-- 1 bruce libvirtd 4955792 Nov  8 18:56 kernel
-rw-rw-r-- 1 bruce libvirtd 3714968 Nov  8 18:56 ramdisk

The mount process is in uninterruptible sleep state. I see this issue after a couple of days of usage and end up rebooting and re-stacking. Does anyone know what might be going on here?

edit retag flag offensive close merge delete


Have a look at the log file for nova-compute (/var/log/nova/nova-compute.log). There will most probably be more info on the problem.

fetahi gravatar imagefetahi ( 2013-11-09 00:27:47 -0600 )edit

I don't have any errors/exceptions. I found two warnings though: "WARNING nova.compute.manager [-] Found 3 in the database and 2 on the hypervisor." and "WARNING nova.virt.libvirt.vif [req-65080806-a8ba-467f-b09f-88a5a5498a67 admin TenantOne] Deprecated: The LibvirtHybridOVSBridgeDriver VIF driver is now deprecated and will be removed in the next release. Please use the LibvirtGenericVIFDriver VIF driver, together with a network plugin that reports the 'vif_type' attribute"

dushyant gravatar imagedushyant ( 2013-11-09 00:55:49 -0600 )edit

Grep for the id of the instance that is stuck in spawning state and see if you can find any hints. If it has been a while since this happened, the error may have also moved to nova-compute.log.1. Finally, set debug=True in /etc/nova/nova.conf and try to recreate the error for more info.

fetahi gravatar imagefetahi ( 2013-11-09 01:21:30 -0600 )edit

I had done that already...actually immediately after I saw the error. I see only DEBUG and INFO messages. "21:42:59.293 1185 DEBUG nova.compute.manager [-] [instance: 4aeefbdf-4688-4abd-959e-2fca7f3106f5] Updated the info_cache for instance _heal_instance_info_cache /opt/stack/nova/nova/compute/" and "2013-11-08 21:43:53.815 1185 INFO nova.compute.manager [-] [instance: 4aeefbdf-4688-4abd-959e-2fca7f3106f5] During sync_power_state the instance has a pending task. Skip." I had set debug = True in nova.conf.

dushyant gravatar imagedushyant ( 2013-11-09 01:33:24 -0600 )edit

Look into the rotated log files (nova-compute.log.*) if you do not find the messages related to the VM starting, then you should try to recreate the problem again.

fetahi gravatar imagefetahi ( 2013-11-09 05:42:08 -0600 )edit