Ask Your Question

Failed to scheduler_run_instance: No valid host was found [closed]

asked 2013-12-18 04:44:31 -0500

kemra102 gravatar image

updated 2014-01-22 15:12:29 -0500

Evgeny gravatar image

Hi Folks,

I have a CentOS 6.5 based setup of OpenStack Havana consisting of a single controller and 6 compute nodes running nova-compute & nova-networking (multi-homed networking setup).

I was having some issues booting instances with nova using shared storage for /var/lib/nova/instances. However someone suggested trying to do it on local storage and see if the error persists.

So I removed the shared storage from /etc/fstab and rebooted the compute nodes. Now with the compute nodes running on local storage for Nova I booted up an instance and the nova list shows this info:

[root@openstackadm ~]# nova list
| ID                                   | Name        | Status  | Task State | Power State | Networks          |
| aa0d81c0-401b-4cbb-a566-2cbe665f2c7e | tdh3        | ERROR   | None       | NOSTATE     | vmnet=  |

NOTE: The vmnet is unassigned and disappears from the nova list output after a few seconds.

I check /var/log/noca/scheduler.log on the controller and I found the following relevant snippet:

[root@openstackadm ~]# tail -n2 /var/log/nova/scheduler.log 
2013-12-18 09:33:36.918 2266 WARNING nova.scheduler.utils [req-97c7e24a-250c-4098-844b-554357c34b06 5fc621f9f24941febb883f78d2d350ec c22af115b6a84dc78b1ae19385c6cd23] Failed to scheduler_run_instance: No valid host was found. Exceeded max scheduling attempts 3 for instance aa0d81c0-401b-4cbb-a566-2cbe665f2c7e
2013-12-18 09:33:36.919 2266 WARNING nova.scheduler.utils [req-97c7e24a-250c-4098-844b-554357c34b06 5fc621f9f24941febb883f78d2d350ec c22af115b6a84dc78b1ae19385c6cd23] [instance: aa0d81c0-401b-4cbb-a566-2cbe665f2c7e] Setting instance to ERROR state.

I did a little googling around and the only real answer I could find on the error is that it might be to do with a lack of resources on the available compute nodes. However this cannot be the case as no other instances are currently running and the nodes are fairly decent boxes.

I tried setting scheduler_default_filters=AllHostsFilter in /etc/nova/nova.conf and restarted the scheduler service but this has had no effect.

edit retag flag offensive reopen merge delete

Closed for the following reason the question is answered, right answer was accepted by dheeru
close date 2014-01-13 11:17:40.484423


This log snippet is just giving symptom of the issue. Can you look at the following two logs and findout what is happening ? /var/log/nova/nova-compute.log /var/log/libvirt/libvirtd.log

dheeru gravatar imagedheeru ( 2013-12-18 05:35:10 -0500 )edit

Hi dheeru, I've emailed them to you.

kemra102 gravatar imagekemra102 ( 2013-12-18 06:54:32 -0500 )edit

The error could be due to multiple reasons: 1. compute service not running (or compute heartbeat not getting updated or getting updated in wrong db)- check each 'nova-compute' in 'nova-manage service list' 2. none of the Compute hosts have the resources required to spawn instance of requested flavor

unmesh-gurjar gravatar imageunmesh-gurjar ( 2013-12-19 05:27:57 -0500 )edit

I have checked using 'nova-manage service list' and all the servers report enabled and working. Also all of the compute nodes are completely free (i.e. no instances at all running right now) and are decently specced too so adding a tiny (1vcpu, 512MB RAM, 10GB disk) should not be an issue.

kemra102 gravatar imagekemra102 ( 2014-01-02 04:21:03 -0500 )edit

Could you double check if the image you are trying to spawn the instance from is supported by the hypervisor(s) the Compute nodes have.

unmesh-gurjar gravatar imageunmesh-gurjar ( 2014-01-02 04:41:20 -0500 )edit

1 answer

Sort by ยป oldest newest most voted

answered 2014-01-10 10:57:11 -0500

kemra102 gravatar image

Other logs showed image was not found.

Image showed in Glance but when checking the storage it was not present.

Deleting from Glance and re-uploading the image allowed an instance to boot.

Thanks to dheeru for the solution.

edit flag offensive delete link more

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower


Asked: 2013-12-18 04:44:31 -0500

Seen: 10,134 times

Last updated: Jan 10 '14