Ask Your Question
0

nova scheduler: No valid host was found. There are not enough hosts available.

asked 2016-11-01 10:13:40 -0500

doc24 gravatar image

updated 2016-11-01 18:51:23 -0500

We have deployed multi-node OpenStack (Juno: http://docs.openstack.org/juno/instal...) based on Virtualbox. Our compute and network nodes are on guest VMs, whereas our controller node is on host machine. We are facing an error, "No valid host was found. There are not enough hosts available" when we try to launch an instance. According to nova-scheduler logs, the host controller is failing. We presume that it might be an issue of compute hypervisor which is not shown in hypervisor-list. Any idea how this issue can be resolved?

nova service-list

+----+------------------+------------+----------+---------+-------+----------------------------+-----------------+
| Id | Binary           | Host       | Zone     | Status  | State | Updated_at                 | Disabled Reason |
+----+------------------+------------+----------+---------+-------+----------------------------+-----------------+
| 1  | nova-cert        | controller | internal | enabled | up    | 2016-11-01T14:53:57.000000 | -               |
| 2  | nova-consoleauth | controller | internal | enabled | up    | 2016-11-01T14:53:58.000000 | -               |
| 3  | nova-scheduler   | controller | internal | enabled | up    | 2016-11-01T14:53:59.000000 | -               |
| 4  | nova-conductor   | controller | internal | enabled | up    | 2016-11-01T14:53:59.000000 | -               |
| 5  | nova-compute     | compute    | nova     | enabled | up    | 2016-11-01T14:54:03.000000 | -               |
| 6  | nova-compute     | controller | nova     | enabled | up    | 2016-11-01T14:53:59.000000 | -               |
+----+------------------+------------+----------+---------+-------+----------------------------+-----------------+

nova hypervisor-list

+----+---------------------+-------+---------+
| ID | Hypervisor hostname | State | Status  |
+----+---------------------+-------+---------+
| 1  | controller          | up    | enabled |
+----+---------------------+-------+---------+

2016-11-02 00:06:46.832 10638 WARNING nova.scheduler.filters.compute_filter [req-710be1c4-185e-425d-bc34-8e8b47183c8d 9224f04451e74a8daafb9fba6a7ac047 03a58bc4d34c4c0e9c52b679cd659d6c - - -] (controller, controller) ram:6848 disk:360448 io_ops:0 instances:1 has not been heard from in a while
2016-11-02 00:06:46.832 10638 INFO nova.filters [req-710be1c4-185e-425d-bc34-8e8b47183c8d 9224f04451e74a8daafb9fba6a7ac047 03a58bc4d34c4c0e9c52b679cd659d6c - - -] Filter ComputeFilter returned 0 hosts
2016-11-02 00:06:46.832 10638 DEBUG nova.scheduler.filter_scheduler [req-710be1c4-185e-425d-bc34-8e8b47183c8d 9224f04451e74a8daafb9fba6a7ac047 03a58bc4d34c4c0e9c52b679cd659d6c - - -] There are 0 hosts available but 1 instances requested to build. select_destinations /usr/lib/python2.7/dist-packages/nova/scheduler/filter_scheduler.py:77
edit retag flag offensive close merge delete

Comments

Why Juno? More people could help you if you used a more recent, supported version of OpenStack. In Mitaka, the message "has not been heard from in a while" is issued when the service is down. So the question is, why is is up according to the service-list command.

Bernd Bausch gravatar imageBernd Bausch ( 2016-11-01 18:58:32 -0500 )edit

A service is declared down if no heartbeat comes from it for over a minute (75 secs I think). Perhaps you need to run nova service-list a second time? Also check the nova-compute.log on the compute hosts and the nova-api and nova-conductor logs.

Bernd Bausch gravatar imageBernd Bausch ( 2016-11-01 19:01:38 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
0

answered 2016-11-01 21:16:20 -0500

doc24 gravatar image

updated 2016-11-01 23:16:59 -0500

We have run nova service-list several times, and the service appears to be up every time. The compute-node service is on the controller node as well as on the compute node (we are not sure if it has to be on one of the nodes only) because nova-compute.log are different on compute node and controller node. Not sure if nova-compute service should be there on the controller node or not.

---------------------nova-compute.log on compute node------------------------------------------------------------

2016-11-02 10:29:30.676 17047 AUDIT nova.compute.resource_tracker [-] Auditing locally available compute resources
2016-11-02 10:29:31.000 17047 WARNING nova.compute.resource_tracker [-] No service record for host compute


2016-11-02 10:31:58.311 13672 AUDIT nova.service [-] Starting compute node (version 2014.2.4)
2016-11-02 10:31:58.312 13672 DEBUG nova.virt.libvirt.driver [-] Connecting to libvirt: qemu:///system _get_new_connection /us
r/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py:766
2016-11-02 10:31:58.421 13672 DEBUG nova.virt.libvirt.driver [-] Registering for lifecycle events <nova.virt.libvirt.driver.Li
bvirtDriver object at 0x7f3aeca36d10> _get_new_connection /usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py:783
2016-11-02 10:31:58.434 13672 DEBUG nova.virt.libvirt.driver [-] Registering for connection events: <nova.virt.libvirt.driver.
LibvirtDriver object at 0x7f3aeca36d10> _get_new_connection /usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py:794
2016-11-02 10:31:58.446 13672 DEBUG nova.virt.libvirt.config [-] Generated XML ('<cpu>\n  <arch>x86_64</arch>\n  <model>core2d
uo</model>\n  <vendor>Intel</vendor>\n  <topology sockets="1" cores="1" threads="1"/>\n  <feature name="lahf_lm"/>\n  <feature
 name="rdtscp"/>\n</cpu>\n',)  to_xml /usr/lib/python2.7/dist-packages/nova/virt/libvirt/config.py:82
2016-11-02 10:31:58.477 13672 DEBUG nova.virt.libvirt.driver [-] Starting native event thread _init_events /usr/lib/python2.7/
dist-packages/nova/virt/libvirt/driver.py:705
2016-11-02 10:31:58.484 13672 DEBUG nova.virt.libvirt.driver [-] Starting green dispatch thread _init_events /usr/lib/python2.
7/dist-packages/nova/virt/libvirt/driver.py:710
2016-11-02 10:31:58.644 13672 DEBUG nova.virt.libvirt.driver [-] Updating host stats update_status /usr/lib/python2.7/dist-pac
kages/nova/virt/libvirt/driver.py:6496
2016-11-02 10:31:58.796 13672 AUDIT nova.compute.resource_tracker [-] Auditing locally available compute resources
2016-11-02 10:31:58.806 13672 DEBUG nova.virt.libvirt.driver [-] Updating host stats update_status /usr/lib/python2.7/dist-pac
kages/nova/virt/libvirt/driver.py:6496
2016-11-02 10:31:58.959 13672 DEBUG nova.compute.resource_tracker [-] Hypervisor: free ram (MB): 1282 _report_hypervisor_resou
rce_view /usr/lib/python2.7/dist-packages/nova/compute/resource_tracker.py:488
2016-11-02 10:31:58.959 13672 DEBUG nova.compute.resource_tracker [-] Hypervisor: free disk (GB): 42 _report_hypervisor_resour
ce_view /usr/lib/python2.7/dist-packages/nova/compute/resource_tracker.py:489
2016-11-02 10:31:58.960 13672 DEBUG nova.compute.resource_tracker [-] Hypervisor: free VCPUs: 1 _report_hypervisor_resource_vi
ew /usr/lib/python2.7/dist-packages/nova/compute/resource_tracker.py:494
2016-11-02 10:31:58.960 13672 DEBUG nova.compute.resource_tracker [-] Hypervisor: assignable PCI devices: [] _report_hyperviso
r_resource_view ...
(more)
edit flag offensive delete link more

Comments

You wonder if nova-compute should be on both the compute node and the controller? This is entirely up to you. nova-compute must run on those nodes where you want to run VMs (except for baremetal instances). Those nodes need to have a hypervisor like libvirt and KVM.

Bernd Bausch gravatar imageBernd Bausch ( 2016-11-01 23:31:12 -0500 )edit

Are hypervisors installed on both? If not, perhaps this can lead to a situation where Nova thinks a node is a compute node but it can't run VMs.

Bernd Bausch gravatar imageBernd Bausch ( 2016-11-01 23:32:17 -0500 )edit

Thank you so much @Bernd Bausch for your answers. The issue has been fixed and I can launch instances without any errors now. The nova-compute.conf file had virt_type = kvm. I changed it to qemu, restarted all the compute services, and everything worked fine.

doc24 gravatar imagedoc24 ( 2016-11-02 01:58:10 -0500 )edit

This is a well known issue try to change kvm by qemu in the nova.conf file (/etc/nova/nova.conf)

yas gravatar imageyas ( 2016-11-02 02:14:10 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2016-11-01 10:13:40 -0500

Seen: 2,372 times

Last updated: Nov 01 '16