No valid host was found. There are not enough hosts available.

asked 2016-05-27 02:07:55 -0500

mgmead
91 ●2 ●2 ●6

I have tried and tried to make this work. I am following Mitaka example install. I have create controller, and compute1 I have set up the network, I have set up horizon. I have also set up nova on both machines as per the instructions. I can ping my gateway, I can do EVERYTHING (list images, create instances, etc.) - but I can't get it to boot. I ALWAYS get "No valid host was found. There are not enough hosts available."

I'm at my wit's end. I DO NOT KNOW WHERE TO LOOK.

Please help.

edit retag flag offensive close merge delete

Comments

Wow, someone else who uses the "wit's end" trope. That's my thoughts exactly. Useless errors in logfiles. "no valid host" yet the developer(s) thought it cute to not say what criteria is failing because this has to be as difficult as possible, it seems..

tiger ( 2017-05-01 13:03:27 -0500 )edit

Although this question is old (why does it turn up in my feed today?), let me counter tiger's comment. The failing criteria are logged in the scheduler's log file.

Users are not supposed to know the underlying infrastructure, which might explain why they are not given any more details.

Bernd Bausch ( 2018-06-25 00:50:29 -0500 )edit

add a comment

9 answers

Sort by » oldest newest most voted

answered 2016-05-29 01:22:12 -0500

mgmead
91 ●2 ●2 ●6

THANKS SO MUCH! Just hearing a voice in the wilderness gave me hope :)

So, none of those were the answer -- and sadly - as is the case with ALL of the "no valid host" posts ... each one is seemingly unique.

BUT! WANDERING SUFFERERS READ BELOW FOR TROUBLESHOOTING TIPS!

What was the case for me is that I'm using some older 2950s ... which probably DO have hardware acceleration (they passed the egrep -c '(vmx|svm)' /proc/cpuinfo test...) but my guess is that there's a BIOS twiddle on these older boxes that is not set correctly - so even though it reports "vmx or svm" are present -- they aren't "active" in some way.

But once jdiaz let me know that I'm not alone in a wasteland of CPU pain ... I took a look and found that ... (I checked /var/log/nova-conductor.log on the controller and found could not find capabilities for domaintype=kvm ... which was a new string, searching on THAT lead me to the MOST VALUABLE COMMAND virt-host-validate, which lists all the success/fails for the virtualization host. From there, I foudn that /dev/kvm was failing ... so I set nova-compute.conf to use qemu (in the [libvirt] section on compute1).

Anyway -- props to jdiaz for just being supportive. But like I promised, since I'm a veteran who just lived through this, I'm gonna write a LOT of troubleshooting steps you can take to try and fix this VERY ambiguous error message ... I can't guarantee these will work, but if you are diligent and poke around - you'll potentially find what you need.

STEPS TO TAKE WHEN YOU GET THE NO VALID HOST ERROR MESSAGE

Don't panic, it's in there - if dashboard is up and running in a sane way, you're not nuts
Understand the difference between what the controller needs and what the compute machine needs. If you've gotten the dreaded message on the dashboard, that means that the controller is complaining. But odds are good that it's complaining about the compute node.
There are three main areas where the compute node can let you down: A - the HARDWARE is not compatible B - the IMAGE FORMAT is not compatible C - the environment is not compatible
Go to the compute node and run virt-host-validate -- do it, do it NOW! This will tell you if your hardware is compatible for things like hardware acceleration (that's what got me), whether various extended services are running, and so forth. This powerful command will let you know whether the MACHINE, the METAL, is doing it's job.
If that doesn't spawn a solution, go to your image, and your image "spawning" command (openstack image create "[imagename]" --file [image file] --disk-format [qcow2, e.g] --container-format bare --public) and confirm that it has the right disk-format (apropos of what jdiaz said above)
If THAT doesn't work - it's quite possible that you aren't allocating enough disk, ram or other environment details ...

(more)

edit flag offensive delete link

Comments

@mgmead : Your answer really helped me to fix the problem of launch instance ERROR status. Kudos buddy..... :) :) :)

King Gul ( 2016-10-13 11:10:15 -0500 )edit

Got the same error, virt-host-validate showed /dev/kvm Fail but that was not the issue in my case. Checked /var/log/nova/nova-compute.log: 'Network requires port_security_enabled and subnet associated in order to apply security groups.' Enabled the previously disabled (by me) security and it works.

Wajahat ( 2017-07-26 11:00:50 -0500 )edit

virt-host-validate really saved me a lot of headache. One quick reboot and BIOS config change and BOOM SCIENCE!!! Thanks for the great troubleshooting advice! :-)

NetGeek1999 ( 2018-03-25 05:23:44 -0500 )edit

add a comment

answered 2016-05-28 20:26:13 -0500

jdiaz9

165 ●4 ●5 ●13

There are many things that could be wrong here. I'll list a few to give you an idea.

Image is not compatible with hypervisor. If using KVM make sure image is QCOW2.
Make sure nova service-list has the compute node as enabled and up. If it is down you might need to restart the nova-compute service on the compute node and or make sure that you have your networking from controller to compute node configured correctly. Can you ping controller node from compute node?
Check that rabbitmq is ok.
Make sure your instance flavor requirements are below the limits of the compute node.
Make sure libvirt is up on compute node.

These are some of the things that come to mind.

edit flag offensive delete link

Comments

any idea on networking setup for controller and compute

rdang846 ( 2019-07-01 20:27:59 -0500 )edit

add a comment

answered 2017-03-30 14:37:24 -0500

Jay Montoya

36 ●1 ●2 http://www.gtatelecom....

Hi all,

I got this error and doing some research and config changes, I found that the problem is on my hardware. I successfully installed an OpenStack Ocata, but when I started to launch different instances, I got the No valid host was found. There are not enough hosts available error, after long time of trying different solutions I got to the origin of the problem.

After running the virt-host-validate command on my server I found that Hardware Virtualization is not supported on my hardware, here is the command output:

[root@rdo ~(keystone_admin)]# virt-host-validate
  QEMU: Checking for hardware virtualization                                 : FAIL (Only emulated CPUs are available, performance will be significantly limited)
  QEMU: Checking if device /dev/vhost-net exists                             : PASS
  QEMU: Checking if device /dev/net/tun exists                               : PASS
  QEMU: Checking for cgroup 'memory' controller support                      : PASS
  QEMU: Checking for cgroup 'memory' controller mount-point                  : PASS
  QEMU: Checking for cgroup 'cpu' controller support                         : PASS
  QEMU: Checking for cgroup 'cpu' controller mount-point                     : PASS
  QEMU: Checking for cgroup 'cpuacct' controller support                     : PASS
  QEMU: Checking for cgroup 'cpuacct' controller mount-point                 : PASS
  QEMU: Checking for cgroup 'cpuset' controller support                      : PASS
  QEMU: Checking for cgroup 'cpuset' controller mount-point                  : PASS
  QEMU: Checking for cgroup 'devices' controller support                     : PASS
  QEMU: Checking for cgroup 'devices' controller mount-point                 : PASS
  QEMU: Checking for cgroup 'blkio' controller support                       : PASS
  QEMU: Checking for cgroup 'blkio' controller mount-point                   : PASS
  QEMU: Checking for device assignment IOMMU support                         : WARN (Unknown if this platform has IOMMU support)
   LXC: Checking for Linux >= 2.6.26                                         : PASS
   LXC: Checking for namespace ipc                                           : PASS
   LXC: Checking for namespace mnt                                           : PASS
   LXC: Checking for namespace pid                                           : PASS
   LXC: Checking for namespace uts                                           : PASS
   LXC: Checking for namespace net                                           : PASS
   LXC: Checking for namespace user                                          : PASS
   LXC: Checking for cgroup 'memory' controller support                      : PASS
   LXC: Checking for cgroup 'memory' controller mount-point                  : PASS
   LXC: Checking for cgroup 'cpu' controller support                         : PASS
   LXC: Checking for cgroup 'cpu' controller mount-point                     : PASS
   LXC: Checking for cgroup 'cpuacct' controller support                     : PASS
   LXC: Checking for cgroup 'cpuacct' controller mount-point                 : PASS
   LXC: Checking for cgroup 'cpuset' controller support                      : PASS
   LXC: Checking for cgroup 'cpuset' controller mount-point                  : PASS
   LXC: Checking for cgroup 'devices' controller support                     : PASS
   LXC: Checking for cgroup 'devices' controller mount-point                 : PASS
   LXC: Checking for cgroup 'blkio' controller support                       : PASS
   LXC: Checking for cgroup 'blkio' controller mount-point                   : PASS

Being this a hardware problem, do someone know if there is a solution to use the same server with some kind of upgrade? Or maybe this hardware definitely will not be usable even for a small OpenStack PoC?

Thanks for you answers.

edit flag offensive delete link

Comments

@Jay Montoya, Did you resolve the problem, I am facing exactly the same issue !?

kibnelbachyr ( 2017-04-04 05:32:45 -0500 )edit

@kibnelbachyr: make the virt-type as qemu in your nova.conf file and restart the services.

DarkKnight ( 2017-04-21 04:10:48 -0500 )edit

@kibnelbachyr: were you able to solve? I am facing the same when i try to boot an instance with nova or openstackclient

adnavare ( 2017-04-26 19:22:12 -0500 )edit

I am running inside devstack

adnavare ( 2017-04-26 19:23:10 -0500 )edit

@kibnelbachyr I was not able to solve this, the computer itself does not support virtualization, changed computer and problem was gone.

Jay Montoya ( 2017-04-30 12:32:33 -0500 )edit

add a comment

answered 2016-10-06 15:36:50 -0500

CodesMcCodes
11 ●1 ●1

I ran up against the No valid host was found. There are not enough hosts available error and couldn't figure out why. I knew it wasn't due to image format because I had previously used this instance. The instance that launched didn't have enough ram because I accidentally gave it 4MB instead of 4GB so I decided to delete and recreate the instance to resize the ram. BTW. The reason I chose to delete and recreate is because I was running into the same error when trying to resize. Because, by default, OpenStack will try to resize the instance on a different host. So anyways when trying to recreate I ran into this error and totally hit a dead end.

I tried everything mgmead suggested.

The key for me was this in the /var/log/nova/nova-scheduler.log

2016-10-06 15:13:37.839 30218 INFO nova.filters [req-3b92a5a1-2a51-4d9c-b98d-cfa2bf6b2e5e 5d7a59d920084480b4beb0d145f13cd8 925b92a5b70a4ce9965a9d3ecc7e87c6 - - -] Filter ComputeCapabilitiesFilter returned 0 hosts 2016-10-06 15:13:37.840 30218 INFO nova.filters [req-3b92a5a1-2a51-4d9c-b98d-cfa2bf6b2e5e 5d7a59d920084480b4beb0d145f13cd8 925b92a5b70a4ce9965a9d3ecc7e87c6 - - -] Filtering removed all hosts for the request with instance ID '3d52306e-6232-4584-9513-03dcba0d9df9'. Filter results: ['RetryFilter: (start: 3, end: 3)', 'AvailabilityZoneFilter: (start: 3, end: 3)', 'RamFilter: (start: 3, end: 3)', 'DiskFilter: (start: 3, end: 3)', 'ComputeFilter: (start: 3, end: 2)', 'ComputeCapabilitiesFilter: (start: 2, end: 0)']

So basically before running the ComputeCapabilitiesFilter filter there were 2 candidate hosts, but after there were no hosts left. But why??? after googling ComputeCapabilitiesFilter I realized that it was testing for extra-specs that I accidentally added when trying to change my flavor incorrectly.

[root@control01 ~]# openstack flavor show lb
+----------------------------+--------------------------------------+
| Field                      | Value                                |
+----------------------------+--------------------------------------+
| OS-FLV-DISABLED:disabled   | False                                |
| OS-FLV-EXT-DATA:ephemeral  | 0                                    |
| disk                       | 25                                   |
| id                         | 155ef9c8-e685-4236-a766-ee82f0ca8089 |
| name                       | lb                                   |
| os-flavor-access:is_public | True                                 |
| properties                 | ram='4096'                           |
| ram                        | 4096                                 |
| rxtx_factor                | 1.0                                  |
| swap                       |                                      |
| vcpus                      | 4                                    |
+----------------------------+--------------------------------------+

Here you can see in the properties that I had ram='4096'. No bueno. Deleted the metadata for this flavor in the dashboard and I was finally able to start a new instance.

Hope this helps someone.

edit flag offensive delete link

Comments

+1 for advice to consider nova-scheduler.log; that process is in charge of determining where to launch a new instance. If you see it considering all your compute hosts and come back with "RetryFilter returned 0 hosts", it's worth checking their own nova-compute.log to see if they reveal anything

sxc731 ( 2017-08-26 12:14:00 -0500 )edit

Happened the same to me: was playing with host aggregates and properties. Then I created a setup where my flavor did match none (based on extra properties).

xian ( 2018-05-30 15:32:11 -0500 )edit

add a comment

answered 2017-04-05 08:51:19 -0500

jbheren
1 ●2 ●3 ●4

When this happens, I simply connect to one compute node and restart the nova-scheduler service.

juju ssh nova-compute/0
sudo /etc/init.d/nova-scheduler restart

edit flag offensive delete link

add a comment

answered 2017-04-06 08:29:09 -0500

CurDown
1

check the core service is running by typing command " netstat -an | grep LISTENING". In the controller node,it should contains listening port 8778(placement_api service), 8774(compute-service),9292(Image service),9696(network),5000(Identify service),5672(rabbitmq server), 11211( memcache server),35357(Identify service) at least if you don't modify the default config. if you install Ocata by offical guide line by line ,You must start placement-api service manually。
In compute node,you can run command "virt-host-validate" to check your host that whether it supports hardware virtualization.If fails ,edit the file "/etc/nova/nova.conf",set virt_type=qemu.
Ensure your host owns enough cpu,Memory,disk resources.
if All the steps is ok ,Open Debug log message By set debug=true int /etc/nova/nova.conf。you can find more information in the directory /var/log/nova/

edit flag offensive delete link

add a comment

answered 2018-02-19 09:08:30 -0500

scampo
13 ●3 ●5 ●10

updated 2018-02-19 09:17:48 -0500

Hi all,

I got this error while I was working with DEVSTACK.

I successfully installed Devstack based on the latest stable version (Pike) and when i tried to launch an instance I got the error message "message: No valid host was found. There are not enough hosts available. code: 500".

I tried to execute the virt-host-validate command with the result shown below:

:~$ virt-host-validate
QEMU: Checking for hardware virtualization                                 : PASS
QEMU: Checking if device /dev/kvm exists                                   : PASS
QEMU: Checking if device /dev/kvm is accessible                            : FAIL (Check /dev/kvm is world writable or you are in a group that is allowed to access it)
QEMU: Checking if device /dev/vhost-net exists                             : PASS
QEMU: Checking if device /dev/net/tun exists                               : PASS
QEMU: Checking for cgroup 'memory' controller support                      : PASS
QEMU: Checking for cgroup 'memory' controller mount-point                  : PASS
QEMU: Checking for cgroup 'cpu' controller support                         : PASS
QEMU: Checking for cgroup 'cpu' controller mount-point                     : PASS
QEMU: Checking for cgroup 'cpuacct' controller support                     : PASS
QEMU: Checking for cgroup 'cpuacct' controller mount-point                 : PASS
QEMU: Checking for cgroup 'cpuset' controller support                      : PASS
QEMU: Checking for cgroup 'cpuset' controller mount-point                  : PASS
QEMU: Checking for cgroup 'devices' controller support                     : PASS
QEMU: Checking for cgroup 'devices' controller mount-point                 : PASS
QEMU: Checking for cgroup 'blkio' controller support                       : PASS
QEMU: Checking for cgroup 'blkio' controller mount-point                   : PASS
QEMU: Checking for device assignment IOMMU support                         : PASS
QEMU: Checking if IOMMU is enabled by kernel                               : WARN (IOMMU appears to be disabled in kernel. Add intel_iommu=on to kernel cmdline arguments)
LXC: Checking for Linux >= 2.6.26                                         : PASS
LXC: Checking for namespace ipc                                           : PASS
LXC: Checking for namespace mnt                                           : PASS
LXC: Checking for namespace pid                                           : PASS
LXC: Checking for namespace uts                                           : PASS
LXC: Checking for namespace net                                           : PASS
LXC: Checking for namespace user                                          : PASS
LXC: Checking for cgroup 'memory' controller support                      : PASS
LXC: Checking for cgroup 'memory' controller mount-point                  : PASS
LXC: Checking for cgroup 'cpu' controller support                         : PASS
LXC: Checking for cgroup 'cpu' controller mount-point                     : PASS
LXC: Checking for cgroup 'cpuacct' controller support                     : PASS
LXC: Checking for cgroup 'cpuacct' controller mount-point                 : PASS
LXC: Checking for cgroup 'cpuset' controller support                      : PASS
LXC: Checking for cgroup 'cpuset' controller mount-point                  : PASS
LXC: Checking for cgroup 'devices' controller support                     : PASS
LXC: Checking for cgroup 'devices' controller mount-point                 : PASS
LXC: Checking for cgroup 'blkio' controller support                       : PASS
LXC: Checking for cgroup 'blkio' controller mount-point                   : PASS

Any tips to solve the failure got while "Checking if device /dev/kvm is accessible"?

I tried, without success, following the tips in this page: https://help.ubuntu.com/community/KVM/Installation (https://help.ubuntu.com/community/KVM...).

Thanks

edit flag offensive delete link

Comments

You should create a new question instead of submitting an answer to an existing, 9-months-old question.

In a nutshell, when launching an instance, Nova looks for a host with the correct properties and enough resources to run the instance. If there is no such host, you get this error.

Bernd Bausch ( 2018-02-19 17:48:15 -0500 )edit

For example, you might not have enough memory or disk space. You will find details in the nova-scheduler log file, in particular when you enable DEBUG logging, and possibly the placement log and the nova-compute log.

Bernd Bausch ( 2018-02-19 17:50:14 -0500 )edit

Whether the warning about IOMMU is really a problem is not clear to me. Normally, DevStack relies on software-only qemu as hypervisor, AFAIK, and doesn't need the KVM driver.

In any case, the answer seems to be "Add intel_iommu=on to kernel cmdline arguments".

Bernd Bausch ( 2018-02-19 17:53:16 -0500 )edit

Thanks for the suggestions! I reposted my question here https://ask.openstack.org/en/question/112654/devstack-no-valid-host-was-found-there-are-not-enough-hosts-available-code-500-failure-checking-if-device-devkvm-is-accessible/ (https://ask.openstack.org/en/question...)

scampo ( 2018-02-20 04:31:25 -0500 )edit

I've already checked the amount of memory or disk space. Fortunately it is enough. Now, I am going to check the log files.

scampo ( 2018-02-20 04:33:07 -0500 )edit

see more comments

answered 2017-05-26 08:24:39 -0500

Saintset
1

nothing worked any Ideas???

edit flag offensive delete link

add a comment

answered 2018-06-06 04:33:23 -0500

FrancescoDeRose
1

Good Morning, when I create a nova instance with a flavor a get an Error : No valid host was found. There are not enough hosts available Code 500 Détails: << File "/opt/stack/nova/nova/conductor/manager.py", line 1113, in schedule_and_build_instances instance_uuids, return_alternates=True) File "/opt/stack/nova/nova/conductor/manager.py", line 715, in _schedule_instances return_alternates=return_alternates) File "/opt/stack/nova/nova/scheduler/utils.py", line 840, in wrapped return func(args, *kwargs) File "/opt/stack/nova/nova/scheduler/client/__init__.py", line 53, in select_destinations instance_uuids, return_objects, return_alternates) File "/opt/stack/nova/nova/scheduler/client/__init__.py", line 37, in __run_method return getattr(self.instance, __name)(args, *kwargs) File "/opt/stack/nova/nova/scheduler/client/query.py", line 42, in select_destinations instance_uuids, return_objects, return_alternates) File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 158, in select_destinations return cctxt.call(ctxt, 'select_destinations', **msg_args) File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 179, in call retry=self.retry) File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 133, in _send retry=retry) File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 584, in send call_monitor_timeout, retry=retry) File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 575, in _send raise result>>

Any help, will be very appreciated. Thanks in advance Francesco

edit flag offensive delete link

add a comment

No valid host was found. There are not enough hosts available.

Comments

9 answers

Comments

Comments

Comments

Comments

Comments

Your Answer

Get to know Ask OpenStack

Question Tools

Stats

Related questions

Feedback About This Page

OpenStack

Community

Documentation

Branding & Legal

No valid host was found. There are not enough hosts available. edit

Comments

9 answers

Comments

Comments

Comments

Comments

Comments

Your Answer

Get to know Ask OpenStack

Question Tools

Stats

Related questions

Feedback About This Page

OpenStack

Community

Documentation

Branding & Legal

No valid host was found. There are not enough hosts available.