Ask Your Question
0

Devstack - No valid host was found. There are not enough hosts available. - code 500 - Failure Checking if device /dev/kvm is accessible

asked 2018-02-20 04:23:32 -0600

scampo gravatar image

Hi all, (I repost my question here as new) I got this error while I was working with DEVSTACK.

I successfully installed Devstack based on the latest stable version (Pike) and when i tried to launch an instance I got the error message "message: No valid host was found. There are not enough hosts available. code: 500".

I tried to execute the virt-host-validate command with the result shown below:

:~$ virt-host-validate
QEMU: Checking for hardware virtualization                                 : PASS
QEMU: Checking if device /dev/kvm exists                                   : PASS
QEMU: Checking if device /dev/kvm is accessible                            : FAIL (Check /dev/kvm is world writable or you are in a group that is allowed to access it)
QEMU: Checking if device /dev/vhost-net exists                             : PASS
QEMU: Checking if device /dev/net/tun exists                               : PASS
QEMU: Checking for cgroup 'memory' controller support                      : PASS
QEMU: Checking for cgroup 'memory' controller mount-point                  : PASS
QEMU: Checking for cgroup 'cpu' controller support                         : PASS
QEMU: Checking for cgroup 'cpu' controller mount-point                     : PASS
QEMU: Checking for cgroup 'cpuacct' controller support                     : PASS
QEMU: Checking for cgroup 'cpuacct' controller mount-point                 : PASS
QEMU: Checking for cgroup 'cpuset' controller support                      : PASS
QEMU: Checking for cgroup 'cpuset' controller mount-point                  : PASS
QEMU: Checking for cgroup 'devices' controller support                     : PASS
QEMU: Checking for cgroup 'devices' controller mount-point                 : PASS
QEMU: Checking for cgroup 'blkio' controller support                       : PASS
QEMU: Checking for cgroup 'blkio' controller mount-point                   : PASS
QEMU: Checking for device assignment IOMMU support                         : PASS
QEMU: Checking if IOMMU is enabled by kernel                               : WARN (IOMMU appears to be disabled in kernel. Add intel_iommu=on to kernel cmdline arguments)
LXC: Checking for Linux >= 2.6.26                                         : PASS
LXC: Checking for namespace ipc                                           : PASS
LXC: Checking for namespace mnt                                           : PASS
LXC: Checking for namespace pid                                           : PASS
LXC: Checking for namespace uts                                           : PASS
LXC: Checking for namespace net                                           : PASS
LXC: Checking for namespace user                                          : PASS
LXC: Checking for cgroup 'memory' controller support                      : PASS
LXC: Checking for cgroup 'memory' controller mount-point                  : PASS
LXC: Checking for cgroup 'cpu' controller support                         : PASS
LXC: Checking for cgroup 'cpu' controller mount-point                     : PASS
LXC: Checking for cgroup 'cpuacct' controller support                     : PASS
LXC: Checking for cgroup 'cpuacct' controller mount-point                 : PASS
LXC: Checking for cgroup 'cpuset' controller support                      : PASS
LXC: Checking for cgroup 'cpuset' controller mount-point                  : PASS
LXC: Checking for cgroup 'devices' controller support                     : PASS
LXC: Checking for cgroup 'devices' controller mount-point                 : PASS
LXC: Checking for cgroup 'blkio' controller support                       : PASS
LXC: Checking for cgroup 'blkio' controller mount-point                   : PASS

Any tips to solve the failure got while "Checking if device /dev/kvm is accessible"?

I tried, without success, following the tips in this page "https://help.ubuntu.com/community/KVM/Installation" and in this question "https://ask.openstack.org/en/question/92701/no-valid-host-was-found-there-are-not-enough-hosts-available"

Thanks

edit retag flag offensive close merge delete

Comments

1

In /etc/nova/nova.conf, check virt_type. If it is set to qemu, which I assume, your problem has nothing to do with KVM. If it’s set to kvm, set it to qemu, restart n-cpu (Nova’s compute service) and try again.

The key to the solution is in the n-sch log (nova scheduler). Look there for clues.

Bernd Bausch gravatar imageBernd Bausch ( 2018-02-20 05:05:47 -0600 )edit
1

For an overview of the Nova scheduler, see https://docs.openstack.org/nova/lates.... In Pike, Nova uses the Placement service to compile a list of hosts with sufficient resources before filtering; see https://docs.openstack.org/nova/lates....

Bernd Bausch gravatar imageBernd Bausch ( 2018-02-20 05:13:02 -0600 )edit
1

Regarding the /dev/kvm error, I suppose you could try making it world-readable, e.g. chmod ugo+w /dev/kvm

Bernd Bausch gravatar imageBernd Bausch ( 2018-02-20 05:15:16 -0600 )edit

Thank you! I'll give you a feedback asap!

scampo gravatar imagescampo ( 2018-02-20 08:34:05 -0600 )edit

we have been struggling with the same error trying to run Windows 10 on OpenStack. Its insane, error is not very clear and solution is inexistent.

finch15 gravatar imagefinch15 ( 2018-02-21 13:10:32 -0600 )edit

2 answers

Sort by » oldest newest most voted
1

answered 2018-02-27 18:21:01 -0600

updated 2018-02-27 18:21:22 -0600

The nova compute service, devstack@n-cpu, registers its host with the Placement Service upon startup. The compute host is a resource provider, and your error message indicates that this resource provider is not found in the Placement Service database.

First, run the command nova-status upgrade check. It should return something like this:

+---------------------------+
| Upgrade Check Results     |
+---------------------------+
| Check: Cells v2           |
| Result: Success           |
| Details: None             |
+---------------------------+
| Check: Placement API      |
| Result: Success           |
| Details: None             |
+---------------------------+
| Check: Resource Providers |
| Result: Success           |
| Details: None             |
+---------------------------+

It might look different in Queens. Both the Placement API and the Resource Providers should be OK; if they are not, perhaps you need to reinstall.

Also check the status and the logs of devstack@n-cpu and devstack@placement-api:

systemctl status devstack@n-cpu devstack@placement-api
journalctl -u devstack@n-cpu 
journalctl -u devstack@placement-api

I hope more clues are found here.

The questions I have:

  • Was n-cpu successful in registering the host?
  • If yes, what became of its placement service record?

You can also check the Placement API and its database. There is no CLI; you have to use the API directly. Get an authentication token and list the resource provides in the Placement Service database:

source ~/devstack/openrc admin admin
TOKEN=$(openstack token issue -c id -f value)
curl -H "OpenStack-API-Version: placement latest" -H "x-auth-token: $TOKEN" http://192.168.5.10/placement/resource_providers | python -m json.tool

(The json.tool part is for pretty-printing)

Alternatively, you can look into the database, but I don't remember the detailed names of the placement database and its tables.

mysql -e "show databases;" | grep placement
mysql
> use NAME_OF_PLACEMENT_DATABASE
> show tables;
> select * from NAME_OF_RESOURCE_PROVIDER_TABLE;
edit flag offensive delete link more

Comments

Hi @Bernd Bausch

I tried to launch the nova-status upgrade check getting the expected result that you shown previously.

scampo gravatar imagescampo ( 2018-03-05 10:56:11 -0600 )edit

Then, I tried to look into the databases, but without success. To more clearify: I looked into resource_provider table for each of these databases: nova-api, nova_cell0 and nova_cell1. But all tables are empty!

scampo gravatar imagescampo ( 2018-03-05 11:03:59 -0600 )edit

I also tried to check the status and the logs of devstack@n-cpu and devstack@placement-api. I hope to answer to your first question pasting a piece of n-cpu logs in the next comment.

scampo gravatar imagescampo ( 2018-03-05 11:16:13 -0600 )edit

n-cpu log

node.node nova-compute[19161]: WARNING nova.virt.libvirt.driver [None
req-162330c2-9062-4505-8fa7-54c1b0993a43 None None] Cannot update service status on host
node.node" since it is not registered.: ComputeHostNotFound_Remote: 
Compute host node.node could not be found.
scampo gravatar imagescampo ( 2018-03-05 11:27:19 -0600 )edit

It seems that n-cpu wasn't successful in registering the host. Is it correct? What can I do? Thanks!

scampo gravatar imagescampo ( 2018-03-05 11:31:07 -0600 )edit
0

answered 2018-02-27 10:49:21 -0600

scampo gravatar image

updated 2018-02-27 11:32:24 -0600

Hi @Bernd Bausch

I set virt_type to qemu value as you suggested.

I restarted the nova compute services and I tried to create a new instance with the same error.

I found the log and I paste a little bit sequence here:

Feb 27 09:36:08 node devstack@n-api.service[959]: Tue Feb 27 09:36:08 2018 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/flavors/m1.small (ip 12.12.12.12) !!!
Feb 27 09:36:08 node devstack@n-api.service[959]: Tue Feb 27 09:36:08 2018 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/flavors/m1.small (ip 12.12.12.12) !!!
Feb 27 09:36:08 node devstack@n-api.service[959]: #033[00;36mINFO nova.api.openstack.wsgi [#033[01;36mNone req-b3f9c7aa-2881-4777-a043-3cd8a451a7f0 #033[00;36mdemo admin#033[00;36m] #033[01;35m#033[00;36mHTTP exception thrown: Flavor m1.small could not be found.#033[00m
Feb 27 09:36:08 node devstack@n-api.service[959]: #033[00;32mDEBUG nova.api.openstack.wsgi [#033[01;36mNone req-b3f9c7aa-2881-4777-a043-3cd8a451a7f0 #033[00;36mdemo admin#033[00;32m] #033[01;35m#033[00;32mReturning 404 to user: Flavor m1.small could not be found.#033[00m #033[00;33m{{(pid=988) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:1029}}#033[00m

I got those messages several times. After I got also:

Feb 27 09:36:10 node devstack@keystone.service[918]: #033[00;32mDEBUG keystone.middleware.auth [#033[01;36mNone req-709db444-27c3-4e33-af86-9a1de2b52532 #033[00;36mNone None#033[00;32m] #033[01;35m#033[00;32mAuthenticating user token#033[00m #033[00;33m{{(pid=931) process_request /usr/local/lib/python2.7/dist-packages/keystonemiddleware/auth_token/__init__.py:400}}#033[00m
Feb 27 09:36:10 node nova-conductor[957]: #033[01;31mERROR nova.conductor.manager [#033[01;36mNone req-92bfafc4-5781-442c-ba51-e9bcf3b5520b #033[00;36mdemo admin#033[01;31m] #033[01;35m#033[01;31mFailed to schedule instances#033[00m: NoValidHost_Remote: No valid host was found.
Feb 27 09:36:10 node nova-conductor[957]: Traceback (most recent call last):
Feb 27 09:36:10 node nova-conductor[957]:   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 232, in inner
Feb 27 09:36:10 node nova-conductor[957]:     return func(*args, **kwargs)
Feb 27 09:36:10 node nova-conductor[957]:   File "/opt/stack/nova/nova/scheduler/manager.py", line 137, in select_destinations
Feb 27 09:36:10 node nova-conductor[957]:     raise exception.NoValidHost(reason="")
Feb 27 09:36:10 node nova-conductor[957]: NoValidHost: No valid host was found.
Feb 27 09:36:10 node nova-conductor[957]: ERROR nova.conductor.manager #033[01;35m#033[00mTraceback (most recent call last):
Feb 27 09:36:10 node nova-conductor[957]: ERROR nova.conductor.manager #033[01;35m#033[00m  File "/opt/stack/nova/nova/conductor/manager.py", line 1049, in schedule_and_build_instances
Feb 27 09:36:10 node nova-conductor[957]: ERROR nova.conductor.manager #033[01 ...
(more)
edit flag offensive delete link more

Comments

API /compute/v2.1/flavors/m1.smallis incorrect and provokes the message in the log file. The API requires the flavor's UUID, not its name (https://developer.openstack.org/api-r...). But I wonder if this is related to your problem.

Bernd Bausch gravatar imageBernd Bausch ( 2018-02-27 17:53:26 -0600 )edit

The real problem has to do with the Placement Service. I need to ask you to look at other logs, namely the Placement Service log devstack@placement-api and the Nova Compute log devstack@n-cpu. I am adding an answer to your question, although I don't have a definite answer yet.

Bernd Bausch gravatar imageBernd Bausch ( 2018-02-27 17:54:19 -0600 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

2 followers

Stats

Asked: 2018-02-20 04:23:32 -0600

Seen: 1,315 times

Last updated: Feb 27 '18