Yet another "no valid host available"

asked 2017-05-01 13:10:17 -0600

tiger gravatar image

I successfully launched four m1.medium. Henceforth, any new instance turns up as ERROR. I'll answer the usual battery of questions that get asked.

[root@kvm8 ~(keystone_admin)]# virt-host-validate
  QEMU: Checking for hardware virtualization                                 : PASS
  QEMU: Checking if device /dev/kvm exists                                   : PASS
  QEMU: Checking if device /dev/kvm is accessible                            : PASS
  QEMU: Checking if device /dev/vhost-net exists                             : PASS
  QEMU: Checking if device /dev/net/tun exists                               : PASS
  QEMU: Checking for cgroup 'memory' controller support                      : PASS
  QEMU: Checking for cgroup 'memory' controller mount-point                  : PASS
  QEMU: Checking for cgroup 'cpu' controller support                         : PASS
  QEMU: Checking for cgroup 'cpu' controller mount-point                     : PASS
  QEMU: Checking for cgroup 'cpuacct' controller support                     : PASS
  QEMU: Checking for cgroup 'cpuacct' controller mount-point                 : PASS
  QEMU: Checking for cgroup 'cpuset' controller support                      : PASS
  QEMU: Checking for cgroup 'cpuset' controller mount-point                  : PASS
  QEMU: Checking for cgroup 'devices' controller support                     : PASS
  QEMU: Checking for cgroup 'devices' controller mount-point                 : PASS
  QEMU: Checking for cgroup 'blkio' controller support                       : PASS
  QEMU: Checking for cgroup 'blkio' controller mount-point                   : PASS
  QEMU: Checking for device assignment IOMMU support                         : PASS
  QEMU: Checking if IOMMU is enabled by kernel                               : WARN (IOMMU appears to be disabled in kernel. Add intel_iommu=on to kernel cmdline arguments)
   LXC: Checking for Linux >= 2.6.26                                         : PASS
   LXC: Checking for namespace ipc                                           : PASS
   LXC: Checking for namespace mnt                                           : PASS
   LXC: Checking for namespace pid                                           : PASS
   LXC: Checking for namespace uts                                           : PASS
   LXC: Checking for namespace net                                           : PASS
   LXC: Checking for namespace user                                          : PASS
   LXC: Checking for cgroup 'memory' controller support                      : PASS
   LXC: Checking for cgroup 'memory' controller mount-point                  : PASS
   LXC: Checking for cgroup 'cpu' controller support                         : PASS
   LXC: Checking for cgroup 'cpu' controller mount-point                     : PASS
   LXC: Checking for cgroup 'cpuacct' controller support                     : PASS
   LXC: Checking for cgroup 'cpuacct' controller mount-point                 : PASS
   LXC: Checking for cgroup 'cpuset' controller support                      : PASS
   LXC: Checking for cgroup 'cpuset' controller mount-point                  : PASS
   LXC: Checking for cgroup 'devices' controller support                     : PASS
   LXC: Checking for cgroup 'devices' controller mount-point                 : PASS
   LXC: Checking for cgroup 'blkio' controller support                       : PASS
   LXC: Checking for cgroup 'blkio' controller mount-point                   : PASS


[root@kvm8 ~(keystone_admin)]# nova service-list
/usr/lib/python2.7/site-packages/novaclient/client.py:278: UserWarning: The 'tenant_id' argument is deprecated in Ocata and its use may result in errors in future releases. As 'project_id' is provided, the 'tenant_id' argument will be ignored.
  warnings.warn(msg)
+----+------------------+----------------------+----------+---------+-------+----------------------------+-----------------+
| Id | Binary           | Host                 | Zone     | Status  | State | Updated_at                 | Disabled Reason |
+----+------------------+----------------------+----------+---------+-------+----------------------------+-----------------+
| 9  | nova-cert        | kvm8.mia.ccncore.net | internal | enabled | up    | 2017-05-01T18:05:33.000000 | -               |
| 10 | nova-conductor   | kvm8.mia.ccncore.net | internal | enabled | up    | 2017-05-01T18:05:39.000000 | -               |
| 24 | nova-scheduler   | kvm8.mia.ccncore.net | internal | enabled | up    | 2017-05-01T18:05:37.000000 | -               |
| 25 | nova-consoleauth | kvm8.mia.ccncore.net | internal | enabled | up    | 2017-05-01T18:05:35.000000 | -               |
| 26 | nova-compute     | kvm8.mia.ccncore.net | nova     | enabled | up    | 2017-05-01T18:05:33.000000 | -               |
+----+------------------+----------------------+----------+---------+-------+----------------------------+-----------------+
[root@kvm8 ~(keystone_admin)]# nova hypervisor-show 1
/usr/lib/python2.7/site-packages/novaclient/client.py:278: UserWarning: The 'tenant_id' argument is deprecated in Ocata and its use may result in errors in future releases. As 'project_id' is provided, the 'tenant_id' argument will be ignored.
  warnings.warn ...
(more)
edit retag flag offensive close merge delete

Comments

OK, have set "debug=True" in /etc/nova/nova.conf Restarted NOVA Now in /var/log/nova/nova.conf I see some clues. Seems like it thought I was out of disk (I am not) so I set disk_allocation_ratio to 3.0. This let me instantiate a fifth VM but now the nova.log says "RetryFilter returns 0 hosts"

tiger gravatar imagetiger ( 2017-05-01 14:29:53 -0600 )edit

The retry filter is used in this scenario:

  • the scheduler found a compute host but
  • the instance failed to start on that host

The retry filter then removes that failed host from the list.

You should find out why the instance failed to start, and why no other hosts pass the filters.

Bernd Bausch gravatar imageBernd Bausch ( 2017-05-01 22:47:48 -0600 )edit

It believes it is out of disk space. How can I force OpenStack to use a recently created volume? I blew away 'cinder-volumes' then recreated it as /dev/loop0 with 'dd if=/dev/zero' to create a 400GB filesystem file. There is yet something else that has to be done for OpenStack to recognize the space

tiger gravatar imagetiger ( 2017-05-04 08:46:22 -0600 )edit