Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Liberty: NoValidHost: No valid host was found. There are not enough hosts available.

I am able to launch "small" instances that require only a small disk, but spawning instances with larger disk fails. We are using a 1 TB shared storage mounted into all 3 compute nodes via NFS. When "exceeding" the disk limit, we get the following exception:

NoValidHost: No valid host was found. There are not enough hosts available.

It seems that all three compute nodes are filtered out by nova:

Starting with 3 host(s) get_filtered_objects /usr/lib/python2.7/dist-packages/nova/filters.py:70
Filter RetryFilter returned 3 host(s) get_filtered_objects /usr/lib/python2.7/dist-packages/nova/filters.py:104
Filter AvailabilityZoneFilter returned 3 host(s) get_filtered_objects /usr/lib/python2.7/dist-packages/nova/filters.py:104
Filter RamFilter returned 3 host(s) get_filtered_objects /usr/lib/python2.7/dist-packages/nova/filters.py:104
(mos-compute02, mos-compute02.localdomain) ram:29103 disk:13312 io_ops:0 instances:2 does not have 81920 MB usable disk, it only has 13312.0 MB usable disk. host_passes /usr/lib/python2.7/dist-packages/nova/scheduler/filters/disk_filter.py:60
(mos-compute01, mos-compute01.localdomain) ram:29611 disk:14336 io_ops:0 instances:1 does not have 81920 MB usable disk, it only has 14336.0 MB usable disk. host_passes /usr/lib/python2.7/dist-packages/nova/scheduler/filters/disk_filter.py:60
(mos-compute03, mos-compute03.localdomain) ram:29103 disk:13312 io_ops:0 instances:2 does not have 81920 MB usable disk, it only has 13312.0 MB usable disk. host_passes /usr/lib/python2.7/dist-packages/nova/scheduler/filters/disk_filter.py:60
Filter DiskFilter returned 0 hosts
Filtering removed all hosts for the request with reservation ID 'r-wyf01la7' and instance ID '4035e2b9-1ccc-454e-b5a5-e0b785e78c92'. Filter results: [('RetryFilter', [(u'mos-compute02', u'mos-compute02.localdomain'), (u'mos-compute01', u'mos-compute01.localdomain'), (u'mos-compute03', u'mos-compute03.localdomain')]), ('AvailabilityZoneFilter', [(u'mos-compute02', u'mos-compute02.localdomain'), (u'mos-compute01', u'mos-compute01.localdomain'), (u'mos-compute03', u'mos-compute03.localdomain')]), ('RamFilter', [(u'mos-compute02', u'mos-compute02.localdomain'), (u'mos-compute01', u'mos-compute01.localdomain'), (u'mos-compute03', u'mos-compute03.localdomain')]), ('DiskFilter', None)] get_filtered_objects /usr/lib/python2.7/dist-packages/nova/filters.py:122
Filtering removed all hosts for the request with reservation ID 'r-wyf01la7' and instance ID '4035e2b9-1ccc-454e-b5a5-e0b785e78c92'. Filter results: ['RetryFilter: (start: 3, end: 3)', 'AvailabilityZoneFilter: (start: 3, end: 3)', 'RamFilter: (start: 3, end: 3)', 'DiskFilter: (start: 3, end: 0)']
There are 0 hosts available but 1 instances requested to build. select_destinations /usr/lib/python2.7/dist-packages/nova/scheduler/filter_scheduler.py:87

nova hypervisor-stats:

+----------------------+-------+
| Property             | Value |
+----------------------+-------+
| count                | 3     |
| current_workload     | 0     |
| disk_available_least | 40    |
| free_disk_gb         | 52    |
| free_ram_mb          | 87817 |
| local_gb             | 114   |
| local_gb_used        | 62    |
| memory_mb            | 96521 |
| memory_mb_used       | 8704  |
| running_vms          | 5     |
| vcpus                | 24    |
| vcpus_used           | 5     |
+----------------------+-------+

The hypervisor tells Nova:

Hypervisor/Node resource view: name=mos-compute01.localdomain free_ram=31216MB free_disk=36GB ...

I think the free_disk is the available physical space on the system disk of the compute node, but how can I tell Nova that the available space for instance disk files is about 1 TB?