"Newton" - Filter RetryFilter returned 0 hosts

asked 2017-07-25 15:58:28 -0500

tiger gravatar image

Running PackStack all-in-one. Can't instantiate a Cirros VM. Set debug=true in /etc/nova/novs.conf, restarted nova services, retried to instantiate and see all the filters it passes (returns 1 host) but then at the end "RetryFilter returned 0 hosts". It mentions NUMA topology but I have already removed / reinserted KVM kernel modules and even rebooted. Any ideas? Logfile follows..

2017-07-25 20:47:38.236 1585 DEBUG nova.filters [req-6f7041ba-e36d-4cdb-8e63-d2a8ce878ea7 ff0b41615dd749e39267a5f6ead17b19 32bbb96442094b91813ec556dfd4ff30 - - -] Starting with 1 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:70
2017-07-25 20:47:38.236 1585 DEBUG nova.scheduler.filters.retry_filter [req-6f7041ba-e36d-4cdb-8e63-d2a8ce878ea7 ff0b41615dd749e39267a5f6ead17b19 32bbb96442094b91813ec556dfd4ff30 - - -] Re-scheduling is disabled host_passes /usr/lib/python2.7/site-packages/nova/scheduler/filters/retry_filter.py:34
2017-07-25 20:47:38.236 1585 DEBUG nova.filters [req-6f7041ba-e36d-4cdb-8e63-d2a8ce878ea7 ff0b41615dd749e39267a5f6ead17b19 32bbb96442094b91813ec556dfd4ff30 - - -] Filter RetryFilter returned 1 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:104
2017-07-25 20:47:38.237 1585 DEBUG nova.filters [req-6f7041ba-e36d-4cdb-8e63-d2a8ce878ea7 ff0b41615dd749e39267a5f6ead17b19 32bbb96442094b91813ec556dfd4ff30 - - -] Filter AvailabilityZoneFilter returned 1 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:104
2017-07-25 20:47:38.237 1585 DEBUG nova.filters [req-6f7041ba-e36d-4cdb-8e63-d2a8ce878ea7 ff0b41615dd749e39267a5f6ead17b19 32bbb96442094b91813ec556dfd4ff30 - - -] Filter RamFilter returned 1 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:104
2017-07-25 20:47:38.237 1585 DEBUG nova.filters [req-6f7041ba-e36d-4cdb-8e63-d2a8ce878ea7 ff0b41615dd749e39267a5f6ead17b19 32bbb96442094b91813ec556dfd4ff30 - - -] Filter DiskFilter returned 1 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:104
2017-07-25 20:47:38.238 1585 DEBUG nova.filters [req-6f7041ba-e36d-4cdb-8e63-d2a8ce878ea7 ff0b41615dd749e39267a5f6ead17b19 32bbb96442094b91813ec556dfd4ff30 - - -] Filter ComputeFilter returned 1 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:104
2017-07-25 20:47:38.238 1585 DEBUG nova.filters [req-6f7041ba-e36d-4cdb-8e63-d2a8ce878ea7 ff0b41615dd749e39267a5f6ead17b19 32bbb96442094b91813ec556dfd4ff30 - - -] Filter ComputeCapabilitiesFilter returned 1 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:104
2017-07-25 20:47:38.238 1585 DEBUG nova.filters [req-6f7041ba-e36d-4cdb-8e63-d2a8ce878ea7 ff0b41615dd749e39267a5f6ead17b19 32bbb96442094b91813ec556dfd4ff30 - - -] Filter ImagePropertiesFilter returned 1 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:104
2017-07-25 20:47:38.239 1585 DEBUG nova.filters [req-6f7041ba-e36d-4cdb-8e63-d2a8ce878ea7 ff0b41615dd749e39267a5f6ead17b19 32bbb96442094b91813ec556dfd4ff30 - - -] Filter ServerGroupAntiAffinityFilter returned 1 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:104
2017-07-25 20:47:38.239 1585 DEBUG nova.filters [req-6f7041ba-e36d-4cdb-8e63-d2a8ce878ea7 ff0b41615dd749e39267a5f6ead17b19 32bbb96442094b91813ec556dfd4ff30 - - -] Filter ServerGroupAffinityFilter returned 1 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:104
2017-07-25 20:47:38.239 1585 DEBUG nova.filters [req-6f7041ba-e36d-4cdb-8e63-d2a8ce878ea7 ff0b41615dd749e39267a5f6ead17b19 32bbb96442094b91813ec556dfd4ff30 - - -] Filter CoreFilter returned 1 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:104
2017-07-25 20:47:38.240 1585 DEBUG nova.scheduler.filter_scheduler [req-6f7041ba-e36d-4cdb-8e63-d2a8ce878ea7 ff0b41615dd749e39267a5f6ead17b19 32bbb96442094b91813ec556dfd4ff30 - - -] Filtered [(kvm8.ims1.mia.ccncore.net, kvm8.ims1.mia.ccncore.net) ram: 97779MB disk: 514048MB io_ops: 0 instances: 0] _schedule /usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py:115
2017-07-25 20:47:38.240 1585 DEBUG nova.scheduler.filter_scheduler [req-6f7041ba-e36d-4cdb-8e63-d2a8ce878ea7 ff0b41615dd749e39267a5f6ead17b19 32bbb96442094b91813ec556dfd4ff30 - - -] Weighed [WeighedHost [host: (kvm8.ims1.mia.ccncore.net, kvm8.ims1.mia.ccncore.net) ram: 97779MB disk: 514048MB io_ops: 0 instances: 0, weight: 0.0]] _schedule /usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py:120
2017-07-25 20:47:38.240 1585 DEBUG nova.scheduler.filter_scheduler [req-6f7041ba-e36d-4cdb-8e63-d2a8ce878ea7 ...
(more)
edit retag flag offensive close merge delete

Comments

Do you see the 5 seconds time difference between 20:47:38 and 20:47:43 near the end? During this time, kvm8 has tried to run the instance but failed, so that the filter process starts again with "Starting with 1 host". The retry filter removes the failed host to prevent an infinite failure loop.

Bernd Bausch gravatar imageBernd Bausch ( 2017-07-25 17:22:58 -0500 )edit

Check nova-compute and possibly other logs to understand why kvm8 fails to launch the VM.

Bernd Bausch gravatar imageBernd Bausch ( 2017-07-25 17:25:26 -0500 )edit