Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Openstack Pike: how to re-create the cell_v0 db? (another variation of the dreaded "NoValidHost" error)

Hi folks,

I've been tearing my hair out over this problem for the last few days now. It looks like I have some stray / leftover entries in my cell_v0 db that prevent me from re-initializing the db properly again, the end result being that I cannot launch any instances on my compute node (set up using packstack on centos7, all in one so it's a single physical server setup for now).

I've tried stopping the compute services including nova-api, dropping the nova_cell0 db and setting it up again according to the pike documentation, but running nova-manage cell_v0 discover_hosts doesn't find any compute nodes and the compute_nodes table in nova_cell0 remains empty. (I don't even know if this is an error or not)

When trying to launch an instance, I see the following entries in my nova logs (lDEBUG=True in nova.conf):

2017-10-31 08:29:32.215 5052 WARNING nova.image.glance [req-919c1e78-4e87-4ce6-b73b-25546c3f539e 59d48c491afd46de9ca19f3f536ed35c f46090ede3d346f4957aafa7492e8e87 - default default] No protocol specified in for api_server 'http://10.129.36.244:9292', please update [glance] api_servers with fully qualified url including scheme (http / https): NoValidHost_Remote: No valid host was found. There are not enough hosts available.

Filtering debug output also shows no hosts being selected even at the top of the filter chain:

nova-scheduler.log:2017-10-31 08:29:32.077 7830 INFO nova.filters [req-919c1e78-4e87-4ce6-b73b-25546c3f539e 59d48c491afd46de9ca19f3f536ed35c f46090ede3d346f4957aafa7492e8e87 - default default] Filter RetryFilter returned 0 hosts
nova-scheduler.log:2017-10-31 08:29:32.077 7830 DEBUG nova.filters [req-919c1e78-4e87-4ce6-b73b-25546c3f539e 59d48c491afd46de9ca19f3f536ed35c f46090ede3d346f4957aafa7492e8e87 - default default] Filtering removed all hosts for the request with instance ID '7e6aa5ce-3a47-4fff-85ff-65cebdb48e53'. Filter results: [('RetryFilter', None)] get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:129

My hypervisor gets listed fine though, all services appear to be running ok:

 +--------------------------------------+-----------------------------------+-------+---------+
    | ID                                   | Hypervisor hostname               | State | Status  |
    +--------------------------------------+-----------------------------------+-------+---------+
    | 906b994a-8459-4a49-8afe-e1a77f5b5fbd | host.example.com | up    | enabled |
    +--------------------------------------+-----------------------------------+-------+---------+

+--------------------------------------+------------------+-----------------------------------+----------+---------+-------+----------------------------+-----------------+-------------+
| Id                                   | Binary           | Host                              | Zone     | Status  | State | Updated_at                 | Disabled Reason | Forced down |
+--------------------------------------+------------------+-----------------------------------+----------+---------+-------+----------------------------+-----------------+-------------+

| cb2caa6e-284b-417f-af0d-11393730eda1 | nova-conductor   | denios03.arvato-systems-media.net | internal | enabled | up    | 2017-10-31T08:24:09.000000 | -               | False       |
| d688efa3-8d0f-4a41-8f93-7e7a4fefdb92 | nova-compute     | host.example.com | nova     | enabled | up    | 2017-10-31T08:24:06.000000 | -               | False       |
| 873ae55a-d26a-4b64-9223-e399eac314ed | nova-scheduler   | host.example.com | internal | enabled | up    | 2017-10-31T08:24:06.000000 | -               | False       |
| d4d858b8-52ad-444a-8afa-e2bd900227b7 | nova-consoleauth | host.example.com| internal | enabled | up    | 2017-10-31T08:24:09.000000 | -               | False       |
+--------------------------------------+------------------+-----------------------------------+----------+---------+-------+----------------------------+-----------------+-------------+

I'm at my wit's end with this issue, I've even set up another openstack pike instance from packstack (ironically on another openstack "newton" host that runs fine and which isn't using the new placement stuff) to see if it's ok that the nova_placement DB has no tables at all (it doesn't) etc, so any help on this would be much appreciated. I

All the best, Uwe