Openstack Pike: how to re-create the cell_v0 db? (another variation of the dreaded "NoValidHost" error)

asked 2017-10-31 03:40:04 -0500

hoover1967 gravatar image

Hi folks,

I've been tearing my hair out over this problem for the last few days now. It looks like I have some stray / leftover entries in my cell_v0 db that prevent me from re-initializing the db properly again, the end result being that I cannot launch any instances on my compute node (set up using packstack on centos7, all in one so it's a single physical server setup for now).

I've tried stopping the compute services including nova-api, dropping the nova_cell0 db and setting it up again according to the pike documentation, but running nova-manage cell_v0 discover_hosts doesn't find any compute nodes and the compute_nodes table in nova_cell0 remains empty. (I don't even know if this is an error or not)

When trying to launch an instance, I see the following entries in my nova logs (lDEBUG=True in nova.conf):

2017-10-31 08:29:32.215 5052 WARNING nova.image.glance [req-919c1e78-4e87-4ce6-b73b-25546c3f539e 59d48c491afd46de9ca19f3f536ed35c f46090ede3d346f4957aafa7492e8e87 - default default] No protocol specified in for api_server 'http://10.129.36.244:9292', please update [glance] api_servers with fully qualified url including scheme (http / https): NoValidHost_Remote: No valid host was found. There are not enough hosts available.

Filtering debug output also shows no hosts being selected even at the top of the filter chain:

nova-scheduler.log:2017-10-31 08:29:32.077 7830 INFO nova.filters [req-919c1e78-4e87-4ce6-b73b-25546c3f539e 59d48c491afd46de9ca19f3f536ed35c f46090ede3d346f4957aafa7492e8e87 - default default] Filter RetryFilter returned 0 hosts
nova-scheduler.log:2017-10-31 08:29:32.077 7830 DEBUG nova.filters [req-919c1e78-4e87-4ce6-b73b-25546c3f539e 59d48c491afd46de9ca19f3f536ed35c f46090ede3d346f4957aafa7492e8e87 - default default] Filtering removed all hosts for the request with instance ID '7e6aa5ce-3a47-4fff-85ff-65cebdb48e53'. Filter results: [('RetryFilter', None)] get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:129

My hypervisor gets listed fine though, all services appear to be running ok:

 +--------------------------------------+-----------------------------------+-------+---------+
    | ID                                   | Hypervisor hostname               | State | Status  |
    +--------------------------------------+-----------------------------------+-------+---------+
    | 906b994a-8459-4a49-8afe-e1a77f5b5fbd | host.example.com | up    | enabled |
    +--------------------------------------+-----------------------------------+-------+---------+

+--------------------------------------+------------------+-----------------------------------+----------+---------+-------+----------------------------+-----------------+-------------+
| Id                                   | Binary           | Host                              | Zone     | Status  | State | Updated_at                 | Disabled Reason | Forced down |
+--------------------------------------+------------------+-----------------------------------+----------+---------+-------+----------------------------+-----------------+-------------+

| cb2caa6e-284b-417f-af0d-11393730eda1 | nova-conductor   | denios03.arvato-systems-media.net | internal | enabled | up    | 2017-10-31T08:24:09.000000 | -               | False       |
| d688efa3-8d0f-4a41-8f93-7e7a4fefdb92 | nova-compute     | host.example.com | nova     | enabled | up    | 2017-10-31T08:24:06.000000 | -               | False       |
| 873ae55a-d26a-4b64-9223-e399eac314ed | nova-scheduler   | host.example.com | internal | enabled | up    | 2017-10-31T08:24:06.000000 | -               | False       |
| d4d858b8-52ad-444a-8afa-e2bd900227b7 | nova-consoleauth | host.example.com| internal | enabled | up    | 2017-10-31T08:24:09.000000 | -               | False       |
+--------------------------------------+------------------+-----------------------------------+----------+---------+-------+----------------------------+-----------------+-------------+

I'm at my wit's end with this issue, I've even set up another openstack pike instance from packstack (ironically on another openstack "newton" host that runs fine and which isn't using the new placement stuff) to see if it's ok that the nova_placement DB has no tables at all (it doesn't) etc, so any help on this would be much appreciated. I

All the best, Uwe

edit retag flag offensive close merge delete

Comments

But this error message is referred to glance: "No protocol specified in for api_server 'http://10.129.36.244:9292', please update [glance] api_servers with fully qualified url including scheme" I would recommend to first fix this issue because nova needs to contact glance.

eblock gravatar imageeblock ( 2017-11-01 07:04:59 -0500 )edit

Although nova-manage cell_v0 discover_hosts did work for me the table "compute_nodes" in nova_cell0 database is also empty. I recently upgraded to Ocata.

eblock gravatar imageeblock ( 2017-11-01 07:08:13 -0500 )edit