I have been running a small Packstack Stein setup with 1 Controller/Network node and 6 Compute nodes. Everything was working fine for a week or so, and then I rebooted the Controller node. After that reboot, none of the VMs I had created will bind to their DHCP addresses. Also, when I try to create a new VM, Horizon gives this error:

Error: Failed to perform requested operation on instance "demo-ubuntu-1", the instance has an error status: Please try again later [Error: Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for instance a69324fb-6d8f-48db-879e-3ef7eb662e72. Last exception: Binding failed for port d5b75470-312f-43b2-9227-b7d9addd58ed, please check neutron logs for more information.].

The Neutron server.log shows this corresponding error:

2019-07-26 20:22:25.053 6556 ERROR neutron.plugins.ml2.managers [req-b95bda3

d-b022-464e-a0c9-90d8030e5141 66c1ccba9ff34e05bc28e503c3058f3d 4ab1c3975b3e4

6fba252f1fe38fd067d - default default] Failed to bind port d5b75470-312f-43b

2-9227-b7d9addd58ed on host shp-vmc-06-man for vnic_

type normal using segments [{'network_id': '64d106db-376b-4736-9e38-f820a9fc

87e3', 'segmentation_id': 50, 'physical_network': None, 'id': 'efe44ba5-d98f

-44d3-ac40-8b33d06fe6f3', 'network_type':

More Neutron server.log info from DEBUG mode:

2019-07-29 11:34:08.910 6536 DEBUG neutron.plugins.ml2.managers [req-07fc39e0-b191-4b8a-83e0-b7c2c8599498 66c1ccba9ff34e05bc28e503c3058f3d 4ab1c3975b3e46fba252f1fe38fd067d - default default] Attempting to bind port caa7070a-8e71-482c-87d9-a0e9deac5106 on host shp-vmc-06-man for vnic_type normal with profile  bind_port /usr/lib/python2.7/site-packages/neutron/plugins/ml2/managers.py:765
2019-07-29 11:34:08.911 6536 DEBUG neutron.plugins.ml2.managers [req-07fc39e0-b191-4b8a-83e0-b7c2c8599498 66c1ccba9ff34e05bc28e503c3058f3d 4ab1c3975b3e46fba252f1fe38fd067d - default default] Attempting to bind port caa7070a-8e71-482c-87d9-a0e9deac5106 by drivers ovn on host shp-vmc-06-man at level 0 using segments [{'network_id': '64d106db-376b-4736-9e38-f820a9fc87e3', 'segmentation_id': 50, 'physical_network': None, 'id': 'efe44ba5-d98f-44d3-ac40-8b33d06fe6f3', 'network_type': u'geneve'}] _bind_port_level /usr/lib/python2.7/site-packages/neutron/plugins/ml2/managers.py:792
2019-07-29 11:34:08.912 6536 DEBUG networking_ovn.ml2.mech_driver [req-07fc39e0-b191-4b8a-83e0-b7c2c8599498 66c1ccba9ff34e05bc28e503c3058f3d 4ab1c3975b3e46fba252f1fe38fd067d - default default] Refusing to bind port caa7070a-8e71-482c-87d9-a0e9deac5106 due to no OVN chassis for host: shp-vmc-06-man bind_port /usr/lib/python2.7/site-packages/networking_ovn/ml2/mech_driver.py:634

Again, the DHCP binding was working fine until I rebooted... Any ideas on what could be wrong, or where I could start looking?

I am not running Cinder in this experiment.

Here is the output of openstack compute service list:

[root@shp-vmc-00-man ~]# openstack compute service list
+----+------------------+----------------+----------+---------+-------+----------------------------+
| ID | Binary           | Host           | Zone     | Status  | State | Updated At                 |
+----+------------------+----------------+----------+---------+-------+----------------------------+
| 10 | nova-conductor   | shp-vmc-00-man | internal | enabled | up    | 2019-07-29T14:14:59.000000 |
| 21 | nova-scheduler   | shp-vmc-00-man | internal | enabled | up    | 2019-07-29T14:14:59.000000 |
| 31 | nova-consoleauth | shp-vmc-00-man | internal | enabled | up    | 2019-07-29T14:14:58.000000 |
| 32 | nova-compute     | shp-vmc-01-man | nova     | enabled | up    | 2019-07-29T14:15:00.000000 |
| 33 | nova-compute     | shp-vmc-02-man | nova     | enabled | up    | 2019-07-29T14:14:57.000000 |
| 34 | nova-compute     | shp-vmc-04-man | nova     | enabled | up    | 2019-07-29T14:14:57.000000 |
| 35 | nova-compute     | shp-vmc-05-man | nova     | enabled | up    | 2019-07-29T14:14:57.000000 |
| 36 | nova-compute     | shp-vmc-06-man | nova     | enabled | up    | 2019-07-29T14:15:00.000000 |
| 37 | nova-compute     | shp-vmc-03-vm  | nova     | enabled | up    | 2019-07-29T14:15:00.000000 |
+----+------------------+----------------+----------+---------+-------+----------------------------+

Here is the output of openstack network agent list:

[root@shp-vmc-00-man ~]# openstack network agent list

+--------------------------------------+----------------------+----------------+-------------------+-------+-------+-------------------------------+
| ID                                   | Agent Type           | Host           | Availability Zone | Alive | State | Binary                        |
+--------------------------------------+----------------------+----------------+-------------------+-------+-------+-------------------------------+
| 13b83a90-acd9-4c77-84ed-1f7caa2e2a5e | DHCP agent           | shp-vmc-00-man | nova              | XXX   | UP    | neutron-dhcp-agent            |
| dbf1018e-4178-48d0-a360-b315790897ef | OVN Controller agent | localhost      | n/a               | :-)   | UP    | ovn-controller                |
| 40dad5f7-b8d4-4c67-8376-2ba6672cf1fa | OVN Metadata agent   | localhost      | n/a               | :-)   | UP    | networking-ovn-metadata-agent |
| 37564b7a-8ea7-47e1-a2d3-dbd692728f05 ...