Failed to bind port on host
Hi, just made a fresh installation of the Liberty release on Ubuntu 14.04 and I am running into some network problem that I dont understand them. I think the biggest problem is: Failed to bind port 802862a2-223f-4e32-bc9c-b5821351f70b on host hv02
Services are running as seen:
neutron agent-list
+--------------------------------------+--------------------+---------------------+-------+----------------+---------------------------+
| id | agent_type | host | alive | admin_state_up | binary |
+--------------------------------------+--------------------+---------------------+-------+----------------+---------------------------+
| 0a1eb1ea-ec5d-4edd-b2eb-f8adf8545365 | L3 agent | controller02 | :-) | True | neutron-l3-agent |
| 0cb6076a-135a-42d6-8d73-4bd59f1dbffe | Metadata agent | controller01 | :-) | True | neutron-metadata-agent |
| 505c0d2d-15f9-4948-9e16-e9a9b0a01d87 | Linux bridge agent | controller02 | :-) | True | neutron-linuxbridge-agent |
| 637dc6d3-3c26-42c9-86a3-0246aa357c4f | DHCP agent | controller02 | :-) | True | neutron-dhcp-agent |
| adadaaa9-bce5-477e-a734-73a3c08f4fb8 | DHCP agent | controller01 | :-) | True | neutron-dhcp-agent |
| b7985827-ac34-4f38-9245-dc1767c3c008 | Linux bridge agent | hv03 | :-) | True | neutron-linuxbridge-agent |
| ef78d68b-20b5-4ea1-804e-f7befa8e2634 | Linux bridge agent | controller01 | :-) | True | neutron-linuxbridge-agent |
| f2038985-9f83-47e8-969d-2e91e04e02b7 | Metadata agent | controller02 | :-) | True | neutron-metadata-agent |
| f82682d9-018d-4cc4-9f94-8302e17be17a | L3 agent | controller01 | :-) | True | neutron-l3-agent |
| f8f419fc-38d9-470d-925e-fcd45778e32d | Linux bridge agent | hv01 | :-) | True | neutron-linuxbridge-agent |
| ff70f7cf-8c74-4642-b14c-32a2a23d91a9 | Linux bridge agent | hv02 | :-) | True | neutron-linuxbridge-agent |
+--------------------------------------+--------------------+---------------------+-------+----------------+---------------------------+
neutron net-list
+--------------------------------------+----------+------------------------------------------------------+
| id | name | subnets |
+--------------------------------------+----------+------------------------------------------------------+
| 78bba7fe-a27d-4000-bd28-704660d9c680 | Internal | d68bb402-da64-42db-897b-51f94e725e3f 172.16.101.0/24 |
+--------------------------------------+----------+------------------------------------------------------+
nova service-list
+----+------------------+---------------------+----------+---------+-------+----------------------------+-----------------+
| Id | Binary | Host | Zone | Status | State | Updated_at | Disabled Reason |
+----+------------------+---------------------+----------+---------+-------+----------------------------+-----------------+
| 1 | nova-cert | controller01 | internal | enabled | up | 2016-06-01T15:15:47.000000 | - |
| 2 | nova-cert | controller02 | internal | enabled | up | 2016-06-01T15:15:51.000000 | - |
| 3 | nova-consoleauth | controller01 | internal | enabled | up | 2016-06-01T15:15:51.000000 | - |
| 4 | nova-consoleauth | controller02 | internal | enabled | down | 2016-06-01T12:38:49.000000 | - |
| 5 | nova-scheduler | controller01 | internal | enabled | up | 2016-06-01T15:15:49.000000 | - |
| 6 | nova-scheduler | controller02 | internal | enabled | up | 2016-06-01T15:15:52.000000 | - |
| 7 | nova-conductor | controller01 | internal | enabled | up | 2016-06-01T15:15:53.000000 | - |
| 8 | nova-conductor | controller02 | internal | enabled | up | 2016-06-01T15:15:49.000000 | - |
| 9 | nova-compute | hv01-k58 | nova | enabled | up | 2016-06-01T15:15:48.000000 | - |
| 10 | nova-compute | hv02-k58 | nova | enabled | up | 2016-06-01T15:15:52.000000 | - |
| 11 | nova-compute | hv03-k58 | nova | enabled | up | 2016-06-01T15:15:50.000000 | - |
+----+------------------+---------------------+----------+---------+-------+----------------------------+-----------------+
When I start a machine I get following error messages:
neutron-server.log (controller node):
[...]
2016-06-01 16:24:10.164 9575 ERROR neutron.plugins.ml2.managers [req-00f88547-b3a3-40bc-9d1e-4bebdefa6fce 859ec55610584042aedd429d341a5242 77cddd1adacb4becb6c0a52188cf34bb - - -] Failed to bind port 802862a2-223f-4e32-bc9c-b5821351f70b on host hv02
2016-06-01 16:24:10.164 9575 ERROR neutron.plugins.ml2.managers [req-00f88547-b3a3-40bc-9d1e-4bebdefa6fce 859ec55610584042aedd429d341a5242 77cddd1adacb4becb6c0a52188cf34bb - - -] Failed to bind port 802862a2-223f-4e32-bc9c-b5821351f70b on host hv02
2016-06-01 16:24:10.165 9575 INFO neutron.plugins.ml2.plugin [req-00f88547-b3a3-40bc-9d1e-4bebdefa6fce 859ec55610584042aedd429d341a5242 77cddd1adacb4becb6c0a52188cf34bb - - -] Attempt 2 to bind port 802862a2-223f-4e32-bc9c-b5821351f70b
[...]
nova-conductor.log (controller node):
[...]
2016-06-01 16:24:12.142 1656 ERROR nova.scheduler.utils [req-ec201652-17bc-486c-b5be-7846112d692b d7d8a3b5814540a4b01da3412db9c49c 20f9af47fde64253a5975e072e09ab2e - - -] [instance: 1a0145bb-55f6-4ea9-9667-9e96876ea17e] Error from last host: hv02 (node hv02): [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1905, in _do_build_and_run_instance\n filter_properties)\n', u' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2058, in _build_and_run_instance\n instance_uuid=instance.uuid, reason=six.text_type(e))\n', u'RescheduledException: Build of instance 1a0145bb-55f6-4ea9-9667-9e96876ea17e was re-scheduled: Binding failed for port 802862a2-223f-4e32-bc9c-b5821351f70b, please check neutron logs for more information.\n']
[...]
nova-compute.log (compute node):
[...]
2016-06-01 16:24:24.577 5889 ERROR nova.compute.manager [req-ec201652-17bc-486c-b5be-7846112d692b d7d8a3b5814540a4b01da3412db9c49c 20f9af47fde64253a5975e072e09ab2e - - -] [instance: 1a0145bb-55f6-4ea9-9667-9e96876ea17e] Instance failed to spawn
2016-06-01 16:24:24.577 5889 ERROR nova.compute.manager [instance: 1a0145bb-55f6-4ea9-9667-9e96876ea17e] Traceback (most recent call last):
2016-06-01 16:24:24.577 5889 ERROR nova.compute.manager [instance: 1a0145bb-55f6-4ea9-9667-9e96876ea17e] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2156, in _build_resources
2016-06-01 ...
What is interesting that no bridge device is getting created on the controller node.
check
neutron agent-show $UUID-of-lbagent
, and the linuxbrige agent logs. Is the network vxlan or vlan or flat?Hi darragh-oreilly,
agent list: http://paste.openstack.org/show/507263/ linuxbridge message: http://paste.openstack.org/show/507264/
The network type is flat.
any idea why flat network is not working? I cant see the error.
try deleting the flat net and recreating it - this should get rid of the ports with binding failed