neutron (Juno) l3 HA test failed
Hello. I'm testing neutron (Juno) L3 HA feature with following configurations, with one controller node and two network nodes. All the nodes are Centos 7 minimal installation.
controller node
/etc/neutron/neutron.conf
[DEFAULT]
verbose = True
core_plugin = ml2
service_plugins = router
auth_strategy = keystone
l3_ha = True
max_l3_agents_per_router = 3
min_l3_agents_per_router = 2
notify_nova_on_port_status_changes = True
notify_nova_on_port_data_changes = True
nova_url = http://controller:8774/v2
nova_region_name = regionOne
nova_admin_username = nova
nova_admin_tenant_id = 3c5abd3469af433db8fe2047c6d62033
nova_admin_password = NOVA_PASS
rabbit_host=localhost
rabbit_userid=guest
rabbit_password=RABBIT_PASS
rpc_backend=rabbit
[matchmaker_redis]
[matchmaker_ring]
[quotas]
[agent]
root_helper = sudo neutron-rootwrap /etc/neutron/rootwrap.conf
[keystone_authtoken]
auth_uri=http://controller:5000/v2.0
identity_uri=http://controller:35357
admin_tenant_name = service
admin_user = neutron
admin_password = NEUTRON_PASS
[database]
connection = mysql://neutron:NEUTRON_PASS@controller/neutron
[service_providers]
/etc/neutron/plugin.ini
[ml2]
type_drivers = flat,vlan
tenant_network_types = vlan
mechanism_drivers = openvswitch
[ml2_type_flat]
[ml2_type_vlan]
network_vlan_ranges = default:400:1000
[ml2_type_gre]
[ml2_type_vxlan]
[securitygroup]
enable_security_group = True
network node
/etc/neutron/neutron.conf
[DEFAULT]
verbose = True
core_plugin = ml2
service_plugins = router
auth_strategy = neutron
l3_ha = True
max_l3_agents_per_router = 3
min_l3_agents_per_router = 2
rabbit_host=controller
rabbit_password=RABBIT_PASS
rpc_backend=rabbit
[matchmaker_redis]
[matchmaker_ring]
[quotas]
[agent]
root_helper = sudo neutron-rootwrap /etc/neutron/rootwrap.conf
[keystone_authtoken]
auth_uri = http://controller:5000/v2.0
identity_uri = http://controller:35357
admin_tenant_name = service
admin_user = neutron
admin_password = NEUTRON_PASS
[database]
[service_providers]
/etc/neutron/plugin.ini
[ml2]
type_drivers = flat,vlan
tenant_network_types = vlan
mechanism_drivers = openvswitch
[ml2_type_flat]
[ml2_type_vlan]
network_vlan_ranges = default:400:1000
[ml2_type_gre]
[ml2_type_vxlan]
[securitygroup]
enable_security_group = True
/etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini
[ovs]
tenant_network_type = vlan
network_vlan_ranges = default:400:1000
enable_tunneling = False
bridge_mappings = default:br-ens2f0,external:br-ex
[agent]
[securitygroup]
firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
enable_security_group = True
/etc/neutron/l3_agent.ini
[DEFAULT]
verbose = True
interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver
use_namespaces = True
gateway_external_network_id = ea0937e7-9cc3-4f4e-ba84-b29d1b718a84
external_network_bridge = br-ex
ha_confs_path = $state_path/ha_confs
ha_vrrp_auth_type = PASS
ha_vrrp_auth_password = ABC
ha_vrrp_advert_int = 2
With these configurations, on creating a router with HA enabled, following messages are shown on the controller.
+-----------------------+--------------------------------------+
| Field | Value |
+-----------------------+--------------------------------------+
| admin_state_up | True |
| distributed | False |
| external_gateway_info | |
| ha | True |
| id | f3ace776-91d6-4528-b603-9011db11f470 |
| name | demo-router |
| routes | |
| status | ACTIVE |
| tenant_id | b2e4e2e598614b5dbd878ae976728630 |
+-----------------------+--------------------------------------+
However, in the /var/log/neutron/server.log, you can find error messages indicating port binding failures:
2014-12-29 11:57:17.702 26747 INFO neutron.db.l3_hamode_db [req-7b7135d6-e3a8-4841-baca-0b786d0f1b78 None] Number of available agents lower than max_l3_agents_per_router. L3 agents available: 2
2014-12-29 11:57:17.950 26747 INFO neutron.wsgi [req-7b7135d6-e3a8-4841-baca-0b786d0f1b78 None] 10.24.148.21 - - [29/Dec/2014 11:57:17] "POST /v2.0/routers.json HTTP/1.1" 201 448 0.320873
2014-12-29 11:57:18.181 26747 WARNING neutron.plugins.ml2.managers [req-8c31ef8e-2452-42af-b590-0015b45a325e None] Failed to bind port 7ec5cb76-94c6-4e2e-a687-606d8fb34ce7 on host network1
2014-12-29 11:57:18.206 26747 WARNING neutron.plugins.ml2.plugin [req-8c31ef8e-2452-42af-b590-0015b45a325e None] In _notify_port_updated(), no bound segment for port 7ec5cb76-94c6-4e2e-a687-606d8fb34ce7 on network 46725d33-cd6d-418b-9574-d45cf7e6e340
2014-12-29 11:57:18.208 26747 WARNING neutron.plugins.ml2.managers [req-ae7d921b-8f89-466c-9023-73d8edb720ca None] Failed to bind port bc42bc0d-d1f7-4440-8704-757a47cee268 on host network2
2014-12-29 11:57:18.227 26747 WARNING neutron.plugins.ml2.plugin [req-ae7d921b-8f89-466c-9023-73d8edb720ca None] In _notify_port_updated(), no bound segment for port bc42bc0d-d1f7-4440-8704-757a47cee268 on network 46725d33-cd6d-418b-9574-d45cf7e6e340
2014-12-29 11:57:20.756 26747 WARNING neutron.plugins.ml2.rpc [req-3b378bc3-7378-4d61-b4dd-832afbe0f941 None] Device 7ec5cb76-94c6-4e2e-a687-606d8fb34ce7 requested by agent ovs-agent-network1 on network 46725d33-cd6d-418b-9574-d45cf7e6e340 not bound, vif_type: binding_failed
2014-12-29 11:57:21.574 ...