Ask Your Question
1

neutron (Juno) l3 HA test failed

asked 2014-12-28 21:35:28 -0500

Byungjoon Lee gravatar image

updated 2014-12-29 20:21:16 -0500

Hello. I'm testing neutron (Juno) L3 HA feature with following configurations, with one controller node and two network nodes. All the nodes are Centos 7 minimal installation.

controller node

/etc/neutron/neutron.conf

[DEFAULT]
verbose = True
core_plugin = ml2
service_plugins = router
auth_strategy = keystone
l3_ha = True
max_l3_agents_per_router = 3
min_l3_agents_per_router = 2
notify_nova_on_port_status_changes = True
notify_nova_on_port_data_changes = True
nova_url = http://controller:8774/v2
nova_region_name = regionOne
nova_admin_username = nova
nova_admin_tenant_id = 3c5abd3469af433db8fe2047c6d62033
nova_admin_password = NOVA_PASS
rabbit_host=localhost
rabbit_userid=guest
rabbit_password=RABBIT_PASS
rpc_backend=rabbit
[matchmaker_redis]
[matchmaker_ring]
[quotas]
[agent]
root_helper = sudo neutron-rootwrap /etc/neutron/rootwrap.conf
[keystone_authtoken]
auth_uri=http://controller:5000/v2.0
identity_uri=http://controller:35357
admin_tenant_name = service
admin_user = neutron
admin_password = NEUTRON_PASS
[database]
connection = mysql://neutron:NEUTRON_PASS@controller/neutron
[service_providers]

/etc/neutron/plugin.ini

[ml2]
type_drivers = flat,vlan
tenant_network_types = vlan
mechanism_drivers = openvswitch
[ml2_type_flat]
[ml2_type_vlan]
network_vlan_ranges = default:400:1000
[ml2_type_gre]
[ml2_type_vxlan]
[securitygroup]
enable_security_group = True

network node

/etc/neutron/neutron.conf

[DEFAULT]
verbose = True
core_plugin = ml2
service_plugins = router
auth_strategy = neutron
l3_ha = True
max_l3_agents_per_router = 3
min_l3_agents_per_router = 2
rabbit_host=controller
rabbit_password=RABBIT_PASS
rpc_backend=rabbit
[matchmaker_redis]
[matchmaker_ring]
[quotas]
[agent]
root_helper = sudo neutron-rootwrap /etc/neutron/rootwrap.conf
[keystone_authtoken]
auth_uri = http://controller:5000/v2.0
identity_uri = http://controller:35357
admin_tenant_name = service
admin_user = neutron
admin_password = NEUTRON_PASS
[database]
[service_providers]

/etc/neutron/plugin.ini

[ml2]
type_drivers = flat,vlan
tenant_network_types = vlan
mechanism_drivers = openvswitch
[ml2_type_flat]
[ml2_type_vlan]
network_vlan_ranges = default:400:1000
[ml2_type_gre]
[ml2_type_vxlan]
[securitygroup]
enable_security_group = True

/etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini

[ovs]
tenant_network_type = vlan
network_vlan_ranges = default:400:1000
enable_tunneling = False
bridge_mappings = default:br-ens2f0,external:br-ex
[agent]
[securitygroup]
firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
enable_security_group = True

/etc/neutron/l3_agent.ini

[DEFAULT]
verbose = True
interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver
use_namespaces = True
gateway_external_network_id = ea0937e7-9cc3-4f4e-ba84-b29d1b718a84
external_network_bridge = br-ex
ha_confs_path = $state_path/ha_confs
ha_vrrp_auth_type = PASS
ha_vrrp_auth_password = ABC
ha_vrrp_advert_int = 2

With these configurations, on creating a router with HA enabled, following messages are shown on the controller.

+-----------------------+--------------------------------------+
| Field                 | Value                                |
+-----------------------+--------------------------------------+
| admin_state_up        | True                                 |
| distributed           | False                                |
| external_gateway_info |                                      |
| ha                    | True                                 |
| id                    | f3ace776-91d6-4528-b603-9011db11f470 |
| name                  | demo-router                          |
| routes                |                                      |
| status                | ACTIVE                               |
| tenant_id             | b2e4e2e598614b5dbd878ae976728630     |
+-----------------------+--------------------------------------+

However, in the /var/log/neutron/server.log, you can find error messages indicating port binding failures:

2014-12-29 11:57:17.702 26747 INFO neutron.db.l3_hamode_db [req-7b7135d6-e3a8-4841-baca-0b786d0f1b78 None] Number of available agents lower than max_l3_agents_per_router. L3 agents available: 2
2014-12-29 11:57:17.950 26747 INFO neutron.wsgi [req-7b7135d6-e3a8-4841-baca-0b786d0f1b78 None] 10.24.148.21 - - [29/Dec/2014 11:57:17] "POST /v2.0/routers.json HTTP/1.1" 201 448 0.320873
2014-12-29 11:57:18.181 26747 WARNING neutron.plugins.ml2.managers [req-8c31ef8e-2452-42af-b590-0015b45a325e None] Failed to bind port 7ec5cb76-94c6-4e2e-a687-606d8fb34ce7 on host network1
2014-12-29 11:57:18.206 26747 WARNING neutron.plugins.ml2.plugin [req-8c31ef8e-2452-42af-b590-0015b45a325e None] In _notify_port_updated(), no bound segment for port 7ec5cb76-94c6-4e2e-a687-606d8fb34ce7 on network 46725d33-cd6d-418b-9574-d45cf7e6e340
2014-12-29 11:57:18.208 26747 WARNING neutron.plugins.ml2.managers [req-ae7d921b-8f89-466c-9023-73d8edb720ca None] Failed to bind port bc42bc0d-d1f7-4440-8704-757a47cee268 on host network2
2014-12-29 11:57:18.227 26747 WARNING neutron.plugins.ml2.plugin [req-ae7d921b-8f89-466c-9023-73d8edb720ca None] In _notify_port_updated(), no bound segment for port bc42bc0d-d1f7-4440-8704-757a47cee268 on network 46725d33-cd6d-418b-9574-d45cf7e6e340
2014-12-29 11:57:20.756 26747 WARNING neutron.plugins.ml2.rpc [req-3b378bc3-7378-4d61-b4dd-832afbe0f941 None] Device 7ec5cb76-94c6-4e2e-a687-606d8fb34ce7 requested by agent ovs-agent-network1 on network 46725d33-cd6d-418b-9574-d45cf7e6e340 not bound, vif_type: binding_failed
2014-12-29 11:57:21.574 ...
(more)
edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted
0

answered 2014-12-29 22:55:32 -0500

Byungjoon Lee gravatar image

Adding to @rahulrajvn 's answer, I think the more robust way of fixing the aforementioned configuration error is to fix the /usr/lib/systemd/system/neutron-openvswitch-agent.service file like following:

...
ExecStart=/usr/bin/neutron-openvswitch-agent --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugin.ini --config-file /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini --log-file /var/log/neutron/openvswitch-agent.log
....

That is, add the ovs_neutron_plugin.ini into the config file path of neutron-openvswitch-agent. This method has been tested.

edit flag offensive delete link more
1

answered 2014-12-28 23:42:33 -0500

rahulrajvn gravatar image

updated 2014-12-29 04:18:36 -0500

This happens mainly because of neutron configurations, Please make sure that following steps are done in corresponding server's.

in the neutron controller node the softlink is made correctly.

 ln -s /etc/neutron/plugins/ml2/ml2_conf.ini /etc/neutron/plugin.ini

and in network node

 ln -s /etc/neutron/plugins/ml2/ml2_conf.ini /etc/neutron/plugin.ini

cp /usr/lib/systemd/system/neutron-openvswitch-agent.service /usr/lib/systemd/system/neutron-openvswitch-agent.service.orig
sed -i 's,plugins/openvswitch/ovs_neutron_plugin.ini,plugin.ini,g' /usr/lib/systemd/system/neutron-openvswitch-agent.service

I was not able to go through the entire updates, but for the issue between the neutron router's have you tried disabling the firewall and disabling the selinux in all the server's. its not a exact solution but we can at-least remove firewall and selinux from our further troubleshooting. Also make sure that keepalived is installed in both neutron server.

edit flag offensive delete link more

Comments

Thank you. For your help, one problem disappeared, but another raised. If possible, could you please read the updated part of the question?

Byungjoon Lee gravatar imageByungjoon Lee ( 2014-12-29 00:15:27 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

2 followers

Stats

Asked: 2014-12-28 21:35:28 -0500

Seen: 740 times

Last updated: Dec 29 '14