DVR: Failed updating arp entry
Using neutron DVR in Kilo, new router namespace is unable to ping DHCP servers. Problem occurs on one host only. All other hosts with the same network can reach the dhcp servers. And all other networks on the same host can reach their DHCP servers. I see the following error in the l3-agent log file.
2015-07-20 08:56:17.302 94525 ERROR neutron.agent.linux.utils [req-36b3da49-9d7f-4c83-90c0-b7e969493636 ]
Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-5f1d4930-46d4-42f6-a580-3eefd4e24bf8', 'ip', '-4', 'neigh', 'replace', '172.16.0.8', 'lladdr', 'fa:16:3e:3b:67:69', 'nud', 'permanent', 'dev', 'qr-cef89d3d-68']
Exit code: 1
Stdin:
Stdout:
Stderr: Cannot find device "qr-cef89d3d-68"
2015-07-20 08:56:17.303 94525 ERROR neutron.agent.l3.dvr_router [req-36b3da49-9d7f-4c83-90c0-b7e969493636 ] DVR: Failed updating arp entry
2015-07-20 08:56:17.303 94525 TRACE neutron.agent.l3.dvr_router Traceback (most recent call last):
2015-07-20 08:56:17.303 94525 TRACE neutron.agent.l3.dvr_router File "/usr/lib/python2.7/site-packages/neutron/agent/l3/dvr_router.py", line 195, in _update_arp_entry
2015-07-20 08:56:17.303 94525 TRACE neutron.agent.l3.dvr_router device.neigh.add(ip, mac)
2015-07-20 08:56:17.303 94525 TRACE neutron.agent.l3.dvr_router File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 538, in add
2015-07-20 08:56:17.303 94525 TRACE neutron.agent.l3.dvr_router 'dev', self.name))
2015-07-20 08:56:17.303 94525 TRACE neutron.agent.l3.dvr_router File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 222, in _as_root
2015-07-20 08:56:17.303 94525 TRACE neutron.agent.l3.dvr_router use_root_namespace=use_root_namespace)
2015-07-20 08:56:17.303 94525 TRACE neutron.agent.l3.dvr_router File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 69, in _as_root
2015-07-20 08:56:17.303 94525 TRACE neutron.agent.l3.dvr_router log_fail_as_error=self.log_fail_as_error)
2015-07-20 08:56:17.303 94525 TRACE neutron.agent.l3.dvr_router File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 78, in _execute
2015-07-20 08:56:17.303 94525 TRACE neutron.agent.l3.dvr_router log_fail_as_error=log_fail_as_error)
2015-07-20 08:56:17.303 94525 TRACE neutron.agent.l3.dvr_router File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 137, in execute
2015-07-20 08:56:17.303 94525 TRACE neutron.agent.l3.dvr_router raise RuntimeError(m)
2015-07-20 08:56:17.303 94525 TRACE neutron.agent.l3.dvr_router RuntimeError:
2015-07-20 08:56:17.303 94525 TRACE neutron.agent.l3.dvr_router Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-5f1d4930-46d4-42f6-a580-3eefd4e24bf8', 'ip', '-4', 'neigh', 'replace', '172.16.0.8', 'lladdr', 'fa:16:3e:3b:67:69', 'nud', 'permanent', 'dev', 'qr-cef89d3d-68']
2015-07-20 08:56:17.303 94525 TRACE neutron.agent.l3.dvr_router Exit code: 1
2015-07-20 08:56:17.303 94525 TRACE neutron.agent.l3.dvr_router Stdin:
2015-07-20 08:56:17.303 94525 TRACE neutron.agent.l3.dvr_router Stdout:
2015-07-20 08:56:17.303 94525 TRACE neutron.agent.l3.dvr_router Stderr: Cannot find device "qr-cef89d3d-68"
2015-07-20 08:56:17.303 94525 TRACE neutron.agent.l3.dvr_router
2015-07-20 08 ...