Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

neutron-openvswitch-agent.service not starting

Hi, I am following a openstack liberty setup document and in compute-vm node I am unable to start neutron-openvswitch-agent.service. neutron is running fine in controller, nova is also good in both.

I am using CentOS 7 and here log:

[root@compute-vm ~]# systemctl status neutron-openvswitch-agent.service ● neutron-openvswitch-agent.service - OpenStack Neutron Open vSwitch Agent Loaded: loaded (/usr/lib/systemd/system/neutron-openvswitch-agent.service; enabled; vendor preset: disabled) Active: failed (Result: exit-code) since Tue 2017-04-18 10:45:07 CEST; 114ms ago Process: 2648 ExecStart=/usr/bin/neutron-openvswitch-agent --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/openvswitch_agent.ini --config-dir /etc/neutron/conf.d/common --config-dir /etc/neutron/conf.d/neutron-openvswitch-agent --log-file /var/log/neutron/openvswitch-agent.log (code=exited, status=1/FAILURE) Process: 2641 ExecStartPre=/usr/bin/neutron-enable-bridge-firewall.sh (code=exited, status=0/SUCCESS) Main PID: 2648 (code=exited, status=1/FAILURE)

Apr 18 10:44:53 compute-vm.dkrz.de systemd[1]: Starting OpenStack Neutron Op.... Apr 18 10:44:53 compute-vm.dkrz.de neutron-enable-bridge-firewall.sh[2641]: n... Apr 18 10:44:53 compute-vm.dkrz.de neutron-enable-bridge-firewall.sh[2641]: n... Apr 18 10:44:53 compute-vm.dkrz.de neutron-enable-bridge-firewall.sh[2641]: n... Apr 18 10:44:53 compute-vm.dkrz.de systemd[1]: Started OpenStack Neutron Ope.... Apr 18 10:44:54 compute-vm.dkrz.de neutron-openvswitch-agent[2648]: No handle... Apr 18 10:45:07 compute-vm.dkrz.de systemd[1]: neutron-openvswitch-agent.ser...E Apr 18 10:45:07 compute-vm.dkrz.de systemd[1]: Unit neutron-openvswitch-agen.... Apr 18 10:45:07 compute-vm.dkrz.de systemd[1]: neutron-openvswitch-agent.ser.... Hint: Some lines were ellipsized, use -l to show in full. [root@compute-vm ~]# cat /var/log/neutron/openvswitch-agent.log | tail -80 2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_vsctl.py", line 63, in run_vsctl 2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl log_fail_as_error=False).rstrip() 2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 159, in execute 2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl raise RuntimeError(m) 2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl RuntimeError: 2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl Command: ['neutron-rootwrap-xen-dom0', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=ofport', 'list', 'Interface', 'xapi1'] 2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 96 2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl 2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl 2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl [-] Unable to execute ['ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=ofport', 'list', 'Interface', 'xapi1']. 2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl Traceback (most recent call last): 2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_vsctl.py", line 63, in run_vsctl 2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl log_fail_as_error=False).rstrip() 2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 159, in execute 2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl raise RuntimeError(m) 2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl RuntimeError: 2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl Command: ['neutron-rootwrap-xen-dom0', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=ofport', 'list', 'Interface', 'xapi1'] 2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 96

...

2017-04-18 10:45:06.612 2648 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 96 2017-04-18 10:45:06.612 2648 ERROR neutron.agent.ovsdb.impl_vsctl 2017-04-18 10:45:06.612 2648 ERROR neutron.agent.ovsdb.impl_vsctl 2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib [-] Timed out retrieving ofport on port xapi1. Exception: RetryError[Attempts: 11, Value: None] 2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib Traceback (most recent call last): 2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib File "/usr/lib/python2.7/site-packages/neutron/agent/common/ovs_lib.py", line 262, in get_port_ofport 2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib ofport = self._get_port_ofport(port_name) 2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib File "/usr/lib/python2.7/site-packages/neutron/agent/common/ovs_lib.py", line 83, in wrapped 2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib return new_fn(args, *kwargs) 2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib File "/usr/lib/python2.7/site-packages/retrying.py", line 68, in wrapped_f 2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib return Retrying(dargs, *dkw).call(f, args, *kw) 2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib File "/usr/lib/python2.7/site-packages/retrying.py", line 231, in call 2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib raise RetryError(attempt) 2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib RetryError: RetryError[Attempts: 11, Value: None] 2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib 2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl [-] Unable to execute ['ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', 'set-fail-mode', 'xapi1', 'secure']. 2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl Traceback (most recent call last): 2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_vsctl.py", line 63, in run_vsctl 2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl log_fail_as_error=False).rstrip() 2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 159, in execute 2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl raise RuntimeError(m) 2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl RuntimeError: 2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl Command: ['neutron-rootwrap-xen-dom0', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', 'set-fail-mode', 'xapi1', 'secure'] 2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 96 2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl 2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl 2017-04-18 10:45:07.155 2648 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [-] Command: ['neutron-rootwrap-xen-dom0', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', 'set-fail-mode', 'xapi1', 'secure'] Exit code: 96

neutron-openvswitch-agent.service not starting

Hi, I am following a openstack liberty setup document and in compute-vm node I am unable to start neutron-openvswitch-agent.service. neutron is running fine in controller, nova is also good in both.

I am using CentOS 7 and here log:

 [root@compute-vm ~]#  systemctl status neutron-openvswitch-agent.service
● neutron-openvswitch-agent.service - OpenStack Neutron Open vSwitch Agent
   Loaded: loaded (/usr/lib/systemd/system/neutron-openvswitch-agent.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Tue 2017-04-18 10:45:07 CEST; 114ms ago
  Process: 2648 ExecStart=/usr/bin/neutron-openvswitch-agent --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/openvswitch_agent.ini --config-dir /etc/neutron/conf.d/common --config-dir /etc/neutron/conf.d/neutron-openvswitch-agent --log-file /var/log/neutron/openvswitch-agent.log (code=exited, status=1/FAILURE)
  Process: 2641 ExecStartPre=/usr/bin/neutron-enable-bridge-firewall.sh (code=exited, status=0/SUCCESS)
 Main PID: 2648 (code=exited, status=1/FAILURE)

status=1/FAILURE) Apr 18 10:44:53 compute-vm.dkrz.de systemd[1]: Starting OpenStack Neutron Op.... Apr 18 10:44:53 compute-vm.dkrz.de neutron-enable-bridge-firewall.sh[2641]: n... Apr 18 10:44:53 compute-vm.dkrz.de neutron-enable-bridge-firewall.sh[2641]: n... Apr 18 10:44:53 compute-vm.dkrz.de neutron-enable-bridge-firewall.sh[2641]: n... Apr 18 10:44:53 compute-vm.dkrz.de systemd[1]: Started OpenStack Neutron Ope.... Apr 18 10:44:54 compute-vm.dkrz.de neutron-openvswitch-agent[2648]: No handle... Apr 18 10:45:07 compute-vm.dkrz.de systemd[1]: neutron-openvswitch-agent.ser...E Apr 18 10:45:07 compute-vm.dkrz.de systemd[1]: Unit neutron-openvswitch-agen.... Apr 18 10:45:07 compute-vm.dkrz.de systemd[1]: neutron-openvswitch-agent.ser.... Hint: Some lines were ellipsized, use -l to show in full. [root@compute-vm ~]# cat /var/log/neutron/openvswitch-agent.log | tail -80 2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_vsctl.py", line 63, in run_vsctl 2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl log_fail_as_error=False).rstrip() 2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 159, in execute 2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl raise RuntimeError(m) 2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl RuntimeError: 2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl Command: ['neutron-rootwrap-xen-dom0', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=ofport', 'list', 'Interface', 'xapi1'] 2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 96 2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl 2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl 2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl [-] Unable to execute ['ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=ofport', 'list', 'Interface', 'xapi1']. 2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl Traceback (most recent call last): 2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_vsctl.py", line 63, in run_vsctl 2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl log_fail_as_error=False).rstrip() 2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 159, in execute 2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl raise RuntimeError(m) 2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl RuntimeError: 2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl Command: ['neutron-rootwrap-xen-dom0', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=ofport', 'list', 'Interface', 'xapi1'] 2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 96

...

96 ... 2017-04-18 10:45:06.612 2648 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 96 2017-04-18 10:45:06.612 2648 ERROR neutron.agent.ovsdb.impl_vsctl 2017-04-18 10:45:06.612 2648 ERROR neutron.agent.ovsdb.impl_vsctl 2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib [-] Timed out retrieving ofport on port xapi1. Exception: RetryError[Attempts: 11, Value: None] 2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib Traceback (most recent call last): 2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib File "/usr/lib/python2.7/site-packages/neutron/agent/common/ovs_lib.py", line 262, in get_port_ofport 2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib ofport = self._get_port_ofport(port_name) 2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib File "/usr/lib/python2.7/site-packages/neutron/agent/common/ovs_lib.py", line 83, in wrapped 2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib return new_fn(args, *kwargs) new_fn(*args, **kwargs) 2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib File "/usr/lib/python2.7/site-packages/retrying.py", line 68, in wrapped_f 2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib return Retrying(dargs, *dkw).call(f, args, *kw) Retrying(*dargs, **dkw).call(f, *args, **kw) 2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib File "/usr/lib/python2.7/site-packages/retrying.py", line 231, in call 2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib raise RetryError(attempt) 2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib RetryError: RetryError[Attempts: 11, Value: None] 2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib 2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl [-] Unable to execute ['ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', 'set-fail-mode', 'xapi1', 'secure']. 2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl Traceback (most recent call last): 2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_vsctl.py", line 63, in run_vsctl 2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl log_fail_as_error=False).rstrip() 2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 159, in execute 2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl raise RuntimeError(m) 2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl RuntimeError: 2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl Command: ['neutron-rootwrap-xen-dom0', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', 'set-fail-mode', 'xapi1', 'secure'] 2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 96 2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl 2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl 2017-04-18 10:45:07.155 2648 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [-] Command: ['neutron-rootwrap-xen-dom0', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', 'set-fail-mode', 'xapi1', 'secure'] Exit code: 96

96

neutron-openvswitch-agent.service not starting

Hi, I am following a openstack liberty setup document and in compute-vm node I am unable to start neutron-openvswitch-agent.service. neutron is running fine in controller, nova is also good in both.

I am using CentOS 7 and here log:

 [root@compute-vm ~]#  systemctl status neutron-openvswitch-agent.service
● neutron-openvswitch-agent.service - OpenStack Neutron Open vSwitch Agent
   Loaded: loaded (/usr/lib/systemd/system/neutron-openvswitch-agent.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Tue 2017-04-18 10:45:07 CEST; 114ms ago
  Process: 2648 ExecStart=/usr/bin/neutron-openvswitch-agent --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/openvswitch_agent.ini --config-dir /etc/neutron/conf.d/common --config-dir /etc/neutron/conf.d/neutron-openvswitch-agent --log-file /var/log/neutron/openvswitch-agent.log (code=exited, status=1/FAILURE)
  Process: 2641 ExecStartPre=/usr/bin/neutron-enable-bridge-firewall.sh (code=exited, status=0/SUCCESS)
 Main PID: 2648 (code=exited, status=1/FAILURE)

Apr 18 10:44:53 compute-vm.dkrz.de systemd[1]: Starting OpenStack Neutron Op....
Apr 18 10:44:53 compute-vm.dkrz.de neutron-enable-bridge-firewall.sh[2641]: n...
Apr 18 10:44:53 compute-vm.dkrz.de neutron-enable-bridge-firewall.sh[2641]: n...
Apr 18 10:44:53 compute-vm.dkrz.de neutron-enable-bridge-firewall.sh[2641]: n...
Apr 18 10:44:53 compute-vm.dkrz.de systemd[1]: Started OpenStack Neutron Ope....
Apr 18 10:44:54 compute-vm.dkrz.de neutron-openvswitch-agent[2648]: No handle...
Apr 18 10:45:07 compute-vm.dkrz.de systemd[1]: neutron-openvswitch-agent.ser...E
Apr 18 10:45:07 compute-vm.dkrz.de systemd[1]: Unit neutron-openvswitch-agen....
Apr 18 10:45:07 compute-vm.dkrz.de systemd[1]: neutron-openvswitch-agent.ser....
Hint: Some lines were ellipsized, use -l to show in full.
[root@compute-vm ~]# cat /var/log/neutron/openvswitch-agent.log | tail -80
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_vsctl.py", line 63, in run_vsctl
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl     log_fail_as_error=False).rstrip()
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 159, in execute
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl     raise RuntimeError(m)
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl RuntimeError: 
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl Command: ['neutron-rootwrap-xen-dom0', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=ofport', 'list', 'Interface', 'xapi1']
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 96
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl 
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl 
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl [-] Unable to execute ['ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=ofport', 'list', 'Interface', 'xapi1'].
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl Traceback (most recent call last):
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_vsctl.py", line 63, in run_vsctl
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl     log_fail_as_error=False).rstrip()
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 159, in execute
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl     raise RuntimeError(m)
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl RuntimeError: 
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl Command: ['neutron-rootwrap-xen-dom0', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=ofport', 'list', 'Interface', 'xapi1']
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 96

...

2017-04-18 10:45:06.612 2648 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 96
2017-04-18 10:45:06.612 2648 ERROR neutron.agent.ovsdb.impl_vsctl 
2017-04-18 10:45:06.612 2648 ERROR neutron.agent.ovsdb.impl_vsctl 
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib [-] Timed out retrieving ofport on port xapi1. Exception: RetryError[Attempts: 11, Value: None]
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib Traceback (most recent call last):
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib   File "/usr/lib/python2.7/site-packages/neutron/agent/common/ovs_lib.py", line 262, in get_port_ofport
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib     ofport = self._get_port_ofport(port_name)
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib   File "/usr/lib/python2.7/site-packages/neutron/agent/common/ovs_lib.py", line 83, in wrapped
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib     return new_fn(*args, **kwargs)
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib   File "/usr/lib/python2.7/site-packages/retrying.py", line 68, in wrapped_f
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib     return Retrying(*dargs, **dkw).call(f, *args, **kw)
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib   File "/usr/lib/python2.7/site-packages/retrying.py", line 231, in call
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib     raise RetryError(attempt)
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib RetryError: RetryError[Attempts: 11, Value: None]
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib 
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl [-] Unable to execute ['ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', 'set-fail-mode', 'xapi1', 'secure'].
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl Traceback (most recent call last):
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_vsctl.py", line 63, in run_vsctl
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl     log_fail_as_error=False).rstrip()
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 159, in execute
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl     raise RuntimeError(m)
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl RuntimeError: 
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl Command: ['neutron-rootwrap-xen-dom0', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', 'set-fail-mode', 'xapi1', 'secure']
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 96
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl 
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl 
2017-04-18 10:45:07.155 2648 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [-] 
Command: ['neutron-rootwrap-xen-dom0', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', 'set-fail-mode', 'xapi1', 'secure']
Exit code: 96

neutron-openvswitch-agent.service not starting

Hi, I am following a openstack liberty setup document and in compute-vm node I am unable to start neutron-openvswitch-agent.service. neutron is running fine in controller, nova is also good in both.

I am using CentOS 7 and here log:

 [root@compute-vm ~]#  systemctl status neutron-openvswitch-agent.service
● neutron-openvswitch-agent.service - OpenStack Neutron Open vSwitch Agent
   Loaded: loaded (/usr/lib/systemd/system/neutron-openvswitch-agent.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Tue 2017-04-18 10:45:07 CEST; 114ms ago
  Process: 2648 ExecStart=/usr/bin/neutron-openvswitch-agent --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/openvswitch_agent.ini --config-dir /etc/neutron/conf.d/common --config-dir /etc/neutron/conf.d/neutron-openvswitch-agent --log-file /var/log/neutron/openvswitch-agent.log (code=exited, status=1/FAILURE)
  Process: 2641 ExecStartPre=/usr/bin/neutron-enable-bridge-firewall.sh (code=exited, status=0/SUCCESS)
 Main PID: 2648 (code=exited, status=1/FAILURE)

Apr 18 10:44:53 compute-vm.dkrz.de systemd[1]: Starting OpenStack Neutron Op....
Apr 18 10:44:53 compute-vm.dkrz.de neutron-enable-bridge-firewall.sh[2641]: n...
Apr 18 10:44:53 compute-vm.dkrz.de neutron-enable-bridge-firewall.sh[2641]: n...
Apr 18 10:44:53 compute-vm.dkrz.de neutron-enable-bridge-firewall.sh[2641]: n...
Apr 18 10:44:53 compute-vm.dkrz.de systemd[1]: Started OpenStack Neutron Ope....
Apr 18 10:44:54 compute-vm.dkrz.de neutron-openvswitch-agent[2648]: No handle...
Apr 18 10:45:07 compute-vm.dkrz.de systemd[1]: neutron-openvswitch-agent.ser...E
Apr 18 10:45:07 compute-vm.dkrz.de systemd[1]: Unit neutron-openvswitch-agen....
Apr 18 10:45:07 compute-vm.dkrz.de systemd[1]: neutron-openvswitch-agent.ser....
Hint: Some lines were ellipsized, use -l to show in full.
[root@compute-vm ~]# cat /var/log/neutron/openvswitch-agent.log | tail -80
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_vsctl.py", line 63, in run_vsctl
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl     log_fail_as_error=False).rstrip()
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 159, in execute
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl     raise RuntimeError(m)
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl RuntimeError: 
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl Command: ['neutron-rootwrap-xen-dom0', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=ofport', 'list', 'Interface', 'xapi1']
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 96
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl 
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl 
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl [-] Unable to execute ['ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=ofport', 'list', 'Interface', 'xapi1'].
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl Traceback (most recent call last):
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_vsctl.py", line 63, in run_vsctl
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl     log_fail_as_error=False).rstrip()
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 159, in execute
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl     raise RuntimeError(m)
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl RuntimeError: 
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl Command: ['neutron-rootwrap-xen-dom0', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=ofport', 'list', 'Interface', 'xapi1']
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 96

...

2017-04-18 10:45:06.612 2648 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 96
2017-04-18 10:45:06.612 2648 ERROR neutron.agent.ovsdb.impl_vsctl 
2017-04-18 10:45:06.612 2648 ERROR neutron.agent.ovsdb.impl_vsctl 
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib [-] Timed out retrieving ofport on port xapi1. Exception: RetryError[Attempts: 11, Value: None]
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib Traceback (most recent call last):
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib   File "/usr/lib/python2.7/site-packages/neutron/agent/common/ovs_lib.py", line 262, in get_port_ofport
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib     ofport = self._get_port_ofport(port_name)
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib   File "/usr/lib/python2.7/site-packages/neutron/agent/common/ovs_lib.py", line 83, in wrapped
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib     return new_fn(*args, **kwargs)
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib   File "/usr/lib/python2.7/site-packages/retrying.py", line 68, in wrapped_f
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib     return Retrying(*dargs, **dkw).call(f, *args, **kw)
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib   File "/usr/lib/python2.7/site-packages/retrying.py", line 231, in call
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib     raise RetryError(attempt)
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib RetryError: RetryError[Attempts: 11, Value: None]
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib 
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl [-] Unable to execute ['ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', 'set-fail-mode', 'xapi1', 'secure'].
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl Traceback (most recent call last):
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_vsctl.py", line 63, in run_vsctl
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl     log_fail_as_error=False).rstrip()
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 159, in execute
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl     raise RuntimeError(m)
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl RuntimeError: 
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl Command: ['neutron-rootwrap-xen-dom0', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', 'set-fail-mode', 'xapi1', 'secure']
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 96
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl 
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl 
2017-04-18 10:45:07.155 2648 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [-] 
Command: ['neutron-rootwrap-xen-dom0', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', 'set-fail-mode', 'xapi1', 'secure']
Exit code: 96

output of ovs-vsctl --timeout=10 --oneline --format=json -- --columns=ofport list Interface xapi1

On compute-vm :

=ofport list Interface xapi1tl --timeout=10 --oneline --format=json -- --columns 
ovs-vsctl: no row "xapi1" in table Interface
=ofport list Interface xapi7
ovs-vsctl: no row "xapi7" in table Interface
[root@compute-vm ~]#

---------on xen server [root@openstack02 ~]# xe network-list name-label=openstack-int-network params=bridge bridge ( RO) : xapi7

neutron-openvswitch-agent.service not starting

Hi, I am following a openstack liberty setup document and in compute-vm node I am unable to start neutron-openvswitch-agent.service. neutron is running fine in controller, nova is also good in both.

I am using CentOS 7 and here log:

 [root@compute-vm ~]#  systemctl status neutron-openvswitch-agent.service
● neutron-openvswitch-agent.service - OpenStack Neutron Open vSwitch Agent
   Loaded: loaded (/usr/lib/systemd/system/neutron-openvswitch-agent.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Tue 2017-04-18 10:45:07 CEST; 114ms ago
  Process: 2648 ExecStart=/usr/bin/neutron-openvswitch-agent --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/openvswitch_agent.ini --config-dir /etc/neutron/conf.d/common --config-dir /etc/neutron/conf.d/neutron-openvswitch-agent --log-file /var/log/neutron/openvswitch-agent.log (code=exited, status=1/FAILURE)
  Process: 2641 ExecStartPre=/usr/bin/neutron-enable-bridge-firewall.sh (code=exited, status=0/SUCCESS)
 Main PID: 2648 (code=exited, status=1/FAILURE)

Apr 18 10:44:53 compute-vm.dkrz.de systemd[1]: Starting OpenStack Neutron Op....
Apr 18 10:44:53 compute-vm.dkrz.de neutron-enable-bridge-firewall.sh[2641]: n...
Apr 18 10:44:53 compute-vm.dkrz.de neutron-enable-bridge-firewall.sh[2641]: n...
Apr 18 10:44:53 compute-vm.dkrz.de neutron-enable-bridge-firewall.sh[2641]: n...
Apr 18 10:44:53 compute-vm.dkrz.de systemd[1]: Started OpenStack Neutron Ope....
Apr 18 10:44:54 compute-vm.dkrz.de neutron-openvswitch-agent[2648]: No handle...
Apr 18 10:45:07 compute-vm.dkrz.de systemd[1]: neutron-openvswitch-agent.ser...E
Apr 18 10:45:07 compute-vm.dkrz.de systemd[1]: Unit neutron-openvswitch-agen....
Apr 18 10:45:07 compute-vm.dkrz.de systemd[1]: neutron-openvswitch-agent.ser....
Hint: Some lines were ellipsized, use -l to show in full.
[root@compute-vm ~]# cat /var/log/neutron/openvswitch-agent.log | tail -80
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_vsctl.py", line 63, in run_vsctl
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl     log_fail_as_error=False).rstrip()
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 159, in execute
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl     raise RuntimeError(m)
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl RuntimeError: 
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl Command: ['neutron-rootwrap-xen-dom0', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=ofport', 'list', 'Interface', 'xapi1']
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 96
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl 
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl 
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl [-] Unable to execute ['ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=ofport', 'list', 'Interface', 'xapi1'].
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl Traceback (most recent call last):
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_vsctl.py", line 63, in run_vsctl
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl     log_fail_as_error=False).rstrip()
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 159, in execute
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl     raise RuntimeError(m)
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl RuntimeError: 
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl Command: ['neutron-rootwrap-xen-dom0', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=ofport', 'list', 'Interface', 'xapi1']
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 96

...

2017-04-18 10:45:06.612 2648 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 96
2017-04-18 10:45:06.612 2648 ERROR neutron.agent.ovsdb.impl_vsctl 
2017-04-18 10:45:06.612 2648 ERROR neutron.agent.ovsdb.impl_vsctl 
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib [-] Timed out retrieving ofport on port xapi1. Exception: RetryError[Attempts: 11, Value: None]
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib Traceback (most recent call last):
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib   File "/usr/lib/python2.7/site-packages/neutron/agent/common/ovs_lib.py", line 262, in get_port_ofport
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib     ofport = self._get_port_ofport(port_name)
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib   File "/usr/lib/python2.7/site-packages/neutron/agent/common/ovs_lib.py", line 83, in wrapped
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib     return new_fn(*args, **kwargs)
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib   File "/usr/lib/python2.7/site-packages/retrying.py", line 68, in wrapped_f
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib     return Retrying(*dargs, **dkw).call(f, *args, **kw)
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib   File "/usr/lib/python2.7/site-packages/retrying.py", line 231, in call
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib     raise RetryError(attempt)
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib RetryError: RetryError[Attempts: 11, Value: None]
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib 
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl [-] Unable to execute ['ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', 'set-fail-mode', 'xapi1', 'secure'].
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl Traceback (most recent call last):
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_vsctl.py", line 63, in run_vsctl
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl     log_fail_as_error=False).rstrip()
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 159, in execute
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl     raise RuntimeError(m)
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl RuntimeError: 
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl Command: ['neutron-rootwrap-xen-dom0', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', 'set-fail-mode', 'xapi1', 'secure']
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 96
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl 
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl 
2017-04-18 10:45:07.155 2648 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [-] 
Command: ['neutron-rootwrap-xen-dom0', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', 'set-fail-mode', 'xapi1', 'secure']
Exit code: 96

output of ovs-vsctl --timeout=10 --oneline --format=json -- --columns=ofport list Interface xapi1:

On compute-vm :

=ofport list Interface xapi1tl --timeout=10 --oneline --format=json -- --columns 
ovs-vsctl: no row "xapi1" in table Interface
=ofport list Interface xapi7
ovs-vsctl: no row "xapi7" in table Interface
[root@compute-vm ~]#

---------on xen server server

[root@openstack02 ~]# xe network-list name-label=openstack-int-network params=bridge
 bridge ( RO)    : xapi7

xapi7

neutron-openvswitch-agent.service not starting

Hi, I am following a openstack liberty setup document and in compute-vm node I am unable to start neutron-openvswitch-agent.service. neutron is running fine in controller, nova is also good in both.

I am using CentOS 7 and here log:

 [root@compute-vm ~]#  systemctl status neutron-openvswitch-agent.service
● neutron-openvswitch-agent.service - OpenStack Neutron Open vSwitch Agent
   Loaded: loaded (/usr/lib/systemd/system/neutron-openvswitch-agent.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Tue 2017-04-18 10:45:07 CEST; 114ms ago
  Process: 2648 ExecStart=/usr/bin/neutron-openvswitch-agent --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/openvswitch_agent.ini --config-dir /etc/neutron/conf.d/common --config-dir /etc/neutron/conf.d/neutron-openvswitch-agent --log-file /var/log/neutron/openvswitch-agent.log (code=exited, status=1/FAILURE)
  Process: 2641 ExecStartPre=/usr/bin/neutron-enable-bridge-firewall.sh (code=exited, status=0/SUCCESS)
 Main PID: 2648 (code=exited, status=1/FAILURE)

Apr 18 10:44:53 compute-vm.dkrz.de systemd[1]: Starting OpenStack Neutron Op....
Apr 18 10:44:53 compute-vm.dkrz.de neutron-enable-bridge-firewall.sh[2641]: n...
Apr 18 10:44:53 compute-vm.dkrz.de neutron-enable-bridge-firewall.sh[2641]: n...
Apr 18 10:44:53 compute-vm.dkrz.de neutron-enable-bridge-firewall.sh[2641]: n...
Apr 18 10:44:53 compute-vm.dkrz.de systemd[1]: Started OpenStack Neutron Ope....
Apr 18 10:44:54 compute-vm.dkrz.de neutron-openvswitch-agent[2648]: No handle...
Apr 18 10:45:07 compute-vm.dkrz.de systemd[1]: neutron-openvswitch-agent.ser...E
Apr 18 10:45:07 compute-vm.dkrz.de systemd[1]: Unit neutron-openvswitch-agen....
Apr 18 10:45:07 compute-vm.dkrz.de systemd[1]: neutron-openvswitch-agent.ser....
Hint: Some lines were ellipsized, use -l to show in full.
[root@compute-vm ~]# cat /var/log/neutron/openvswitch-agent.log | tail -80
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_vsctl.py", line 63, in run_vsctl
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl     log_fail_as_error=False).rstrip()
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 159, in execute
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl     raise RuntimeError(m)
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl RuntimeError: 
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl Command: ['neutron-rootwrap-xen-dom0', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=ofport', 'list', 'Interface', 'xapi1']
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 96
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl 
2017-04-18 10:45:00.271 2648 ERROR neutron.agent.ovsdb.impl_vsctl 
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl [-] Unable to execute ['ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=ofport', 'list', 'Interface', 'xapi1'].
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl Traceback (most recent call last):
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_vsctl.py", line 63, in run_vsctl
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl     log_fail_as_error=False).rstrip()
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 159, in execute
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl     raise RuntimeError(m)
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl RuntimeError: 
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl Command: ['neutron-rootwrap-xen-dom0', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=ofport', 'list', 'Interface', 'xapi1']
2017-04-18 10:45:01.903 2648 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 96

...

2017-04-18 10:45:06.612 2648 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 96
2017-04-18 10:45:06.612 2648 ERROR neutron.agent.ovsdb.impl_vsctl 
2017-04-18 10:45:06.612 2648 ERROR neutron.agent.ovsdb.impl_vsctl 
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib [-] Timed out retrieving ofport on port xapi1. Exception: RetryError[Attempts: 11, Value: None]
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib Traceback (most recent call last):
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib   File "/usr/lib/python2.7/site-packages/neutron/agent/common/ovs_lib.py", line 262, in get_port_ofport
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib     ofport = self._get_port_ofport(port_name)
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib   File "/usr/lib/python2.7/site-packages/neutron/agent/common/ovs_lib.py", line 83, in wrapped
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib     return new_fn(*args, **kwargs)
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib   File "/usr/lib/python2.7/site-packages/retrying.py", line 68, in wrapped_f
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib     return Retrying(*dargs, **dkw).call(f, *args, **kw)
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib   File "/usr/lib/python2.7/site-packages/retrying.py", line 231, in call
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib     raise RetryError(attempt)
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib RetryError: RetryError[Attempts: 11, Value: None]
2017-04-18 10:45:06.613 2648 ERROR neutron.agent.common.ovs_lib 
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl [-] Unable to execute ['ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', 'set-fail-mode', 'xapi1', 'secure'].
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl Traceback (most recent call last):
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_vsctl.py", line 63, in run_vsctl
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl     log_fail_as_error=False).rstrip()
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 159, in execute
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl     raise RuntimeError(m)
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl RuntimeError: 
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl Command: ['neutron-rootwrap-xen-dom0', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', 'set-fail-mode', 'xapi1', 'secure']
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl Exit code: 96
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl 
2017-04-18 10:45:07.154 2648 ERROR neutron.agent.ovsdb.impl_vsctl 
2017-04-18 10:45:07.155 2648 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [-] 
Command: ['neutron-rootwrap-xen-dom0', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', 'set-fail-mode', 'xapi1', 'secure']
Exit code: 96

output of ovs-vsctl --timeout=10 --oneline --format=json -- --columns=ofport list Interface xapi1:

On compute-vm :

=ofport list Interface xapi1tl --timeout=10 --oneline --format=json -- --columns 
ovs-vsctl: no row "xapi1" in table Interface
=ofport list Interface xapi7
ovs-vsctl: no row "xapi7" in table Interface
[root@compute-vm ~]#

---------on xen server

[root@openstack02 ~]# xe network-list name-label=openstack-int-network params=bridge
bridge ( RO)    : xapi7

xe network-list output

uuid ( RO)                : 1277a952-e56e-642a-2ce8-b9d8a6b5edc8
name-label ( RW): Pool-wide network associated with eth3
name-description ( RW): 
bridge ( RO): xenbr3


uuid ( RO)                : f7436411-dd0d-e1ae-007a-9338cb1ff1fb
name-label ( RW): Pool-wide network associated with eth2
name-description ( RW): 
bridge ( RO): xenbr2


uuid ( RO)                : c3fd73bb-72d6-66f4-4e24-771d3ba6796a
name-label ( RW): DKRZ_30
name-description ( RW): DKRZ
bridge ( RO): xapi6


uuid ( RO)                : 55efa0c1-c563-b257-aa9c-1b841eac5084
name-label ( RW): OpenStack VM Integration Network
name-description ( RW): 
bridge ( RO): xapi5


  uuid ( RO)                : a83473f0-c098-b901-0315-3a68290212eb
  name-label ( RW): OS_PUB
  name-description ( RW): 
  bridge ( RO): xapi0


  uuid ( RO)                : f3a6f9dd-0604-73d1-b19f-af33f2d453fb
  name-label ( RW): Bond 2+3
  name-description ( RW): 
  bridge ( RO): xapi1


 uuid ( RO)                : 42fac584-7342-c46f-5b03-a8ab4107a70e
 name-label ( RW): OpenStack VM Network
 name-description ( RW): 
 bridge ( RO): xapi3


uuid ( RO)                : 74685b95-8a00-145d-029d-7d4ed03ff5e5
name-label ( RW): Pool-wide network associated with eth0
name-description ( RW): 
bridge ( RO): xenbr0


uuid ( RO)                : 2db651e4-4c91-7817-2973-92f24b8b6cd1
 name-label ( RW): Pool-wide network associated with eth1
 name-description ( RW): 
 bridge ( RO): xenbr1


 uuid ( RO)                : a72e647a-d9b6-161a-c26c-ae4ca9cecbac
 name-label ( RW): openstack-int-network
 name-description ( RW): 
 bridge ( RO): xapi7


 uuid ( RO)                : a7927e08-7060-c2dc-efb2-d0dba8be0f57
 name-label ( RW): OpenStack Public Network
 name-description ( RW): 
 bridge ( RO): xapi4


 uuid ( RO)                : 8ba7c161-3039-4147-8965-ff214b714406
  name-label ( RW): Host internal management network
  name-description ( RW): Network on which guests will be assigned a private link-local IP address which can be     used to talk XenAPI
  bridge ( RO): xenapi