Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

neutron-server high CPU usage L3 try find compute node agent

Hello everyone,

I ask this question, to see if someone can help me ...

I have a controlling node and a calculation node.

The controlling node also acts as a network node. I have used the following installation guide, for ubuntu and Pike version.

https://docs.openstack.org/pike/install/

The problem is the following:

When someone performs simulations and in this case self-saved with heat. After a while in the controlling node the neutron-server service reaches almost 100% of CPU usage.

In that case there is a slowdown in the cloud and when launching instances.


To get back down CPU usage I have to restart the neutron-linuxbridge-agent service of the calculation node. But before that I see that the following appears in the logs of the compute node.

ERROR neutron.plugins.ml2.drivers.agent._common_agent Traceback (most recent call last): 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 336, in treat_devices_removed 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent cfg.CONF.host) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/neutron/agent/rpc.py", line 139, in update_device_down 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent agent_id=agent_id, host=host) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/neutron/common/rpc.py", line 162, in call 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent return self._original_context.call(ctxt, method, *kwargs) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 169, in call 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent retry=self.retry) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 123, in _send 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent timeout=timeout, retry=retry) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 578, in send 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent retry=retry) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 569, in _send 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent raise result 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent RemoteError: Remote error: AgentNotFoundByTypeHost Agent with agent_type=L3 agent and host=urano could not be found 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming\n res = self.dispatcher.dispatch(message)\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch\n result = func(ctxt, *new_args)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/rpc.py", line 234, in update_device_down\n n_const.PORT_STATUS_DOWN, host)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/rpc.py", line 331, in notify_l2pop_port_wiring\n l2pop_driver.obj.update_port_down(port_context)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/l2pop/mech_driver.py", line 253, in update_port_down\n admin_context, agent_host, [port[\'device_id\']]):\n', u' File "/usr/lib/python2.7/dist-packages/neutron/db/l3_agentschedulers_db.py", line 303, in list_router_ids_on_host\n context, constants.AGENT_TYPE_L3, host)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/db/agents_db.py", line 291, in _get_agent_by_type_and_host\n host=host)\n', u'AgentNotFoundByTypeHost: Agent with agent_type=L3 agent and host=urano could not be found\n']. 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent

The calculation node is called "urano" and when it restarts the aforementioned service it stabilizes again ...

It looks like you are looking for an L3 agent that is not installed in the calculation node and is not reflected as necessary in the installation guide. I am using option 2 for the neutron configuration.

Regards,

neutron-server high CPU usage L3 try find compute node agent

Hello everyone,

I ask this question, to see if someone can help me ...

I have a controlling node and a calculation compute node.

The controlling node also acts as a network node. I have used the following installation guide, for ubuntu and Pike version.

https://docs.openstack.org/pike/install/

The problem is the following:

When someone performs simulations and in this case self-saved with heat. After a while in the controlling node the neutron-server service reaches almost 100% of CPU usage.

In that case there is a slowdown in the cloud and when launching instances.


To get back down CPU usage I have to restart the neutron-linuxbridge-agent service of the calculation node. But before that I see that the following appears in the logs of the compute node.

ERROR neutron.plugins.ml2.drivers.agent._common_agent Traceback (most recent call last): 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 336, in treat_devices_removed 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent cfg.CONF.host) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/neutron/agent/rpc.py", line 139, in update_device_down 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent agent_id=agent_id, host=host) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/neutron/common/rpc.py", line 162, in call 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent return self._original_context.call(ctxt, method, *kwargs) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 169, in call 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent retry=self.retry) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 123, in _send 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent timeout=timeout, retry=retry) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 578, in send 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent retry=retry) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 569, in _send 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent raise result 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent RemoteError: Remote error: AgentNotFoundByTypeHost Agent with agent_type=L3 agent and host=urano could not be found 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming\n res = self.dispatcher.dispatch(message)\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch\n result = func(ctxt, *new_args)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/rpc.py", line 234, in update_device_down\n n_const.PORT_STATUS_DOWN, host)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/rpc.py", line 331, in notify_l2pop_port_wiring\n l2pop_driver.obj.update_port_down(port_context)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/l2pop/mech_driver.py", line 253, in update_port_down\n admin_context, agent_host, [port[\'device_id\']]):\n', u' File "/usr/lib/python2.7/dist-packages/neutron/db/l3_agentschedulers_db.py", line 303, in list_router_ids_on_host\n context, constants.AGENT_TYPE_L3, host)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/db/agents_db.py", line 291, in _get_agent_by_type_and_host\n host=host)\n', u'AgentNotFoundByTypeHost: Agent with agent_type=L3 agent and host=urano could not be found\n']. 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent

The calculation compute node is called "urano" and when it restarts the aforementioned service it stabilizes again ...

It looks like you are looking for an L3 agent that is not installed in the calculation compute node and is not reflected as necessary in the installation guide. I am using option 2 for the neutron configuration.

Regards,

neutron-server high CPU usage L3 try find compute node agent

Hello everyone,

I ask this question, to see if someone can help me ...

I have a controlling node and a compute node.

The controlling node also acts as a network node. I have used the following installation guide, for ubuntu and Pike version.

https://docs.openstack.org/pike/install/

The problem is the following:

When someone performs simulations and in this case self-saved auto-scaling with heat. heat 30 instances. After a while in the controlling node the neutron-server service reaches almost 100% of CPU usage.

In that case there is a slowdown in the cloud and when launching instances.


To get back down CPU usage I have to restart the neutron-linuxbridge-agent service of the calculation node. But before that I see that the following appears in the logs of the compute node.

ERROR neutron.plugins.ml2.drivers.agent._common_agent Traceback (most recent call last): 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 336, in treat_devices_removed 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent cfg.CONF.host) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/neutron/agent/rpc.py", line 139, in update_device_down 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent agent_id=agent_id, host=host) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/neutron/common/rpc.py", line 162, in call 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent return self._original_context.call(ctxt, method, *kwargs) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 169, in call 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent retry=self.retry) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 123, in _send 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent timeout=timeout, retry=retry) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 578, in send 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent retry=retry) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 569, in _send 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent raise result 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent RemoteError: Remote error: AgentNotFoundByTypeHost Agent with agent_type=L3 agent and host=urano could not be found 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming\n res = self.dispatcher.dispatch(message)\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch\n result = func(ctxt, *new_args)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/rpc.py", line 234, in update_device_down\n n_const.PORT_STATUS_DOWN, host)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/rpc.py", line 331, in notify_l2pop_port_wiring\n l2pop_driver.obj.update_port_down(port_context)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/l2pop/mech_driver.py", line 253, in update_port_down\n admin_context, agent_host, [port[\'device_id\']]):\n', u' File "/usr/lib/python2.7/dist-packages/neutron/db/l3_agentschedulers_db.py", line 303, in list_router_ids_on_host\n context, constants.AGENT_TYPE_L3, host)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/db/agents_db.py", line 291, in _get_agent_by_type_and_host\n host=host)\n', u'AgentNotFoundByTypeHost: Agent with agent_type=L3 agent and host=urano could not be found\n']. 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent

The compute node is called "urano" and when it restarts the aforementioned service it stabilizes again ...

It looks like you are looking for an L3 agent that is not installed in the compute node and is not reflected as necessary in the installation guide. I am using option 2 for the neutron configuration.

Regards,

neutron-server high CPU usage L3 try find compute node agent

Hello everyone,

I ask this question, to see if someone can help me ...

I have a controlling node and a compute node.

The controlling node also acts as a network node. I have used the following installation guide, for ubuntu and Pike version.

https://docs.openstack.org/pike/install/

The problem is the following:

When someone performs simulations and in this case auto-scaling with heat 30 instances. After a while in the controlling node the neutron-server service reaches almost 100% of CPU usage.

In that case there is a slowdown in the cloud and when launching instances.


To get back down CPU usage I have to restart the neutron-linuxbridge-agent service of the calculation node. But before that I see that the following appears in the logs of the compute node.

ERROR neutron.plugins.ml2.drivers.agent._common_agent Traceback (most recent call last): 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 336, in treat_devices_removed 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent cfg.CONF.host) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/neutron/agent/rpc.py", line 139, in update_device_down 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent agent_id=agent_id, host=host) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/neutron/common/rpc.py", line 162, in call 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent return self._original_context.call(ctxt, method, *kwargs) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 169, in call 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent retry=self.retry) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 123, in _send 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent timeout=timeout, retry=retry) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 578, in send 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent retry=retry) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 569, in _send 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent raise result 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent RemoteError: Remote error: AgentNotFoundByTypeHost Agent with agent_type=L3 agent and host=urano could not be found 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming\n res = self.dispatcher.dispatch(message)\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch\n result = func(ctxt, *new_args)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/rpc.py", line 234, in update_device_down\n n_const.PORT_STATUS_DOWN, host)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/rpc.py", line 331, in notify_l2pop_port_wiring\n l2pop_driver.obj.update_port_down(port_context)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/l2pop/mech_driver.py", line 253, in update_port_down\n admin_context, agent_host, [port[\'device_id\']]):\n', u' File "/usr/lib/python2.7/dist-packages/neutron/db/l3_agentschedulers_db.py", line 303, in list_router_ids_on_host\n context, constants.AGENT_TYPE_L3, host)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/db/agents_db.py", line 291, in _get_agent_by_type_and_host\n host=host)\n', u'AgentNotFoundByTypeHost: Agent with agent_type=L3 agent and host=urano could not be found\n']. 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent

The compute node is called "urano" and when it restarts the aforementioned service it stabilizes again ...

It looks like you are looking for an L3 agent that is not installed in the compute node and is not reflected as necessary in the installation guide. I am using option 2 for the neutron configuration.

Regards,

neutron-server high CPU usage L3 try find compute node agent

Hello everyone,

I ask this question, to see if someone can help me ...

I have a controlling node and a compute node.

The controlling node also acts as a network node. I have used the following installation guide, for ubuntu and Pike version.

https://docs.openstack.org/pike/install/

The problem is the following:

When someone performs simulations and in this case auto-scaling with heat 30 instances. After a while in the controlling node the neutron-server service reaches almost 100% of CPU usage.

In that case there is a slowdown in the cloud and when launching instances.


To get back down CPU usage I have to restart the neutron-linuxbridge-agent service of the calculation compute node. But before that I see that the following appears in the logs of the compute node.

ERROR neutron.plugins.ml2.drivers.agent._common_agent Traceback (most recent call last): 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 336, in treat_devices_removed 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent cfg.CONF.host) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/neutron/agent/rpc.py", line 139, in update_device_down 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent agent_id=agent_id, host=host) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/neutron/common/rpc.py", line 162, in call 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent return self._original_context.call(ctxt, method, *kwargs) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 169, in call 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent retry=self.retry) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 123, in _send 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent timeout=timeout, retry=retry) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 578, in send 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent retry=retry) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 569, in _send 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent raise result 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent RemoteError: Remote error: AgentNotFoundByTypeHost Agent with agent_type=L3 agent and host=urano could not be found 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming\n res = self.dispatcher.dispatch(message)\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch\n result = func(ctxt, *new_args)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/rpc.py", line 234, in update_device_down\n n_const.PORT_STATUS_DOWN, host)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/rpc.py", line 331, in notify_l2pop_port_wiring\n l2pop_driver.obj.update_port_down(port_context)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/l2pop/mech_driver.py", line 253, in update_port_down\n admin_context, agent_host, [port[\'device_id\']]):\n', u' File "/usr/lib/python2.7/dist-packages/neutron/db/l3_agentschedulers_db.py", line 303, in list_router_ids_on_host\n context, constants.AGENT_TYPE_L3, host)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/db/agents_db.py", line 291, in _get_agent_by_type_and_host\n host=host)\n', u'AgentNotFoundByTypeHost: Agent with agent_type=L3 agent and host=urano could not be found\n']. 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent

The compute node is called "urano" and when it restarts the aforementioned service it stabilizes again ...

It looks like you are looking for an L3 agent that is not installed in the compute node and is not reflected as necessary in the installation guide. I am using option 2 for the neutron configuration.

Regards,

neutron-server high CPU usage L3 try find compute node agent

Hello everyone,

I ask this question, to see if someone can help me ...

I have a controlling controller node and a compute node.

The controlling controller node also acts as a network node. I have used the following installation guide, for ubuntu and Pike version.

https://docs.openstack.org/pike/install/

The problem is the following:

When someone performs simulations and in this case auto-scaling with heat 30 instances. After a while in the controlling controller node the neutron-server service reaches almost 100% of CPU usage.

In that case there is a slowdown in the cloud and when launching instances.


To get back down CPU usage I have to restart the neutron-linuxbridge-agent service of the compute node. But before that I see that the following appears in the logs of the compute node.

ERROR neutron.plugins.ml2.drivers.agent._common_agent Traceback (most recent call last): 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 336, in treat_devices_removed 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent cfg.CONF.host) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/neutron/agent/rpc.py", line 139, in update_device_down 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent agent_id=agent_id, host=host) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/neutron/common/rpc.py", line 162, in call 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent return self._original_context.call(ctxt, method, *kwargs) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 169, in call 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent retry=self.retry) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 123, in _send 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent timeout=timeout, retry=retry) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 578, in send 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent retry=retry) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 569, in _send 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent raise result 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent RemoteError: Remote error: AgentNotFoundByTypeHost Agent with agent_type=L3 agent and host=urano could not be found 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming\n res = self.dispatcher.dispatch(message)\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch\n result = func(ctxt, *new_args)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/rpc.py", line 234, in update_device_down\n n_const.PORT_STATUS_DOWN, host)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/rpc.py", line 331, in notify_l2pop_port_wiring\n l2pop_driver.obj.update_port_down(port_context)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/l2pop/mech_driver.py", line 253, in update_port_down\n admin_context, agent_host, [port[\'device_id\']]):\n', u' File "/usr/lib/python2.7/dist-packages/neutron/db/l3_agentschedulers_db.py", line 303, in list_router_ids_on_host\n context, constants.AGENT_TYPE_L3, host)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/db/agents_db.py", line 291, in _get_agent_by_type_and_host\n host=host)\n', u'AgentNotFoundByTypeHost: Agent with agent_type=L3 agent and host=urano could not be found\n']. 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent

The compute node is called "urano" and when it restarts the aforementioned service it stabilizes again ...

It looks like you are looking for an L3 agent that is not installed in the compute node and is not reflected as necessary in the installation guide. I am using option 2 for the neutron configuration.

Regards,

neutron-server high CPU usage L3 try find compute node agent

Hello everyone,

I ask this question, to see if someone can help me ...

I have a controller node and a compute node.

The controller node also acts as a network node. I have used the following installation guide, for ubuntu and Pike version.

https://docs.openstack.org/pike/install/

The problem is the following:

When someone performs simulations and in this case auto-scaling with heat 30 instances. After a while in the controller node the neutron-server service reaches almost 100% of CPU usage.

In that case there is a slowdown in the cloud and when launching instances.


To get back down CPU usage I have to restart the neutron-linuxbridge-agent service of the compute node. But before that I see that the following appears in the logs of the compute node.

ERROR neutron.plugins.ml2.drivers.agent._common_agent Traceback (most recent call last): 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 336, in treat_devices_removed 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent cfg.CONF.host) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/neutron/agent/rpc.py", line 139, in update_device_down 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent agent_id=agent_id, host=host) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/neutron/common/rpc.py", line 162, in call 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent return self._original_context.call(ctxt, method, *kwargs) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 169, in call 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent retry=self.retry) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 123, in _send 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent timeout=timeout, retry=retry) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 578, in send 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent retry=retry) 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 569, in _send 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent raise result 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent RemoteError: Remote error: AgentNotFoundByTypeHost Agent with agent_type=L3 agent and host=urano could not be found 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming\n res = self.dispatcher.dispatch(message)\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch\n result = func(ctxt, *new_args)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/rpc.py", line 234, in update_device_down\n n_const.PORT_STATUS_DOWN, host)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/rpc.py", line 331, in notify_l2pop_port_wiring\n l2pop_driver.obj.update_port_down(port_context)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/l2pop/mech_driver.py", line 253, in update_port_down\n admin_context, agent_host, [port[\'device_id\']]):\n', u' File "/usr/lib/python2.7/dist-packages/neutron/db/l3_agentschedulers_db.py", line 303, in list_router_ids_on_host\n context, constants.AGENT_TYPE_L3, host)\n', u' File "/usr/lib/python2.7/dist-packages/neutron/db/agents_db.py", line 291, in _get_agent_by_type_and_host\n host=host)\n', u'AgentNotFoundByTypeHost: Agent with agent_type=L3 agent and host=urano could not be found\n']. 2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent

The compute node is called "urano" and when it restarts the aforementioned service it stabilizes again ...

It looks like you are looking for an L3 agent that is not installed in the compute node and is not reflected as necessary in the installation guide. I am using option 2 for the neutron configuration.

Regards,

neutron-server high CPU usage L3 try find compute node agent

Hello everyone,

I ask this question, to see if someone can help me ...

I have a controller node and a compute node.

The controller node also acts as a network node. I have used the following installation guide, for ubuntu and Pike version.

https://docs.openstack.org/pike/install/

The problem is the following:

When someone performs simulations and in this case auto-scaling with heat 30 instances. After a while in the controller node the neutron-server service reaches almost 100% of CPU usage.

In that case there is a slowdown in the cloud and when launching instances.


To get back down CPU usage I have to restart the neutron-linuxbridge-agent service of the compute node. But before that I see that the following appears in the logs of the compute node.

ERROR neutron.plugins.ml2.drivers.agent._common_agent Traceback (most recent call last):
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 336, in treat_devices_removed
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent     cfg.CONF.host)
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/rpc.py", line 139, in update_device_down
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent     agent_id=agent_id, host=host)
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/neutron/common/rpc.py", line 162, in call
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent     return self._original_context.call(ctxt, method, *kwargs)
**kwargs)
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 169, in call
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent     retry=self.retry)
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 123, in _send
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent     timeout=timeout, retry=retry)
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 578, in send
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent     retry=retry)
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 569, in _send
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent     raise result
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent RemoteError: Remote error: AgentNotFoundByTypeHost Agent with agent_type=L3 agent and host=urano could not be found
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent [u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', u'  File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', u'  File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch\n    result = func(ctxt, *new_args)\n', **new_args)\n', u'  File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/rpc.py", line 234, in update_device_down\n    n_const.PORT_STATUS_DOWN, host)\n', u'  File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/rpc.py", line 331, in notify_l2pop_port_wiring\n    l2pop_driver.obj.update_port_down(port_context)\n', u'  File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/l2pop/mech_driver.py", line 253, in update_port_down\n    admin_context, agent_host, [port[\'device_id\']]):\n', u'  File "/usr/lib/python2.7/dist-packages/neutron/db/l3_agentschedulers_db.py", line 303, in list_router_ids_on_host\n    context, constants.AGENT_TYPE_L3, host)\n', u'  File "/usr/lib/python2.7/dist-packages/neutron/db/agents_db.py", line 291, in _get_agent_by_type_and_host\n    host=host)\n', u'AgentNotFoundByTypeHost: Agent with agent_type=L3 agent and host=urano could not be found\n'].
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent

neutron.plugins.ml2.drivers.agent._common_agent

The compute node is called "urano" and when it restarts the aforementioned service it stabilizes again ...

It looks like you are looking for an L3 agent that is not installed in the compute node and is not reflected as necessary in the installation guide. I am using option 2 for the neutron configuration.

Regards,

neutron-server high CPU usage L3 try find compute node agent

Hello everyone,

I ask this question, to see if someone can help me ...

I have a controller node and a compute node.

The controller node also acts as a network node. I have used the following installation guide, for ubuntu and Pike version.

https://docs.openstack.org/pike/install/

The problem is the following:

When someone performs simulations and in this case auto-scaling with heat 30 instances. After a while in the controller node the neutron-server service reaches almost 100% of CPU usage.

In that case there is a slowdown in the cloud and when launching instances.


To get back down CPU usage I have to restart the neutron-linuxbridge-agent service of the compute node. But before that I see that the following appears in the logs of the compute node.

ERROR neutron.plugins.ml2.drivers.agent._common_agent Traceback (most recent call last):
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 336, in treat_devices_removed
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent     cfg.CONF.host)
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/rpc.py", line 139, in update_device_down
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent     agent_id=agent_id, host=host)
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/neutron/common/rpc.py", line 162, in call
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent     return self._original_context.call(ctxt, method, **kwargs)
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 169, in call
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent     retry=self.retry)
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 123, in _send
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent     timeout=timeout, retry=retry)
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 578, in send
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent     retry=retry)
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 569, in _send
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent     raise result
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent RemoteError: Remote error: AgentNotFoundByTypeHost Agent with agent_type=L3 agent and host=urano could not be found
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent [u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', u'  File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', u'  File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch\n    result = func(ctxt, **new_args)\n', u'  File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/rpc.py", line 234, in update_device_down\n    n_const.PORT_STATUS_DOWN, host)\n', u'  File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/rpc.py", line 331, in notify_l2pop_port_wiring\n    l2pop_driver.obj.update_port_down(port_context)\n', u'  File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/l2pop/mech_driver.py", line 253, in update_port_down\n    admin_context, agent_host, [port[\'device_id\']]):\n', u'  File "/usr/lib/python2.7/dist-packages/neutron/db/l3_agentschedulers_db.py", line 303, in list_router_ids_on_host\n    context, constants.AGENT_TYPE_L3, host)\n', u'  File "/usr/lib/python2.7/dist-packages/neutron/db/agents_db.py", line 291, in _get_agent_by_type_and_host\n    host=host)\n', u'AgentNotFoundByTypeHost: Agent with agent_type=L3 agent and host=urano could not be found\n'].
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent

The compute node is called "urano" and when it restarts the aforementioned service it stabilizes again ...

It looks like you are looking for an L3 agent that is not installed in the compute node and is not reflected as necessary in the installation guide. I am using option 2 for the neutron configuration.

Regards,

neutron-server high CPU usage L3 try find compute node agentagent. neutron: Error, "AgentNotFoundByTypeHost: Agent with agent_type=L3 agent and host=urano could not be found.

Hello everyone,

I ask this question, to see if someone can help me ...

I have a controller node and a compute node.

The controller node also acts as a network node. I have used the following installation guide, for ubuntu and Pike version.

https://docs.openstack.org/pike/install/

The problem is the following:

When someone performs simulations and in this case auto-scaling with heat 30 instances. After a while in the controller node the neutron-server service reaches almost 100% of CPU usage.

In that case there is a slowdown in the cloud and when launching instances.


To get back down CPU usage I have to restart the neutron-linuxbridge-agent service of the compute node. But before that I see that the following appears in the logs of the compute node.

ERROR neutron.plugins.ml2.drivers.agent._common_agent Traceback (most recent call last):
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 336, in treat_devices_removed
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent     cfg.CONF.host)
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/rpc.py", line 139, in update_device_down
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent     agent_id=agent_id, host=host)
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/neutron/common/rpc.py", line 162, in call
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent     return self._original_context.call(ctxt, method, **kwargs)
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 169, in call
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent     retry=self.retry)
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 123, in _send
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent     timeout=timeout, retry=retry)
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 578, in send
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent     retry=retry)
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 569, in _send
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent     raise result
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent RemoteError: Remote error: AgentNotFoundByTypeHost Agent with agent_type=L3 agent and host=urano could not be found
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent [u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', u'  File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', u'  File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch\n    result = func(ctxt, **new_args)\n', u'  File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/rpc.py", line 234, in update_device_down\n    n_const.PORT_STATUS_DOWN, host)\n', u'  File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/rpc.py", line 331, in notify_l2pop_port_wiring\n    l2pop_driver.obj.update_port_down(port_context)\n', u'  File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/l2pop/mech_driver.py", line 253, in update_port_down\n    admin_context, agent_host, [port[\'device_id\']]):\n', u'  File "/usr/lib/python2.7/dist-packages/neutron/db/l3_agentschedulers_db.py", line 303, in list_router_ids_on_host\n    context, constants.AGENT_TYPE_L3, host)\n', u'  File "/usr/lib/python2.7/dist-packages/neutron/db/agents_db.py", line 291, in _get_agent_by_type_and_host\n    host=host)\n', u'AgentNotFoundByTypeHost: Agent with agent_type=L3 agent and host=urano could not be found\n'].
2017-12-30 10:33:38.088 1574 ERROR neutron.plugins.ml2.drivers.agent._common_agent

The compute node is called "urano" and when it restarts the aforementioned service it stabilizes again ...

It looks like you are looking for an L3 agent that is not installed in the compute node and is not reflected as necessary in the installation guide. I am using option 2 for the neutron configuration.

Regards,