Random timeouts while quering metadata

asked 2017-05-28 14:48:20 -0600

baurzhan-konurbayev gravatar image

Hi guys,

I have VMware Integrated Openstack setup with recent Openstack release.

An issue happens when I try to spawn 30 instances simultenously. The problem is that metadata service sometimes is unavailable for some reason and the most likely it could be due to overloading for some of services or some other OS limitations.

Tried to increase metadata_workers up to 8 both on Neutron server and Controllers, but still no luck.

What I could find only is timeout messages in metadata proxy.

Please, see neutron-ns-metadata-proxylog:

latest _proxy_request /usr/lib/python2.7/dist-packages/neutron/agent/metadata/namespace_proxy.py:93
2017-05-28 18:36:05.508 1875 INFO neutron.wsgi [-] 192.168.3.154 - - [28/May/2017 18:36:05] "GET / HTTP/1.1" 200 215 0.031583
2017-05-28 18:36:05.517 1875 DEBUG neutron.wsgi [-] (1875) timed out ('192.168.3.158', 38478) process_request /usr/lib/python2.7/dist-packages/eventlet/wsgi.py:724
2017-05-28 18:36:05.517 1875 DEBUG neutron.wsgi [-] (1875) accepted ('192.168.3.152', 41824) server /usr/lib/python2.7/dist-packages/eventlet/wsgi.py:867
2017-05-28 18:36:05.549 1875 DEBUG neutron.wsgi [-] (1875) timed out ('192.168.3.146', 59032) process_request /usr/lib/python2.7/dist-packages/eventlet/wsgi.py:724
2017-05-28 18:36:05.549 1875 DEBUG neutron.wsgi [-] (1875) accepted ('192.168.3.158', 38586) server /usr/lib/python2.7/dist-packages/eventlet/wsgi.py:867
2017-05-28 18:36:05.549 1875 DEBUG neutron.agent.metadata.namespace_proxy [-] Request: GET / HTTP/1.0

Couldn't find any other errors in neutron logs, nova API logs, Ubuntu logs.

I have HA setup with two DHCP agents.

Appreciate any help.

Thanks.

edit retag flag offensive close merge delete