Ask Your Question
1

Why does Metadata Agent keep falling over?

asked 2014-11-15 11:52:12 -0500

jproulx gravatar image

updated 2014-11-15 11:55:16 -0500

I've been running Icehouse neutron-metadata-agent (with ML2 OVS and network namespaces) since I upgraded from Havana in August.

Within the last few days it's started periodically failing. Restarting the metadata-agent usually fixes, sometimes I also need to restart the dhcp-agent but I'm not 100% sure that is required. Service works sometimes for hours sometimes minutes then goes back into failed state.

The neutron-ns-metadataproxy-<uuid>.log shows this for all requests when failing:

2014-11-15 12:32:57.316 30563 INFO neutron.wsgi [-] (30563) accepted ('10.10.161.17', 40973)

2014-11-15 12:32:57.317 30563 DEBUG neutron.agent.metadata.namespace_proxy [-] Request: GET /latest/meta-data/ HTTP/1.0
Accept: */*
Content-Type: text/plain
Host: 169.254.169.254 __call__ /usr/lib/python2.7/dist-packages/neutron/agent/metadata/namespace_proxy.py:68
2014-11-15 12:32:57.318 30563 ERROR neutron.agent.metadata.namespace_proxy [-] Unexpected error.
2014-11-15 12:32:57.318 30563 TRACE neutron.agent.metadata.namespace_proxy Traceback (most recent call last):
2014-11-15 12:32:57.318 30563 TRACE neutron.agent.metadata.namespace_proxy   File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/namespace_proxy.py", line 74, in __call__
2014-11-15 12:32:57.318 30563 TRACE neutron.agent.metadata.namespace_proxy     req.body)
2014-11-15 12:32:57.318 30563 TRACE neutron.agent.metadata.namespace_proxy   File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/namespace_proxy.py", line 105, in _proxy_request
2014-11-15 12:32:57.318 30563 TRACE neutron.agent.metadata.namespace_proxy     connection_type=UnixDomainHTTPConnection)
2014-11-15 12:32:57.318 30563 TRACE neutron.agent.metadata.namespace_proxy   File "/usr/lib/python2.7/dist-packages/httplib2/__init__.py", line 1569, in request
2014-11-15 12:32:57.318 30563 TRACE neutron.agent.metadata.namespace_proxy     (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
2014-11-15 12:32:57.318 30563 TRACE neutron.agent.metadata.namespace_proxy   File "/usr/lib/python2.7/dist-packages/httplib2/__init__.py", line 1316, in _request
2014-11-15 12:32:57.318 30563 TRACE neutron.agent.metadata.namespace_proxy     (response, content) = self._conn_request(conn, request_uri, method, body, headers)
2014-11-15 12:32:57.318 30563 TRACE neutron.agent.metadata.namespace_proxy   File "/usr/lib/python2.7/dist-packages/httplib2/__init__.py", line 1251, in _conn_request
2014-11-15 12:32:57.318 30563 TRACE neutron.agent.metadata.namespace_proxy     conn.connect()
2014-11-15 12:32:57.318 30563 TRACE neutron.agent.metadata.namespace_proxy   File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/namespace_proxy.py", line 48, in connect
2014-11-15 12:32:57.318 30563 TRACE neutron.agent.metadata.namespace_proxy     self.sock.connect(cfg.CONF.metadata_proxy_socket)
2014-11-15 12:32:57.318 30563 TRACE neutron.agent.metadata.namespace_proxy   File "/usr/lib/python2.7/dist-packages/eventlet/greenio.py", line 192, in connect
2014-11-15 12:32:57.318 30563 TRACE neutron.agent.metadata.namespace_proxy     while not socket_connect(fd, address):
2014-11-15 12:32:57.318 30563 TRACE neutron.agent.metadata.namespace_proxy   File "/usr/lib/python2.7/dist-packages/eventlet/greenio.py", line 39, in socket_connect
2014-11-15 12:32:57.318 30563 TRACE neutron.agent.metadata.namespace_proxy     raise socket.error(err, errno.errorcode[err])
2014-11-15 12:32:57.318 30563 TRACE neutron.agent ...
(more)
edit retag flag offensive close merge delete

Comments

does the metadata-agent log have anything?

darragh-oreilly gravatar imagedarragh-oreilly ( 2015-02-18 01:46:29 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
0

answered 2015-02-18 01:18:38 -0500

updated 2015-02-18 01:21:07 -0500

ECONNREFUSED is irrefutable - nothing was negotiated, there will be no IO.

Check to see if perchance SElinux is enabled, or if a firewall rule is precluding the connection.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2014-11-15 11:52:12 -0500

Seen: 437 times

Last updated: Feb 18 '15