metadata server - 500 Internal Server error

asked 2016-12-14 08:48:06 -0500

vathanlal gravatar image

updated 2016-12-23 04:45:43 -0500

Hello I created an Openstack newton cluster I have cirros and ubuntu VMs running in the cluster. I can ssh into cirros instance using cirros and cubswin:) . I can ping the 8.8.8.8 from the cirros instance and I can ssh into the cirros instance from any node in my network. But when I run curl http://169.254.169.254 from inside the instance it is giving me

500 Internal Server Error

Remote metadata server experienced an internal server error.

My /var/log/neutron/neutron-metadata-agent.log has the below error also

2016-12-23 11:33:02.111 25058 DEBUG neutron.agent.metadata.agent [-] Request: GET / HTTP/1.0^M
Accept-Encoding: gzip, deflate^M
Content-Length: 0^M
Content-Type: text/plain^M
Host: 169.254.169.254^M
User-Agent: Python-httplib2/0.9.1 (gzip)^M
X-Forwarded-For: 192.168.1.11^M
X-Neutron-Router-Id: baa1e7f8-1915-46ec-a46d-f90c4d1bfc1c __call__ /usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py:86
2016-12-23 11:33:02.117 25058 DEBUG oslo_messaging._drivers.amqpdriver [-] CALL msg_id: 3996b10a61f7415ca8a4721c5ab008b0 exchange 'neutron' topic 'q-plugin' _send /usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:448
2016-12-23 11:33:02.199 25058 DEBUG oslo_messaging._drivers.amqpdriver [-] received reply msg_id: 3996b10a61f7415ca8a4721c5ab008b0 __call__ /usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:296
2016-12-23 11:33:02.201 25058 DEBUG oslo_messaging._drivers.amqpdriver [-] CALL msg_id: f80218ffb0d2404281a02185d71d8125 exchange 'neutron' topic 'q-plugin' _send /usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:448
2016-12-23 11:33:02.268 25058 DEBUG oslo_messaging._drivers.amqpdriver [-] received reply msg_id: f80218ffb0d2404281a02185d71d8125 __call__ /usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:296
2016-12-23 11:33:02.282 25058 ERROR neutron.agent.metadata.agent [-] Unexpected error.
2016-12-23 11:33:02.282 25058 ERROR neutron.agent.metadata.agent Traceback (most recent call last):
2016-12-23 11:33:02.282 25058 ERROR neutron.agent.metadata.agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 90, in __call__
2016-12-23 11:33:02.282 25058 ERROR neutron.agent.metadata.agent     return self._proxy_request(instance_id, tenant_id, req)
2016-12-23 11:33:02.282 25058 ERROR neutron.agent.metadata.agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 220, in _proxy_request
2016-12-23 11:33:02.282 25058 ERROR neutron.agent.metadata.agent     raise Exception(_('Unexpected response code: %s') % resp.status)
2016-12-23 11:33:02.282 25058 ERROR neutron.agent.metadata.agent Exception: Unexpected response code: 300
2016-12-23 11:33:02.282 25058 ERROR neutron.agent.metadata.agent
2016-12-23 11:33:02.285 25058 INFO eventlet.wsgi.server [-] 192.168.1.11,<local> - - [23/Dec/2016 11:33:02] "GET / HTTP/1.1" 500 229 0.173503

when I tried to ssh into ubuntu instance using mykey it is giving me error as below

OpenSSH_7.2p2 Ubuntu-4ubuntu2.1, OpenSSL 1.0.2g  1 Mar 2016
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to 141.30.74.132 [141.30.74.132] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1 ...
(more)
edit retag flag offensive close merge delete

Comments

-double check configuration, especially service_neutron_metadata_proxy /secret_key stuff -put both Neutron and Nova to debug -try to catch traffic by tcpdump between neutron-metadata-agent and nova metadata service if they are on different hosts

volenbovsky gravatar imagevolenbovsky ( 2016-12-16 11:19:13 -0500 )edit

My configuration is exactly the same as in the Document.. iam getting Remote metadata server experienced an internal server error. while executing curl http://169.254.169.254 from the instance.. I dont know how to troubleshoot this issue..

vathanlal gravatar imagevathanlal ( 2016-12-22 07:43:50 -0500 )edit