"500 error" failed to get instance-id of datasource

asked 2014-06-06 04:06:07 -0600

Deeptanshu
231 ●31 ●32 ●45

updated 2014-06-06 11:39:59 -0600

smaffulli
6981 ●38 ●68 ●102 http://maffulli.net/

We are unable to access the instance with key pair. On checking the logs we found that meta data is not working properly. We are getting 500 internal error while running the curl command from inside the instance. We are running neutron-metadata-agent on neutron server. I have configured the meta data as per suggestions still no success. We have "service" tenant in openstack , there is no tenant with "services" . We follow the openstack guide for installing Icehouse on ubuntu 12.04. We are running ML2+GRE+OVS

There is no /etc/neutron/metadata_agent.ini on controller node. Should it be present on controller ?

Please find the logs below and /etc/neutron/metadata_agent.ini file content from neutron server.

Neutron /etc/neutron/metadata_agent.ini

verbose = True
auth_url = http://192.168.10.2:5000/v2.0
auth_region = RegionOne
admin_tenant_name = service
admin_user = neutron
admin_password = de65e97a4e135
nova_metadata_ip =  192.168.10.2
metadata_proxy_shared_secret = 63797d4fereh48f7c8
nova_metadata_port = 8775
metadata_workers = 0

Nova.conf output from controller

# cat /etc/nova/nova.conf | grep metadata | grep -v ^#

enabled_apis=ec2,osapi_compute,metadata
service_neutron_metadata_proxy = true
neutron_metadata_proxy_shared_secret = 63797d4fereh48f7c8
metadata_host = 192.168.10.2
metadata_listen = 0.0.0.0
metadata_workers=2
metadata_listen_port=8775

curl http://169.254.169.254/2009-04-04/instance-id

<head>
<title>500 Internal Server Error</title>
</head>
<body>
500 Internal Server Error
Remote metadata server experienced an internal server error.

On controller

$ netstat -lntp | grep 8775
tcp 0 0 0.0.0.0:8775 0.0.0.0:* LISTEN 28072/python

$ netstat -lntp | grep 9697

$ ps -ef | grep 28072
nova 28072 1 1 11:23 ? 00:00:14 /usr/bin/python /usr/bin/nova-api --config-file=/etc/nova/nova.conf
nova 28157 28072 0 11:23 ? 00:00:00 /usr/bin/python /usr/bin/nova-api --config-file=/etc/nova/nova.conf
nova 28158 28072 0 11:23 ? 00:00:00 /usr/bin/python /usr/bin/nova-api --config-file=/etc/nova/nova.conf
nova 28159 28072 0 11:23 ? 00:00:00 /usr/bin/python /usr/bin/nova-api --config-file=/etc/nova/nova.conf
nova 28160 28072 0 11:23 ? 00:00:00 /usr/bin/python /usr/bin/nova-api --config-file=/etc/nova/nova.conf
nova 28161 28072 0 11:23 ? 00:00:00 /usr/bin/python /usr/bin/nova-api --config-file=/etc/nova/nova.conf
nova 28162 28072 0 11:23 ? 00:00:00 /usr/bin/python /usr/bin/nova-api --config-file=/etc/nova/nova.conf
nova 28163 28072 0 11:23 ? 00:00:00 /usr/bin/python /usr/bin/nova-api --config-file=/etc/nova/nova.conf
nova 28164 28072 0 11:23 ? 00:00:00 /usr/bin/python /usr/bin/nova-api --config-file=/etc/nova/nova.conf
nova 28180 28072 0 11:23 ? 00:00:01 /usr/bin/python /usr/bin/nova-api --config-file=/etc/nova/nova.conf
nova 28181 28072 0 11:23 ? 00:00:02 /usr/bin/python /usr/bin/nova-api --config-file=/etc/nova/nova.conf
nova 28182 28072 0 11:23 ? 00:00:00 /usr/bin/python /usr/bin/nova-api --config-file=/etc/nova/nova.conf
nova 28183 28072 0 11:23 ? 00:00:00 /usr/bin/python /usr ...

(more)

edit retag flag offensive close merge delete

Comments

If neutron-metadata-agent runs on Controller. Answer is yes
Port 5000 is incorrect, not space. Focus your attention.

dbaxps ( 2014-06-06 04:59:50 -0600 )edit

Hi,

Will Iptables cause any issue on compute controller or neutron ?

Deeptanshu ( 2014-06-06 05:51:21 -0600 )edit

What reports /var/log/nova/nova-compute.log on Compute Node during VM booting UP ? Please,post.

Does it report :-
2014-06-03 10:11:53.730 10125 TRACE nova.compute.manager [instance: 2bda1669-85ab-4c10-8bcc-ab9e522dae05]     raise exception.VirtualInterfaceCreateException()
2014-06-03 10:11:53.730 10125 TRACE nova.compute.manager [instance: 2bda1669-85ab-4c10-8bcc-ab9e522dae05] VirtualInterfaceCreateException: Virtual Interface creation failed

dbaxps ( 2014-06-06 08:03:29 -0600 )edit

guys: thank you both for the time you're taking to fix this issue. Please try to keep this site usable: update questions and answers while you go, use comments only for short comments (should be limited to 200 chars)

smaffulli ( 2014-06-06 11:41:01 -0600 )edit

add a comment

2 answers

Sort by » oldest newest most voted

answered 2014-06-06 04:33:17 -0600

dbaxps

10788 ●57 ●112 ●224 http://bderzhavets.blo...

updated 2014-06-06 11:46:28 -0600

smaffulli
6981 ●38 ●68 ●102 http://maffulli.net/

There should be no metadata_agent.ini file on the controller. It is only available on neutron node If neutron-metadata-agent doesn't run on Controller.

Pay attention to auth_url in both files. I also see 2 mistakes in metadata_agent.ini on Neutron Server

auth_url = http://192.168.10.2 : 5000/v2.0 - should be 35357/v2.0
admin_tenant_name = service - should be services
nova_metadata_port = 8775 one more mistake

Some suggestions on how to debug this issue further:

Since you are running ML2+GRE+OVS setup, check configurations here http://kashyapc.fedorapeople.org/virt...

Iptables can cause issues on compute controller or neutron, view http://kashyapc.fedorapeople.org/virt... for iptables tuning

Another thing I would try is to switch to 9697 in nova.conf on Controller, in metadata_agent.ini on Neutron Server. That's what I did on Two Node IceHouse Neutron OVS&GRE Setup to succeed. Just attempt to match Neutron metadata port and Nova metadata port . But in your case on different boxes. Check outputs and configs per http://bderzhavets.blogspot.com/2014/...

Review the Notes for (manually) setting up a 2-node minimal IceHouse RDO setup in virtual machines

Don't forget to restart services after each configuration change.

Review on Neutron Server:

[root@icehouse1 neutron(keystone_admin)]# cat metadata_agent.conf
[DEFAULT]
debug = False
>>> PAY ATTENTION HERE:   **auth_url = http://192.168.1.127:35357/v2.0**
auth_region = RegionOne
auth_insecure = False
admin_tenant_name = services
admin_user = neutron
admin_password = 5f11f559abc94440
nova_metadata_ip = 192.168.1.127
nova_metadata_port = 8775
metadata_proxy_shared_secret =227f7bbc8b6f4f74
metadata_workers = 0

On Controller Node:

[root@icehouse2 neutron]# cat metadata_agent.conf
[DEFAULT]
>>> PAY ATTENTION HERE **auth_url = http://localhost:5000/v2.0**<br>
auth_region = RegionOne
admin_tenant_name = %SERVICE_TENANT_NAME%
admin_user = %SERVICE_USER%
admin_password = %SERVICE_PASSWORD%

Make sure nova.conf on Compute has following entries

[root@icehouse2 nova]# cat nova.conf  | grep metadata_
metadata_host = 192.168.1.127
metadata_listen = 0.0.0.0
metadata_listen_port = 8775
service_neutron_metadata_proxy = True
neutron_metadata_proxy_shared_secret = 227f7bbc8b6f4f74

edit flag offensive delete link

Comments

In case of IceHouse Neutron ML2&OVS&GRE Setup file metadata_agent.ini should be the same on Neutron Sever and on Compute Nodes. What corresponds Kashyap notices, mentioned above
http://kashyapc.fedorapeople.org/virt...
On Compute Node

[root@icehouse2  neutron(keystone_admin)]# cat metadata_agent.ini
[DEFAULT]
debug = False
>>> PAY ATTENTION HERE:   **auth_url = http://192.168.1.127:35357/v2.0**
auth_region = RegionOne
auth_insecure = False
admin_tenant_name = services
admin_user = neutron
admin_password = 5f11f559abc94440
nova_metadata_ip = 192.168.1.127
nova_metadata_port = 8775
metadata_proxy_shared_secret =227f7bbc8b6f4f74
metadata_workers = 0

dbaxps ( 2014-06-10 08:24:21 -0600 )edit

add a comment

answered 2015-07-12 22:52:53 -0600

haichuan0227
1 ●1

all my config is fine, this not work for me.

is there any other solution ?

edit flag offensive delete link

add a comment

"500 error" failed to get instance-id of datasource

Comments

2 answers

Comments

Get to know Ask OpenStack

Question Tools

Stats

Related questions

Feedback About This Page

OpenStack

Community

Documentation

Branding & Legal

"500 error" failed to get instance-id of datasource edit

Comments

2 answers

Comments

Get to know Ask OpenStack

Question Tools

Stats

Related questions

Feedback About This Page

OpenStack

Community

Documentation

Branding & Legal

"500 error" failed to get instance-id of datasource