Ask Your Question
3

What causes the meta-data service timing out on some nodes?

asked 2013-05-10 18:33:19 -0500

Trent gravatar image

updated 2013-05-10 18:50:43 -0500

smaffulli gravatar image

Hey guys- Has anyone seen this sort of thing and have any idea what is up? The weird thing is that is only happens on some of the compute nodes in this cluster not all of them- Thanks!

cloudinit startlocal running: Fri, 10 May 2013 23:29:23 +0000. up 2.99 seconds
no instance data found in startlocal
ciinfo: lo    : 1 127.0.0.1       255.0.0.0       .
ciinfo: eth0  : 1 10.160.4.152    255.255.255.0   fa:16:3e:eb:d9:13
ciinfo: route0: 0.0.0.0         10.160.4.150    0.0.0.0         eth0   UG
ciinfo: route1: 10.160.4.0      0.0.0.0         255.255.255.0   eth0   U
cloudinit start running: Fri, 10 May 2013 23:29:24 +0000. up 3.44 seconds
20130510 23:30:15,787  util.py[WARNING]: 'http://169.254.169.254/20090404/metadata/instanceid' failed [51/120s]: socket timeout [timed out]
20130510 23:31:06,841  util.py[WARNING]: 'http://169.254.169.254/20090404/metadata/instanceid' failed [102/120s]: socket timeout [timed out]
20130510 23:31:23,860  util.py[WARNING]: 'http://169.254.169.254/20090404/metadata/instanceid' failed [119/120s]: socket timeout [timed out]
20130510 23:31:24,862  DataSourceEc2.py[CRITICAL]: giving up on md after 120 seconds
edit retag flag offensive close merge delete

Comments

Which version of OpenStack are you running? Earlier versions (eg Diablo, Essex) had some problems with the scalability of the metadata service that are fixed in Folsom and beyond.

fifieldt gravatar imagefifieldt ( 2013-05-12 00:03:50 -0500 )edit

I'm getting same error all time, but if I try to curl this: http://169.254.169.254/2009-04-04/meta-data/instance-id it works, with dashes. Why they are missing on instance boot I didn't figure out.

Alen Komljen gravatar imageAlen Komljen ( 2013-05-24 07:29:56 -0500 )edit

About the missing dashes AFAIK is just a bug in the log and they are not really missing. I used to have problems because of the controller node being overloaded. After the instance boot can you curl http://192.254.169.254/2009-04-04/meta-data/instance-id ?

fmanco gravatar imagefmanco ( 2013-06-15 09:18:44 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
0

answered 2013-05-10 20:37:23 -0500

updated 2013-05-10 20:37:41 -0500

I'd tcpdump on the interface of the HV and track down why these packages aren't getting there. You should be able to run curl 169.254.169.254 from your instance to generate another request. Also is there anything coming up in the logs?

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

Stats

Asked: 2013-05-10 18:33:19 -0500

Seen: 1,187 times

Last updated: May 10 '13