Ask Your Question
0

Instance metadata not found (http error [404]) for instances started on a compute node

asked 2012-02-27 20:43:17 -0500

alshishtawy gravatar image

I installed OpenStack Diablo release on two nodes: a controller node with (api, compute, network, scheduler, vncproxy) and a compute node with (compute, network). I use FlatDHCPManager.

The problem is that instances started on the compute node can not find meta data (they can reach the api server) I can ping those instances but not ssh. The console output contains the following error: ... cloud-init start-local running: Mon, 27 Feb 2012 19:48:47 +0000. up 1.95 seconds no instance data found in start-local cloud-init-nonet waiting 120 seconds for a network device. ci-info: lo : 1 127.0.0.1 255.0.0.0
ci-info: eth0 : 1 192.168.22.40 255.255.255.0 02:16:3e:70:79:48 ci-info: route-0: 0.0.0.0 192.168.22.20 0.0.0.0 eth0 UG ci-info: route-1: 192.168.22.0 0.0.0.0 255.255.255.0 eth0 U cloud-init start running: Mon, 27 Feb 2012 19:48:59 +0000. up 14.12 seconds 2012-02-27 19:49:09,381 - DataSourceEc2.py[WARNING]: 'http://169.254.169.254' failed: http error [404] 2012-02-27 19:49:10,487 - DataSourceEc2.py[WARNING]: 'http://169.254.169.254' failed: http error [404] 2012-02-27 19:49:11,591 - DataSourceEc2.py[WARNING]: 'http://169.254.169.254' failed: http error [404] ...

instances on the controller node run correctly. I also use the flag --ec2_dmz_host on the compute node

edit retag flag offensive close merge delete

6 answers

Sort by ยป oldest newest most voted
0

answered 2012-03-03 05:55:51 -0500

For multi-node installation, the root cause for both http error[404] or url error are, I think, request bad url upon vm initialization when starting up on compute mode, which is resulted from bad nova.conf configuration, esp. the network config.

When I changed nova.conf at each of the compute node as follows, all vms can be set up correctly on each of node, and ssh to the vm would be fine as well.

--network_manager=nova.network.manager.FlatDHCPManager --fixed_range=10.100.80.0/24 --network_size=256 --flat_network_dhcp_start=10.100.80.2 --public_interface=eth1 --flat_interface=eth0 --flat_network_bridge=br100 --flat_injected=false

P.S., the scenario for my deployment is as fallows, control: 10.100.79.136 compute node: 10.100.79.136, 10.100.79.137.

and I created the private network as: nova-manage network create private --fixed_range_v4=10.100.80.0/24 --num_networks=1 --network_size=256 --multi_host=T

edit flag offensive delete link more
0

answered 2012-02-27 20:58:31 -0500

alshishtawy gravatar image

When I remove the flag --ec2_dmz_host from the compute node the console output becomes:

cloud-init start-local running: Mon, 27 Feb 2012 20:54:39 +0000. up 1.91 seconds no instance data found in start-local ci-info: lo : 1 127.0.0.1 255.0.0.0
ci-info: eth0 : 1 192.168.22.41 255.255.255.0 02:16:3e:52:1f:dd ci-info: route-0: 0.0.0.0 192.168.22.20 0.0.0.0 eth0 UG ci-info: route-1: 192.168.22.0 0.0.0.0 255.255.255.0 eth0 U cloud-init start running: Mon, 27 Feb 2012 20:54:53 +0000. up 15.14 seconds 2012-02-27 20:55:03,278 - DataSourceEc2.py[WARNING]: 'http://169.254.169.254' failed: url error [[Errno 111] Connection refused] 2012-02-27 20:55:04,282 - DataSourceEc2.py[WARNING]: 'http://169.254.169.254' failed: url error [[Errno 111] Connection refused]

So it seems in the original setup, with the --ec2_dmz_host flag, instances can reach the api server. Why can't find metadata?

edit flag offensive delete link more
0

answered 2012-02-27 21:12:55 -0500

vishvananda gravatar image

Your traffic is probably being snatted, so the metadata server gets the wrong source ip and can't look up info. Try setting --dmz_cidr to the ip address of your api host e.g. --dmz_cidr=192.168.22.1/32 (or wherever your api host is running)

On Feb 27, 2012, at 1:00 PM, Ahmad Al-Shishtawy wrote:

Question #189026 on OpenStack Compute (nova) changed: https://answers.launchpad.net/nova/+q...

Ahmad Al-Shishtawy gave more information on the question: When I remove the flag --ec2_dmz_host from the compute node the console output becomes:

cloud-init start-local running: Mon, 27 Feb 2012 20:54:39 +0000. up 1.91 seconds no instance data found in start-local ci-info: lo : 1 127.0.0.1 255.0.0.0
ci-info: eth0 : 1 192.168.22.41 255.255.255.0 02:16:3e:52:1f:dd ci-info: route-0: 0.0.0.0 192.168.22.20 0.0.0.0 eth0 UG ci-info: route-1: 192.168.22.0 0.0.0.0 255.255.255.0 eth0 U cloud-init start running: Mon, 27 Feb 2012 20:54:53 +0000. up 15.14 seconds 2012-02-27 20:55:03,278 - DataSourceEc2.py[WARNING]: 'http://169.254.169.254' failed: url error [[Errno 111] Connection refused] 2012-02-27 20:55:04,282 - DataSourceEc2.py[WARNING]: 'http://169.254.169.254' failed: url error [[Errno 111] Connection refused]

So it seems in the original setup, with the --ec2_dmz_host flag, instances can reach the api server. Why can't find metadata?


You received this question notification because you are a member of Nova Core, which is an answer contact for OpenStack Compute (nova).

edit flag offensive delete link more
0

answered 2012-02-27 21:47:10 -0500

alshishtawy gravatar image

When I set --dmz_cidr=<api host="" ip="">/32 I get:

2012-02-27 21:35:50,279 - DataSourceEc2.py[WARNING]: 'http://169.254.169.254' failed: url error [timed out]

When I set --dmz_cidr=192.168.22.1/32 I get same error: 2012-02-27 21:41:28,384 - DataSourceEc2.py[WARNING]: 'http://169.254.169.254' failed: http error [404]

I forgot to mention that I have one nic (eth0)

edit flag offensive delete link more
0

answered 2012-03-01 13:16:33 -0500

Same issue here, after I added iptables nova-network-PREROUTING

cloud-init start-local running: Thu, 01 Mar 2012 07:59:09 +0000. up 1.70 seconds no instance data found in start-local ci-info: lo : 1 127.0.0.1 255.0.0.0
ci-info: eth0 : 1 10.100.80.3 255.255.255.0 02:16:3e:6f:92:f3 ci-info: route-0: 0.0.0.0 10.100.80.1 0.0.0.0 eth0 UG ci-info: route-1: 10.100.80.0 0.0.0.0 255.255.255.0 eth0 U cloud-init start running: Thu, 01 Mar 2012 07:59:11 +0000. up 3.03 seconds 2012-03-01 07:59:11,365 - DataSourceEc2.py[WARNING]: 'http://169.254.169.254' failed: http error [404] 2012-03-01 07:59:12,433 - DataSourceEc2.py[WARNING]: 'http://169.254.169.254' failed: http error [404] 2012-03-01 07:59:13,509 - DataSourceEc2.py[WARNING]: 'http://169.254.169.254' failed: http error [404] 2012-03-01 07:59:14,577 - DataSourceEc2.py[WARNING]: 'http://169.254.169.254' failed: http error [404]

edit flag offensive delete link more
0

answered 2012-07-18 16:10:41 -0500

kermit666 gravatar image

See a possible solution here:

http://askubuntu.com/questions/165115/ubuntu-cloud-instances-in-openstack-cant-import-public-ssh-key (http://askubuntu.com/questions/165115...)

Basically, you should try restarting nova-api and rebooting your instance:

$ sudo restart nova-api
edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2012-02-27 20:43:17 -0500

Seen: 626 times

Last updated: Jul 18 '12