Ask Your Question
1

Can't reach metadata server from VM running in single instance setup

asked 2013-07-01 09:22:34 -0500

Adrian Smith gravatar image

updated 2013-07-05 08:44:40 -0500

I've setup a single node Nova VM using these instructions from ilearnstack. The problem is that instances launched in this environment aren't able to access the metadata server. With CirrOS the error is,

cloudsetup: checking http://169.254.169.254/20090404/metadata/instanceid
wget: can't connect to remote host (169.254.169.254): No route to host

The VM is successfully receiving an IP address.

Since this setup doesn't seem to run an L3 agent I tried setting enable_isolated_metadata = True in /etc/quantum/dhcp_agent.ini (as per this message). That didn't make any difference.

I'm guessing there's something fundamentally wrong with what I've done as I can't ping the VM from within the host (I presume that should be possible?).

Edit: some additional details and questions: Routing table included below. When I launch a VM I can't ping the VM from the host node (I've updated the security groups to allow ICMP messages). I'm using vlan tenant network types. An eth1.1000 device is created by quantum for this purpose. I was wondering though, will the L2 agent drop incoming messages on eth1 if they're not tagged with the 1000 vlan id (which presumably they won't if I'm pinging from the host)?

Some details about the environment,

The Nova VM has three NICs,

# Host-only network
auto eth0
iface eth0 inet static
address 10.10.100.51
netmask 255.255.255.0

# Internal network
auto eth1
iface eth1 inet static
address 192.168.20.10
netmask 255.255.255.0

# NAT network
auto eth2
iface eth2 inet dhcp

Other Details,

  • Using Linux Bridge
  • When I launched a VM the host node got 5 new network devices, ns-c056d062-7e (10.0.0.3), tap31db1cf1-05, tapc056d062-7e, brq08772967-38 (bridging the previous devices) and eth1.1000

Bridge Details,

# brctl show
bridge name      bridge id          STP enabled   interfaces
brq08772967-38   8000.080027872ea9  no            eth1.1000
                                                  tap57bc3d92-15
                                                  tapc056d062-7e

Routes on Host:

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.0.4.2        0.0.0.0         UG    100    0        0 eth2
10.0.4.0        0.0.0.0         255.255.255.0   U     0      0        0 eth2
10.10.100.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
192.168.20.0    0.0.0.0         255.255.255.0   U     0      0        0 eth1
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr0

This routing table looks wrong to me. The VM I created is attached to eth1 (well eth1.1000). I guess this means I should have a route for the VM network (10.0.0.0/24) going to eth1. I created that route using route add -net 10.0.0.0/24 dev eth1, and tried pinging again. I still don't get a response. Using tcpdump -n -i eth1 I can see the arp requests. However there's no ... (more)

edit retag flag offensive close merge delete

Comments

Just for troubleshooting can you put firewall_driver=nova.virt.firewall.NoopFirewallDriver in nova.conf? after changing it restart nova services, then try to ping the VM. I suspect something is blocking at the iptables level need to isolate the issue.

Ashokb gravatar imageAshokb ( 2013-07-03 13:10:17 -0500 )edit

I set the firewall_driver, restarted all services and created a new VM. Still not luck unfortunately. I still can't ping or reach the VM from the host. I also tried using "quantum security-group-rule-create" to allow ICMP and SSH traffic but no joy.

Adrian Smith gravatar imageAdrian Smith ( 2013-07-05 07:29:58 -0500 )edit

Ok as a final try do a #iptables -F. then try to ping . It could work . I do hope that vm received the dhcp ip .

Ashokb gravatar imageAshokb ( 2013-07-05 20:54:28 -0500 )edit

2 answers

Sort by ยป oldest newest most voted
2

answered 2013-07-01 10:23:39 -0500

Arfghl gravatar image

I think it is not possible for Cirros image to contact the metdata service on a flat network because the DHCP option 121 is ot available for this image. Try with an Ubuntu image for exemple and besure that you have got this line in your nova.conf ([NETWORK]): service_quantum_metadata_proxy=true

edit flag offensive delete link more

Comments

I've tried this with no luck unfortunately. The error is,

util.py[WARNING]: 'http://169.254.169.254/20090404/metadata/instanceid' failed [3/120s]: url error [[Errno 113] No route to host]

I presume the setting 'quantum_metadata_proxy_shared_secret' would have no bearing at this stage?

Adrian Smith gravatar imageAdrian Smith ( 2013-07-02 10:46:03 -0500 )edit

Try to see in your nova-api log if the metadata-server get a request from your VM. And be sure you can reach your VM from your compute node because Openstack does not configure this for you.

Arfghl gravatar imageArfghl ( 2013-07-03 02:57:22 -0500 )edit

There doesn't appear to be anything metadata related in the nova-api log. Your second suggestion is interesting. I can't ping the VM from the host. I've included more details/questions above as space is limited here.

Adrian Smith gravatar imageAdrian Smith ( 2013-07-03 10:55:31 -0500 )edit

Doesn't an APIPA address suggest a NIC / HA which isn't getting either DHCP or a manually defined IP address, and therefore it parks itself over in 169.254 where it's out of the way and won't interfere with IP traffic? http://compnetworking.about.com/cs/protocolsdhcp/g/bldef_apipa.htm

Kiloseven gravatar imageKiloseven ( 2013-10-15 12:58:57 -0500 )edit
1

answered 2013-07-01 12:26:11 -0500

Ashokb gravatar image

Don't use Cirros it has an open bug regarding stateless route pushing. At present metadata push for Cirros can only happen through router. If you use Precise Ubuntu image then the push can happen through dhcp.

https://bugs.launchpad.net/cirros/+bug/1190372

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

Stats

Asked: 2013-07-01 09:22:34 -0500

Seen: 2,633 times

Last updated: Jul 05 '13