Ask Your Question
0

cloud-init not assigning IP

asked 2015-04-21 11:29:24 -0500

bobyakov gravatar image

updated 2015-04-21 11:49:24 -0500

Hi GUys,

Have a problem accessing any instance via ssh. Getting host unreachable. I have made no changes and this issue just started out of the blue, was working fine before. Running ubuntu 14.04 and Icehouse

See below error when instance is booting, I suspect issue is related to change in /etc/network/interface to /etc/network/interface.d Since I can't access instance have no idea how to get around this. PLEASE help!!!!

Error:

  cloud-init-nonet[7.17]: waiting 10 seconds for network device
 * Starting set console keymap[74G[ OK ]
 * Starting Signal sysvinit that virtual filesystems are mounted[74G[ OK ]
 * Starting Signal sysvinit that virtual filesystems are mounted[74G[ OK ]
 * Starting set sysctls from /etc/sysctl.conf[74G[ OK ]
 * Starting Bridge udev events into upstart[74G[ OK ]
 * Stopping set sysctls from /etc/sysctl.conf[74G[ OK ]
 * Starting Signal sysvinit that remote filesystems are mounted[74G[ OK ]
 * Stopping set console keymap[74G[ OK ]
 * Starting device node and kernel event manager[74G[ OK ]
 * Starting load modules from /etc/modules[74G[ OK ]
 * Starting cold plug devices[74G[ OK ]
 * Starting log initial device creation[74G[ OK ]
 * Stopping load modules from /etc/modules[74G[ OK ]
 * Starting Uncomplicated firewall[74G[ OK ]
 * Starting configure network device security[74G[ OK ]
 * Starting configure network device security[74G[ OK ]
 * Starting Mount network filesystems[74G[ OK ]
 * Stopping Mount network filesystems[74G[ OK ]
 * Starting Bridge socket events into upstart[74G[ OK ]
 * Starting configure network device[74G[ OK ]
 * Stopping cold plug devices[74G[ OK ]
 * Stopping log initial device creation[74G[ OK ]
 * Starting enable remaining boot-time encrypted block devices[74G[ OK ]
cloud-init-nonet[17.19]: waiting 120 seconds for network device
cloud-init-nonet[137.19]: gave up waiting for a network device.
Cloud-init v. 0.7.5 running 'init' at Tue, 21 Apr 2015 16:21:30 +0000. Up 137.37 seconds.
ci-info: +++++++++++++++++++++++Net device info+++++++++++++++++++++++
ci-info: +--------+------+-----------+-----------+-------------------+
ci-info: | Device |  Up  |  Address  |    Mask   |     Hw-Address    |
ci-info: +--------+------+-----------+-----------+-------------------+
ci-info: |   lo   | True | 127.0.0.1 | 255.0.0.0 |         .         |
ci-info: |  eth0  | True |     .     |     .     | fa:16:3e:5b:87:1e |
ci-info: +--------+------+-----------+-----------+-------------------+
ci-info: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!Route info failed!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
2015-04-21 16:21:30,479 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [0/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by <class 'socket.error'>: [Errno 101] Network is unreachable)]
2015-04-21 16:21:31,485 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [1/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by <class 'socket.error'>: [Errno 101] Network is unreachable)]
2015-04-21 16:21:32,489 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [2/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by <class 'socket.error'>: [Errno 101] Network is

draft saved... tags: unreachable)]

edit retag flag offensive close merge delete

Comments

Since yesterday, we are facing the same problem in our HA Liberty deployment. Have you found a solution to this problem?

Herr-Herner gravatar imageHerr-Herner ( 2016-01-11 23:40:25 -0500 )edit

I deleted the interface on compute host and restarted instance. It created new interface and started working. I never found root cause.

bobyakov gravatar imagebobyakov ( 2016-01-12 10:17:05 -0500 )edit

6 answers

Sort by ยป oldest newest most voted
2

answered 2015-04-21 12:23:35 -0500

It looks like a DHCP issue, have you seen this issue with any other instances? The link below may be able to elaborate on the issue. There are quite a few DHCP related questions on this site with answers, so I would recommend reading through a few to have a stronger understanding of what variables are in play. Good luck.

https://ask.openstack.org/en/question...

edit flag offensive delete link more

Comments

That is the exact issue i'm having , he does not have an answer either.

bobyakov gravatar imagebobyakov ( 2015-04-21 13:06:28 -0500 )edit
0

answered 2015-12-25 09:20:45 -0500

Here are some ways to get fixed IP: 1. you define network with DHCP enable, and then create the instance with nova boot command and specify static IP. 2. you define network with DHCP disable, but define the router (L3) and take use of router for metadata service. 3. you define network with DHCP disable and no L3, you can define the static IP in boot script and take use of nova boot command with config_driver parameter.

edit flag offensive delete link more
0

answered 2015-04-21 14:09:10 -0500

bobyakov gravatar image

updated 2015-04-22 10:50:16 -0500

Hi All, Still don't know the root cause, but I deleted the internal interface on the virtual router on the one project and recreated it and that seems to have fixed the issue. Everything must be configured correctly because it is working now.

Any ideas as to why this happened are welcome in case it happens again.

Update<<<<< Created an additional network and new interface on virtual router, same image on new network works. Old network with existing instances does not. Tried deleting interface on old network and recreating , same issue.

edit flag offensive delete link more

Comments

Issue came back after a day. I suspect it is related to virtual ports on compute node. One instance is working one is not on same compute node different networks.. Don't know enough how to troubleshoot, any input would be grateful.

bobyakov gravatar imagebobyakov ( 2015-04-24 15:06:43 -0500 )edit
0

answered 2015-04-21 12:12:25 -0500

rahulrajvn gravatar image

Hi Try restarting the neutron-metadata service in neutron server and check it logs whether the request from new servers are reaching the neutron-metadata service.

edit flag offensive delete link more

Comments

1

I think the issue is that the instance is not getting IP therefore unable to reach metadata, does that sound correct?

bobyakov gravatar imagebobyakov ( 2015-04-21 12:55:45 -0500 )edit

Ya seems you are right. Have you tried setting the MTU for the dhcp service and your NICS. which is mentioned as optional under "To configure the DHCP agent" in the following Document. http://docs.openstack.org/juno/instal... Setting Mtu for NIC

rahulrajvn gravatar imagerahulrajvn ( 2015-04-21 13:46:44 -0500 )edit

Yes I lowered it to 1462 had issue with connecting when I first built it. Lowering mtu fixed that.

bobyakov gravatar imagebobyakov ( 2015-04-21 14:10:13 -0500 )edit
0

answered 2015-04-21 13:31:30 -0500

jdiaz9 gravatar image

There are a variety of issues that can lead to this problem. Initially, I would suggest you look over your network topology to check and see if the subnet is connected to your router. If you happen to have a link missing then I would suggest you attach the subnet to your router by attaching the interface to it.

I would also suggest you check if DHCP is enabled on your subnet. neutron subnet-show <subnet_id> will show you just that. If DHCP happens to be disabled this would be your problem. (you would have to manually create the port and assign it manually to the instance if that is the case).

Another possibility is that the issue is specific to your DHCP agent. It might be the case that the neutron DHCP agent was too slow in granting a lease to the instance and cloud-init gave out the warnings you see above. Usually, a reboot would resolve that issue and allow cloud-init to complete successfully.

Good luck.

edit flag offensive delete link more

Comments

Still have 1 instance not working on another project, yes enable_dhcp = True

bobyakov gravatar imagebobyakov ( 2015-04-21 14:16:29 -0500 )edit
0

answered 2015-04-21 12:38:28 -0500

dodi gravatar image

On your network topology, have you attached a port from your router to your internal network, the requirements are:

  1. install the neutron-metadata and make sure its in the nova.conf and add metadata in the following line: i.e. enabled_apis=ec2,osapi_compute,metadata

you should also have the following:

neutron_metadata_proxy_shared_secret=(your shared secret) service_neutron_metadata_proxy = True metadata_listen=0.0.0.0

Since the metadata communicates through the virtual router you will need to attach the internal subnet/network to the router where it can get the metadata attributes.

edit flag offensive delete link more

Comments

on the neutron network server should the value for metadata_listen= controller ? Currently I have it pointed to HAProxy which then points to controller. The weird thing is this was all working recently, and some VM's still work, most do not.

bobyakov gravatar imagebobyakov ( 2015-04-21 12:57:54 -0500 )edit

Also on neutron server the metadata_agent.ini file has value nova_metadata_ip= not metadata_listen

bobyakov gravatar imagebobyakov ( 2015-04-21 13:02:22 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2015-04-21 11:29:24 -0500

Seen: 8,960 times

Last updated: Dec 25 '15