Ask Your Question
0

VM Can't get DHCP IP

asked 2014-07-24 16:18:56 -0500

RStack gravatar image

updated 2014-07-24 18:27:12 -0500

I'm running OpenStack RDO with the following setup.

  • 1x Controller Node ( Neutron - Nova - glance - Horizon - Keystone - GRE + FlatDHCPManager)
  • 10x Compute Nodes ( Nova Compute - KVM )
  • OS : CentOS 6 - 64bit

Suddenly all vm's on the compute nodes can't get new IP address from the controller node. It was working properly hours ago without any modifications from me. I tried to restart neutron-dhcp-agent - qpidd and dnsmasq with no luck.

This log from the controller node - many errors

http://paste.openstack.org/show/88016/

And this from the compute node

http://paste.openstack.org/show/88017/

Realtime compute node log after restarting vm network service

http://paste.openstack.org/show/88039/

Installed packages on Controller Node : rpm -qa | grep -i opensta

openstack-selinux-0.1.3-2.el6ost.noarch
openstack-puppet-modules-2013.2-9.1.el6.noarch
openstack-ceilometer-api-2013.2.3-2.el6.noarch
openstack-packstack-2013.2.1-0.36.dev1013.el6.noarch
openstack-nova-scheduler-2013.2.3-1.el6.noarch
openstack-ceilometer-common-2013.2.3-2.el6.noarch
python-django-openstack-auth-1.1.2-1.el6.noarch
openstack-ceilometer-central-2013.2.3-2.el6.noarch
openstack-ceilometer-collector-2013.2.3-2.el6.noarch
openstack-neutron-openvswitch-2013.2.3-9.el6.noarch
openstack-nova-common-2013.2.3-1.el6.noarch
openstack-packstack-puppet-2013.2.1-0.36.dev1013.el6.noarch
openstack-glance-2013.2.3-2.el6.noarch
openstack-nova-conductor-2013.2.3-1.el6.noarch
openstack-nova-novncproxy-2013.2.3-1.el6.noarch
openstack-nova-cert-2013.2.3-1.el6.noarch
openstack-keystone-2013.2.3-3.el6.noarch
openstack-neutron-2013.2.3-9.el6.noarch
openstack-ceilometer-alarm-2013.2.3-2.el6.noarch
openstack-dashboard-2013.2.3-1.el6.noarch
openstack-nova-api-2013.2.3-1.el6.noarch
openstack-nova-console-2013.2.3-1.el6.noarch
openstack-utils-2013.2-2.el6.noarch

On Controller Node

 >> ps aux | grep -i dhcp
nobody   12639  0.0  0.0  12884   864 ?        S    Jul19   0:00 dnsmasq --no-hosts --no-resolv --strict-order --bind-interfaces --interface=tap7139b265-41 --except-interface=lo --pid-file=/var/lib/neutron/dhcp/c19ca2ea-8278-4069-bfea-dadd92961cac/pid --dhcp-hostsfile=/var/lib/neutron/dhcp/c19ca2ea-8278-4069-bfea-dadd92961cac/host --dhcp-optsfile=/var/lib/neutron/dhcp/c19ca2ea-8278-4069-bfea-dadd92961cac/opts --leasefile-ro --dhcp-range=tag0,10.0.0.0,static,86400s --dhcp-lease-max=256 --conf-file= --domain=openstacklocal
neutron  24884  0.0  0.1 273748 32140 ?        S    Jul24   0:00 /usr/bin/python /usr/bin/neutron-dhcp-agent --log-file /var/log/neutron/dhcp-agent.log --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/dhcp_agent.ini

>> ip netns
qrouter-id
qdhcp-id

>> ip netns exec qdhcp-network-id ip a
16: tap7139b265-41: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether fa:16:3e:2e:18:35 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.3/24 brd 10.0.0.255 scope global tap7139b265-41
    inet6 fe80::f816:3eff:fe2e:1835/64 scope link 
       valid_lft forever preferred_lft forever
17: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
>> ip netns exec qdhcp-network-id ifconfig
lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:4 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4 errors:0 dropped ...
(more)
edit retag flag offensive close merge delete

Comments

I notice in http://paste.openstack.org/show/88017/ that there are AMQP connection failures and then reconnection. I don't have a solution but I suggest looking at your message queue & see if there are errors there.

jproulx gravatar imagejproulx ( 2014-07-24 18:10:00 -0500 )edit

@jproulx I restarted qpidd several times with no luck. What's the log file path please should i review ?

RStack gravatar imageRStack ( 2014-07-24 18:17:08 -0500 )edit

@jproulx This is a realtime compute node log after restarting vm network service

http://paste.openstack.org/show/88039/

RStack gravatar imageRStack ( 2014-07-24 18:27:45 -0500 )edit

Edit the question to add more details and remove the unnecesary ones. Make the question readable to others as you go on debugging.

smaffulli gravatar imagesmaffulli ( 2014-07-25 11:08:51 -0500 )edit

@vasanth-rajasekaran i have disabled IPtables with the same problem.

RStack gravatar imageRStack ( 2014-07-25 21:37:01 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
3

answered 2014-07-25 04:59:04 -0500

vasanth-rajasekaran gravatar image

Stop the Iptables services in both controller and compute nodes with following commands: iptables -F service iptables stop

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

2 followers

Stats

Asked: 2014-07-24 16:18:56 -0500

Seen: 2,294 times

Last updated: Jul 24 '14