Revision history [back]

click to hide/show revision 1
initial version

Neutron - DHCP Agent is unable to enable dhcp

I am not able to login to the VM that I am have created. I am using a CentOS 6.4 + Havana on 3 node installation. On the network node, I see message in the dhcp-agent.log file - unable to enable dhcp. Any idea why?

2014-02-04 17:40:00.172 10487 ERROR neutron.agent.dhcp_agent [-] Unable to enable dhcp. 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent Traceback (most recent call last): 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.6/site-packages/neutron/agent/dhcp_agent.py", line 126, in call_driver 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent getattr(driver, action)(**action_kwargs) 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.6/site-packages/neutron/agent/linux/dhcp.py", line 167, in enable 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent reuse_existing=True) 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.6/site-packages/neutron/agent/linux/dhcp.py", line 705, in setup 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent namespace=network.namespace) 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.6/site-packages/neutron/agent/linux/interface.py", line 174, in plug 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent namespace2=namespace) 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.6/site-packages/neutron/agent/linux/ip_lib.py", line 126, in add_veth 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent self.ensure_namespace(namespace2) 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.6/site-packages/neutron/agent/linux/ip_lib.py", line 136, in ensure_namespace 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent ip = self.netns.add(name) 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.6/site-packages/neutron/agent/linux/ip_lib.py", line 447, in add 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent self._as_root('add', name, use_root_namespace=True) 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.6/site-packages/neutron/agent/linux/ip_lib.py", line 217, in _as_root 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent kwargs.get('use_root_namespace', False)) 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.6/site-packages/neutron/agent/linux/ip_lib.py", line 70, in _as_root 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent namespace) 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.6/site-packages/neutron/agent/linux/ip_lib.py", line 81, in _execute 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent root_helper=root_helper) 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.6/site-packages/neutron/agent/linux/utils.py", line 62, in execute 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent raise RuntimeError(m) 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent RuntimeError: 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'add', 'qdhcp-14b6e9ee-50d4-44fd-916e-91260ff62f32'] 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent Exit code: 255 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent Stdout: '' 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent Stderr: 'Bind /proc/self/ns/net -> /var/run/netns/qdhcp-14b6e9ee-50d4-44fd-916e-91260ff62f32 failed: No such file or directory\n' 2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent 2014-02-04 17:40:00.280 10487 DEBUG neutron.openstack.common.rpc.amqp [-] Making asynchronous cast on q-plugin... cast /usr/lib/python2.6/site-packages/neutron/openstack/common/rpc/amqp.py:559 2014-02-04 17:40:00.280 10487 DEBUG neutron.openstack.common.rpc.amqp [-] UNIQUE_ID is c03bacfe49a94ee6a8e038f696be74c5. _add_unique_id /usr/lib/python2.6/site-packages/neutron/openstack/common/rpc/amqp.py:339 2014-02-04 17:40:00.283 10487 DEBUG qpid.messaging.io.ops [-] SENT[25d9cf8]: ExchangeQuery(name='neutron', id=serial(0), sync=True) write_op /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:685 2014-02-04 17:40:00.285 10487 DEBUG qpid.messaging.io.raw [-] SENT[25d9cf8]: '\x0f\x01\x00\x1a\x00\x01\x00\x00\x00\x00\x00\x00\x07\x03\x01\x01\x01\x00\x07neutron\x0f\x01\x00\x1a\x00\x01\x00\x00\x00\x00\x00\x00\x08\x04\x01\x01\x01\x00\x07neutron' writeable /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:480 2014-02-04 17:40:00.285 10487 DEBUG qpid.messaging.io.raw [-] READ[25d9cf8]: '\x0f\x01\x00,\x00\x01\x00\x00\x00\x00\x00\x00\x03\x02\x01\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x12\x07\x01\t\x00\x05topic\x00\x00\x00\x04\x00\x00\x00\x00\x0f\x00\x00\x1a\x00\x00\x00\x00\x00\x00\x00\x00\x02\n\x01\x00\x00\x08\x00\x00\x00\x00\x00\x00\x00\x00\x0f\x01\x00\x1e\x00\x01\x00\x00\x00\x00\x00\x00\x03\x02\x01\x00\x03\x00\x00\x00\x00\x01\x00\x00\x00\x04\x08\x01\x00\x00\x0f\x00\x00\x1a\x00\x00\x00\x00\x00\x00\x00\x00\x02\n\x01\x00\x00\x08\x00\x00\x00\x00\x00\x00\x00\x01' readable /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:416 2014-02-04 17:40:00.287 10487 DEBUG qpid.messaging.io.ops [-] RCVD[25d9cf8]: ExecutionResult(command_id=serial(0), value=ExchangeQueryResult(type=u'topic', arguments={}), id=serial(0)) write /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:653 2014-02-04 17:40:00.287 10487 DEBUG qpid.messaging.io.ops [-] RCVD[25d9cf8]: SessionCompleted(commands=[0-0]) write /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:653 2014-02-04 17:40:00.288 10487 DEBUG qpid.messaging.io.ops [-] RCVD[25d9cf8]: ExecutionResult(command_id=serial(1), value=QueueQueryResult(), id=serial(1)) write /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:653 2014-02-04 17:40:00.289 10487 DEBUG qpid.messaging.io.ops [-] RCVD[25d9cf8]: SessionCompleted(commands=[0-1]) write /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:653 2014-02-04 17:40:00.291 10487 DEBUG qpid.messaging.io.ops [-] SENT[25d9cf8]: MessageTransfer(destination='neutron', id=serial(2), sync=True, headers=(DeliveryProperties(routing_key='q-plugin'), MessageProperties(content_type='amqp/map', application_headers={'qpid.subject': 'q-plugin'})), payload='\x00\x00\x02\xde\x00\x00\x00\x02\x0coslo.message\x95\x02\xb7{"_context_roles": ["admin"], "_context_read_deleted": "no", "args": {"agent_state": {"agent_state": {"binary": "neutron-dhcp-agent", "topic": "dhcp_agent", "host": "Openstack-Network", "agent_type": "DHCP agent", "configurations": {"subnets": 0, "use_namespaces": true, "dhcp_lease_duration": 86400, "dhcp_driver": "neutron.agent.linux.dhcp.Dnsmasq", "networks": 0, "ports": 0}}}, "time": "2014-02-05T01:40:00.279910"}, "namespace": null, "_unique_id": "c03bacfe49a94ee6a8e038f696be74c5", "_context_timestamp": "2014-02-05 01:40:00.279761", "_context_is_admin": true, "version": "1.0", "_context_project_id": null, "_context_tenant_id": null, "_context_user_id": null, "method": "report_state"}\x0coslo.version\x95\x00\x032.0') write_op /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:685 2014-02-04 17:40:00.292 10487 DEBUG qpid.messaging [-] SENT[25ec2d8]: Message(properties={'qpid.subject': 'q-plugin'}, content={'oslo.message': '{"_context_roles": ["admin"], "_context_read_deleted": "no", "args": {"agent_state": {"agent_state": {"binary": "neutron-dhcp-agent", "topic": "dhcp_agent", "host": "Openstack-Network", "agent_type": "DHCP agent", "configurations": {"subnets": 0, "use_namespaces": true, "dhcp_lease_duration": 86400, "dhcp_driver": "neutron.agent.linux.dhcp.Dnsmasq", "networks": 0, "ports": 0}}}, "time": "2014-02-05T01:40:00.279910"}, "namespace": null, "_unique_id": "c03bacfe49a94ee6a8e038f696be74c5", "_context_timestamp": "2014-02-05 01:40:00.279761", "_context_is_admin": true, "version": "1.0", "_context_project_id": null, "_context_tenant_id": null, "_context_user_id": null, "method": "report_state"}', 'oslo.version': '2.0'}) send /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:1282 2014-02-04 17:40:00.293 10487 DEBUG qpid.messaging.io.raw [-] SENT[25d9cf8]: '\x0b\x01\x00\x1a\x00\x01\x00\x00\x00\x00\x00\x00\x04\x01\x01\x01\x01\x00\x07neutron\x03\x02\x00N\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\r\x04\x01\x00\x02\x08q-plugin\x00\x00\x00-\x04\x03\x10\x01\x08amqp/map\x00\x00\x00\x1c\x00\x00\x00\x01\x0cqpid.subject\x95\x00\x08q-plugin\x07\x03\x02\xee\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x02\xde\x00\x00\x00\x02\x0coslo.message\x95\x02\xb7{"_context_roles": ["admin"], "_context_read_deleted": "no", "args": {"agent_state": {"agent_state": {"binary": "neutron-dhcp-agent", "topic": "dhcp_agent", "host": "Openstack-Network", "agent_type": "DHCP agent", "configurations": {"subnets": 0, "use_namespaces": true, "dhcp_lease_duration": 86400, "dhcp_driver": "neutron.agent.linux.dhcp.Dnsmasq", "networks": 0, "ports": 0}}}, "time": "2014-02-05T01:40:00.279910"}, "namespace": null, "_unique_id": "c03bacfe49a94ee6a8e038f696be74c5", "_context_timestamp": "2014-02-05 01:40:00.279761", "_context_is_admin": true, "version": "1.0", "_context_project_id": null, "_context_tenant_id": null, "_context_user_id": null, "method": "report_state"}\x0coslo.version\x95\x00\x032.0' writeable /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:480 2014-02-04 17:40:00.294 10487 DEBUG qpid.messaging.io.raw [-] READ[25d9cf8]: '\x0f\x00\x00\x1a\x00\x00\x00\x00\x00\x00\x00\x00\x02\n\x01\x00\x00\x08\x00\x00\x00\x00\x00\x00\x00\x02' readable /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:416 2014-02-04 17:40:00.294 10487 DEBUG qpid.messaging.io.ops [-] RCVD[25d9cf8]: SessionCompleted(commands=[0-2]) write /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:653 2014-02-04 17:40:00.294 10487 DEBUG qpid.messaging [-] RACK[25ec2d8]: Message(properties={'qpid.subject': 'q-plugin'}, content={'oslo.message': '{"_context_roles": ["admin"], "_context_read_deleted": "no", "args": {"agent_state": {"agent_state": {"binary": "neutron-dhcp-agent", "topic": "dhcp_agent", "host": "Openstack-Network", "agent_type": "DHCP agent", "configurations": {"subnets": 0, "use_namespaces": true, "dhcp_lease_duration": 86400, "dhcp_driver": "neutron.agent.linux.dhcp.Dnsmasq", "networks": 0, "ports": 0}}}, "time": "2014-02-05T01:40:00.279910"}, "namespace": null, "_unique_id": "c03bacfe49a94ee6a8e038f696be74c5", "_context_timestamp": "2014-02-05 01:40:00.279761", "_context_is_admin": true, "version": "1.0", "_context_project_id": null, "_context_tenant_id": null, "_context_user_id": null, "method": "report_state"}', 'oslo.version': '2.0'}) msg_acked /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:1271 2014-02-04 17:40:00.295 10487 DEBUG qpid.messaging.io.ops [-] SENT[25d9cf8]: SessionDetach(name='ab612efb-6727-41c1-a542-edc60a14633d:24') write_op /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:685 2014-02-04 17:40:00.295 10487 DEBUG qpid.messaging.io.raw [-] SENT[25d9cf8]: "\x0f\x00\x009\x00\x00\x00\x00\x00\x00\x00\x00\x02\x03\x01\x00\x00'ab612efb-6727-41c1-a542-edc60a14633d:24" writeable /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:480 2014-02-04 17:40:00.296 10487 DEBUG qpid.messaging.io.raw [-] READ[25d9cf8]: "\x0f\x00\x00:\x00\x00\x00\x00\x00\x00\x00\x00\x02\x04\x03\x00\x00'ab612efb-6727-41c1-a542-edc60a14633d:24\x00" readable /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:416 2014-02-04 17:40:00.296 10487 DEBUG qpid.messaging.io.ops [-] RCVD[25d9cf8]: SessionDetached(name='ab612efb-6727-41c1-a542-edc60a14633d:24', code=0) write /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:653 2014-02-04 17:40:00.297 10487 DEBUG qpid.messaging.io.ops [-] SENT[25d9cf8]: SessionAttach(name='ab612efb-6727-41c1-a542-edc60a14633d:25') write_op /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:685 2014-02-04 17:40:00.297 10487 DEBUG qpid.messaging.io.ops [-] SENT[25d9cf8]: SessionCommandPoint(command_id=serial(0), command_offset=0) write_op /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:685 2014-02-04 17:40:00.297 10487 DEBUG qpid.messaging.io.raw [-] SENT[25d9cf8]: "\x0f\x00\x009\x00\x00\x00\x00\x00\x00\x00\x00\x02\x01\x01\x00\x00'ab612efb-6727-41c1-a542-edc60a14633d:25\x0f\x00\x00\x1c\x00\x00\x00\x00\x00\x00\x00\x00\x02\x07\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" writeable /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:480 2014-02-04 17:40:00.298 10487 DEBUG qpid.messaging.io.raw [-] READ[25d9cf8]: "\x0f\x00\x009\x00\x00\x00\x00\x00\x00\x00\x00\x02\x02\x01\x00\x00'ab612efb-6727-41c1-a542-edc60a14633d:25\x0f\x00\x00\x1c\x00\x00\x00\x00\x00\x00\x00\x00\x02\x07\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" readable /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:416 2014-02-04 17:40:00.298 10487 DEBUG qpid.messaging.io.ops [-] RCVD[25d9cf8]: SessionAttached(name='ab612efb-6727-41c1-a542-edc60a14633d:25') write /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:653 2014-02-04 17:40:00.298 10487 DEBUG qpid.messaging.io.ops [-] RCVD[25d9cf8]: SessionCommandPoint(command_id=serial(0), command_offset=0) write /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:653