Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Running a router inside an OpenStack VM

I'm trying to make a router VM (actually an IPsec VPN server) inside OpenStack. The OpenStack is an installation of Mirantis 7 (why not newer? 8 from some reason causes kernel OOPS to the same image that works on 7, and 9 that was just released... not tested yet).

We're using a Ceph-based installation and "Neutron with tunnelling segmentation".

Looking inside a VM at the output of 'arp -n' shows that it sees MAC addresses of other VMs in its network, so I have imagined that a simple 'route add 10.0.0.0/24 gw [other VM IP]' in Linux, then pinging 10.0.0.1 would result in those packets appearing on tcpdump running on [other VM IP] output. That does not seem to be the case.

Googling a bit, I found this document: https://wiki.openstack.org/wiki/Neutron/ML2PortSecurityExtensionDriver - which sounds relevant (I have a similar behaviour in GCE - which is solvable simply by creating VMs with a "IP Forwarding" flag enabled) - so I tried that (or at least I think I did, the instructions assume I know the precise steps, while I am merely guessing), by editing /etc/neutron/plugins/ml2/ml2_conf.ini on each and every one of the controllers in my Mirantis cluster, and adding the line:

extension_drivers = port_security

and then executing:

service neutron-server restart

which, by the way, did not have any affect at the beginning. I am assuming that maybe just one of the 3 is primary (active and 2 standby's), and as long as one with the old config was running, that kept things alive.

When I did the same on the last controller - all hell broke lose - and API calls to get the network status (openstack ... network show [netID]) returned that the API returned an error.

Then I started examining the logs, I found out that the below exception was thrown every time I tried the API call:

<166>Jul 11 16:49:35 node-25 neutron-metadata-agent 2016-07-11 16:49:35.993 17687 INFO eventlet.wsgi.server [-] (17687) accepted ''
<163>Jul 11 16:49:36 node-25 neutron-metadata-agent 2016-07-11 16:49:36.074 17687 ERROR neutron.agent.metadata.agent [-] Unexpected error.
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent Traceback (most recent call last):
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 109, in __call__
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent     instance_id, tenant_id = self._get_instance_and_tenant_id(req)
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 216, in _get_instance_and_tenant_id
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent     ports = self._get_ports(remote_address, network_id, router_id)
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 204, in _get_ports
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent     networks = self._get_router_networks(router_id)
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent   File "/usr/lib/python2.7/dist-packages/neutron/common/utils.py", line 101, in __call__
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent     return self._get_from_cache(target_self, *args, **kwargs)
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent   File "/usr/lib/python2.7/dist-packages/neutron/common/utils.py", line 79, in _get_from_cache
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent     item = self.func(target_self, *args, **kwargs)
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 153, in _get_router_networks
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent     internal_ports = self._get_ports_from_server(router_id=router_id)
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 127, in _get_ports_from_server
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent     return self.plugin_rpc.get_ports(self.context, filters)
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 70, in get_ports
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent     return cctxt.call(context, 'get_ports', filters=filters)
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent   File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 156, in call
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent     retry=self.retry)
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent   File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 90, in _send
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent     timeout=timeout, retry=retry)
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 350, in send
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent     retry=retry)
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 341, in _send
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent     raise result
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent TypeError: 'NoneType' object has no attribute '__getitem__'
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent Traceback (most recent call last):
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent 
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent   File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent     executor_callback))
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent 
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent   File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent     executor_callback)
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent 
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent   File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent     result = func(ctxt, **new_args)
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent 
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent   File "/usr/lib/python2.7/dist-packages/neutron/api/rpc/handlers/metadata_rpc.py", line 43, in get_ports
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent     return self.plugin.get_ports(context, filters=filters)
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent 
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent   File "/usr/lib/python2.7/dist-packages/neutron/db/db_base_plugin_v2.py", line 1979, in get_ports
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent     items = [self._make_port_dict(c, fields) for c in query]
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent 
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent   File "/usr/lib/python2.7/dist-packages/neutron/db/db_base_plugin_v2.py", line 936, in _make_port_dict
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent     attributes.PORTS, res, port)
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent 
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent   File "/usr/lib/python2.7/dist-packages/neutron/db/common_db_mixin.py", line 162, in _apply_dict_extend_functions
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent     func(*args)
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent 
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent   File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/plugin.py", line 493, in _ml2_md_extend_port_dict
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent     self.extension_manager.extend_port_dict(session, portdb, result)
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent 
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent   File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/managers.py", line 796, in extend_port_dict
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent     driver.obj.extend_port_dict(session, base_model, result)
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent 
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent   File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/extensions/port_security.py", line 63, in extend_port_dict
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent     self._extend_port_security_dict(result, db_data)
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent 
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent   File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/extensions/port_security.py", line 67, in _extend_port_security_dict
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent     db_data['port_security'][psec.PORTSECURITY])
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent 
2016-07-11 16:49:36.074 17687 TRACE neutron.agent.metadata.agent TypeError: 'NoneType' object has no attribute '__getitem__'

So I have two alternative questions here: If the issue I think I'm experiencing makes sense (it's not supposed to work without making the change I want to change) - then - any idea why it causes this exception?

And if I'm wrong with my assumption that it's not supposed to work without this change (or in other words - VMs SHOULD be able to route traffic to foreign subnets to another VM in the same LAN, i.e. spoof the destination) - then any idea what I may be doing wrong while trying to make this work?

Thanks!