Dear all, i am facing a serious issue with my openstack setup. I am running openstack Icehouse on Ubuntu 12.0.4.1 and used Mirantis Fuel 5.1.1 to deploy. I have a HA architecture with three controllers/network nodes. Moreover i am using one L3 agent to run two external networks. The above setup was working well for some months now when suddenly some of my users complained that they cannot access their VM's from the internet (via floating IP).

Debugging the problem i found out that not all of my namespaces are created on all of my network/controller nodes. For example i have node-1, node-2 and node-3. When the L3 agent runs on either node-2 or node-3 only one qrouter namespace is created/loaded, when the L3 agent runs on node-1 i have 19 q router namespaces. Same goes for the qdhcp namespaces. When they run on node-1 and node-3 i have 25 qdhcp namespaces. Node-2 loads only 1.

I checked the mysql tables and all networks and routers are on all nodes. Has anybody else faced something like this?? I am in desperate need for help:)