Ask Your Question
0

OVS ignoring new floating IPs

asked 2019-02-28 07:50:40 -0500

plobo gravatar image

Hi all,

Hoping you'll be able to help shed some light or point me in the right direction.

I'm currently faced with a situation very similar to what is described in this https://ask.openstack.org/en/question/101254/instance-not-accessible-from-public-network (question) with a minor difference. Whereas in the previous question, the OP couldn't connect to any floating IP, I am able to connect to floating IPs which already existed.

A little background

  • Cluster deployed with Kolla Ansible using Rocky.
  • Cluster comprised of:
    • 2 Compute nodes;
    • 3 Control/Network nodes;
    • 4 Storage nodes;
    • 4 VLANs configured for neutron_external, tenant_networks and 2x ceph networks

Cluster was working fine a couple of weeks ago. Instances were created without issues and floating IPs assigned and working well. Yesterday, when creating a new instance and assigning a floating IP, all seemed to work normally except I can't access the instance via floating IP. Internal networking is ok (I can ping and ssh between instances) and I can access the outside network from the instance. Just can't access via floating IP.

When troubleshooting, I noticed that the neutron_openvswitch_agent container had been restarted a week ago and seems to coincide with the time this issue may have started. Also, on the compute/control nodes, the vxlan_sys_4789 interface appears as disconnected/down.

I'd appreciate some pointers in the right direction as to how I can further troubleshoot this and correct this issue.

Like I said, existing floating IPs are working just fine. I even removed a floating IP from an existing instance and assigned it to the new instance and it works (I can access new instance using old floating IP), so this seems to be isolated to newly created floating IPs.

Thanks in advance,
Pedro

edit retag flag offensive close merge delete

Comments

can you check for the subnet masks what you using for floating IP. may be it have crossed the range.

TijoV gravatar imageTijoV ( 2019-03-01 03:39:19 -0500 )edit

Don't think that's the issue as we're using a /24 and only have 10 used IPs. Like I said in original post, existing IPs are working fine. I can even move them from 1 instance to another and they work. New floating IPs don't however so seems like something is not getting configured along the way

plobo gravatar imageplobo ( 2019-03-01 04:39:45 -0500 )edit

Other new floating ips works in the same compute host? might be an issue with ovs on the specific node?

Eduardo Gonzalez gravatar imageEduardo Gonzalez ( 2019-03-01 05:22:50 -0500 )edit

Yeah, thinking that too. Any existing floating IP is working just fine, so has to be something specific to new floating IPs and OVS

plobo gravatar imageplobo ( 2019-03-01 08:13:24 -0500 )edit

Likely something ovs-vswitchd->ovsdb missmatch or something, maybe namespaces wasnt cleared properly by l3 or something else. Need more info to track the issue

Eduardo Gonzalez gravatar imageEduardo Gonzalez ( 2019-03-04 04:25:28 -0500 )edit

1 answer

Sort by » oldest newest most voted
0

answered 2019-03-01 09:24:02 -0500

plobo gravatar image

Network restart and restarting kolla containers did nothing, however rebooting the server solved the issue. I can only assume that for some odd reason, osvdb wasn't pickling up changes made…?

I really hate not knowing what was causing this (especially since I fear it may happen again), so while this situation is "solved" (had to try reboot since I had some users with urgency) I'd still love some pointers on what possibly was happening and how to best troubleshoot and solve this in future scenarios.

Now with a little less "pressure" I can look into this a little more calmly so would appreciate the pointers and guidance.

Thanks, P.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2019-02-28 07:50:40 -0500

Seen: 27 times

Last updated: Mar 01