Ask Your Question
0

packets not reaching VM

asked 2016-06-22 04:13:31 -0500

fresher gravatar image

updated 2016-06-22 23:22:47 -0500

We have a Openstack Juno setup with 1 controller+neutron node and 3 compute nodes. 1 VM (LB) has ipvsadm installed and two VMs act as back end server.

On the server with ipvsadm I have eth0:0 IP as 192.168.1.21 which acts as application IP. The ipvsadm uses round robin scheme. This is done using commands as below:

sudo ipvsadm -A -t 192.168.1.21:6000 -s rr
sudo ipvsadm -a -t 192.168.1.21:6000 -r 192.168.1.77:6000 -g
sudo ipvsadm -a -t 192.168.1.21:6000 -r 192.168.1.79:6000 -g

where 192.168.1.77 and 192.168.1.79 are back end server VM IP.

The problem is that the packets go out of the LB VM but never reach the back end server.

In the tcpdumps on various interfaces show that the packet reach till qbr of the LB VM but donot reach the qvo interface of LB VM. Are there any rules that get applied here which block these packets. The packets from the client VM are sent to back end server by the LB VM by changing the destination MAC of the packets.

The packets that leave LB VM to reach back end VM have source as the client VM IP and destination IP as 192.168.1.21 (application IP) and the src MAC of LB VM and dst MAC of backend server VM. Is this the reason for the packets to be blocked. Is there any way to allow these packets to flow to the back end server?

Please help us regarding this.

The host has many VMs thus a huge list when iptables -S is done. The LB VM IP is 192.168.1.75 

-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N neutron-filter-top
-N neutron-openvswi-FORWARD
-N neutron-openvswi-INPUT
-N neutron-openvswi-OUTPUT
-N neutron-openvswi-i10a4632b-9
-N neutron-openvswi-i131bf200-3
-N neutron-openvswi-i319ad1b9-b
-N neutron-openvswi-i3fb9dd53-d
-N neutron-openvswi-i4f240dc1-6
-N neutron-openvswi-i578e7d84-b
-N neutron-openvswi-i5c7b6b9f-d
-N neutron-openvswi-i5e93d25e-1
-N neutron-openvswi-i5fcd8d22-3
-N neutron-openvswi-i610385e2-d
-N neutron-openvswi-i93321459-0
-N neutron-openvswi-ib094e27b-1
-N neutron-openvswi-ib9882cfd-4
-N neutron-openvswi-ic2dad3ae-c
-N neutron-openvswi-ie8321530-4
-N neutron-openvswi-local
-N neutron-openvswi-o10a4632b-9
-N neutron-openvswi-o131bf200-3
-N neutron-openvswi-o319ad1b9-b
-N neutron-openvswi-o3fb9dd53-d
-N neutron-openvswi-o4f240dc1-6
-N neutron-openvswi-o578e7d84-b
-N neutron-openvswi-o5c7b6b9f-d
-N neutron-openvswi-o5e93d25e-1
-N neutron-openvswi-o5fcd8d22-3
-N neutron-openvswi-o610385e2-d
-N neutron-openvswi-o93321459-0
-N neutron-openvswi-ob094e27b-1
-N neutron-openvswi-ob9882cfd-4
-N neutron-openvswi-oc2dad3ae-c
-N neutron-openvswi-oe8321530-4
-N neutron-openvswi-s10a4632b-9
-N neutron-openvswi-s131bf200-3
-N neutron-openvswi-s319ad1b9-b
-N neutron-openvswi-s3fb9dd53-d
-N neutron-openvswi-s4f240dc1-6
-N neutron-openvswi-s578e7d84-b
-N neutron-openvswi-s5c7b6b9f-d
-N neutron-openvswi-s5e93d25e-1
-N neutron-openvswi-s5fcd8d22-3
-N neutron-openvswi-s610385e2-d
-N neutron-openvswi-s93321459-0
-N neutron-openvswi-sb094e27b-1
-N neutron-openvswi-sb9882cfd-4
-N neutron-openvswi-sc2dad3ae-c
-N neutron-openvswi-se8321530-4
-N neutron-openvswi-sg-chain
-N neutron-openvswi-sg-fallback
-A INPUT -j neutron-openvswi-INPUT
-A FORWARD -j neutron-filter-top
-A FORWARD -j neutron-openvswi-FORWARD
-A OUTPUT -j neutron-filter-top
-A OUTPUT -j neutron-openvswi-OUTPUT
-A neutron-filter-top -j neutron-openvswi-local
-A neutron-openvswi-FORWARD -m physdev --physdev-out tap5fcd8d22-30 --physdev-is-bridged -j neutron-openvswi-sg-chain
-A neutron-openvswi-FORWARD -m physdev --physdev-in tap5fcd8d22-30 --physdev-is-bridged -j neutron-openvswi-sg-chain
-A neutron-openvswi-FORWARD -m physdev --physdev-out tap319ad1b9-bb --physdev-is-bridged -j neutron-openvswi-sg-chain
-A neutron-openvswi-FORWARD -m physdev --physdev-in tap319ad1b9-bb --physdev-is-bridged -j neutron-openvswi-sg-chain
-A neutron-openvswi-FORWARD -m physdev --physdev-out tape8321530-46 --physdev-is-bridged -j neutron-openvswi-sg-chain
-A neutron-openvswi-FORWARD -m physdev --physdev-in tape8321530-46 --physdev-is-bridged -j neutron-openvswi-sg-chain
-A neutron-openvswi-FORWARD -m physdev --physdev-out tap93321459-0e --physdev-is-bridged -j neutron-openvswi-sg-chain
-A neutron-openvswi-FORWARD -m ...
(more)
edit retag flag offensive close merge delete
add a comment

1 answer

Sort by ยป oldest newest most voted
0

answered 2016-06-22 05:49:12 -0500

hkominos gravatar image

I dont know what ipvsadm is but from judging from the where the tcpdump stops i can make a guess. as you can see from http://dischord.org/2015/03/09/troubl... The packets leave your vm. go through qbr which is a linux bridge which in connection with IPTABLES sets the traffic rules and then the packets go through the qvb-qvo virtual ethernet pair. So 1)I would guess that the packets stop at the linux bridge and not at qvo 2)I dont think its possible to see a packet at qvb and not at qvo since its just a "wire"

Bottom line ->Start with the iptables rules+ I assume you have already specified the rules in default rules in the horizon interface for new vms

edit flag offensive delete link more

Comments

I did not specify any rules. The VM use the default security group which has ICMP and TCP allowed for ssh. before launching any VM I had done:

nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0 nova secgroup-add-rule default tcp 22 22 0.0.0.0/0

Do I need to add some more rules for LB VM?

fresher gravatar imagefresher ( 2016-06-22 06:49:38 -0500 )edit

The should be automatically updated. For a start make sure with tcpdump that the packets stop at the linux bridge and not at qvb. Then look at the iptables -S in your LB HOST (not the VM) and lets take it from there

hkominos gravatar imagehkominos ( 2016-06-22 08:17:14 -0500 )edit

I have edited the question with the result of iptables -S. Yes I have verified using tcpdump that the packets reach qbr but donot reach qvb and qvo. Can we add some rule in neutron-openvswi chain of the LB VM on compute node to prevent the drop of these packets?

fresher gravatar imagefresher ( 2016-06-22 23:23:56 -0500 )edit

If yes please guide us on how can we configure such a rule. As i can see a drop rule in the chain which drops anything other than packet with IP and MAC of the LB VM. But our packet has a different IP. The rule addition would be required at the backend server VM neutron-openvswi chain as well?

fresher gravatar imagefresher ( 2016-06-23 02:22:07 -0500 )edit

Well my ability with Iptables is really limited so i think you are alone from here. Grab a manual and understand them and edit the rules . From what i see this is a good start -A neutron-openvswi-sb094e27b-1 -s 192.168.1.75/32 -m mac --mac-source FA:16:3E:3E:49:F0 -j RETURN

hkominos gravatar imagehkominos ( 2016-06-23 04:27:10 -0500 )edit
see more comments

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

subscribe to rss feed

Stats

Asked: 2016-06-22 04:13:31 -0500

Seen: 446 times

Last updated: Jun 22 '16

Related questions

Why is the br-tun bridge operated in Flow mode?

ovs gre tunnel problem. Dosent show up after ovs agent restart.

kolla-ansible config adds l3-agent qG-interface in br-int (==> No inet access for VMs)

Neutron OVS ERROR on controller

how to conf the quantum of f3

Openstack network error - RTNETLINK answers: File exists

The server has either erred or is incapable of performing the requested operation

Neutron-ovs-cleanup delete interfaces

neutron public subnet wrong IP

Create a Network Toplogy via Dashboard