Ask Your Question
0

packets not reaching VM

asked 2016-06-22 04:13:31 -0600

fresher gravatar image

updated 2016-06-22 23:22:47 -0600

We have a Openstack Juno setup with 1 controller+neutron node and 3 compute nodes. 1 VM (LB) has ipvsadm installed and two VMs act as back end server.

On the server with ipvsadm I have eth0:0 IP as 192.168.1.21 which acts as application IP. The ipvsadm uses round robin scheme. This is done using commands as below:

sudo ipvsadm -A -t 192.168.1.21:6000 -s rr
sudo ipvsadm -a -t 192.168.1.21:6000 -r 192.168.1.77:6000 -g
sudo ipvsadm -a -t 192.168.1.21:6000 -r 192.168.1.79:6000 -g

where 192.168.1.77 and 192.168.1.79 are back end server VM IP.

The problem is that the packets go out of the LB VM but never reach the back end server.

In the tcpdumps on various interfaces show that the packet reach till qbr of the LB VM but donot reach the qvo interface of LB VM. Are there any rules that get applied here which block these packets. The packets from the client VM are sent to back end server by the LB VM by changing the destination MAC of the packets.

The packets that leave LB VM to reach back end VM have source as the client VM IP and destination IP as 192.168.1.21 (application IP) and the src MAC of LB VM and dst MAC of backend server VM. Is this the reason for the packets to be blocked. Is there any way to allow these packets to flow to the back end server?

Please help us regarding this.

The host has many VMs thus a huge list when iptables -S is done. The LB VM IP is 192.168.1.75 

-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N neutron-filter-top
-N neutron-openvswi-FORWARD
-N neutron-openvswi-INPUT
-N neutron-openvswi-OUTPUT
-N neutron-openvswi-i10a4632b-9
-N neutron-openvswi-i131bf200-3
-N neutron-openvswi-i319ad1b9-b
-N neutron-openvswi-i3fb9dd53-d
-N neutron-openvswi-i4f240dc1-6
-N neutron-openvswi-i578e7d84-b
-N neutron-openvswi-i5c7b6b9f-d
-N neutron-openvswi-i5e93d25e-1
-N neutron-openvswi-i5fcd8d22-3
-N neutron-openvswi-i610385e2-d
-N neutron-openvswi-i93321459-0
-N neutron-openvswi-ib094e27b-1
-N neutron-openvswi-ib9882cfd-4
-N neutron-openvswi-ic2dad3ae-c
-N neutron-openvswi-ie8321530-4
-N neutron-openvswi-local
-N neutron-openvswi-o10a4632b-9
-N neutron-openvswi-o131bf200-3
-N neutron-openvswi-o319ad1b9-b
-N neutron-openvswi-o3fb9dd53-d
-N neutron-openvswi-o4f240dc1-6
-N neutron-openvswi-o578e7d84-b
-N neutron-openvswi-o5c7b6b9f-d
-N neutron-openvswi-o5e93d25e-1
-N neutron-openvswi-o5fcd8d22-3
-N neutron-openvswi-o610385e2-d
-N neutron-openvswi-o93321459-0
-N neutron-openvswi-ob094e27b-1
-N neutron-openvswi-ob9882cfd-4
-N neutron-openvswi-oc2dad3ae-c
-N neutron-openvswi-oe8321530-4
-N neutron-openvswi-s10a4632b-9
-N neutron-openvswi-s131bf200-3
-N neutron-openvswi-s319ad1b9-b
-N neutron-openvswi-s3fb9dd53-d
-N neutron-openvswi-s4f240dc1-6
-N neutron-openvswi-s578e7d84-b
-N neutron-openvswi-s5c7b6b9f-d
-N neutron-openvswi-s5e93d25e-1
-N neutron-openvswi-s5fcd8d22-3
-N neutron-openvswi-s610385e2-d
-N neutron-openvswi-s93321459-0
-N neutron-openvswi-sb094e27b-1
-N neutron-openvswi-sb9882cfd-4
-N neutron-openvswi-sc2dad3ae-c
-N neutron-openvswi-se8321530-4
-N neutron-openvswi-sg-chain
-N neutron-openvswi-sg-fallback
-A INPUT -j neutron-openvswi-INPUT
-A FORWARD -j neutron-filter-top
-A FORWARD -j neutron-openvswi-FORWARD
-A OUTPUT -j neutron-filter-top
-A OUTPUT -j neutron-openvswi-OUTPUT
-A neutron-filter-top -j neutron-openvswi-local
-A neutron-openvswi-FORWARD -m physdev --physdev-out tap5fcd8d22-30 --physdev-is-bridged -j neutron-openvswi-sg-chain
-A neutron-openvswi-FORWARD -m physdev --physdev-in tap5fcd8d22-30 --physdev-is-bridged -j neutron-openvswi-sg-chain
-A neutron-openvswi-FORWARD -m physdev --physdev-out tap319ad1b9-bb --physdev-is-bridged -j neutron-openvswi-sg-chain
-A neutron-openvswi-FORWARD -m physdev --physdev-in tap319ad1b9-bb --physdev-is-bridged -j neutron-openvswi-sg-chain
-A neutron-openvswi-FORWARD -m physdev --physdev-out tape8321530-46 --physdev-is-bridged -j neutron-openvswi-sg-chain
-A neutron-openvswi-FORWARD -m physdev --physdev-in tape8321530-46 --physdev-is-bridged -j neutron-openvswi-sg-chain
-A neutron-openvswi-FORWARD -m physdev --physdev-out tap93321459-0e --physdev-is-bridged -j neutron-openvswi-sg-chain
-A neutron-openvswi-FORWARD -m ...
(more)
edit retag flag offensive close merge delete
add a comment

1 answer

Sort by ยป oldest newest most voted
0

answered 2016-06-22 05:49:12 -0600

hkominos gravatar image

I dont know what ipvsadm is but from judging from the where the tcpdump stops i can make a guess. as you can see from http://dischord.org/2015/03/09/troubl... The packets leave your vm. go through qbr which is a linux bridge which in connection with IPTABLES sets the traffic rules and then the packets go through the qvb-qvo virtual ethernet pair. So 1)I would guess that the packets stop at the linux bridge and not at qvo 2)I dont think its possible to see a packet at qvb and not at qvo since its just a "wire"

Bottom line ->Start with the iptables rules+ I assume you have already specified the rules in default rules in the horizon interface for new vms

edit flag offensive delete link more

Comments

I did not specify any rules. The VM use the default security group which has ICMP and TCP allowed for ssh. before launching any VM I had done:

nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0 nova secgroup-add-rule default tcp 22 22 0.0.0.0/0

Do I need to add some more rules for LB VM?

fresher gravatar imagefresher ( 2016-06-22 06:49:38 -0600 )edit

The should be automatically updated. For a start make sure with tcpdump that the packets stop at the linux bridge and not at qvb. Then look at the iptables -S in your LB HOST (not the VM) and lets take it from there

hkominos gravatar imagehkominos ( 2016-06-22 08:17:14 -0600 )edit

I have edited the question with the result of iptables -S. Yes I have verified using tcpdump that the packets reach qbr but donot reach qvb and qvo. Can we add some rule in neutron-openvswi chain of the LB VM on compute node to prevent the drop of these packets?

fresher gravatar imagefresher ( 2016-06-22 23:23:56 -0600 )edit

If yes please guide us on how can we configure such a rule. As i can see a drop rule in the chain which drops anything other than packet with IP and MAC of the LB VM. But our packet has a different IP. The rule addition would be required at the backend server VM neutron-openvswi chain as well?

fresher gravatar imagefresher ( 2016-06-23 02:22:07 -0600 )edit

Well my ability with Iptables is really limited so i think you are alone from here. Grab a manual and understand them and edit the rules . From what i see this is a good start -A neutron-openvswi-sb094e27b-1 -s 192.168.1.75/32 -m mac --mac-source FA:16:3E:3E:49:F0 -j RETURN

hkominos gravatar imagehkominos ( 2016-06-23 04:27:10 -0600 )edit
see more comments

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

subscribe to rss feed

Stats

Asked: 2016-06-22 04:13:31 -0600

Seen: 487 times

Last updated: Jun 22 '16

Related questions

Problems with OVS: No Connectivity

specify OVS path

Connection lost between OVS patch ports?

kolla-ansible config adds l3-agent qG-interface in br-int (==> No inet access for VMs)

OVS Offloading with Bonding

create security group

neutron dhcp issues

dhcp worked but cannot ping or ssh to instance

Gateway always down [closed]

Instance gets dhcp IP but fails to connect to metadata server