Ask Your Question
0

Why would an OVS bridge not forward ARP

asked 2012-09-20 23:07:13 -0500

sunil-srivastava gravatar image

Bridge "br-eth0" Port "br-eth0" Interface "br-eth0" type: internal Port "eth0" Interface "eth0" Port "phy-br-eth0" Interface "phy-br-eth0" Bridge br-int Port "tap55d1e5e8-ab" tag: 1 Interface "tap55d1e5e8-ab" type: internal Port "qr-4b50a17d-3c" tag: 1 Interface "qr-4b50a17d-3c" type: internal Port "int-br-eth0" Interface "int-br-eth0" Port "tape8d6e0a5-52" tag: 1 Interface "tape8d6e0a5-52" Port "tap6176588e-48" tag: 1 Interface "tap6176588e-48" Port br-int Interface br-int type: internal

I can see ARP packets sent from int-br-eth0 to phy-br-eth0 but not to upstream eth0.

So we cannot ping from one VM (or DHCP NetNS) on one machine to another VM on another machine.

I see the ping triggering ARPs. The Tx counter of int-br-eth0 and Rx counter of phy-br-eth0 were also corelated with ping.

edit retag flag offensive close merge delete

25 answers

Sort by ยป oldest newest most voted
0

answered 2012-09-21 00:36:35 -0500

Hi Sunil,

Is the host (or switch attached to eth0) configured to recieve a packet with a vlan tag on it? If a packet is sent from [tape8d6e0a5-52 or tap6176588e-48] the ARP request will enter int-br-eth0(and a vlan tag of 1 will be added to the packet). Then the request will enter br-eth0 with this vlan tag and then exit eth0.

The other option is that: if you do a ovs-dpctl show, eth0 cordinates to port 5 in which case the packets won't be forwarded on due to the drop rule in your flow table for br-eth0.

edit flag offensive delete link more
0

answered 2012-09-21 00:48:17 -0500

sunil-srivastava gravatar image

Hi Aaron,

Please see this outout. It is phy-br-eth0.

stack@esg-dell-c4-s11:~$ sudo ovs-ofctl show br-eth0 OFPT_FEATURES_REPLY (xid=0x1): ver:0x1, dpid:000000219bc9d983 n_tables:255, n_buffers:256 features: capabilities:0xc7, actions:0xfff 5(phy-br-eth0): addr:06:ee:0f:8c:92:b3 config: 0 state: 0 current: 10GB-FD COPPER 8(eth0): addr:00:21:9b:c9:d9:83 config: 0 state: 0 current: 1GB-FD FIBER AUTO_NEG advertised: 1GB-FD AUTO_NEG supported: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER FIBER AUTO_NEG LOCAL(br-eth0): addr:00:21:9b:c9:d9:83 config: PORT_DOWN state: LINK_DOWN OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0

edit flag offensive delete link more
0

answered 2012-09-21 00:53:06 -0500

sunil-srivastava gravatar image

The switch is connected to eth0 and configured to take VLAN #1 and set in trunk mode.

edit flag offensive delete link more
0

answered 2012-09-21 01:18:44 -0500

Looks like br-eth0 is down, I'm not sure if that would stop if from forwarding packets. Can you try ifconfig br-eth0 up; and see if that changes anything? You're sure if you tcpdump on eth0 you don't see any of these arps?

Also:

cookie=0x0, duration=30902.643s, table=0, n_packets=7780, n_bytes=329156, priority=2,in_port=5 actions=drop

would block the returning ARP reply (though if it's not making it out eth0, that doesn't matter yet).

Are you using a particular plugin and it's not working as expected?

edit flag offensive delete link more
0

answered 2012-09-21 02:33:04 -0500

gongysh gravatar image

can u list the network and show the network you are using: quantum net-list quantum net-show

and make sure the ovs-quantum-agent is active. It seems your flows in ovs bridge are not set well.

edit flag offensive delete link more
0

answered 2012-09-21 07:01:47 -0500

sunil-srivastava gravatar image

Someone else has been on the setup and things have changed a bit.

He said he entered a flow entry - but I did not have time to follow up. I am in R/O mode. :-)

br-eth0 was brought up and that may have changed the behavior.

Now ARP is reach the other machine, and I can see the traffic on eth0, phy-br-eth0 and int-br-eth0.

But there is not ARP reply. The ARPs are not getting to any of the TAP interfaces (but I only did once).

The br-int seems to be dropping now.

BTW, one TAP interface is not there as VM was brought down but ovs-vsctl has it.

stack@esg-dell-c4-s10:~/devstack$ ovs-dpctl show br-eth0 flows system@br-eth0: lookups: hit:658100 missed:119465 lost:0 flows: 27 port 0: br-eth0 (internal) port 6: eth0 port 9: phy-br-eth0 ovs-dpctl: opening datapath flows failed (No such device)

stack@esg-dell-c4-s10:~/devstack$ sudo ovs-ofctl dump-flows br-eth0 NXST_FLOW reply (xid=0x4): cookie=0x0, duration=54070.378s, table=0, n_packets=80, n_bytes=6552, priority=2,in_port=9 actions=drop cookie=0x0, duration=54070.663s, table=0, n_packets=632974, n_bytes=79155773, priority=1 actions=NORMAL

stack@esg-dell-c4-s10:~/devstack$ ovs-dpctl show br-eth0 flows system@br-eth0: lookups: hit:660125 missed:119838 lost:0 flows: 30 port 0: br-eth0 (internal) port 6: eth0 port 9: phy-br-eth0 ovs-dpctl: opening datapath flows failed (No such device)

stack@esg-dell-c4-s10:~/devstack$ sudo ovs-ofctl dump-flows br-int NXST_FLOW reply (xid=0x4): cookie=0x0, duration=54307.045s, table=0, n_packets=608686, n_bytes=77906470, priority=2,in_port=18 actions=drop cookie=0x0, duration=54307.359s, table=0, n_packets=14395, n_bytes=2748282, priority=1 actions=NORMAL

stack@esg-dell-c4-s10:~/devstack$ ovs-dpctl show br-int flows system@br-int: lookups: hit:631616 missed:143682 lost:0 flows: 26 port 0: br-int (internal) Sep 21 03:00:32|00001|netdev_linux|WARN|/sys/class/net/tap26583155-34/carrier: open failed: No such file or directory port 1: tap26583155-34 (internal) port 14: tapd1802d22-b4 port 15: tapfa0e7fcf-8d port 16: tap5eb27feb-05 port 18: int-br-eth0 ovs-dpctl: opening datapath flows failed (No such device)

edit flag offensive delete link more
0

answered 2012-09-21 05:20:44 -0500

sunil-srivastava gravatar image

Hi Aaron,

I am leaving on some trip, and would not have access.

The br-eth0 is up and tried that still but did not work.

stack@esg-dell-c4-s11:~/gitstack/devstack$ ifconfig br-eth0 Link encap:Ethernet HWaddr 00:21:9b:c9:d9:83 inet6 addr: fe80::221:9bff:fec9:d983/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:660411 errors:0 dropped:1883 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:84199314 (84.1 MB) TX bytes:468 (468.0 B)

Sunil.

edit flag offensive delete link more
0

answered 2012-09-21 05:24:04 -0500

sunil-srivastava gravatar image

Hi Yong,

That is not the root cause. Still.

stack@esg-dell-c4-s11:~/gitstack/devstack$ quantum net-list +--------------------------------------+---------+--------------------------------------+ | id | name | subnets | +--------------------------------------+---------+--------------------------------------+ | 68f76ec1-407b-4e42-a089-d0e6553473f8 | ext_net | 09851d25-806f-492c-b708-bf03838d77b3 | | fa8f9c5e-e41a-4f80-955c-94b3a45b9dcb | net1 | 31ed889f-f3f5-4faa-bb51-1d92344c91a3 | +--------------------------------------+---------+--------------------------------------+

stack@esg-dell-c4-s11:~/gitstack/devstack$ quantum net-show net1 +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | True | | id | fa8f9c5e-e41a-4f80-955c-94b3a45b9dcb | | name | net1 | | provider:network_type | local | | provider:physical_network | | | provider:segmentation_id | | | router:external | False | | shared | False | | status | ACTIVE | | subnets | 31ed889f-f3f5-4faa-bb51-1d92344c91a3 | | tenant_id | b0d8717a0f8b4cf8bdff8d84156622af |

stack@esg-dell-c4-s11:~/gitstack/devstack$ quantum net-show ext_net +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | True | | id | 68f76ec1-407b-4e42-a089-d0e6553473f8 | | name | ext_net | | provider:network_type | local | | provider:physical_network | | | provider:segmentation_id | | | router:external | True | | shared | False | | status | ACTIVE | | subnets | 09851d25-806f-492c-b708-bf03838d77b3 | | tenant_id | 44cb33fdc72b44ad8e200a1326199895 | +---------------------------+--------------------------------------+

+---------------------------+--------------------------------------+

edit flag offensive delete link more
0

answered 2012-09-21 06:10:35 -0500

Sunil, One last thing. If you leave the ping running and then provide the output of

ovs-dpctl dump-flows br-int ovs-dpctl dump-flows br-tun

That will show the active flow entires in the kernel. Did you try running tcpdump on eth0 to see if you see arp packets there? You never said how you know that they are not making it out eth0. You just said you were unable to ping. (The drop flow entry you provided blocks the returning replies so ping definitely will not work).

Aaron

edit flag offensive delete link more
0

answered 2012-09-21 06:26:42 -0500

sunil-srivastava gravatar image

Hi Aron,

Here is the TCP Dumps.

The following output show there is no link issue between phy-br-eth0 and int-br-eth0.

(1)

root@esg-dell-c4-s11:~# ping 10.0.0.2
PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data. 64 bytes from 10.0.0.2: icmp_req=1 ttl=64 time=0.056 ms 64 bytes from 10.0.0.2: icmp_req=2 ttl=64 time=0.052 ms 64 bytes from 10.0.0.2: icmp_req=3 ttl=64 time=0.032 ms 64 bytes from 10.0.0.2: icmp_req=4 ttl=64 time=0.041 ms 64 bytes from 10.0.0.2: icmp_req=5 ttl=64 time=0.048 ms --- 10.0.0.2 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4006ms

The above triggers this below on int-br-eth0

stack@esg-dell-c4-s11:~/devstack$ sudo tcpdump -i int-br-eth0 arp and src 10.0.0.2 tcpdump: WARNING: int-br-eth0: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on int-br-eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 00:41:55.755545 ARP, Request who-has http://inenbasavbl1c.corp.emc.com tell http://usxxstephc2mbp1.corp.emc.com , length 28 00:41:56.753793 ARP, Request who-has http://inenbasavbl1c.corp.emc.com tell http://usxxstephc2mbp1.corp.emc.com , length 28 00:41:57.753782 ARP, Request who-has http://inenbasavbl1c.corp.emc.com tell http://usxxstephc2mbp1.corp.emc.com , length 28 00:41:58.771011 ARP, Request who-has http://inenbasavbl1c.corp.emc.com tell http://usxxstephc2mbp1.corp.emc.com , length 28 00:41:59.769790 ARP, Request who-has http://inenbasavbl1c.corp.emc.com tell http://usxxstephc2mbp1.corp.emc.com , length 28 00:42:00.769796 ARP, Request who-has http://inenbasavbl1c.corp.emc.com tell http://usxxstephc2mbp1.corp.emc.com , length 28

(2)

And again on phy-br-eth0.

root@esg-dell-c4-s11:~# ping 10.0.0.3 PING 10.0.0.3 (10.0.0.3) 56(84) bytes of data. From 10.0.0.2 icmp_seq=1 Destination Host Unreachable From 10.0.0.2 icmp_seq=2 Destination Host Unreachable From 10.0.0.2 icmp_seq=3 Destination Host Unreachable ^C --- 10.0.0.3 ping statistics --- 5 packets transmitted, 0 received, +3 errors, 100% packet loss, time 4024ms

stack@esg-dell-c4-s11:~/devstack$ sudo tcpdump -i phy-br-eth0 arp and src 10.0.0.2 tcpdump: WARNING: phy-br-eth0: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on phy-br-eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 00:43:35.871097 ARP, Request who-has http://inenbasavbl1c.corp.emc.com tell http://usxxstephc2mbp1.corp.emc.com , length 28 00:43:36.869787 ARP, Request who-has http://inenbasavbl1c.corp.emc.com tell http://usxxstephc2mbp1.corp.emc.com , length 28 00:43:37.873777 ARP, Request who-has http://inenbasavbl1c.corp.emc.com tell http://usxxstephc2mbp1.corp.emc.com , length 28 00:43:38.887008 ARP, Request who-has http://inenbasavbl1c.corp.emc.com tell http://usxxstephc2mbp1.corp.emc.com , length ... (more)

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2012-09-20 23:07:13 -0500

Seen: 1,846 times

Last updated: Sep 26 '12