Neutron Network node not Routing
Pulling my hair out on this one.... I can't get my Neutron network node to do a simple route of a packet to my controller node! I'm sure its something stupid, but I can't see it I guess. /proc/sys/net/ipv4/ip_forward has '1' as its content. /etc/sysctl.conf looks like this:
net.ipv4.conf.all.rp_filter=0
net.ipv4.conf.default.rp_filter=0
net.ipv4.ip_forward=1
I can get to all my instances on my compute nodes, so I know the bridging/tunneling is working. All three networks (External on br-ex, Tunnel on eth2, Management on eth0, eth1 assigned to br-ex) show up in my 'ip a' output. I can ping the Compute and Controller nodes just fine from the Network node, but outside connections coming in on the External network stop at the Network node.
What am I doing wrong??
Thanks for your time!
JD
UPDATE1:
# ovs-vsctl show
3ec0fc70-90ce-4e89-818f-5fdab99bf08a
Bridge br-int
fail_mode: secure
Port br-int
Interface br-int
type: internal
Port "tap221f3067-b6"
tag: 3
Interface "tap221f3067-b6"
type: internal
Port int-br-ex
Interface int-br-ex
type: patch
options: {peer=phy-br-ex}
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port "tap114ed438-14"
tag: 1
Interface "tap114ed438-14"
type: internal
Port "tapac8c15d2-cc"
tag: 2
Interface "tapac8c15d2-cc"
type: internal
Port "qr-dec6f704-48"
tag: 3
Interface "qr-dec6f704-48"
type: internal
Bridge br-ex
Port phy-br-ex
Interface phy-br-ex
type: patch
options: {peer=int-br-ex}
Port "qg-710689ae-67"
Interface "qg-710689ae-67"
type: internal
Port br-ex
Interface br-ex
type: internal
Port "eth1"
Interface "eth1"
Bridge br-tun
fail_mode: secure
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
Port "gre-0a000119"
Interface "gre-0a000119"
type: gre
options: {df_default="true", in_key=flow, local_ip="10.0.1.28", out_key=flow, remote_ip="10.0.1.25"}
Port br-tun
Interface br-tun
type: internal
ovs_version: "2.0.2"
# ifconfig
br-ex Link encap:Ethernet HWaddr 00:1e:90:13:9a:f6
inet addr:10.147.29.28 Bcast:10.147.29.255 Mask:255.255.255.0
inet6 addr: fe80::b803:34ff:fe88:d5c3/64 Scope:Link
UP BROADCAST RUNNING MTU:1500 Metric:1
RX packets:10273 errors:0 dropped:0 overruns:0 frame:0
TX packets:5682 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1114956 (1.1 MB) TX bytes:1168138 (1.1 MB)
br-int Link encap:Ethernet HWaddr c2:81:7e:1a:7f:4f
inet6 addr: fe80::8401:81ff:fe90:8376/64 Scope:Link
UP BROADCAST RUNNING MTU:1500 Metric:1
RX packets:68 errors:0 dropped:0 overruns:0 frame:0
TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:5148 (5.1 KB) TX bytes:648 (648.0 B)
br-tun Link encap:Ethernet HWaddr 0e:44:0c:98:9f:49
inet6 addr: fe80::dc18:d2ff:fec6:2d2c/64 Scope:Link
UP BROADCAST RUNNING MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:648 (648.0 B)
eth0 Link encap:Ethernet HWaddr 6c:19:8f:9a:5f:37
inet addr:XXX.147.45.28 Bcast:XXX.147.45.255 Mask:255.255.255.0
inet6 addr: fe80::6e19:8fff:fe9a:5f37/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:12973 errors:0 dropped:0 overruns:0 frame:0
TX packets:21699 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1677191 (1.6 MB) TX bytes:4041026 (4.0 MB)
eth1 Link encap:Ethernet HWaddr 00:1e:90:13:9a:f6
inet6 addr: fe80::21e:90ff:fe13:9af6/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:17951 errors:0 dropped:0 overruns:0 frame:0
TX packets:10330 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1846931 (1.8 MB) TX bytes:1648586 (1.6 MB)
eth2 Link encap:Ethernet HWaddr 14:cc:20:04:68:dd
inet addr:10.0.1.28 Bcast:10.0.1.255 Mask:255.255.255.0
inet6 addr: fe80::16cc:20ff:fe04:68dd/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:4913 errors:0 dropped:0 overruns:0 frame:0
TX packets:7825 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:682532 (682.5 KB) TX bytes:983492 (983.4 KB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:22 errors:0 dropped:0 overruns:0 frame:0
TX packets:22 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:5232 (5.2 KB) TX bytes:5232 (5.2 KB)
# ip netns exec qrouter-c939264f-e3ee-45d5-a885-11ad94e04c12 ifconfig
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:11 errors:0 dropped:0 overruns:0 frame:0
TX packets:11 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1232 (1.2 KB) TX bytes:1232 (1.2 KB)
qg-710689ae-67 Link encap:Ethernet HWaddr fa:16:3e:80:98:cc
inet addr:10.147.29.210 Bcast:10.147.29.255 Mask:255.255.255.0
inet6 addr: fe80::f816:3eff:fe80:98cc/64 Scope:Link
UP BROADCAST RUNNING MTU:1500 Metric:1
RX packets:8312 errors:0 dropped:0 overruns:0 frame:0
TX packets:4671 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:922167 (922.1 KB) TX bytes:481654 (481.6 KB)
qr-dec6f704-48 Link encap:Ethernet HWaddr fa:16:3e:6d:51:ce
inet addr:10.10.10.1 Bcast:10.10.10.255 Mask:255.255.255.0
inet6 addr: fe80::f816:3eff:fe6d:51ce/64 Scope:Link
UP BROADCAST RUNNING MTU:1500 Metric:1
RX packets:4275 errors:0 dropped:0 overruns:0 frame:0
TX packets:7261 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:467422 (467.4 KB) TX bytes:674930 (674.9 KB)
