Connection between network namespace and public interface doesn't work

asked 2015-07-03 14:52:43 -0500

Florin gravatar image

After rebooting my neutron node, all services ended up coming up just fine, except for the fact that I can't ping/ssh into any floating IPs anymore. I started troubleshooting and here's what I found:

[root@OSneutron ~]# brctl show
bridge name     bridge id               STP enabled     interfaces
brq10c40db7-aa          8000.001e67487f61       no              enp5s0f1
                                                        tap4b4b1551-85
                                                        tap73c34e57-99
brq549d3e87-13          8000.001e67487f65       no              enp1s0.120
                                                        tap49ca1f2d-0d
                                                        tap615154c2-7b
brqd8d6d3f0-88          8000.001e67487f65       no              enp1s0.130
                                                        tap052488a7-d1
                                                        tapdf4c69c5-db
brqf66306a4-d9          8000.001e67487f65       no              enp1s0.110
                                                        tap1c780f31-7e
                                                        tap6ec4c0b1-c1
                                                        tapea8ec78d-e8

Each bridge except for the first one corresponds to one tenant. Enp1s0 is the data interface. The first bridge connects to the enp5s0f1 interface which is the public interface. Everything looked fine up to this point. I started looking into the network name spaces:

[root@OSneutron ~]# ip netns show
qrouter-8589c99a-3325-40d0-9fb4-c5e142bdfeb6
qrouter-0bad7947-83e4-4778-86ae-6a8cab2e3e2f
qrouter-854ae272-4af9-43ab-816b-00ef2f6feb85
qdhcp-f66306a4-d92c-4a00-b18f-1acfceca1243
qdhcp-d8d6d3f0-888a-4ace-a245-5b5fd18a0360
qdhcp-549d3e87-133e-4f0e-98be-b6260b224775
[root@OSneutron ~]# ip netns exec qrouter-854ae272-4af9-43ab-816b-00ef2f6feb85 ifconfig
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 0  (Local Loopback)
        RX packets 32  bytes 3584 (3.5 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 32  bytes 3584 (3.5 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

qg-baa82bfb-05: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.99.1.3  netmask 255.255.0.0  broadcast 10.99.255.255
        inet6 fe80::f816:3eff:fe50:5b03  prefixlen 64  scopeid 0x20<link>
        ether fa:16:3e:50:5b:03  txqueuelen 1000  (Ethernet)
        RX packets 8  bytes 648 (648.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 124  bytes 5652 (5.5 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

qr-1c780f31-7e: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.110.0.1  netmask 255.255.0.0  broadcast 10.110.255.255
        inet6 fe80::f816:3eff:fed1:a0e4  prefixlen 64  scopeid 0x20<link>
        ether fa:16:3e:d1:a0:e4  txqueuelen 1000  (Ethernet)
        RX packets 320  bytes 18790 (18.3 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 97  bytes 6774 (6.6 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

I choose one of the tenants and decided to try to ping the floating IP of one of the VMs:

[root@OSneutron ~]# ip netns exec qrouter-854ae272-4af9-43ab-816b-00ef2f6feb85 ping 10.99.1.5
PING 10.99.1.5 (10.99.1.5) 56(84) bytes of data.
64 bytes from 10.99.1.5: icmp_seq=1 ttl=64 time=0.271 ms
64 bytes from 10.99.1.5: icmp_seq=2 ttl=64 time=0.248 ms
64 bytes from 10.99.1.5: icmp_seq=3 ttl=64 time=0.293 ms

If I try to ping the OSController or the OSneutron node from this namespace it doesn't work:

[root@OSneutron ~]# ip netns exec qrouter-854ae272-4af9-43ab-816b-00ef2f6feb85 ping 10.99.0.1
PING 10.99.0.1 (10.99.0.1) 56(84 ...
(more)
edit retag flag offensive close merge delete

Comments

brq10c40db7-aa is missiing tapbaa82bfb-05 - probably because you deleted the bridge. Maybe restarting the l3-agent will put it back. You should check the l3-agent logs.

darragh-oreilly gravatar imagedarragh-oreilly ( 2015-07-07 14:33:44 -0500 )edit

@darragh-oreilly I realized this a couple of days after. You should change your comment to an answer so I can accept it. Thanks!

Florin gravatar imageFlorin ( 2015-07-07 14:54:26 -0500 )edit