Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Cannot ping from an instance to external network, nor external interface of the router

Hello all,

I have installed Openstack-Opendaylight integration, with one Openstack controller node, one Openstack networking-compute node, and one Opendaylight controller node.

I have created a topology with:

  • An external network (192.168.40.0/24)
  • A private network (10.20.1.0/24)
  • A router connecting these two, with default gw to the external network
  • Three instances (from cirros image) connecting to the private network

Please refer to the images for have a better idea about the topology. (https://i.imgur.com/abwmhrc.png, and https://i.imgur.com/pSuwvnU.png).

Everything is up normally. I can actually ping or ssh from one machine in external network (192.168.40.0/24) to an instance using its floating IP. However, from the instance I cannot ping or ssh to the external network. The instance couldn't even ping the external interface on the router.

Below are some information in the system.

Network configuration on the networking-compute node.

root@compute1:/home/tsp# ifconfig
eth0      Link encap:Ethernet  HWaddr b8:2a:72:e1:12:e6  
          inet addr:157.159.68.xx  Bcast:157.159.68.255  Mask:255.255.255.0
          inet6 addr: fe80::ba2a:72ff:fee1:12e6/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:44054 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3330 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:6137839 (6.1 MB)  TX bytes:428835 (428.8 KB)
          Interrupt:53 

eth1      Link encap:Ethernet  HWaddr b8:2a:72:e1:12:e7  
          inet addr:192.168.40.31  Bcast:192.168.40.255  Mask:255.255.255.0
          inet6 addr: fe80::ba2a:72ff:fee1:12e7/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:68677 errors:0 dropped:30 overruns:0 frame:0
          TX packets:91633 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:15612102 (15.6 MB)  TX bytes:73601177 (73.6 MB)
          Interrupt:55 

eth2      Link encap:Ethernet  HWaddr b8:2a:72:e1:12:e8  
          inet addr:10.20.30.31  Bcast:10.20.30.255  Mask:255.255.255.0
          inet6 addr: fe80::ba2a:72ff:fee1:12e8/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2478 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:158592 (158.5 KB)  TX bytes:680 (680.0 B)
          Interrupt:56 

eth3      Link encap:Ethernet  HWaddr b8:2a:72:e1:12:e9  
          inet6 addr: fe80::ba2a:72ff:fee1:12e9/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:3818 errors:0 dropped:0 overruns:0 frame:0
          TX packets:413 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:328022 (328.0 KB)  TX bytes:50024 (50.0 KB)
          Interrupt:57 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:92063 errors:0 dropped:0 overruns:0 frame:0
          TX packets:92063 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1 
          RX bytes:6113813 (6.1 MB)  TX bytes:6113813 (6.1 MB)

tap47bc9e84-86 Link encap:Ethernet  HWaddr fe:16:3e:c6:80:a0  
          inet6 addr: fe80::fc16:3eff:fec6:80a0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1450  Metric:1
          RX packets:6609 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5872 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:642382 (642.3 KB)  TX bytes:578315 (578.3 KB)

tap92098fbe-05 Link encap:Ethernet  HWaddr fe:16:3e:f3:e9:03  
          inet6 addr: fe80::fc16:3eff:fef3:e903/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1450  Metric:1
          RX packets:4376 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5188 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:416800 (416.8 KB)  TX bytes:509285 (509.2 KB)

virbr0    Link encap:Ethernet  HWaddr 52:54:00:2b:8d:c5  
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

vxlan_sys_4789 Link encap:Ethernet  HWaddr 5a:94:e7:a9:de:98  
          inet6 addr: fe80::5894:e7ff:fea9:de98/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:65485  Metric:1
          RX packets:6652 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6652 errors:0 dropped:8 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:483627 (483.6 KB)  TX bytes:483627 (483.6 KB)

Openvswitch-switch start up script.

#!/bin/bash

sudo service openvswitch-switch start
sudo ovs-vsctl set-manager tcp:192.168.40.27:6640
sudo ovs-vsctl set Open_vSwitch . other_config:local_ip=10.20.30.31
sudo ovs-vsctl add-br br-ex
sudo ovs-vsctl add-port br-ex eth3
sudo ovs-vsctl show

Openvswitch show

root@compute1:/home/tsp# ovs-vsctl show
33f2ea6d-8dc9-4593-a5f1-74ef4150cc49
    Manager "tcp:192.168.40.27:6640"
        is_connected: true
    Bridge br-ex
        Port "eth3"
            Interface "eth3"
        Port br-ex
            Interface br-ex
                type: internal
        Port "qg-a15fb460-28"
            Interface "qg-a15fb460-28"
                type: internal
    Bridge br-int
        Controller "tcp:192.168.40.27:6653"
            is_connected: true
        fail_mode: secure
        Port "tap08000ed3-f1"
            Interface "tap08000ed3-f1"
                type: internal
        Port br-int
            Interface br-int
                type: internal
        Port "tuncf3b0226a9e"
            Interface "tuncf3b0226a9e"
                type: vxlan
                options: {key=flow, local_ip="10.20.30.31", remote_ip="10.20.30.31"}
        Port "qr-f354b832-e2"
            Interface "qr-f354b832-e2"
                type: internal
        Port "tap92098fbe-05"
            Interface "tap92098fbe-05"
        Port "tap47bc9e84-86"
            Interface "tap47bc9e84-86"
        Port "tap67097bab-99"
            Interface "tap67097bab-99"
                type: internal
    ovs_version: "2.6.1"

Router config

root@compute1:/home/tsp# alias router1="ip netns exec qrouter-e3427579-a439-41a7-9178-5c0e79913723"
root@compute1:/home/tsp# router1 ifconfig 
lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:6 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1 
          RX bytes:639 (639.0 B)  TX bytes:639 (639.0 B)

qg-a15fb460-28 Link encap:Ethernet  HWaddr fa:16:3e:7b:2e:31  
          inet addr:192.168.40.102  Bcast:192.168.40.255  Mask:255.255.255.0
          inet6 addr: fe80::f816:3eff:fe7b:2e31/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:796 errors:0 dropped:30 overruns:0 frame:0
          TX packets:392 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1 
          RX bytes:114344 (114.3 KB)  TX bytes:45672 (45.6 KB)

qr-f354b832-e2 Link encap:Ethernet  HWaddr fa:16:3e:5f:1d:69  
          inet addr:10.20.1.1  Bcast:10.20.1.255  Mask:255.255.255.0
          inet6 addr: fe80::f816:3eff:fe5f:1d69/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1450  Metric:1
          RX packets:393 errors:0 dropped:0 overruns:0 frame:0
          TX packets:549 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1 
          RX bytes:38976 (38.9 KB)  TX bytes:56732 (56.7 KB)

root@compute1:/home/tsp# router1 route
-n Kernel IP routing table Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.40.10   0.0.0.0         UG    0      0        0 qg-a15fb460-28
10.20.1.0       0.0.0.0         255.255.255.0   U     0      0        0 qr-f354b832-e2
192.168.40.0    0.0.0.0         255.255.255.0   U     0      0        0 qg-a15fb460-28 root@compute1:/home/tsp# router1 iptables -S
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N neutron-filter-top
-N neutron-l3-agent-FORWARD
-N neutron-l3-agent-INPUT
-N neutron-l3-agent-OUTPUT
-N neutron-l3-agent-local
-N neutron-l3-agent-scope
-A INPUT -j neutron-l3-agent-INPUT
-A FORWARD -j neutron-filter-top
-A FORWARD -j neutron-l3-agent-FORWARD
-A FORWARD -i qr-f354b832-e2 -o qg-a15fb460-28 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i qr-f354b832-e2 -o qg-a15fb460-28 -j ACCEPT
-A OUTPUT -j neutron-filter-top
-A OUTPUT -j neutron-l3-agent-OUTPUT
-A neutron-filter-top -j neutron-l3-agent-local
-A neutron-l3-agent-FORWARD -j neutron-l3-agent-scope
-A neutron-l3-agent-INPUT -m mark --mark 0x1/0xffff -j ACCEPT
-A neutron-l3-agent-INPUT -p tcp -m tcp --dport 9697 -j DROP root@compute1:/home/tsp# router1 iptables -t nat -S
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-N neutron-l3-agent-OUTPUT
-N neutron-l3-agent-POSTROUTING
-N neutron-l3-agent-PREROUTING
-N neutron-l3-agent-float-snat
-N neutron-l3-agent-snat
-N neutron-postrouting-bottom
-A PREROUTING -j neutron-l3-agent-PREROUTING
-A PREROUTING -j ACCEPT
-A OUTPUT -j neutron-l3-agent-OUTPUT
-A POSTROUTING -j neutron-l3-agent-POSTROUTING
-A POSTROUTING -j neutron-postrouting-bottom
-A POSTROUTING -j MASQUERADE
-A neutron-l3-agent-OUTPUT -d 192.168.40.105/32 -j DNAT --to-destination 10.20.1.13
-A neutron-l3-agent-POSTROUTING ! -i qg-a15fb460-28 ! -o qg-a15fb460-28 -m conntrack ! --ctstate DNAT -j ACCEPT
-A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 9697
-A neutron-l3-agent-PREROUTING -d 192.168.40.105/32 -j DNAT --to-destination 10.20.1.13
-A neutron-l3-agent-float-snat -s 10.20.1.13/32 -j SNAT --to-source 192.168.40.105
-A neutron-l3-agent-snat -j neutron-l3-agent-float-snat
-A neutron-l3-agent-snat -o qg-a15fb460-28 -j SNAT --to-source
192.168.40.102
-A neutron-l3-agent-snat -m mark ! --mark 0x2/0xffff -m conntrack --ctstate DNAT -j SNAT --to-source 192.168.40.102
-A neutron-postrouting-bottom -m comment --comment "Perform source NAT on outgoing traffic." -j neutron-l3-agent-snat

Instance configuration.

$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.20.1.1       0.0.0.0         UG    0      0        0 eth0
10.20.1.0       0.0.0.0         255.255.255.0   U     0      0        0 eth0
169.254.169.254 10.20.1.2       255.255.255.255 UGH   0      0        0 eth0
$ ifconfig
eth0      Link encap:Ethernet  HWaddr FA:16:3E:C6:80:A0  
          inet addr:10.20.1.13  Bcast:10.20.1.255  Mask:255.255.255.0
          inet6 addr: fe80::f816:3eff:fec6:80a0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1450  Metric:1
          RX packets:5373 errors:0 dropped:2 overruns:0 frame:0
          TX packets:7364 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:514659 (502.5 KiB)  TX bytes:715652 (698.8 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

I am stuck in this problem for a week already. I believe that this is due to some misconfiguration somewhere in the system, but couldn't find it. Please help. Thank you very much.