Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

openvswitch and br-ex

Hi,

I’m using openvswitch and create the br-ex bridge for external access.

I added physical port eth1 to br-ex:

localadmin@qa4:~/devstack$ sudo ovs-vsctl show 4b5380e3-d619-470b-be99-deda868283c9 Bridge br-ex Port br-ex Interface br-ex type: internal Port "eth1" Interface "eth1" Port "qg-c4bb44ec-94" Interface "qg-c4bb44ec-94" type: internal

Config in /etc/network/interfaces as follows:

auto eth1 iface eth1 inet manual up ifconfig $IFACE 0.0.0.0 up up ip link set $IFACE promisc on down ip link set $IFACE promisc off down ifconfig $IFACE down

auto br-ex iface br-ex inet static address 172.29.173.4 netmask 255.255.255.192 gateway 172.29.173.1 dns-nameservers 172.29.74.154

Network node host IP is 172.29.172.161/25 at eth0 and routing table as follows:

Kernel IP routing table Destination Gateway Genmask Flags MSS Window irtt Iface 0.0.0.0 172.29.172.129 0.0.0.0 UG 0 0 0 eth0 10.0.0.0 172.29.173.5 255.255.255.0 UG 0 0 0 br-ex 172.29.172.128 0.0.0.0 255.255.255.128 U 0 0 0 eth0 172.29.173.0 0.0.0.0 255.255.255.224 U 0 0 0 br-ex 172.29.173.0 0.0.0.0 255.255.255.192 U 0 0 0 br-ex 192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0

There is one VM with tenant address 10.0.0.2 and floating IP 172.29.173.15:

localadmin@qa4:~/devstack$ nova list +--------------------------------------+------+--------+------------+-------------+---------------------------------+ | ID | Name | Status | Task State | Power State | Networks | +--------------------------------------+------+--------+------------+-------------+---------------------------------+ | e721103e-763a-4a6c-8b3d-294c1cb55726 | vm1 | ACTIVE | - | Running | private=10.0.0.2, 172.29.173.15 | +--------------------------------------+------+--------+------------+-------------+————————————————+

From the Network host, I can ping the VM tenant address, router and DHCP interfaces:

localadmin@qa4:~/devstack$ ping 10.0.0.2 -c 3 PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data. 64 bytes from 10.0.0.2: icmp_seq=1 ttl=63 time=3.79 ms 64 bytes from 10.0.0.2: icmp_seq=2 ttl=63 time=0.704 ms 64 bytes from 10.0.0.2: icmp_seq=3 ttl=63 time=0.565 ms

--- 10.0.0.2 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2001ms rtt min/avg/max/mdev = 0.565/1.688/3.796/1.491 ms localadmin@qa4:~/devstack$ ping 10.0.0.1 -c 3 PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data. 64 bytes from 10.0.0.1: icmp_seq=1 ttl=64 time=0.140 ms 64 bytes from 10.0.0.1: icmp_seq=2 ttl=64 time=0.064 ms 64 bytes from 10.0.0.1: icmp_seq=3 ttl=64 time=0.068 ms

--- 10.0.0.1 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 1999ms rtt min/avg/max/mdev = 0.064/0.090/0.140/0.036 ms localadmin@qa4:~/devstack$ ping 10.0.0.3 -c 3 PING 10.0.0.3 (10.0.0.3) 56(84) bytes of data. 64 bytes from 10.0.0.3: icmp_seq=1 ttl=63 time=0.775 ms 64 bytes from 10.0.0.3: icmp_seq=2 ttl=63 time=0.335 ms 64 bytes from 10.0.0.3: icmp_seq=3 ttl=63 time=0.103 ms

--- 10.0.0.3 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 1998ms rtt min/avg/max/mdev = 0.103/0.404/0.775/0.279 ms

Is this expected behavior? I thought I need to be in the linux IP namespace in order to ping successfully:

sudo ip netns exec qrouter-1683019f-6322-4d0a-bc43-c7a890b5e053 ping 10.0.0.2

When I ping the floating IP 172.29.173.15, it fails:

localadmin@qa4:~/devstack$ ping 172.29.173.15 -c 3 PING 172.29.173.15 (172.29.173.15) 56(84) bytes of data.

--- 172.29.173.15 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 1999ms

However, at the Compute node hosting the VM, tcpdump shows the ping src address is 172.29.173.1 and it is successful:

16:21:11.095676 IP 172.29.173.1 > 10.0.0.2: ICMP echo request, id 24273, seq 1, length 64 16:21:11.096704 IP 10.0.0.2 > 172.29.173.1: ICMP echo reply, id 24273, seq 1, length 64 16:21:12.094577 IP 172.29.173.1 > 10.0.0.2: ICMP echo request, id 24273, seq 2, length 64 16:21:12.094934 IP 10.0.0.2 > 172.29.173.1: ICMP echo reply, id 24273, seq 2, length 64 16:21:13.103541 IP 172.29.173.1 > 10.0.0.2: ICMP echo request, id 24273, seq 3, length 64 16:21:13.103892 IP 10.0.0.2 > 172.29.173.1: ICMP echo reply, id 24273, seq 3, length 64

After I remove “eth1” from br-ex, ping the floating IP becomes successful:

localadmin@qa4:~/devstack$ sudo ovs-vsctl del-port br-ex eth1 localadmin@qa4:~/devstack$ sudo ovs-vsctl show 4b5380e3-d619-470b-be99-deda868283c9 Bridge br-ex Port br-ex Interface br-ex type: internal Port "qg-c4bb44ec-94" Interface "qg-c4bb44ec-94" type: internal

localadmin@qa4:~/devstack$ ping 172.29.173.15 -c 3 PING 172.29.173.15 (172.29.173.15) 56(84) bytes of data. 64 bytes from 172.29.173.15: icmp_seq=1 ttl=63 time=2.34 ms 64 bytes from 172.29.173.15: icmp_seq=2 ttl=63 time=0.663 ms 64 bytes from 172.29.173.15: icmp_seq=3 ttl=63 time=0.631 ms

--- 172.29.173.15 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2002ms rtt min/avg/max/mdev = 0.631/1.213/2.345/0.800 ms

Note that at the Compute node, tcpdump still shows the ping src address is 172.29.173.1.

Ping the VM tenant, router and DHCP addresses still successful:

localadmin@qa4:~/devstack$ ping 10.0.0.1 -c 3 PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data. 64 bytes from 10.0.0.1: icmp_seq=1 ttl=64 time=0.565 ms 64 bytes from 10.0.0.1: icmp_seq=2 ttl=64 time=0.059 ms 64 bytes from 10.0.0.1: icmp_seq=3 ttl=64 time=0.083 ms

--- 10.0.0.1 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 1998ms rtt min/avg/max/mdev = 0.059/0.235/0.565/0.233 ms localadmin@qa4:~/devstack$ ping 10.0.0.2 -c 3 PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data. 64 bytes from 10.0.0.2: icmp_seq=1 ttl=63 time=1.90 ms 64 bytes from 10.0.0.2: icmp_seq=2 ttl=63 time=0.623 ms 64 bytes from 10.0.0.2: icmp_seq=3 ttl=63 time=0.669 ms

--- 10.0.0.2 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.623/1.066/1.907/0.595 ms localadmin@qa4:~/devstack$ ping 10.0.0.3 -c 3 PING 10.0.0.3 (10.0.0.3) 56(84) bytes of data. 64 bytes from 10.0.0.3: icmp_seq=1 ttl=63 time=0.563 ms 64 bytes from 10.0.0.3: icmp_seq=2 ttl=63 time=0.107 ms 64 bytes from 10.0.0.3: icmp_seq=3 ttl=63 time=0.078 ms

--- 10.0.0.3 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 1998ms rtt min/avg/max/mdev = 0.078/0.249/0.563/0.222 ms

Questions: Can I ping the VM tenant and floating IP from the Network host? Or do I need to be in the linux namespace? Do I need to add a physical port to br-ex? Why am I seeing the strange behavior as described above? Thanks, Danny

openvswitch and br-ex

Hi,

I’m using openvswitch and create the br-ex bridge for external access.

I added physical port eth1 to br-ex:

localadmin@qa4:~/devstack$ sudo ovs-vsctl show
4b5380e3-d619-470b-be99-deda868283c9
    Bridge br-ex
        Port br-ex
            Interface br-ex
                type: internal
        Port "eth1"
            Interface "eth1"
        Port "qg-c4bb44ec-94"
            Interface "qg-c4bb44ec-94"
                type: internal

internal

Config in /etc/network/interfaces as follows:

auto eth1
iface eth1 inet manual
        up ifconfig $IFACE 0.0.0.0 up
        up ip link set $IFACE promisc on
        down ip link set $IFACE promisc off
        down ifconfig $IFACE down 

auto br-ex iface br-ex inet static address 172.29.173.4 netmask 255.255.255.192 gateway 172.29.173.1 dns-nameservers 172.29.74.154

172.29.74.154

Network node host IP is 172.29.172.161/25 at eth0 and routing table as follows:

Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         172.29.172.129  0.0.0.0         UG        0 0          0 eth0
10.0.0.0        172.29.173.5    255.255.255.0   UG        0 0          0 br-ex
172.29.172.128  0.0.0.0         255.255.255.128 U         0 0          0 eth0
172.29.173.0    0.0.0.0         255.255.255.224 U         0 0          0 br-ex
172.29.173.0    0.0.0.0         255.255.255.192 U         0 0          0 br-ex
192.168.122.0   0.0.0.0         255.255.255.0   U         0 0          0 virbr0

virbr0

There is one VM with tenant address 10.0.0.2 and floating IP 172.29.173.15:

localadmin@qa4:~/devstack$ nova list
+--------------------------------------+------+--------+------------+-------------+---------------------------------+
| ID                                   | Name | Status | Task State | Power State | Networks                        |
+--------------------------------------+------+--------+------------+-------------+---------------------------------+
| e721103e-763a-4a6c-8b3d-294c1cb55726 | vm1  | ACTIVE | -          | Running     | private=10.0.0.2, 172.29.173.15 |
+--------------------------------------+------+--------+------------+-------------+————————————————+ 

+--------------------------------------+------+--------+------------+-------------+————————————————+

From the Network host, I can ping the VM tenant address, router and DHCP interfaces:

localadmin@qa4:~/devstack$ ping 10.0.0.2 -c 3
PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data.
64 bytes from 10.0.0.2: icmp_seq=1 ttl=63 time=3.79 ms
64 bytes from 10.0.0.2: icmp_seq=2 ttl=63 time=0.704 ms
64 bytes from 10.0.0.2: icmp_seq=3 ttl=63 time=0.565 ms

ms --- 10.0.0.2 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2001ms rtt min/avg/max/mdev = 0.565/1.688/3.796/1.491 ms localadmin@qa4:~/devstack$ ping 10.0.0.1 -c 3 PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data. 64 bytes from 10.0.0.1: icmp_seq=1 ttl=64 time=0.140 ms 64 bytes from 10.0.0.1: icmp_seq=2 ttl=64 time=0.064 ms 64 bytes from 10.0.0.1: icmp_seq=3 ttl=64 time=0.068 ms

ms --- 10.0.0.1 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 1999ms rtt min/avg/max/mdev = 0.064/0.090/0.140/0.036 ms localadmin@qa4:~/devstack$ ping 10.0.0.3 -c 3 PING 10.0.0.3 (10.0.0.3) 56(84) bytes of data. 64 bytes from 10.0.0.3: icmp_seq=1 ttl=63 time=0.775 ms 64 bytes from 10.0.0.3: icmp_seq=2 ttl=63 time=0.335 ms 64 bytes from 10.0.0.3: icmp_seq=3 ttl=63 time=0.103 ms

ms --- 10.0.0.3 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 1998ms rtt min/avg/max/mdev = 0.103/0.404/0.775/0.279 ms

ms

Is this expected behavior? I thought I need to be in the linux IP namespace in order to ping successfully:

sudo ip netns exec qrouter-1683019f-6322-4d0a-bc43-c7a890b5e053 ping 10.0.0.2

10.0.0.2

When I ping the floating IP 172.29.173.15, it fails:

localadmin@qa4:~/devstack$ ping 172.29.173.15 -c 3
PING 172.29.173.15 (172.29.173.15) 56(84) bytes of data.

data. --- 172.29.173.15 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 1999ms

1999ms

However, at the Compute node hosting the VM, tcpdump shows the ping src address is 172.29.173.1 and it is successful:

16:21:11.095676 IP 172.29.173.1 > 10.0.0.2: ICMP echo request, id 24273, seq 1, length 64
16:21:11.096704 IP 10.0.0.2 > 172.29.173.1: ICMP echo reply, id 24273, seq 1, length 64
16:21:12.094577 IP 172.29.173.1 > 10.0.0.2: ICMP echo request, id 24273, seq 2, length 64
16:21:12.094934 IP 10.0.0.2 > 172.29.173.1: ICMP echo reply, id 24273, seq 2, length 64
16:21:13.103541 IP 172.29.173.1 > 10.0.0.2: ICMP echo request, id 24273, seq 3, length 64
16:21:13.103892 IP 10.0.0.2 > 172.29.173.1: ICMP echo reply, id 24273, seq 3, length 64

64

After I remove “eth1” from br-ex, ping the floating IP becomes successful:

localadmin@qa4:~/devstack$ sudo ovs-vsctl del-port br-ex eth1
localadmin@qa4:~/devstack$ sudo ovs-vsctl show
4b5380e3-d619-470b-be99-deda868283c9
    Bridge br-ex
        Port br-ex
            Interface br-ex
                type: internal
        Port "qg-c4bb44ec-94"
            Interface "qg-c4bb44ec-94"
                type: internal

internal localadmin@qa4:~/devstack$ ping 172.29.173.15 -c 3 PING 172.29.173.15 (172.29.173.15) 56(84) bytes of data. 64 bytes from 172.29.173.15: icmp_seq=1 ttl=63 time=2.34 ms 64 bytes from 172.29.173.15: icmp_seq=2 ttl=63 time=0.663 ms 64 bytes from 172.29.173.15: icmp_seq=3 ttl=63 time=0.631 ms

ms --- 172.29.173.15 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2002ms rtt min/avg/max/mdev = 0.631/1.213/2.345/0.800 ms

ms

Note that at the Compute node, tcpdump still shows the ping src address is 172.29.173.1.

Ping the VM tenant, router and DHCP addresses still successful:

localadmin@qa4:~/devstack$ ping 10.0.0.1 -c 3
PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data.
64 bytes from 10.0.0.1: icmp_seq=1 ttl=64 time=0.565 ms
64 bytes from 10.0.0.1: icmp_seq=2 ttl=64 time=0.059 ms
64 bytes from 10.0.0.1: icmp_seq=3 ttl=64 time=0.083 ms

ms --- 10.0.0.1 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 1998ms rtt min/avg/max/mdev = 0.059/0.235/0.565/0.233 ms localadmin@qa4:~/devstack$ ping 10.0.0.2 -c 3 PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data. 64 bytes from 10.0.0.2: icmp_seq=1 ttl=63 time=1.90 ms 64 bytes from 10.0.0.2: icmp_seq=2 ttl=63 time=0.623 ms 64 bytes from 10.0.0.2: icmp_seq=3 ttl=63 time=0.669 ms

ms --- 10.0.0.2 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.623/1.066/1.907/0.595 ms localadmin@qa4:~/devstack$ ping 10.0.0.3 -c 3 PING 10.0.0.3 (10.0.0.3) 56(84) bytes of data. 64 bytes from 10.0.0.3: icmp_seq=1 ttl=63 time=0.563 ms 64 bytes from 10.0.0.3: icmp_seq=2 ttl=63 time=0.107 ms 64 bytes from 10.0.0.3: icmp_seq=3 ttl=63 time=0.078 ms

ms --- 10.0.0.3 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 1998ms rtt min/avg/max/mdev = 0.078/0.249/0.563/0.222 ms

ms

Questions: Can I ping the VM tenant and floating IP from the Network host? Or do I need to be in the linux namespace? Do I need to add a physical port to br-ex? Why am I seeing the strange behavior as described above? Thanks, Danny