Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

openvswitch with rdo repo on centos 7 crash

i have a centos 7 node pulling rpm's from the rdo repo's. i can configure my uplinks such that:

[root@net-pool01 ~]# ovs-vsctl show
6d79b968-b701-4ccd-ac3b-8fab6cdb55d2
    Bridge br-int
        fail_mode: secure
        Port br-int
            Interface br-int
                type: internal
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
    Bridge br-uplink
        Port br-uplink
            Interface br-uplink
                type: internal
        Port "em1"
            Interface "em1"
    Bridge br-tun
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port br-tun
            Interface br-tun
                type: internal
    ovs_version: "2.1.3"
[root@net-pool01 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: em3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether ec:f4:bb:bf:be:bc brd ff:ff:ff:ff:ff:ff
3: em4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether ec:f4:bb:bf:be:bd brd ff:ff:ff:ff:ff:ff
4: em1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP qlen 1000
    link/ether ec:f4:bb:bf:be:b8 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::eef4:bbff:febf:beb8/64 scope link
       valid_lft forever preferred_lft forever
5: em2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether ec:f4:bb:bf:be:ba brd ff:ff:ff:ff:ff:ff
6: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
    link/ether e2:10:68:c9:9f:7a brd ff:ff:ff:ff:ff:ff
7: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
    link/ether d6:6a:f9:58:4d:4f brd ff:ff:ff:ff:ff:ff
8: br-uplink: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
    link/ether ec:f4:bb:bf:be:b8 brd ff:ff:ff:ff:ff:ff
    inet 172.23.66.78/23 brd 172.23.67.255 scope global br-uplink
       valid_lft forever preferred_lft forever
    inet6 2620:114:d000:233f:eef4:bbff:febf:beb8/64 scope global dynamic
       valid_lft 2591852sec preferred_lft 604652sec
    inet6 fe80::eef4:bbff:febf:beb8/64 scope link
       valid_lft forever preferred_lft forever
12: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
    link/ether 8e:02:37:e7:62:45 brd ff:ff:ff:ff:ff:ff

em1 is physically connected to a switch trunk port where:

interface Ethernet102/1/24
  description net-pool01
  no lldp receive
  switchport mode trunk
  switchport trunk native vlan 831
  switchport trunk allowed vlan 831,1441

and i have sysconfig scripts as such:

[root@net-pool01 ~]# cat /etc/sysconfig/network-scripts/ifcfg-em1
# Generated by parse-kickstart
DEVICE=em1
ONBOOT=yes
PERSISTENT_DHCLIENT=0
IPV6INIT=no
IPV6_AUTOCONF=no
BOOTPROTO=none
UUID=0f8cdd94-bf57-4d44-9e8b-9db4e6152d9b
#TYPE=Ethernet
TYPE=OVSPort
OVS_BRIDGE="br-uplink"
NM_CONTROLLED=no
[root@net-pool01 ~]# cat /etc/sysconfig/network-scripts/ifcfg-br-uplink
DEVICE=br-uplink
DEVICETYPE=ovs
TYPE=OVSBridge
BOOTPROTO=static
IPADDR=172.23.66.78
NETMASK=255.255.254.0
GATEWAY=172.23.66.1
ONBOOT=yes
HOTPLUG=no
USERCTL=no
NM_CONTROLLED=no

[root@net-pool01 ~]# cat /etc/sysconfig/network-scripts/ifcfg-br-tun
cat: /etc/sysconfig/network-scripts/ifcfg-br-tun: No such file or directory
[root@net-pool01 ~]# cat /etc/sysconfig/network-scripts/ifcfg-br-int
cat: /etc/sysconfig/network-scripts/ifcfg-br-int: No such file or directory

and everything works; ie i can ping my host at 172.23.66.78.

however, if i reboot the host... i can usually get one or two pings in after it booted, but then it all stops. in the console i get the following message:

[   21.041208] ixgbe 0000:01:00.0 em1: NIC Link is Up 1 Gbps, Flow Control: RX
[   21.049523] IPv6: ADDRCONF(NETDEV_CHANGE): em1: link becomes ready
[   22.193566] device br-tun left promiscuous mode
[   22.227193] ovs-vswitchd[1442]: segfault at 0 ip 00007f034983dab0 sp 00007fff13e1bea8 error 4 in ovs-vswitchd[7f03497cf000+153000]
[   22.677412] device br-tun entered promiscuous mode

CentOS Linux 7 (Core)
Kernel 3.10.0-229.el7.x86_64 on an x86_64

net-pool01 login: [   25.964175] device br-tun left promiscuous mode
[   25.993067] [3103]: segfault at 0 ip 00007f034983dab0 sp 00007fff13e1bea8 error 4 in ovs-vswitchd[7f03497cf000+153000]
[   32.222698] revalidator_19[3278]: segfault at 0 ip 00007f034983dab0 sp 00007f032cff6918 error 4
[   32.222709] revalidator_18[3277]: segfault at 0 ip 00007f034983dab0 sp 00007f032d7f78f8 error 4 in ovs-vswitchd[7f03497cf000+153000]
[   32.245826]  in ovs-vswitchd[7f03497cf000+153000]

CentOS Linux 7 (Core)
Kernel 3.10.0-229.el7.x86_64 on an x86_64

net-pool01 login:

the annoying thing is that if i restart openvswitch it works again (but sometimes crashes again).

[root@net-pool01 ~]# service openvswitch restart
Redirecting to /bin/systemctl restart  openvswitch.service
[   48.992287] device br-tun entered promiscuous mode
[root@net-pool01 ~]# [  114.755370] device br-tun left promiscuous mode
[  114.785829] ovs-vswitchd[3365]: segfault at 0 ip 00007ff09dcb1ab0 sp 00007fff1d681218 error 4 in ovs-vswitchd[7ff09dc43000+153000]
[  115.103826] revalidator_19[3519]: segfault at 0 ip 00007ff09dcb1ab0 sp 00007ff070ff68f8 error 4
[  115.103834] revalidator_23[3523]: segfault at 0 ip 00007ff09dcb1ab0 sp 00007ff04affa8f8 error 4
[  115.103845] revalidator_22[3522]: segfault at 0 ip 00007ff09dcb1ab0 sp 00007ff04b7fb8f8 error 4
[  115.103853] revalidator_24[3524]: segfault at 0 ip 00007ff09dcb1ab0 sp 00007ff04a7f98f8 error 4
[  115.103856]  in ovs-vswitchd[7ff09dc43000+153000]
[  115.103857]
[  115.103864]  in ovs-vswitchd[7ff09dc43000+153000]
[  115.104024]  in ovs-vswitchd[7ff09dc43000+153000]
[  115.160153]  in ovs-vswitchd[7ff09dc43000+153000]

does anyone have any suggestions? (excluding recompiling from source to the latest version of openvswitch). is my configuration extra-ordinary?

cheers,