Ask Your Question
0

Openvswitch agent not creating internal ports after restart

asked 2016-11-16 12:56:09 -0500

samb gravatar image

updated 2016-11-18 21:53:14 -0500

SOLVED

So the problem was that the linux bridge interfaces on the host node were removed for whatever reason after the router failure, and those are not created by neutron, those are created by nova. So instead of messing with neutron and openvswitch, all I had to do was service openstack-nova-compute restart and then a final service neutron-openvswitch-agent restart and everything was fixed.

Original problem:

After a physical router failure we had to restart the network service on our neutron host machine. After doing this and even after restarting the neutron-openvswitch-agent and neutron-l3-agent we are unable to reach VMs on the host. This is not a problem for our other two compute nodes on the same neutron router.

I was able to figure out that openvswitch is not creating ports for the VMs on the neutron host, although they are present in /etc/openvswitch/conf.db . I cannot find errors in any logs.

In the neutron database, the ports on the neutron host have the status of "DOWN".

Is there anyway to get openvswitch to realize it needs to create the ports if restarting it does not work?

I'm not sure exactly what config files or log files are relevant, and I just hope my question isn't too messy. This is for the Mitaka release.

Here is the truncated ovs-vsctl show output on the neutron/compute node where I cannot reach VMs

    Bridge br-ex
    Port "em2"
        Interface "em2"
    Port "qg-15c8ef30-fa"
        Interface "qg-15c8ef30-fa"
            type: internal
    Port phy-br-ex
        Interface phy-br-ex
            type: patch
            options: {peer=int-br-ex}
    Port br-ex
        Interface br-ex
            type: internal
Bridge br-int
    fail_mode: secure
    Port int-br-ex
        Interface int-br-ex
            type: patch
            options: {peer=phy-br-ex}
    Port patch-tun
        Interface patch-tun
            type: patch
            options: {peer=patch-int}
    Port "tap303d9e37-5f"
        tag: 1
        Interface "tap303d9e37-5f"
            type: internal
    Port "qr-943c88cf-6c"
        tag: 1
        Interface "qr-943c88cf-6c"
            type: internal
    Port br-int
        Interface br-int
            type: internal
Bridge br-tun
    fail_mode: secure
    Port patch-int
        Interface patch-int
            type: patch
            options: {peer=patch-tun}
    Port "vxlan-c0a8040b"
        Interface "vxlan-c0a8040b"
            type: vxlan
            options {redacted}

And here is ovs-vsctl show on a compute node that does work.

    Bridge br-ex
    Port br-ex
        Interface br-ex
            type: internal
    Port phy-br-ex
        Interface phy-br-ex
            type: patch
            options: {peer=int-br-ex}
    Port "em1"
        Interface "em1"
Bridge br-int
    fail_mode: secure
    Port "qvoc3aced4c-a1"
        tag: 3
        Interface "qvoc3aced4c-a1"
    Port br-int
        Interface br-int
            type: internal
    Port patch-tun
        Interface patch-tun
            type: patch
            options: {peer=patch-int}
    Port int-br-ex
        Interface int-br-ex
            type: patch
            options: {peer=phy-br-ex}
    Port "qvo8323110f-28"
        tag: 3
        Interface "qvo8323110f-28"
    Port "qvo265e94f3-db"
        tag: 3
        Interface "qvo265e94f3-db"
Bridge br-tun
    fail_mode: secure
    Port patch-int
        Interface patch-int
            type: patch
            options: {peer=patch-tun}
    Port "vxlan-c0a8040b"
        Interface "vxlan-c0a8040b"
            type: vxlan
            options: {redacted}
    Port "vxlan-c0a80402"
        Interface "vxlan-c0a80402"
            type: vxlan
            options: {redacted}
    Port br-tun
        Interface br-tun
            type: internal
ovs_version: "2.5.0"

See how the working node creates the ports on br-int that are like "qvoxxxxxxx-xx"?

I also see in the broken node's openswitch-agent.log but not on the working nodes' log:

2016-11-17 09:43:37.253 32811 WARNING stevedore.named [req-562caef6-a0cf-4c0a-8443-d1e151216af6 - - - - -] Could not load neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver

If I execute ovsdb-client dump on the ... (more)

edit retag flag offensive close merge delete

Comments

Post ovs-vsctl show on Network Node ( running Neutron Server ) and ovs-vsctl show on Compute Node. There is an issue with interprocess communication Neutron Server <=> OVS agent

dbaxps gravatar imagedbaxps ( 2016-11-17 07:21:53 -0500 )edit

I've just edited my question to include them.

samb gravatar imagesamb ( 2016-11-17 07:38:38 -0500 )edit

I also just noticed this is in /var/log/neutron/openvswitch-agent.log

Error received from [ovsdb-client monitor Interface name,ofport,external_ids --format=json]: None
Process [ovsdb-client monitor Interface name,ofport,external_ids --format=json] dies due to the error: None
samb gravatar imagesamb ( 2016-11-17 07:48:46 -0500 )edit

Post openvswitch_agent.ini on both nodes and neutron.conf from Controller Node

dbaxps gravatar imagedbaxps ( 2016-11-17 10:59:42 -0500 )edit
dbaxps gravatar imagedbaxps ( 2016-11-17 11:12:13 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
-1

answered 2016-11-16 20:52:56 -0500

silently gravatar image

try this :

ovs-vsctl add-br br.name

ovs-vsctl add-port br.name  nic.name
edit flag offensive delete link more

Comments

The port between the external bridge and the nic exists, but its not creating ports between the internal bridge and the VMs

samb gravatar imagesamb ( 2016-11-17 07:33:55 -0500 )edit

I think linuxbridge-agent had some error , you can see the log file .

silently gravatar imagesilently ( 2016-12-25 20:51:34 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2016-11-16 12:56:09 -0500

Seen: 2,566 times

Last updated: Nov 18 '16