Ask Your Question
0

Traffic isn't making it to the network node [GRE]

asked 2014-09-24 23:47:37 -0500

Liam Haworth gravatar image

Afternoon Ask OpenStack, (again, problems seem to love me)

I have got a, almost, fully working 3 node openstack setup. I have a control node, a network node and a compute node.

The compute and network node are connected together via a cross over cable and I have confirmed they can talk to each other via it. The problem I am getting is when I spin up a new instance it spawn correctly but as soon as it tries to do DHCP discovery it fails. I decided to have a look at the traffic by adding a snooper port onto br-int on both the compute node and network node ( http://docs.openstack.org/openstack-ops/content/network_troubleshooting.html (http://docs.openstack.org/openstack-o...) ).

From what I can see, the DHCP discover packets are being sent and are hitting br-int on the compute node but nothing is showing at all on br-int on the network node. Since I have no clue on how GRE works and how to check if it is up I am going to put mind out there and say that I think the GRE tunnel isn't forming properly.

Here is the output from openvswitch on both vmhost3 (compute) and os-net (network)

-> vmhost3

cf751e67-6f99-4df7-8ef5-b91be68418b2
    Bridge br-int
        fail_mode: secure
        Port br-int
            Interface br-int
                type: internal
        Port "qvocebdbed6-fe"
            tag: 10
            Interface "qvocebdbed6-fe"
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port "snooper0"
            Interface "snooper0"
    Bridge br-tun
        Port br-tun
            Interface br-tun
                type: internal
        Port "gre-0a000002"
            Interface "gre-0a000002"
                type: gre
                options: {in_key=flow, local_ip="10.0.0.3", out_key=flow, remote_ip="10.0.0.2"}
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
    ovs_version: "2.0.2"

-> os-net

c8dac448-bf2d-4b57-b2e4-a717a65d8df4
    Bridge br-tun
        Port br-tun
            Interface br-tun
                type: internal
        Port "gre-0a000003"
            Interface "gre-0a000003"
                type: gre
                options: {in_key=flow, local_ip="10.0.0.2", out_key=flow, remote_ip="10.0.0.3"}
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
    Bridge br-int
        fail_mode: secure
        Port "snooper0"
            Interface "snooper0"
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port "tapb0728908-62"
            tag: 4095
            Interface "tapb0728908-62"
                type: internal
        Port br-int
            Interface br-int
                type: internal
        Port "qr-e8c25760-2b"
            tag: 1
            Interface "qr-e8c25760-2b"
                type: internal
    Bridge br-ex
        Port "eth2"
            Interface "eth2"
        Port br-ex
            Interface br-ex
                type: internal
        Port "qg-405dd1a6-2b"
            Interface "qg-405dd1a6-2b"
                type: internal
    ovs_version: "2.0.2"

Thank you in advanced for any help!

edit retag flag offensive close merge delete

Comments

How GRE works https://www.hastexo.com/system/files/...
Check on Neutron Node and Compute Node
ovs-ofctl show br-int && ovs-ofctl dump-flows br-int
ovs-ofctl show br-tun && ovs-ofctl dump-flows br-tun

dbaxps gravatar imagedbaxps ( 2014-09-25 00:09:45 -0500 )edit

os-net (network): http://pastebin.com/H8CdzqiS vmhost3 (compute): http://pastebin.com/GZiiCy83

Liam Haworth gravatar imageLiam Haworth ( 2014-09-25 00:23:19 -0500 )edit

VM Started on GRE Compute Node http://textuploader.com/o9u0

dbaxps gravatar imagedbaxps ( 2014-09-25 01:07:27 -0500 )edit

Status on Neutron Server (GRE) . VM started on Compute http://textuploader.com/o9uk

dbaxps gravatar imagedbaxps ( 2014-09-25 01:20:31 -0500 )edit

None of that is making any sense to me nor is it helping with the problem at hand

Liam Haworth gravatar imageLiam Haworth ( 2014-09-25 01:40:24 -0500 )edit

2 answers

Sort by ยป oldest newest most voted
0

answered 2014-09-25 19:10:30 -0500

Liam Haworth gravatar image

The problem is resolved, due to a earlier issue with the ML2 config not being loaded by neutron-server, the network that I was trying to use were made before I fixed that problem and were showing as local networks and not gre networks.

Thank you for the help anyways!

edit flag offensive delete link more
0

answered 2014-09-25 04:33:46 -0500

dbaxps gravatar image

updated 2014-09-25 08:41:11 -0500

Forced to use answer field due to limitations in comment field
Per Status on Neutron Server (GRE) . VM started on Compute http://textuploader.com/o9uk

  # ovs-ofctl dump-flows br-tun
    [...]
 cookie=0x0, duration=357.367s, table=2, n_packets=319, n_bytes=50989, idle_age=2, priority=1,tun_id=0x4 actions=mod_vlan_vid:3,resubmit(,10)
 cookie=0x0, duration=361.872s, table=2, n_packets=19, n_bytes=1320, idle_age=50, priority=1,tun_id=0x3 actions=mod_vlan_vid:1,resubmit(,10)
 cookie=0x0, duration=358.876s, table=2, n_packets=0, n_bytes=0, idle_age=358, priority=1,tun_id=0x2 actions=mod_vlan_vid:2,resubmit(,10)
 cookie=0x0, duration=366.845s, table=3, n_packets=0, n_bytes=0, idle_age=366, priority=0 actions=drop
 cookie=0x0, duration=366.765s, table=10, n_packets=338, n_bytes=52309, idle_age=2, priority=1 actions=learn(table=20,hard_timeout=300,priority=1,NXM_OF_VLAN_TCI[0..11],NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1

Inside of OpenFlowTable 10, we see the important part at the end of the actions parameter: The output of the packet will happen on the virtual switch port 1

[root@icehouse1 ~]# ovs-ofctl show br-tun 
OFPT_FEATURES_REPLY (xid=0x2): dpid:0000ee5708e6e844
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE
 1(patch-int): addr:b2:56:74:b4:e0:55
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 2(gre-c0a80189): addr:be:6e:f3:88:ee:19
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 LOCAL(br-tun): addr:ee:57:08:e6:e8:44
     config:     PORT_DOWN
     state:      LINK_DOWN
     speed: 0 Mbps now, 0 Mbps max
OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0

So port 1 is patch-int. Packets coming in on patch-tun will be forwarded to the patch-int port of the virtual switch.

Yours os-net (network): http://pastebin.com/H8CdzqiS doesn't contain this forwarding step

Now analyze situation on Compute Node when VM starts

ovs-ofctl dump-flows br-tun
[ ... ]
 cookie=0x0, duration=15879.758s, table=10, n_packets=8167, n_bytes=8544625, idle_age=20, priority=1 actions=learn(table=20,hard_timeout=300,priority=1,NXM_OF_VLAN_TCI[0..11],NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1
 cookie=0x0, duration=15879.699s, table=20, n_packets=4, n_bytes=339, idle_age=245, priority=0 actions=resubmit(,21)
 cookie=0x0, duration=244.922s, table=20, n_packets=1831, n_bytes=213987, hard_timeout=300, idle_age=20, hard_age=20, priority=1,vlan_tci=0x0002/0x0fff,dl_dst=fa:16:3e:60:e6:13 actions=load:0->NXM_OF_VLAN_TCI[],load:0x3->NXM_NX_TUN_ID[],output:2

.[root@icehouse2 ~]# ovs-ofctl show br-tun

OFPT_FEATURES_REPLY (xid=0x2): dpid:00005ee03ab7bc40
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE
 1(patch-int): addr:d2:5e:0a:d4:7a:76
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 2(gre-c0a8017f): addr:32:40:5b:34:de:ad
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 LOCAL(br-tun): addr:5e:e0:3a:b7:bc:40 ...
(more)
edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

2 followers

Stats

Asked: 2014-09-24 23:47:37 -0500

Seen: 797 times

Last updated: Sep 25 '14