Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Icehouse Tutorial Neutron ML2 GRE External Gateway IP Not Pingable\Routed To On Ubuntu 14.04 Over Br-Ex

Hi,

Been trying to follow a tutorial to setup a triple node Openstack Icehouse configuration by scratch in a private Virtualbox environment. The guest VMs are using Ubuntu 14.04. So far I've been able to follow along with everything except the networking portion.

This is the tutorial I've been following: Here

I have the following Virtualbox networks attached to the network node running all the Neutron OVS stuff. Neutron service itself is running on a seperate controller node as described in the guide I linked above. Trying to use GRE tunnels for the transport type.

vmnet0: 10.10.10.0/24 Management Network (Virtualbox Host Only)

vmnet1: 10.20.20.0/24 Internal Tunnel Network to Compute Nodes (Virtualbox Host Only)

vmnet2: 10.0.2.0/24 My supposed external interface without an IP configured behind a Virtualbox NAT interface. DHCP off. (Static Nat)

vmnet3: 10.0.5.0/24 Another Nat interface with DHCP just for easy Internet access to install packages... (Regular Nat)

Problem is, installing and setting up the external bridge interface and verifying it's up by pinging the external gateway interface is not working from the network node. I presume this would be possible since the tutorial instructs you to do this?

Here's how I created the external subnet:

neutron subnet-create ext-net --name ext-subnet \
> --allocation-pool start=10.0.2.101,end=10.0.2.200 \                        
> --disable-dhcp --gateway 10.0.2.1 10.0.2.0/24

And I attached it like so;

neutron router-gateway-set demo-router ext-net
Set gateway for router demo-router

Trying to run tracepath just results in the default GW sending the packet to my physical router out in my apartment which obviously isn't going to know where to send it so I don't think I need to look with tcpdump:

tracepath 10.0.2.101
1?: [LOCALHOST]                                         pmtu 1500
1:  10.0.5.2                                              0.664ms 
1:  10.0.5.2                                              0.586ms 
2:  therouter.internal                                    0.909ms asymm 64 
3:  no reply

Here's my routing table on the network node. No modifications done: Shouldn't have Neutron or OVS updated this for me? Do I need to launch an instance before the routing table is updated?

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.0.5.2        0.0.0.0         UG    0      0        0 eth3
10.0.5.0        0.0.0.0         255.255.255.0   U     0      0        0 eth3
10.10.10.0      0.0.0.0         255.255.255.0   U     0      0        0 eth0
10.20.20.0      0.0.0.0         255.255.255.0   U     0      0        0 eth1

Here's more info on my configuration:

/etc/network/interfaces

This file describes the network interfaces available on your system
and how to activate them. For more information, see interfaces(5).

# The loopback network interface
 auto lo
iface lo inet loopback

#VMNET0 10.10.10.0/24
 auto eth0
 iface eth0 inet static
 address 10.10.10.52
 netmask 255.255.255.0
 network 10.10.10.0

 #VMNET2 10.20.20.0/24
 auto eth1
 iface eth1 inet static
 address 10.20.20.51
 netmask 255.255.255.0
 network 10.20.20.0

 # NAT 10.0.2.0/24 The Supposed External interface
 auto eth2
 iface eth2 inet manual
 up ip link set dev $IFACE up
 up ip link set $IFACE promisc on
 down ip link set dev $IFACE down
 down ip link set $IFACE promisc off

 #NAT network just for internet access 
 auto eth3
 iface eth3 inet dhcp

Here's an ifconfig -a on the network node after everything was installed.

br-ex     Link encap:Ethernet  HWaddr 08:00:27:db:50:8f  
      inet6 addr: fe80::5802:28ff:fe6a:44b6/64 Scope:Link
      UP BROADCAST RUNNING  MTU:1500  Metric:1
      RX packets:12 errors:0 dropped:0 overruns:0 frame:0
      TX packets:19 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:0 
      RX bytes:864 (864.0 B)  TX bytes:10110 (10.1 KB)

 br-int    Link encap:Ethernet  HWaddr 3e:43:b3:93:7a:42  
      inet6 addr: fe80::b4dd:7bff:fe7a:10e6/64 Scope:Link
      UP BROADCAST RUNNING  MTU:1500  Metric:1
      RX packets:9 errors:0 dropped:0 overruns:0 frame:0
      TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:0 
      RX bytes:738 (738.0 B)  TX bytes:648 (648.0 B)

  br-tun    Link encap:Ethernet  HWaddr         12:ac:cf:d6:fe:46                                                                                                                                                              
      inet6 addr: fe80::94b8:8fff:feb8:810b/64 Scope:Link                                                                                                                                                        
      UP BROADCAST RUNNING  MTU:1500  Metric:1                                                                                                                                                                   
      RX packets:0 errors:0 dropped:0 overruns:0 frame:0                                                                                                                                                         
      TX packets:8 errors:0 dropped:0 overruns:0 carrier:0                                                                                                                                                       
      collisions:0 txqueuelen:0                                                                                                                                                                                  
      RX bytes:0 (0.0 B)  TX bytes:648 (648.0 B)                                                                                                                                                                 

 eth0      Link encap:Ethernet  HWaddr 08:00:27:9d:de:0b                                                                                                                                                              
      inet addr:10.10.10.52  Bcast:10.10.10.255   Mask:255.255.255.0                                                                                                                                              
      inet6 addr: fe80::a00:27ff:fe9d:de0b/64 Scope:Link                                                                                                                                                         
      UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1                                                                                                                                                         
      RX packets:7206 errors:0 dropped:0 overruns:0 frame:0                                                                                                                                                      
      TX packets:8583 errors:0 dropped:0 overruns:0 carrier:0                                                                                                                                                    
      collisions:0 txqueuelen:1000                                                                                                                                                                               
      RX bytes:643003 (643.0 KB)  TX bytes:2284600 (2.2 MB)                                                                                                                                                      

 eth1      Link encap:Ethernet  HWaddr  08:00:27:99:3b:39                                                                                                                                                              
      inet addr:10.20.20.51  Bcast:10.20.20.255  Mask:255.255.255.0                                                                                                                                              
      inet6 addr: fe80::a00:27ff:fe99:3b39/64 Scope:Link                                                                                                                                                         
      UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1                                                                                                                                                         
      RX packets:1 errors:0 dropped:0 overruns:0 frame:0                                                                                                                                                         
      TX packets:11 errors:0 dropped:0 overruns:0 carrier:0                                                                                                                                                      
      collisions:0 txqueuelen:1000                                                                                                                                                                               
      RX bytes:42 (42.0 B)  TX bytes:934 (934.0 B)                                                                                                                                                               

  eth2      Link encap:Ethernet  HWaddr 08:00:27:db:50:8f                                                                                                                                                              
      inet6 addr: fe80::a00:27ff:fedb:508f/64 Scope:Link                                                                                                                                                         
      UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1                                                                                                                                                 
      RX packets:2 errors:0 dropped:0 overruns:0 frame:0                                                                                                                                                         
      TX packets:28 errors:0 dropped:0 overruns:0 carrier:0                                                                                                                                                      
      collisions:0 txqueuelen:1000 
      RX bytes:84 (84.0 B)  TX bytes:4756 (4.7 KB)

 eth3      Link encap:Ethernet  HWaddr 08:00:27:17:16:ba  
      inet addr:10.0.5.15  Bcast:10.0.5.255  Mask:255.255.255.0
      inet6 addr: fe80::a00:27ff:fe17:16ba/64 Scope:Link
      UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
      RX packets:195 errors:0 dropped:0 overruns:0 frame:0
      TX packets:228 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:1000 
      RX bytes:23490 (23.4 KB)  TX bytes:40415 (40.4 KB)

 lo        Link encap:Local Loopback  
      inet addr:127.0.0.1  Mask:255.0.0.0
      inet6 addr: ::1/128 Scope:Host
      UP LOOPBACK RUNNING  MTU:65536  Metric:1
      RX packets:0 errors:0 dropped:0 overruns:0 frame:0
      TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:0 
      RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

ovs-system Link encap:Ethernet  HWaddr 46:68:99:87:2b:28  
      BROADCAST MULTICAST  MTU:1500  Metric:1
      RX packets:0 errors:0 dropped:0 overruns:0 frame:0
      TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:0 
      RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

Shouldn't there be the 10.0.2.101 IP listening on br-ex? Do I need to add that specifically to the interfaces file and restart networking? Is it specified in another config file somewhere? The tutorial makes it assume that Neutron\OVS will set this up for you. I've seen some configs where the user specifes br-ex in their interfaces file....

Here's an ip a show after everything is setup, again I see no IP listening on br-ex:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
   valid_lft forever preferred_lft forever
inet6 ::1/128 scope host 
   valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 08:00:27:9d:de:0b brd ff:ff:ff:ff:ff:ff
inet 10.10.10.52/24 brd 10.10.10.255 scope global eth0
   valid_lft forever preferred_lft forever
inet6 fe80::a00:27ff:fe9d:de0b/64 scope link 
   valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 08:00:27:99:3b:39 brd ff:ff:ff:ff:ff:ff
inet 10.20.20.51/24 brd 10.20.20.255 scope global eth1
   valid_lft forever preferred_lft forever
inet6 fe80::a00:27ff:fe99:3b39/64 scope link 
   valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master ovs-system state UP group default qlen 1000
link/ether 08:00:27:db:50:8f brd ff:ff:ff:ff:ff:ff
inet6 fe80::a00:27ff:fedb:508f/64 scope link 
   valid_lft forever preferred_lft forever
5: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 08:00:27:17:16:ba brd ff:ff:ff:ff:ff:ff
inet 10.0.5.15/24 brd 10.0.5.255 scope global eth3
   valid_lft forever preferred_lft forever
inet6 fe80::a00:27ff:fe17:16ba/64 scope link 
   valid_lft forever preferred_lft forever
6: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default 
link/ether 46:68:99:87:2b:28 brd ff:ff:ff:ff:ff:ff
7: br-int: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default 
link/ether 3e:43:b3:93:7a:42 brd ff:ff:ff:ff:ff:ff
inet6 fe80::b4dd:7bff:fe7a:10e6/64 scope link 
   valid_lft forever preferred_lft forever
9: br-ex: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default 
link/ether 08:00:27:db:50:8f brd ff:ff:ff:ff:ff:ff
inet6 fe80::5802:28ff:fe6a:44b6/64 scope link 
   valid_lft forever preferred_lft forever
10: br-tun: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default 
link/ether 12:ac:cf:d6:fe:46 brd ff:ff:ff:ff:ff:ff
inet6 fe80::94b8:8fff:feb8:810b/64 scope link 
   valid_lft forever preferred_lft forever

Here's what I get when I do ovs-vsctl on the network node.

ovs-vsctl show
4326918a-a42e-49a9-9b9b-dd01dc9f4271
Bridge br-int
    fail_mode: secure
    Port patch-tun
        Interface patch-tun
            type: patch
            options: {peer=patch-int}
    Port "qr-4db52864-64"
        tag: 1
        Interface "qr-4db52864-64"
            type: internal
    Port br-int
        Interface br-int
            type: internal
Bridge br-tun
    Port br-tun
        Interface br-tun
            type: internal
    Port "gre-0a141434"
        Interface "gre-0a141434"
            type: gre
            options: {in_key=flow, local_ip="10.20.20.51", out_key=flow, remote_ip="10.20.20.52"}
    Port patch-int
        Interface patch-int
            type: patch
            options: {peer=patch-tun}
Bridge br-ex
    Port br-ex
        Interface br-ex
            type: internal
    Port "eth2"
        Interface "eth2"
    Port "qg-bfbbda0d-ab"
        Interface "qg-bfbbda0d-ab"
            type: internal
ovs_version: "2.0.1"

I mean to me, everything seems like it's supposed to be up and running correctly in OVS. I didn't see any errors in the ovs and neutron logs during the subnet and router attachment process. Here's the neutron.conf on the network node:

[DEFAULT]
verbose = True
state_path = /var/lib/neutron
 lock_path = $state_path/lock
  auth_strategy = keystone
 rpc_backend = neutron.openstack.common.rpc.impl_kombu
 rabbit_host = controller
 rabbit_password = guest
 core_plugin = ml2
 service_plugins = router
 allow_overlapping_ips = True
 notification_driver = neutron.openstack.common.notifier.rpc_notifier
[quotas]
[agent]
root_helper = sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf
[keystone_authtoken]
auth_uri = http://controller:5000
auth_host = controller
auth_protocol = http 
auth_port = 35357
admin_tenant_name = service
admin_user = neutron
admin_password = password
signing_dir = $state_path/keystone-signing
[database]
connection = mysql://neutron:password@controller/neutron
[service_providers]
    service_provider=LOADBALANCER:Haproxy:neutron.services.loadbalancer.drivers.haproxy.plugin_driver.HaproxyOnHostPluginDriver:default
service_provider=VPN:openswan:neutron.services.vpn.service_drivers.ipsec.IPsecVPNDriver:default

Can someone let me know what I should be looking at next to troubleshoot this? I've been able to reproduce this behavior by redoing the lab over five times following the tutorial carefully. I can't help but feel the tutorial is missing steps.

Thanks in advance.