Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Three nodes setup with neutron cannot ping either VMs or External gateway

HI, I installed openstack Liberty using juju on Ubuntu 14.04 on three nodes using maas & juju.

The maas controller machine has two interfaces, eth0 with ip 10.0.0.1 and eth1 192.168.29 as gateway to the external network.

Here is my juju deployment config :

keystone:
  admin-password: openstack
  debug: 'true'
  log-level: DEBUG
nova-cloud-controller:
  network-manager: Neutron
nova-compute:
  flat-interface: 'eth0'
  enable-live-migration: 'True'
  migration-auth-type: "none"
  virt-type: kvm
  enable-resize: 'True'
neutron-gateway:
  ext-port: 'eth1'
  bridge-mappings: 'external:br-ex'
  instance-mtu: 1400
glance:
cinder:
openstack-dashboard:
  webroot: "/"

The deployment script

#!/bin/bash
juju deploy --to 0 juju-gui
juju deploy --to lxc:0 mysql
juju deploy --config config.yaml --to lxc:0 keystone
juju deploy --config config.yaml --to lxc:0 nova-cloud-controller
juju deploy --config config.yaml --to lxc:0 glance
juju deploy --to lxc:0 rabbitmq-server
juju deploy --config config.yaml --to lxc:0 openstack-dashboard
juju deploy --config config.yaml --to lxc:0 cinder
juju deploy --config config.yaml nova-compute --constraints "tags=compute"
juju deploy --config config.yaml neutron-gateway --constraints "tags=neutron"
juju add-relation mysql keystone
juju add-relation nova-cloud-controller mysql
juju add-relation nova-cloud-controller rabbitmq-server
juju add-relation nova-cloud-controller glance
juju add-relation nova-cloud-controller keystone
juju add-relation nova-compute nova-cloud-controller
juju add-relation nova-compute mysql
juju add-relation nova-compute rabbitmq-server:amqp
juju add-relation nova-compute glance
juju add-relation glance mysql
juju add-relation glance keystone
juju add-relation glance cinder
juju add-relation mysql cinder
juju add-relation cinder rabbitmq-server
juju add-relation cinder nova-cloud-controller
juju add-relation cinder keystone
juju add-relation openstack-dashboard keystone
juju add-relation neutron-gateway mysql
juju add-relation neutron-gateway:amqp rabbitmq-server:amqp
juju add-relation neutron-gateway nova-cloud-controller
juju set keystone admin-password="openstack"
# display status
juju stat --format=tabular
# Download image
wget http://cloud-images.ubuntu.com/trusty/current/trusty-server-cloudimg-amd64-disk1.img
glance add name="Trusty x86_64" is_public=true container_format=ovf disk_format=qcow2 < trusty-server-cloudimg-amd64-disk1.img
# set default security group
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
# add keypair
nova keypair-add --pub-key ~/.ssh/id_rsa.pub id_rsa
nova image-list

After initial setup, I followed the documentation up to "Verify connectivity" and setup neutron networks as follows :

#create external network and subnet
neutron net-create ext-net --shared --router:external=True
neutron subnet-create ext-net --name ext-subnet \
  --allocation-pool start=192.168.1.230,end=192.168.1.250 \
  --disable-dhcp --gateway 192.168.1.1 192.168.1.0/24

#create tenant network and subnet
neutron net-create demo-net
neutron subnet-create demo-net --name demo-subnet \
  --gateway 10.0.1.1 10.0.1.1/24

# add router
neutron router-create demo-router
neutron router-interface-add demo-router demo-subnet
neutron router-gateway-set demo-router ext-net

The neutron-gateway node has two physical network interfaces :

  • eth0 connected to the juju 10.0.0.0/24 network
  • eth1 connected to the external 192.168.1.0/24 network

When created, the router should get the first address in the specified pool, 192.168.1.230 and says so but this address is unreachable from the controller and from the external network.

ping 192.168.1.230
PING 192.168.1.230 (192.168.1.230) 56(84) bytes of data.
From 192.168.1.29 icmp_seq=1 Destination Host Unreachable
From 192.168.1.29 icmp_seq=2 Destination Host Unreachable
From 192.168.1.29 icmp_seq=3 Destination Host Unreachable

There must me something missing but I don't get it.

On neutron-gateway node

cat /etc/network/interfaces
auto lo
iface lo inet loopback
    dns-nameservers 192.168.1.29
    dns-search maas

iface eth0 inet manual

auto juju-br0
iface juju-br0 inet static
    bridge_ports eth0
    gateway 10.0.0.1
    address 10.0.0.12/24
    mtu 1500

auto eth1
iface eth1 inet manual
    mtu 1500

,

sudo ovs-vsctl show
sudo: unable to resolve host tragic-pickle
fc996723-61c1-4fd0-a38f-1dd0df07eeb8
    Bridge br-tun
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port br-tun
            Interface br-tun
                type: internal
        Port "gre-0a00000b"
            Interface "gre-0a00000b"
                type: gre
                options: {in_key=flow, local_ip="10.0.0.12", out_key=flow, remote_ip="10.0.0.11"}
    Bridge br-ex
        Port br-ex
            Interface br-ex
                type: internal
        Port "tapc4b60553-66"
            Interface "tapc4b60553-66"
        Port "eth1"
            Interface "eth1"
        Port phy-br-ex
            Interface phy-br-ex
    Bridge br-int
        fail_mode: secure
        Port br-int
            Interface br-int
                type: internal
        Port "tapd3a7661a-6c"
            tag: 1
            Interface "tapd3a7661a-6c"
        Port int-br-ex
            Interface int-br-ex
        Port "tapf36bfad8-fe"
            tag: 1
            Interface "tapf36bfad8-fe"
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
    ovs_version: "2.0.2"

,

ifconfig
br-ex     Link encap:Ethernet  HWaddr 68:05:ca:3b:e6:13  
          inet6 addr: fe80::443c:21ff:fe55:7771/64 Scope:Link
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:2011418 errors:0 dropped:16 overruns:0 frame:0
          TX packets:2735 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:248614834 (248.6 MB)  TX bytes:235306 (235.3 KB)

br-int    Link encap:Ethernet  HWaddr 62:6f:d2:ca:3b:43  
          inet6 addr: fe80::f0b2:73ff:fe37:180a/64 Scope:Link
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:44 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:3970 (3.9 KB)  TX bytes:648 (648.0 B)

br-tun    Link encap:Ethernet  HWaddr ba:5a:ae:81:76:4d  
          inet6 addr: fe80::f8e6:5eff:fed0:6199/64 Scope:Link
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:648 (648.0 B)

eth0      Link encap:Ethernet  HWaddr 74:d4:35:fa:0f:37  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:3048127 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3374042 errors:0 dropped:0 overruns:0 carrier:1
          collisions:0 txqueuelen:1000 
          RX bytes:529788654 (529.7 MB)  TX bytes:390302367 (390.3 MB)

eth1      Link encap:Ethernet  HWaddr 68:05:ca:3b:e6:13  
          inet6 addr: fe80::6a05:caff:fe3b:e613/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2274463 errors:0 dropped:1 overruns:0 frame:0
          TX packets:2790 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:273524024 (273.5 MB)  TX bytes:251064 (251.0 KB)
          Interrupt:16 Memory:fddc0000-fdde0000 

int-br-ex Link encap:Ethernet  HWaddr 3e:5b:ca:b3:a6:74  
          inet6 addr: fe80::3c5b:caff:feb3:a674/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:4379 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:542237 (542.2 KB)  TX bytes:648 (648.0 B)

juju-br0  Link encap:Ethernet  HWaddr 74:d4:35:fa:0f:37  
          inet addr:10.0.0.12  Bcast:10.0.0.255  Mask:255.255.255.0
          inet6 addr: fe80::76d4:35ff:fefa:f37/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:3035745 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3371989 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:469223504 (469.2 MB)  TX bytes:389534941 (389.5 MB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:498 errors:0 dropped:0 overruns:0 frame:0
          TX packets:498 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:45274 (45.2 KB)  TX bytes:45274 (45.2 KB)

phy-br-ex Link encap:Ethernet  HWaddr 3e:ed:97:c5:6b:3a  
          inet6 addr: fe80::3ced:97ff:fec5:6b3a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4379 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:648 (648.0 B)  TX bytes:542237 (542.2 KB)

tapd3a7661a-6c Link encap:Ethernet  HWaddr 0a:1b:ad:db:24:f3  
          inet6 addr: fe80::81b:adff:fedb:24f3/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:112 errors:0 dropped:0 overruns:0 frame:0
          TX packets:129 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:10087 (10.0 KB)  TX bytes:11366 (11.3 KB)

tapf36bfad8-fe Link encap:Ethernet  HWaddr 66:4c:65:18:44:62  
          inet6 addr: fe80::644c:65ff:fe18:4462/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:131 errors:0 dropped:0 overruns:0 frame:0
          TX packets:212 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:16576 (16.5 KB)  TX bytes:19812 (19.8 KB)

Thanks for your attention !

Three nodes setup with neutron cannot ping either VMs or External gateway

HI, I installed openstack Liberty using juju on Ubuntu 14.04 on three nodes using maas & juju.

The maas controller machine has two interfaces, eth0 with ip 10.0.0.1 and eth1 192.168.29 as gateway to the external network. network.

Nodes used for openstack have one physical interface named eth0 connected to the maas managed network 10.0.0.1, except the neutron-gateway node that has an eth1 interface connected to the external network 192.168.0.

Here is my juju deployment config :

keystone:
  admin-password: openstack
  debug: 'true'
  log-level: DEBUG
nova-cloud-controller:
  network-manager: Neutron
nova-compute:
  flat-interface: 'eth0'
  enable-live-migration: 'True'
  migration-auth-type: "none"
  virt-type: kvm
  enable-resize: 'True'
neutron-gateway:
  ext-port: 'eth1'
  bridge-mappings: 'external:br-ex'
  instance-mtu: 1400
glance:
cinder:
openstack-dashboard:
  webroot: "/"

The deployment script

#!/bin/bash
juju deploy --to 0 juju-gui
juju deploy --to lxc:0 mysql
juju deploy --config config.yaml --to lxc:0 keystone
juju deploy --config config.yaml --to lxc:0 nova-cloud-controller
juju deploy --config config.yaml --to lxc:0 glance
juju deploy --to lxc:0 rabbitmq-server
juju deploy --config config.yaml --to lxc:0 openstack-dashboard
juju deploy --config config.yaml --to lxc:0 cinder
juju deploy --config config.yaml nova-compute --constraints "tags=compute"
juju deploy --config config.yaml neutron-gateway --constraints "tags=neutron"
juju add-relation mysql keystone
juju add-relation nova-cloud-controller mysql
juju add-relation nova-cloud-controller rabbitmq-server
juju add-relation nova-cloud-controller glance
juju add-relation nova-cloud-controller keystone
juju add-relation nova-compute nova-cloud-controller
juju add-relation nova-compute mysql
juju add-relation nova-compute rabbitmq-server:amqp
juju add-relation nova-compute glance
juju add-relation glance mysql
juju add-relation glance keystone
juju add-relation glance cinder
juju add-relation mysql cinder
juju add-relation cinder rabbitmq-server
juju add-relation cinder nova-cloud-controller
juju add-relation cinder keystone
juju add-relation openstack-dashboard keystone
juju add-relation neutron-gateway mysql
juju add-relation neutron-gateway:amqp rabbitmq-server:amqp
juju add-relation neutron-gateway nova-cloud-controller
juju set keystone admin-password="openstack"
# display status
juju stat --format=tabular
# Download image
wget http://cloud-images.ubuntu.com/trusty/current/trusty-server-cloudimg-amd64-disk1.img
glance add name="Trusty x86_64" is_public=true container_format=ovf disk_format=qcow2 < trusty-server-cloudimg-amd64-disk1.img
# set default security group
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
# add keypair
nova keypair-add --pub-key ~/.ssh/id_rsa.pub id_rsa
nova image-list

After initial setup, I followed the documentation up to "Verify connectivity" and setup neutron networks as follows :

#create external network and subnet
neutron net-create ext-net --shared --router:external=True
neutron subnet-create ext-net --name ext-subnet \
  --allocation-pool start=192.168.1.230,end=192.168.1.250 \
  --disable-dhcp --gateway 192.168.1.1 192.168.1.0/24

#create tenant network and subnet
neutron net-create demo-net
neutron subnet-create demo-net --name demo-subnet \
  --gateway 10.0.1.1 10.0.1.1/24

# add router
neutron router-create demo-router
neutron router-interface-add demo-router demo-subnet
neutron router-gateway-set demo-router ext-net

The neutron-gateway node has two physical network interfaces :

  • eth0 connected to the juju 10.0.0.0/24 network
  • eth1 connected to the external 192.168.1.0/24 network

When created, the router should get the first address in the specified pool, 192.168.1.230 and says so but since this address is unreachable from set in the controller and from the external network./etc/network/interfaces, I am able to ping it.

ping 192.168.1.230
PING 192.168.1.230 (192.168.1.230) 56(84) bytes of data.
From 192.168.1.29 64 bytes from 192.168.1.230: icmp_seq=1 Destination Host Unreachable
From 192.168.1.29 ttl=64 time=0.051 ms
64 bytes from 192.168.1.230: icmp_seq=2 Destination Host Unreachable
From 192.168.1.29 ttl=64 time=0.024 ms
64 bytes from 192.168.1.230: icmp_seq=3 Destination Host Unreachable
ttl=64 time=0.040 ms

I am also able to ping the interfac address specified for the VM network

PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.158 ms
64 bytes from 10.0.1.1: icmp_seq=2 ttl=64 time=0.227 ms
64 bytes from 10.0.1.1: icmp_seq=3 ttl=64 time=0.197 ms
64 bytes from 10.0.1.1: icmp_seq=4 ttl=64 time=0.193 ms

I am able to create a VM, but I cannot ping it, both on the 10.0.1.0 and the 192.168.1.0 networks.

nova list
+--------------------------------------+------+--------+------------+-------------+------------------------------------------+
| ID                                   | Name | Status | Task State | Power State | Networks                                 |
+--------------------------------------+------+--------+------------+-------------+------------------------------------------+
| 0ef5a2bd-cbb6-44c9-b09d-738b553a8367 | test | ACTIVE | -          | Running     | ext-net=192.168.1.234; demo-net=10.0.1.6 |
+--------------------------------------+------+--------+------------+-------------+------------------------------------------+

One more thing is router gateway port has a DOWN status. There must me something missing but I don't get it.

Here is some details that may help, On neutron-gateway node

cat /etc/network/interfaces
auto lo
iface lo inet loopback
    dns-nameservers 192.168.1.29
    dns-search maas

iface eth0 inet manual

auto juju-br0
iface juju-br0 inet static
    bridge_ports eth0
    gateway 10.0.0.1
    address 10.0.0.12/24
    mtu 1500

auto br-ex
iface br-ex inet static
  address 192.168.1.230
  netmask 255.255.255.0
  gateway 192.168.1.1
  dns-nameservers 8.8.8.8

auto eth1
iface eth1 inet manual
#    up ip link set dev $IFACE up
    mtu 1500
#    down ip link set dev $IFACE down

,

sudo ovs-vsctl show
sudo: unable to resolve host tragic-pickle
fc996723-61c1-4fd0-a38f-1dd0df07eeb8
    Bridge br-ex
        Port br-ex
            Interface br-ex
                type: internal
        Port phy-br-ex
            Interface phy-br-ex
        Port "eth1"
            Interface "eth1"
        Port "tap1c34ee72-81"
            Interface "tap1c34ee72-81"
    Bridge br-int
        fail_mode: secure
        Port "tap5f2f6ea9-a8"
            tag: 4095
            Interface "tap5f2f6ea9-a8"
        Port br-int
            Interface br-int
                type: internal
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port "tap3a6679cd-f3"
            tag: 1
            Interface "tap3a6679cd-f3"
        Port int-br-ex
            Interface int-br-ex
    Bridge br-tun
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port br-tun
            Interface br-tun
                type: internal
        Port "gre-0a00000b"
            Interface "gre-0a00000b"
                type: gre
                options: {in_key=flow, local_ip="10.0.0.12", out_key=flow, remote_ip="10.0.0.11"}
    Bridge br-ex
        Port br-ex
patch-int
            Interface br-ex
patch-int
                type: patch
                options: {peer=patch-tun}
        Port br-tun
            Interface br-tun
                type: internal
        Port "tapc4b60553-66"
            Interface "tapc4b60553-66"
        Port "eth1"
            Interface "eth1"
        Port phy-br-ex
            Interface phy-br-ex
    Bridge br-int
        fail_mode: secure
        Port br-int
            Interface br-int
                type: internal
        Port "tapd3a7661a-6c"
            tag: 1
            Interface "tapd3a7661a-6c"
        Port int-br-ex
            Interface int-br-ex
        Port "tapf36bfad8-fe"
            tag: 1
            Interface "tapf36bfad8-fe"
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
    ovs_version: "2.0.2"

,

ifconfig
br-ex     Link encap:Ethernet  HWaddr 68:05:ca:3b:e6:13  
          inet addr:192.168.1.230  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::443c:21ff:fe55:7771/64 fe80::5848:89ff:fee7:35dd/64 Scope:Link
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:2011418 packets:193344 errors:0 dropped:16 dropped:0 overruns:0 frame:0
          TX packets:2735 packets:301 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:248614834 (248.6 bytes:25978737 (25.9 MB)  TX bytes:235306 (235.3 bytes:32000 (32.0 KB)

br-int    Link encap:Ethernet  HWaddr 62:6f:d2:ca:3b:43  
          inet6 addr: fe80::f0b2:73ff:fe37:180a/64 fe80::281b:41ff:fe46:3142/64 Scope:Link
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:44 packets:24 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:3970 (3.9 bytes:1924 (1.9 KB)  TX bytes:648 (648.0 B)

br-tun    Link encap:Ethernet  HWaddr ba:5a:ae:81:76:4d de:e0:7e:d9:2e:46  
          inet6 addr: fe80::f8e6:5eff:fed0:6199/64 fe80::b41f:94ff:fea4:ff87/64 Scope:Link
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:648 (648.0 B)

eth0      Link encap:Ethernet  HWaddr 74:d4:35:fa:0f:37  
          inet6 addr: fe80::76d4:35ff:fefa:f37/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:3048127 packets:253969 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3374042 packets:285181 errors:0 dropped:0 overruns:0 carrier:1
          collisions:0 txqueuelen:1000 
          RX bytes:529788654 (529.7 bytes:56960015 (56.9 MB)  TX bytes:390302367 (390.3 bytes:66650148 (66.6 MB)

eth1      Link encap:Ethernet  HWaddr 68:05:ca:3b:e6:13  
          inet6 addr: fe80::6a05:caff:fe3b:e613/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2274463 packets:216435 errors:0 dropped:1 dropped:0 overruns:0 frame:0
          TX packets:2790 packets:358 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:273524024 (273.5 bytes:28250255 (28.2 MB)  TX bytes:251064 (251.0 bytes:38270 (38.2 KB)
          Interrupt:16 Memory:fddc0000-fdde0000 

int-br-ex Link encap:Ethernet  HWaddr 3e:5b:ca:b3:a6:74 ea:28:33:73:cd:27  
          inet6 addr: fe80::3c5b:caff:feb3:a674/64 fe80::e828:33ff:fe73:cd27/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:4379 packets:192051 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 packets:31 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:542237 (542.2 KB) bytes:23390016 (23.3 MB)  TX bytes:648 (648.0 B)
bytes:2482 (2.4 KB)

juju-br0  Link encap:Ethernet  HWaddr 74:d4:35:fa:0f:37  
          inet addr:10.0.0.12  Bcast:10.0.0.255  Mask:255.255.255.0
          inet6 addr: fe80::76d4:35ff:fefa:f37/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:3035745 packets:253955 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3371989 packets:285175 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:469223504 (469.2 bytes:53403609 (53.4 MB)  TX bytes:389534941 (389.5 bytes:66642750 (66.6 MB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:498 packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:498 packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:45274 (45.2 KB) bytes:0 (0.0 B)  TX bytes:45274 (45.2 KB)
bytes:0 (0.0 B)

phy-br-ex Link encap:Ethernet  HWaddr 3e:ed:97:c5:6b:3a 82:73:1e:a1:49:de  
          inet6 addr: fe80::3ced:97ff:fec5:6b3a/64 fe80::8073:1eff:fea1:49de/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:31 errors:0 dropped:0 overruns:0 frame:0
          TX packets:192051 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:2482 (2.4 KB)  TX bytes:23390016 (23.3 MB)

tap1c34ee72-81 Link encap:Ethernet  HWaddr 1e:35:21:f9:ba:8b  
          inet6 addr: fe80::1c35:21ff:fef9:ba8b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:50 errors:0 dropped:0 overruns:0 frame:0
          TX packets:192065 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:4184 (4.1 KB)  TX bytes:23391103 (23.3 MB)

tap3a6679cd-f3 Link encap:Ethernet  HWaddr de:15:c1:e0:4a:de  
          inet6 addr: fe80::dc15:c1ff:fee0:4ade/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4379 packets:18 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:648 (648.0 B)  TX bytes:542237 (542.2 bytes:1428 (1.4 KB)

tapd3a7661a-6c tap5f2f6ea9-a8 Link encap:Ethernet  HWaddr 0a:1b:ad:db:24:f3 76:9d:30:1a:f0:80  
          inet6 addr: fe80::81b:adff:fedb:24f3/64 fe80::749d:30ff:fe1a:f080/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:112 packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:129 packets:9 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:10087 (10.0 KB) bytes:648 (648.0 B)  TX bytes:11366 (11.3 KB)

tapf36bfad8-fe Link encap:Ethernet  HWaddr 66:4c:65:18:44:62  
          inet6 addr: fe80::644c:65ff:fe18:4462/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:131 errors:0 dropped:0 overruns:0 frame:0
          TX packets:212 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:16576 (16.5 KB)  TX bytes:19812 (19.8 KB)
bytes:726 (726.0 B)

Thanks for your attention !

Three nodes setup with neutron neutron, cannot ping VMs either VMs internal or External gatewayIP

HI, I installed openstack Liberty using juju on Ubuntu 14.04 on three nodes using maas & juju.

The maas controller machine has two interfaces, eth0 with ip 10.0.0.1 and eth1 192.168.29 as gateway to the external network.

Nodes used for openstack have one physical interface named eth0 connected to the maas managed network 10.0.0.1, except the neutron-gateway node that has an eth1 interface connected to the external network 192.168.0.

Here is my juju deployment config :

keystone:
  admin-password: openstack
  debug: 'true'
  log-level: DEBUG
nova-cloud-controller:
  network-manager: Neutron
nova-compute:
  flat-interface: 'eth0'
  enable-live-migration: 'True'
  migration-auth-type: "none"
  virt-type: kvm
  enable-resize: 'True'
neutron-gateway:
  ext-port: 'eth1'
  bridge-mappings: 'external:br-ex'
  instance-mtu: 1400
glance:
cinder:
openstack-dashboard:
  webroot: "/"

The deployment script

#!/bin/bash
juju deploy --to 0 juju-gui
juju deploy --to lxc:0 mysql
juju deploy --config config.yaml --to lxc:0 keystone
juju deploy --config config.yaml --to lxc:0 nova-cloud-controller
juju deploy --config config.yaml --to lxc:0 glance
juju deploy --to lxc:0 rabbitmq-server
juju deploy --config config.yaml --to lxc:0 openstack-dashboard
juju deploy --config config.yaml --to lxc:0 cinder
juju deploy --config config.yaml nova-compute --constraints "tags=compute"
juju deploy --config config.yaml neutron-gateway --constraints "tags=neutron"
juju add-relation mysql keystone
juju add-relation nova-cloud-controller mysql
juju add-relation nova-cloud-controller rabbitmq-server
juju add-relation nova-cloud-controller glance
juju add-relation nova-cloud-controller keystone
juju add-relation nova-compute nova-cloud-controller
juju add-relation nova-compute mysql
juju add-relation nova-compute rabbitmq-server:amqp
juju add-relation nova-compute glance
juju add-relation glance mysql
juju add-relation glance keystone
juju add-relation glance cinder
juju add-relation mysql cinder
juju add-relation cinder rabbitmq-server
juju add-relation cinder nova-cloud-controller
juju add-relation cinder keystone
juju add-relation openstack-dashboard keystone
juju add-relation neutron-gateway mysql
juju add-relation neutron-gateway:amqp rabbitmq-server:amqp
juju add-relation neutron-gateway nova-cloud-controller
juju set keystone admin-password="openstack"
# display status
juju stat --format=tabular
# Download image
wget http://cloud-images.ubuntu.com/trusty/current/trusty-server-cloudimg-amd64-disk1.img
glance add name="Trusty x86_64" is_public=true container_format=ovf disk_format=qcow2 < trusty-server-cloudimg-amd64-disk1.img
# set default security group
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
# add keypair
nova keypair-add --pub-key ~/.ssh/id_rsa.pub id_rsa
nova image-list

After initial setup, I followed the documentation up to "Verify connectivity" and setup neutron networks as follows :

#create external network and subnet
neutron net-create ext-net --shared --router:external=True
neutron subnet-create ext-net --name ext-subnet \
  --allocation-pool start=192.168.1.230,end=192.168.1.250 \
  --disable-dhcp --gateway 192.168.1.1 192.168.1.0/24

#create tenant network and subnet
neutron net-create demo-net
neutron subnet-create demo-net --name demo-subnet \
  --gateway 10.0.1.1 10.0.1.1/24

# add router
neutron router-create demo-router
neutron router-interface-add demo-router demo-subnet
neutron router-gateway-set demo-router ext-net

The neutron-gateway node has two physical network interfaces :

  • eth0 connected to the juju 10.0.0.0/24 network
  • eth1 connected to the external 192.168.1.0/24 network

When created, the router should get the first address in the specified pool, 192.168.1.230 and since this address is set in the /etc/network/interfaces, I am able to ping it.

PING 192.168.1.230 (192.168.1.230) 56(84) bytes of data.
64 bytes from 192.168.1.230: icmp_seq=1 ttl=64 time=0.051 ms
64 bytes from 192.168.1.230: icmp_seq=2 ttl=64 time=0.024 ms
64 bytes from 192.168.1.230: icmp_seq=3 ttl=64 time=0.040 ms

I am also able to ping the interfac address specified for the VM network

PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.158 ms
64 bytes from 10.0.1.1: icmp_seq=2 ttl=64 time=0.227 ms
64 bytes from 10.0.1.1: icmp_seq=3 ttl=64 time=0.197 ms
64 bytes from 10.0.1.1: icmp_seq=4 ttl=64 time=0.193 ms

I am able to create a VM, but I cannot ping it, both on the 10.0.1.0 and the 192.168.1.0 networks.

nova list
+--------------------------------------+------+--------+------------+-------------+------------------------------------------+
| ID                                   | Name | Status | Task State | Power State | Networks                                 |
+--------------------------------------+------+--------+------------+-------------+------------------------------------------+
| 0ef5a2bd-cbb6-44c9-b09d-738b553a8367 | test | ACTIVE | -          | Running     | ext-net=192.168.1.234; demo-net=10.0.1.6 |
+--------------------------------------+------+--------+------------+-------------+------------------------------------------+

One more thing is router gateway port has a DOWN status. There must me something missing but I don't get it.

Here is some details that may help, On neutron-gateway node

ip netns list
qrouter-7ed37eac-f293-43a9-b9c4-1398f1fc08ba
qdhcp-72be7e29-3673-4133-9601-50a0ac061317

,

cat /etc/network/interfaces
auto lo
iface lo inet loopback
    dns-nameservers 192.168.1.29
    dns-search maas

iface eth0 inet manual

auto juju-br0
iface juju-br0 inet static
    bridge_ports eth0
    gateway 10.0.0.1
    address 10.0.0.12/24
    mtu 1500

auto br-ex
iface br-ex inet static
  address 192.168.1.230
  netmask 255.255.255.0
  gateway 192.168.1.1
  dns-nameservers 8.8.8.8

auto eth1
iface eth1 inet manual
#    up ip link set dev $IFACE up
    mtu 1500
#    down ip link set dev $IFACE down

,

sudo ovs-vsctl show
fc996723-61c1-4fd0-a38f-1dd0df07eeb8
    Bridge br-ex
        Port br-ex
            Interface br-ex
                type: internal
        Port phy-br-ex
            Interface phy-br-ex
        Port "eth1"
            Interface "eth1"
        Port "tap1c34ee72-81"
            Interface "tap1c34ee72-81"
    Bridge br-int
        fail_mode: secure
        Port "tap5f2f6ea9-a8"
            tag: 4095
            Interface "tap5f2f6ea9-a8"
        Port br-int
            Interface br-int
                type: internal
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port "tap3a6679cd-f3"
            tag: 1
            Interface "tap3a6679cd-f3"
        Port int-br-ex
            Interface int-br-ex
    Bridge br-tun
        Port "gre-0a00000b"
            Interface "gre-0a00000b"
                type: gre
                options: {in_key=flow, local_ip="10.0.0.12", out_key=flow, remote_ip="10.0.0.11"}
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port br-tun
            Interface br-tun
                type: internal
    ovs_version: "2.0.2"

,

ifconfig
br-ex     Link encap:Ethernet  HWaddr 68:05:ca:3b:e6:13  
          inet addr:192.168.1.230  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::5848:89ff:fee7:35dd/64 Scope:Link
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:193344 errors:0 dropped:0 overruns:0 frame:0
          TX packets:301 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:25978737 (25.9 MB)  TX bytes:32000 (32.0 KB)

br-int    Link encap:Ethernet  HWaddr 62:6f:d2:ca:3b:43  
          inet6 addr: fe80::281b:41ff:fe46:3142/64 Scope:Link
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:24 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:1924 (1.9 KB)  TX bytes:648 (648.0 B)

br-tun    Link encap:Ethernet  HWaddr de:e0:7e:d9:2e:46  
          inet6 addr: fe80::b41f:94ff:fea4:ff87/64 Scope:Link
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:648 (648.0 B)

eth0      Link encap:Ethernet  HWaddr 74:d4:35:fa:0f:37  
          inet6 addr: fe80::76d4:35ff:fefa:f37/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:253969 errors:0 dropped:0 overruns:0 frame:0
          TX packets:285181 errors:0 dropped:0 overruns:0 carrier:1
          collisions:0 txqueuelen:1000 
          RX bytes:56960015 (56.9 MB)  TX bytes:66650148 (66.6 MB)

eth1      Link encap:Ethernet  HWaddr 68:05:ca:3b:e6:13  
          inet6 addr: fe80::6a05:caff:fe3b:e613/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:216435 errors:0 dropped:0 overruns:0 frame:0
          TX packets:358 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:28250255 (28.2 MB)  TX bytes:38270 (38.2 KB)
          Interrupt:16 Memory:fddc0000-fdde0000 

int-br-ex Link encap:Ethernet  HWaddr ea:28:33:73:cd:27  
          inet6 addr: fe80::e828:33ff:fe73:cd27/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:192051 errors:0 dropped:0 overruns:0 frame:0
          TX packets:31 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:23390016 (23.3 MB)  TX bytes:2482 (2.4 KB)

juju-br0  Link encap:Ethernet  HWaddr 74:d4:35:fa:0f:37  
          inet addr:10.0.0.12  Bcast:10.0.0.255  Mask:255.255.255.0
          inet6 addr: fe80::76d4:35ff:fefa:f37/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:253955 errors:0 dropped:0 overruns:0 frame:0
          TX packets:285175 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:53403609 (53.4 MB)  TX bytes:66642750 (66.6 MB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

phy-br-ex Link encap:Ethernet  HWaddr 82:73:1e:a1:49:de  
          inet6 addr: fe80::8073:1eff:fea1:49de/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:31 errors:0 dropped:0 overruns:0 frame:0
          TX packets:192051 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:2482 (2.4 KB)  TX bytes:23390016 (23.3 MB)

tap1c34ee72-81 Link encap:Ethernet  HWaddr 1e:35:21:f9:ba:8b  
          inet6 addr: fe80::1c35:21ff:fef9:ba8b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:50 errors:0 dropped:0 overruns:0 frame:0
          TX packets:192065 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:4184 (4.1 KB)  TX bytes:23391103 (23.3 MB)

tap3a6679cd-f3 Link encap:Ethernet  HWaddr de:15:c1:e0:4a:de  
          inet6 addr: fe80::dc15:c1ff:fee0:4ade/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:18 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:648 (648.0 B)  TX bytes:1428 (1.4 KB)

tap5f2f6ea9-a8 Link encap:Ethernet  HWaddr 76:9d:30:1a:f0:80  
          inet6 addr: fe80::749d:30ff:fe1a:f080/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:9 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:648 (648.0 B)  TX bytes:726 (726.0 B)

Thanks for your attention !

Three nodes setup with neutron, neutron cannot ping either VMs either internal or External IPgateway

HI, I installed openstack Liberty using juju on Ubuntu 14.04 on three nodes using maas & juju.

The maas controller machine has two interfaces, eth0 with ip 10.0.0.1 and eth1 192.168.29 as gateway to the external network.

Nodes used for openstack have one physical interface named eth0 connected to the maas managed network 10.0.0.1, except the neutron-gateway node that has an eth1 interface connected to the external network 192.168.0.

Here is my juju deployment config :

keystone:
  admin-password: openstack
  debug: 'true'
  log-level: DEBUG
nova-cloud-controller:
  network-manager: Neutron
nova-compute:
  flat-interface: 'eth0'
  enable-live-migration: 'True'
  migration-auth-type: "none"
  virt-type: kvm
  enable-resize: 'True'
neutron-gateway:
  ext-port: 'eth1'
  bridge-mappings: 'external:br-ex'
  instance-mtu: 1400
glance:
cinder:
openstack-dashboard:
  webroot: "/"

The deployment script

#!/bin/bash
juju deploy --to 0 juju-gui
juju deploy --to lxc:0 mysql
juju deploy --config config.yaml --to lxc:0 keystone
juju deploy --config config.yaml --to lxc:0 nova-cloud-controller
juju deploy --config config.yaml --to lxc:0 glance
juju deploy --to lxc:0 rabbitmq-server
juju deploy --config config.yaml --to lxc:0 openstack-dashboard
juju deploy --config config.yaml --to lxc:0 cinder
juju deploy --config config.yaml nova-compute --constraints "tags=compute"
juju deploy --config config.yaml neutron-gateway --constraints "tags=neutron"
juju add-relation mysql keystone
juju add-relation nova-cloud-controller mysql
juju add-relation nova-cloud-controller rabbitmq-server
juju add-relation nova-cloud-controller glance
juju add-relation nova-cloud-controller keystone
juju add-relation nova-compute nova-cloud-controller
juju add-relation nova-compute mysql
juju add-relation nova-compute rabbitmq-server:amqp
juju add-relation nova-compute glance
juju add-relation glance mysql
juju add-relation glance keystone
juju add-relation glance cinder
juju add-relation mysql cinder
juju add-relation cinder rabbitmq-server
juju add-relation cinder nova-cloud-controller
juju add-relation cinder keystone
juju add-relation openstack-dashboard keystone
juju add-relation neutron-gateway mysql
juju add-relation neutron-gateway:amqp rabbitmq-server:amqp
juju add-relation neutron-gateway nova-cloud-controller
juju set keystone admin-password="openstack"
# display status
juju stat --format=tabular
# Download image
wget http://cloud-images.ubuntu.com/trusty/current/trusty-server-cloudimg-amd64-disk1.img
glance add name="Trusty x86_64" is_public=true container_format=ovf disk_format=qcow2 < trusty-server-cloudimg-amd64-disk1.img
# set default security group
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
# add keypair
nova keypair-add --pub-key ~/.ssh/id_rsa.pub id_rsa
nova image-list

After initial setup, I followed the documentation up to "Verify connectivity" and setup neutron networks as follows :

#create external network and subnet
neutron net-create ext-net --shared --router:external=True
neutron subnet-create ext-net --name ext-subnet \
  --allocation-pool start=192.168.1.230,end=192.168.1.250 \
  --disable-dhcp --gateway 192.168.1.1 192.168.1.0/24

#create tenant network and subnet
neutron net-create demo-net
neutron subnet-create demo-net --name demo-subnet \
  --gateway 10.0.1.1 10.0.1.1/24

# add router
neutron router-create demo-router
neutron router-interface-add demo-router demo-subnet
neutron router-gateway-set demo-router ext-net

The neutron-gateway node has two physical network interfaces :

  • eth0 connected to the juju 10.0.0.0/24 network
  • eth1 connected to the external 192.168.1.0/24 network

When created, the router should get the first address in the specified pool, 192.168.1.230 and since this address is set in the /etc/network/interfaces, I am able to ping it.

PING 192.168.1.230 (192.168.1.230) 56(84) bytes of data.
64 bytes from 192.168.1.230: icmp_seq=1 ttl=64 time=0.051 ms
64 bytes from 192.168.1.230: icmp_seq=2 ttl=64 time=0.024 ms
64 bytes from 192.168.1.230: icmp_seq=3 ttl=64 time=0.040 ms

I am also able to ping the interfac address specified for the VM network

PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.158 ms
64 bytes from 10.0.1.1: icmp_seq=2 ttl=64 time=0.227 ms
64 bytes from 10.0.1.1: icmp_seq=3 ttl=64 time=0.197 ms
64 bytes from 10.0.1.1: icmp_seq=4 ttl=64 time=0.193 ms

I am able to create a VM, but I cannot ping it, both on the 10.0.1.0 and the 192.168.1.0 networks.

nova list
+--------------------------------------+------+--------+------------+-------------+------------------------------------------+
| ID                                   | Name | Status | Task State | Power State | Networks                                 |
+--------------------------------------+------+--------+------------+-------------+------------------------------------------+
| 0ef5a2bd-cbb6-44c9-b09d-738b553a8367 | test | ACTIVE | -          | Running     | ext-net=192.168.1.234; demo-net=10.0.1.6 |
+--------------------------------------+------+--------+------------+-------------+------------------------------------------+

One more thing is router gateway port has a DOWN status. There must me something missing but I don't get it.

Here is some details that may help, On neutron-gateway node

ip netns list
qrouter-7ed37eac-f293-43a9-b9c4-1398f1fc08ba
qdhcp-72be7e29-3673-4133-9601-50a0ac061317

,

sudo ip netns exec qrouter-7ed37eac-f293-43a9-b9c4-1398f1fc08ba iptables -S -t nat
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-N neutron-postrouting-bottom
-N neutron-vpn-agen-OUTPUT
-N neutron-vpn-agen-POSTROUTING
-N neutron-vpn-agen-PREROUTING
-N neutron-vpn-agen-float-snat
-N neutron-vpn-agen-snat
-A PREROUTING -j neutron-vpn-agen-PREROUTING
-A OUTPUT -j neutron-vpn-agen-OUTPUT
-A POSTROUTING -j neutron-vpn-agen-POSTROUTING
-A POSTROUTING -j neutron-postrouting-bottom
-A neutron-postrouting-bottom -j neutron-vpn-agen-snat
-A neutron-vpn-agen-POSTROUTING ! -i qg-1c34ee72-81 ! -o qg-1c34ee72-81 -m conntrack ! --ctstate DNAT -j ACCEPT
-A neutron-vpn-agen-PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 9697
-A neutron-vpn-agen-snat -j neutron-vpn-agen-float-snat
-A neutron-vpn-agen-snat -s 10.0.1.0/24 -j SNAT --to-source 192.168.1.230

,

cat /etc/network/interfaces
auto lo
iface lo inet loopback
    dns-nameservers 192.168.1.29
    dns-search maas

iface eth0 inet manual

auto juju-br0
iface juju-br0 inet static
    bridge_ports eth0
    gateway 10.0.0.1
    address 10.0.0.12/24
    mtu 1500

auto br-ex
iface br-ex inet static
  address 192.168.1.230
  netmask 255.255.255.0
  gateway 192.168.1.1
  dns-nameservers 8.8.8.8

auto eth1
iface eth1 inet manual
#    up ip link set dev $IFACE up
    mtu 1500
#    down ip link set dev $IFACE down

,

sudo ovs-vsctl show
fc996723-61c1-4fd0-a38f-1dd0df07eeb8
    Bridge br-ex
        Port br-ex
            Interface br-ex
                type: internal
        Port phy-br-ex
            Interface phy-br-ex
        Port "eth1"
            Interface "eth1"
        Port "tap1c34ee72-81"
            Interface "tap1c34ee72-81"
    Bridge br-int
        fail_mode: secure
        Port "tap5f2f6ea9-a8"
            tag: 4095
            Interface "tap5f2f6ea9-a8"
        Port br-int
            Interface br-int
                type: internal
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port "tap3a6679cd-f3"
            tag: 1
            Interface "tap3a6679cd-f3"
        Port int-br-ex
            Interface int-br-ex
    Bridge br-tun
        Port "gre-0a00000b"
            Interface "gre-0a00000b"
                type: gre
                options: {in_key=flow, local_ip="10.0.0.12", out_key=flow, remote_ip="10.0.0.11"}
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port br-tun
            Interface br-tun
                type: internal
    ovs_version: "2.0.2"

,

ifconfig
br-ex     Link encap:Ethernet  HWaddr 68:05:ca:3b:e6:13  
          inet addr:192.168.1.230  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::5848:89ff:fee7:35dd/64 Scope:Link
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:193344 errors:0 dropped:0 overruns:0 frame:0
          TX packets:301 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:25978737 (25.9 MB)  TX bytes:32000 (32.0 KB)

br-int    Link encap:Ethernet  HWaddr 62:6f:d2:ca:3b:43  
          inet6 addr: fe80::281b:41ff:fe46:3142/64 Scope:Link
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:24 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:1924 (1.9 KB)  TX bytes:648 (648.0 B)

br-tun    Link encap:Ethernet  HWaddr de:e0:7e:d9:2e:46  
          inet6 addr: fe80::b41f:94ff:fea4:ff87/64 Scope:Link
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:648 (648.0 B)

eth0      Link encap:Ethernet  HWaddr 74:d4:35:fa:0f:37  
          inet6 addr: fe80::76d4:35ff:fefa:f37/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:253969 errors:0 dropped:0 overruns:0 frame:0
          TX packets:285181 errors:0 dropped:0 overruns:0 carrier:1
          collisions:0 txqueuelen:1000 
          RX bytes:56960015 (56.9 MB)  TX bytes:66650148 (66.6 MB)

eth1      Link encap:Ethernet  HWaddr 68:05:ca:3b:e6:13  
          inet6 addr: fe80::6a05:caff:fe3b:e613/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:216435 errors:0 dropped:0 overruns:0 frame:0
          TX packets:358 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:28250255 (28.2 MB)  TX bytes:38270 (38.2 KB)
          Interrupt:16 Memory:fddc0000-fdde0000 

int-br-ex Link encap:Ethernet  HWaddr ea:28:33:73:cd:27  
          inet6 addr: fe80::e828:33ff:fe73:cd27/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:192051 errors:0 dropped:0 overruns:0 frame:0
          TX packets:31 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:23390016 (23.3 MB)  TX bytes:2482 (2.4 KB)

juju-br0  Link encap:Ethernet  HWaddr 74:d4:35:fa:0f:37  
          inet addr:10.0.0.12  Bcast:10.0.0.255  Mask:255.255.255.0
          inet6 addr: fe80::76d4:35ff:fefa:f37/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:253955 errors:0 dropped:0 overruns:0 frame:0
          TX packets:285175 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:53403609 (53.4 MB)  TX bytes:66642750 (66.6 MB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

phy-br-ex Link encap:Ethernet  HWaddr 82:73:1e:a1:49:de  
          inet6 addr: fe80::8073:1eff:fea1:49de/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:31 errors:0 dropped:0 overruns:0 frame:0
          TX packets:192051 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:2482 (2.4 KB)  TX bytes:23390016 (23.3 MB)

tap1c34ee72-81 Link encap:Ethernet  HWaddr 1e:35:21:f9:ba:8b  
          inet6 addr: fe80::1c35:21ff:fef9:ba8b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:50 errors:0 dropped:0 overruns:0 frame:0
          TX packets:192065 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:4184 (4.1 KB)  TX bytes:23391103 (23.3 MB)

tap3a6679cd-f3 Link encap:Ethernet  HWaddr de:15:c1:e0:4a:de  
          inet6 addr: fe80::dc15:c1ff:fee0:4ade/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:18 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:648 (648.0 B)  TX bytes:1428 (1.4 KB)

tap5f2f6ea9-a8 Link encap:Ethernet  HWaddr 76:9d:30:1a:f0:80  
          inet6 addr: fe80::749d:30ff:fe1a:f080/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:9 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:648 (648.0 B)  TX bytes:726 (726.0 B)

Thanks for your attention !

Three nodes setup with neutron neutron, cannot ping VMs either VMs internal or External gatewayIP

HI, I installed openstack Liberty using juju on Ubuntu 14.04 on three nodes using maas & juju.

The maas controller machine has two interfaces, eth0 with ip 10.0.0.1 and eth1 192.168.29 as gateway to the external network.

Nodes used for openstack have one physical interface named eth0 connected to the maas managed network 10.0.0.1, except the neutron-gateway node that has an eth1 interface connected to the external network 192.168.0.

Here is my juju deployment config :

keystone:
  admin-password: openstack
  debug: 'true'
  log-level: DEBUG
nova-cloud-controller:
  network-manager: Neutron
nova-compute:
  flat-interface: 'eth0'
  enable-live-migration: 'True'
  migration-auth-type: "none"
  virt-type: kvm
  enable-resize: 'True'
neutron-gateway:
  ext-port: 'eth1'
  bridge-mappings: 'external:br-ex'
  instance-mtu: 1400
glance:
cinder:
openstack-dashboard:
  webroot: "/"

The deployment script

#!/bin/bash
juju deploy --to 0 juju-gui
juju deploy --to lxc:0 mysql
juju deploy --config config.yaml --to lxc:0 keystone
juju deploy --config config.yaml --to lxc:0 nova-cloud-controller
juju deploy --config config.yaml --to lxc:0 glance
juju deploy --to lxc:0 rabbitmq-server
juju deploy --config config.yaml --to lxc:0 openstack-dashboard
juju deploy --config config.yaml --to lxc:0 cinder
juju deploy --config config.yaml nova-compute --constraints "tags=compute"
juju deploy --config config.yaml neutron-gateway --constraints "tags=neutron"
juju add-relation mysql keystone
juju add-relation nova-cloud-controller mysql
juju add-relation nova-cloud-controller rabbitmq-server
juju add-relation nova-cloud-controller glance
juju add-relation nova-cloud-controller keystone
juju add-relation nova-compute nova-cloud-controller
juju add-relation nova-compute mysql
juju add-relation nova-compute rabbitmq-server:amqp
juju add-relation nova-compute glance
juju add-relation glance mysql
juju add-relation glance keystone
juju add-relation glance cinder
juju add-relation mysql cinder
juju add-relation cinder rabbitmq-server
juju add-relation cinder nova-cloud-controller
juju add-relation cinder keystone
juju add-relation openstack-dashboard keystone
juju add-relation neutron-gateway mysql
juju add-relation neutron-gateway:amqp rabbitmq-server:amqp
juju add-relation neutron-gateway nova-cloud-controller
juju set keystone admin-password="openstack"
# display status
juju stat --format=tabular
# Download image
wget http://cloud-images.ubuntu.com/trusty/current/trusty-server-cloudimg-amd64-disk1.img
glance add name="Trusty x86_64" is_public=true container_format=ovf disk_format=qcow2 < trusty-server-cloudimg-amd64-disk1.img
# set default security group
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
# add keypair
nova keypair-add --pub-key ~/.ssh/id_rsa.pub id_rsa
nova image-list

After initial setup, I followed the documentation up to "Verify connectivity" and setup neutron networks as follows :

#create external network and subnet
neutron net-create ext-net --shared --router:external=True
neutron subnet-create ext-net --name ext-subnet \
  --allocation-pool start=192.168.1.230,end=192.168.1.250 \
  --disable-dhcp --gateway 192.168.1.1 192.168.1.0/24

#create tenant network and subnet
neutron net-create demo-net
neutron subnet-create demo-net --name demo-subnet \
  --gateway 10.0.1.1 10.0.1.1/24

# add router
neutron router-create demo-router
neutron router-interface-add demo-router demo-subnet
neutron router-gateway-set demo-router ext-net

The neutron-gateway node has two physical network interfaces :

  • eth0 connected to the juju 10.0.0.0/24 network
  • eth1 connected to the external 192.168.1.0/24 network

When created, the router should get the first address in the specified pool, 192.168.1.230 and since this address is set in the /etc/network/interfaces, I am able to ping it.

PING 192.168.1.230 (192.168.1.230) 56(84) bytes of data.
64 bytes from 192.168.1.230: icmp_seq=1 ttl=64 time=0.051 ms
64 bytes from 192.168.1.230: icmp_seq=2 ttl=64 time=0.024 ms
64 bytes from 192.168.1.230: icmp_seq=3 ttl=64 time=0.040 ms

I am also able to ping the interfac address specified for the VM network

PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.158 ms
64 bytes from 10.0.1.1: icmp_seq=2 ttl=64 time=0.227 ms
64 bytes from 10.0.1.1: icmp_seq=3 ttl=64 time=0.197 ms
64 bytes from 10.0.1.1: icmp_seq=4 ttl=64 time=0.193 ms

I am able to create a VM, but I cannot ping it, both on the 10.0.1.0 and the 192.168.1.0 networks.

nova list
+--------------------------------------+------+--------+------------+-------------+------------------------------------------+
| ID                                   | Name | Status | Task State | Power State | Networks                                 |
+--------------------------------------+------+--------+------------+-------------+------------------------------------------+
| 0ef5a2bd-cbb6-44c9-b09d-738b553a8367 | test | ACTIVE | -          | Running     | ext-net=192.168.1.234; demo-net=10.0.1.6 |
+--------------------------------------+------+--------+------------+-------------+------------------------------------------+

One more thing is router gateway port has and loval DHCP port have a DOWN status. There must me something missing but I don't get it.

neutron port-show 1c34ee72-8163-4eca-8cea-65ccd4640cdc
+---------------------+--------------------------------------------------------------------------------------+
| Field               | Value                                                                                |
+---------------------+--------------------------------------------------------------------------------------+
| admin_state_up      | True                                                                                 |
| binding:host_id     | tragic-pickle                                                                        |
| binding:profile     | {}                                                                                   |
| binding:vif_details | {"port_filter": false, "ovs_hybrid_plug": false}                                     |
| binding:vif_type    | ovs                                                                                  |
| binding:vnic_type   | normal                                                                               |
| device_id           | 7ed37eac-f293-43a9-b9c4-1398f1fc08ba                                                 |
| device_owner        | network:router_gateway                                                               |
| extra_dhcp_opts     |                                                                                      |
| fixed_ips           | {"subnet_id": "7f915784-8615-4c39-beaf-7469514f3601", "ip_address": "192.168.1.230"} |
| id                  | 1c34ee72-8163-4eca-8cea-65ccd4640cdc                                                 |
| mac_address         | fa:16:3e:8a:95:f8                                                                    |
| name                |                                                                                      |
| network_id          | fd7742b1-fbca-4e07-a2cf-c4c0fa0cea36                                                 |
| status              | DOWN                                                                                 |
| tenant_id           |                                                                                      |
+---------------------+--------------------------------------------------------------------------------------+

,

neutron port-show 5f2f6ea9-a804-4503-b85f-e4cb4b04af42
+---------------------+---------------------------------------------------------------------------------+
| Field               | Value                                                                           |
+---------------------+---------------------------------------------------------------------------------+
| admin_state_up      | True                                                                            |
| binding:host_id     | tragic-pickle                                                                   |
| binding:profile     | {}                                                                              |
| binding:vif_details | {}                                                                              |
| binding:vif_type    | binding_failed                                                                  |
| binding:vnic_type   | normal                                                                          |
| device_id           | dhcp3d22cbc4-82c2-5ef9-83aa-80edfb56366c-72be7e29-3673-4133-9601-50a0ac061317   |
| device_owner        | network:dhcp                                                                    |
| extra_dhcp_opts     |                                                                                 |
| fixed_ips           | {"subnet_id": "161fba24-f081-4303-80e3-01a03293becb", "ip_address": "10.0.1.2"} |
| id                  | 5f2f6ea9-a804-4503-b85f-e4cb4b04af42                                            |
| mac_address         | fa:16:3e:29:20:5d                                                               |
| name                |                                                                                 |
| network_id          | 72be7e29-3673-4133-9601-50a0ac061317                                            |
| status              | DOWN                                                                            |
| tenant_id           | 7c0f373615b845b4829aac6910f079a8                                                |
+---------------------+---------------------------------------------------------------------------------+

Here is some details that may help, On neutron-gateway node

ip netns list
qrouter-7ed37eac-f293-43a9-b9c4-1398f1fc08ba
qdhcp-72be7e29-3673-4133-9601-50a0ac061317

,

sudo ip netns exec qrouter-7ed37eac-f293-43a9-b9c4-1398f1fc08ba iptables -S -t nat
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-N neutron-postrouting-bottom
-N neutron-vpn-agen-OUTPUT
-N neutron-vpn-agen-POSTROUTING
-N neutron-vpn-agen-PREROUTING
-N neutron-vpn-agen-float-snat
-N neutron-vpn-agen-snat
-A PREROUTING -j neutron-vpn-agen-PREROUTING
-A OUTPUT -j neutron-vpn-agen-OUTPUT
-A POSTROUTING -j neutron-vpn-agen-POSTROUTING
-A POSTROUTING -j neutron-postrouting-bottom
-A neutron-postrouting-bottom -j neutron-vpn-agen-snat
-A neutron-vpn-agen-POSTROUTING ! -i qg-1c34ee72-81 ! -o qg-1c34ee72-81 -m conntrack ! --ctstate DNAT -j ACCEPT
-A neutron-vpn-agen-PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 9697
-A neutron-vpn-agen-snat -j neutron-vpn-agen-float-snat
-A neutron-vpn-agen-snat -s 10.0.1.0/24 -j SNAT --to-source 192.168.1.230

,

cat /etc/network/interfaces
auto lo
iface lo inet loopback
    dns-nameservers 192.168.1.29
    dns-search maas

iface eth0 inet manual

auto juju-br0
iface juju-br0 inet static
    bridge_ports eth0
    gateway 10.0.0.1
    address 10.0.0.12/24
    mtu 1500

auto br-ex
iface br-ex inet static
  address 192.168.1.230
  netmask 255.255.255.0
  gateway 192.168.1.1
  dns-nameservers 8.8.8.8

auto eth1
iface eth1 inet manual
#    up ip link set dev $IFACE up
    mtu 1500
#    down ip link set dev $IFACE down

,

sudo ovs-vsctl show
fc996723-61c1-4fd0-a38f-1dd0df07eeb8
    Bridge br-ex
        Port br-ex
            Interface br-ex
                type: internal
        Port phy-br-ex
            Interface phy-br-ex
        Port "eth1"
            Interface "eth1"
        Port "tap1c34ee72-81"
            Interface "tap1c34ee72-81"
    Bridge br-int
        fail_mode: secure
        Port "tap5f2f6ea9-a8"
            tag: 4095
            Interface "tap5f2f6ea9-a8"
        Port br-int
            Interface br-int
                type: internal
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port "tap3a6679cd-f3"
            tag: 1
            Interface "tap3a6679cd-f3"
        Port int-br-ex
            Interface int-br-ex
    Bridge br-tun
        Port "gre-0a00000b"
            Interface "gre-0a00000b"
                type: gre
                options: {in_key=flow, local_ip="10.0.0.12", out_key=flow, remote_ip="10.0.0.11"}
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port br-tun
            Interface br-tun
                type: internal
    ovs_version: "2.0.2"

,

ifconfig
br-ex     Link encap:Ethernet  HWaddr 68:05:ca:3b:e6:13  
          inet addr:192.168.1.230  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::5848:89ff:fee7:35dd/64 Scope:Link
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:193344 errors:0 dropped:0 overruns:0 frame:0
          TX packets:301 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:25978737 (25.9 MB)  TX bytes:32000 (32.0 KB)

br-int    Link encap:Ethernet  HWaddr 62:6f:d2:ca:3b:43  
          inet6 addr: fe80::281b:41ff:fe46:3142/64 Scope:Link
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:24 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:1924 (1.9 KB)  TX bytes:648 (648.0 B)

br-tun    Link encap:Ethernet  HWaddr de:e0:7e:d9:2e:46  
          inet6 addr: fe80::b41f:94ff:fea4:ff87/64 Scope:Link
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:648 (648.0 B)

eth0      Link encap:Ethernet  HWaddr 74:d4:35:fa:0f:37  
          inet6 addr: fe80::76d4:35ff:fefa:f37/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:253969 errors:0 dropped:0 overruns:0 frame:0
          TX packets:285181 errors:0 dropped:0 overruns:0 carrier:1
          collisions:0 txqueuelen:1000 
          RX bytes:56960015 (56.9 MB)  TX bytes:66650148 (66.6 MB)

eth1      Link encap:Ethernet  HWaddr 68:05:ca:3b:e6:13  
          inet6 addr: fe80::6a05:caff:fe3b:e613/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:216435 errors:0 dropped:0 overruns:0 frame:0
          TX packets:358 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:28250255 (28.2 MB)  TX bytes:38270 (38.2 KB)
          Interrupt:16 Memory:fddc0000-fdde0000 

int-br-ex Link encap:Ethernet  HWaddr ea:28:33:73:cd:27  
          inet6 addr: fe80::e828:33ff:fe73:cd27/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:192051 errors:0 dropped:0 overruns:0 frame:0
          TX packets:31 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:23390016 (23.3 MB)  TX bytes:2482 (2.4 KB)

juju-br0  Link encap:Ethernet  HWaddr 74:d4:35:fa:0f:37  
          inet addr:10.0.0.12  Bcast:10.0.0.255  Mask:255.255.255.0
          inet6 addr: fe80::76d4:35ff:fefa:f37/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:253955 errors:0 dropped:0 overruns:0 frame:0
          TX packets:285175 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:53403609 (53.4 MB)  TX bytes:66642750 (66.6 MB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

phy-br-ex Link encap:Ethernet  HWaddr 82:73:1e:a1:49:de  
          inet6 addr: fe80::8073:1eff:fea1:49de/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:31 errors:0 dropped:0 overruns:0 frame:0
          TX packets:192051 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:2482 (2.4 KB)  TX bytes:23390016 (23.3 MB)

tap1c34ee72-81 Link encap:Ethernet  HWaddr 1e:35:21:f9:ba:8b  
          inet6 addr: fe80::1c35:21ff:fef9:ba8b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:50 errors:0 dropped:0 overruns:0 frame:0
          TX packets:192065 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:4184 (4.1 KB)  TX bytes:23391103 (23.3 MB)

tap3a6679cd-f3 Link encap:Ethernet  HWaddr de:15:c1:e0:4a:de  
          inet6 addr: fe80::dc15:c1ff:fee0:4ade/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:18 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:648 (648.0 B)  TX bytes:1428 (1.4 KB)

tap5f2f6ea9-a8 Link encap:Ethernet  HWaddr 76:9d:30:1a:f0:80  
          inet6 addr: fe80::749d:30ff:fe1a:f080/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:9 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:648 (648.0 B)  TX bytes:726 (726.0 B)

Thanks for your attention !

Three nodes setup with neutron, neutron cannot ping either VMs either internal or External IPgateway

HI, I installed openstack Liberty using juju on Ubuntu 14.04 on three nodes using maas & juju.

The maas controller machine has two interfaces, eth0 with ip 10.0.0.1 and eth1 192.168.29 as gateway to the external network.

Nodes used for openstack have one physical interface named eth0 connected to the maas managed network 10.0.0.1, except the neutron-gateway node that has an eth1 interface connected to the external network 192.168.0.

Here is my juju deployment config :

keystone:
  admin-password: openstack
  debug: 'true'
  log-level: DEBUG
nova-cloud-controller:
  network-manager: Neutron
nova-compute:
  flat-interface: 'eth0'
  enable-live-migration: 'True'
  migration-auth-type: "none"
  virt-type: kvm
  enable-resize: 'True'
neutron-gateway:
  ext-port: 'eth1'
  bridge-mappings: 'external:br-ex'
  instance-mtu: 1400
glance:
cinder:
openstack-dashboard:
  webroot: "/"

The deployment script

#!/bin/bash
juju deploy --to 0 juju-gui
juju deploy --to lxc:0 mysql
juju deploy --config config.yaml --to lxc:0 keystone
juju deploy --config config.yaml --to lxc:0 nova-cloud-controller
juju deploy --config config.yaml --to lxc:0 glance
juju deploy --to lxc:0 rabbitmq-server
juju deploy --config config.yaml --to lxc:0 openstack-dashboard
juju deploy --config config.yaml --to lxc:0 cinder
juju deploy --config config.yaml nova-compute --constraints "tags=compute"
juju deploy --config config.yaml neutron-gateway --constraints "tags=neutron"
juju add-relation mysql keystone
juju add-relation nova-cloud-controller mysql
juju add-relation nova-cloud-controller rabbitmq-server
juju add-relation nova-cloud-controller glance
juju add-relation nova-cloud-controller keystone
juju add-relation nova-compute nova-cloud-controller
juju add-relation nova-compute mysql
juju add-relation nova-compute rabbitmq-server:amqp
juju add-relation nova-compute glance
juju add-relation glance mysql
juju add-relation glance keystone
juju add-relation glance cinder
juju add-relation mysql cinder
juju add-relation cinder rabbitmq-server
juju add-relation cinder nova-cloud-controller
juju add-relation cinder keystone
juju add-relation openstack-dashboard keystone
juju add-relation neutron-gateway mysql
juju add-relation neutron-gateway:amqp rabbitmq-server:amqp
juju add-relation neutron-gateway nova-cloud-controller
juju set keystone admin-password="openstack"
# display status
juju stat --format=tabular
# Download image
wget http://cloud-images.ubuntu.com/trusty/current/trusty-server-cloudimg-amd64-disk1.img
glance add name="Trusty x86_64" is_public=true container_format=ovf disk_format=qcow2 < trusty-server-cloudimg-amd64-disk1.img
# set default security group
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
# add keypair
nova keypair-add --pub-key ~/.ssh/id_rsa.pub id_rsa
nova image-list

After initial setup, I followed the documentation up to "Verify connectivity" and setup neutron networks as follows :

#create external network and subnet
neutron net-create ext-net --shared --router:external=True
neutron subnet-create ext-net --name ext-subnet \
  --allocation-pool start=192.168.1.230,end=192.168.1.250 \
  --disable-dhcp --gateway 192.168.1.1 192.168.1.0/24

#create tenant network and subnet
neutron net-create demo-net
neutron subnet-create demo-net --name demo-subnet \
  --gateway 10.0.1.1 10.0.1.1/24

# add router
neutron router-create demo-router
neutron router-interface-add demo-router demo-subnet
neutron router-gateway-set demo-router ext-net

The neutron-gateway node has two physical network interfaces :

  • eth0 connected to the juju 10.0.0.0/24 network
  • eth1 connected to the external 192.168.1.0/24 network

When created, the router should get the first address in the specified pool, 192.168.1.230 and since this address is set in the /etc/network/interfaces, I am able to ping it.

PING 192.168.1.230 (192.168.1.230) 56(84) bytes of data.
64 bytes from 192.168.1.230: icmp_seq=1 ttl=64 time=0.051 ms
64 bytes from 192.168.1.230: icmp_seq=2 ttl=64 time=0.024 ms
64 bytes from 192.168.1.230: icmp_seq=3 ttl=64 time=0.040 ms

I am also able unable to ping the interfac interface address specified for the VM networknetwork. (I was able to do that once, but dont know why it stopped working)

PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.158 ms
64 bytes from 10.0.1.1: icmp_seq=2 ttl=64 time=0.227 ms
64 bytes from 10.0.1.1: icmp_seq=3 ttl=64 time=0.197 ms
64 bytes from 10.0.1.1: icmp_seq=4 ttl=64 time=0.193 ms
^C
--- 10.0.1.1 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1008ms

I am able to create a VM, but I cannot ping it, both on the 10.0.1.0 and the 192.168.1.0 networks.

nova list
+--------------------------------------+------+--------+------------+-------------+------------------------------------------+
| ID                                   | Name | Status | Task State | Power State | Networks                                 |
+--------------------------------------+------+--------+------------+-------------+------------------------------------------+
| 0ef5a2bd-cbb6-44c9-b09d-738b553a8367 | test | ACTIVE | -          | Running     | ext-net=192.168.1.234; demo-net=10.0.1.6 |
+--------------------------------------+------+--------+------------+-------------+------------------------------------------+

One more thing is router gateway port and loval DHCP port have a DOWN status. There must me something missing but I don't get it.

neutron port-show 1c34ee72-8163-4eca-8cea-65ccd4640cdc
+---------------------+--------------------------------------------------------------------------------------+
| Field               | Value                                                                                |
+---------------------+--------------------------------------------------------------------------------------+
| admin_state_up      | True                                                                                 |
| binding:host_id     | tragic-pickle                                                                        |
| binding:profile     | {}                                                                                   |
| binding:vif_details | {"port_filter": false, "ovs_hybrid_plug": false}                                     |
| binding:vif_type    | ovs                                                                                  |
| binding:vnic_type   | normal                                                                               |
| device_id           | 7ed37eac-f293-43a9-b9c4-1398f1fc08ba                                                 |
| device_owner        | network:router_gateway                                                               |
| extra_dhcp_opts     |                                                                                      |
| fixed_ips           | {"subnet_id": "7f915784-8615-4c39-beaf-7469514f3601", "ip_address": "192.168.1.230"} |
| id                  | 1c34ee72-8163-4eca-8cea-65ccd4640cdc                                                 |
| mac_address         | fa:16:3e:8a:95:f8                                                                    |
| name                |                                                                                      |
| network_id          | fd7742b1-fbca-4e07-a2cf-c4c0fa0cea36                                                 |
| status              | DOWN                                                                                 |
| tenant_id           |                                                                                      |
+---------------------+--------------------------------------------------------------------------------------+

,

neutron port-show 5f2f6ea9-a804-4503-b85f-e4cb4b04af42
+---------------------+---------------------------------------------------------------------------------+
| Field               | Value                                                                           |
+---------------------+---------------------------------------------------------------------------------+
| admin_state_up      | True                                                                            |
| binding:host_id     | tragic-pickle                                                                   |
| binding:profile     | {}                                                                              |
| binding:vif_details | {}                                                                              |
| binding:vif_type    | binding_failed                                                                  |
| binding:vnic_type   | normal                                                                          |
| device_id           | dhcp3d22cbc4-82c2-5ef9-83aa-80edfb56366c-72be7e29-3673-4133-9601-50a0ac061317   |
| device_owner        | network:dhcp                                                                    |
| extra_dhcp_opts     |                                                                                 |
| fixed_ips           | {"subnet_id": "161fba24-f081-4303-80e3-01a03293becb", "ip_address": "10.0.1.2"} |
| id                  | 5f2f6ea9-a804-4503-b85f-e4cb4b04af42                                            |
| mac_address         | fa:16:3e:29:20:5d                                                               |
| name                |                                                                                 |
| network_id          | 72be7e29-3673-4133-9601-50a0ac061317                                            |
| status              | DOWN                                                                            |
| tenant_id           | 7c0f373615b845b4829aac6910f079a8                                                |
+---------------------+---------------------------------------------------------------------------------+

Here is some details that may help, On neutron-gateway node

ip netns list
qrouter-7ed37eac-f293-43a9-b9c4-1398f1fc08ba
qdhcp-72be7e29-3673-4133-9601-50a0ac061317

,

sudo ip netns exec qrouter-7ed37eac-f293-43a9-b9c4-1398f1fc08ba iptables -S -t nat
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-N neutron-postrouting-bottom
-N neutron-vpn-agen-OUTPUT
-N neutron-vpn-agen-POSTROUTING
-N neutron-vpn-agen-PREROUTING
-N neutron-vpn-agen-float-snat
-N neutron-vpn-agen-snat
-A PREROUTING -j neutron-vpn-agen-PREROUTING
-A OUTPUT -j neutron-vpn-agen-OUTPUT
-A POSTROUTING -j neutron-vpn-agen-POSTROUTING
-A POSTROUTING -j neutron-postrouting-bottom
-A neutron-postrouting-bottom -j neutron-vpn-agen-snat
-A neutron-vpn-agen-POSTROUTING ! -i qg-1c34ee72-81 ! -o qg-1c34ee72-81 -m conntrack ! --ctstate DNAT -j ACCEPT
-A neutron-vpn-agen-PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 9697
-A neutron-vpn-agen-snat -j neutron-vpn-agen-float-snat
-A neutron-vpn-agen-snat -s 10.0.1.0/24 -j SNAT --to-source 192.168.1.230

,

cat /etc/network/interfaces
ubuntu@tragic-pickle:~$ cat /etc/network/interfaces
auto lo
iface lo inet loopback
    dns-nameservers 192.168.1.29
    dns-search maas

iface eth0 inet manual

auto juju-br0
iface juju-br0 inet static
    bridge_ports eth0
    gateway 10.0.0.1
    address 10.0.0.12/24
    mtu 1500

## External bridge
auto br-ex
iface br-ex inet static
  address 192.168.1.230
  netmask 255.255.255.0
  gateway 192.168.1.1
  dns-nameservers 8.8.8.8

auto eth1
iface eth1 inet manual
#    up ifconfig $IFACE 0.0.0.0 up
  up ip link set dev $IFACE up
    mtu 1500
#  promisc on
  down ip link set dev $IFACE promisc off
  down ifconfig $IFACE down

,

sudo ovs-vsctl show
fc996723-61c1-4fd0-a38f-1dd0df07eeb8
    Bridge br-ex
        Port br-ex
            Interface br-ex
                type: internal
        Port phy-br-ex
            Interface phy-br-ex
        Port "eth1"
            Interface "eth1"
        Port "tap1c34ee72-81"
            Interface "tap1c34ee72-81"
    Bridge br-int
        fail_mode: secure
        Port "tap5f2f6ea9-a8"
            tag: 4095
            Interface "tap5f2f6ea9-a8"
        Port br-int
            Interface br-int
                type: internal
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port "tap3a6679cd-f3"
            tag: 1
            Interface "tap3a6679cd-f3"
        Port int-br-ex
            Interface int-br-ex
    Bridge br-tun
        Port "gre-0a00000b"
            Interface "gre-0a00000b"
                type: gre
                options: {in_key=flow, local_ip="10.0.0.12", out_key=flow, remote_ip="10.0.0.11"}
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port br-tun
            Interface br-tun
                type: internal
    ovs_version: "2.0.2"

,

ifconfig
br-ex     Link encap:Ethernet  HWaddr 68:05:ca:3b:e6:13  
          inet addr:192.168.1.230  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::5848:89ff:fee7:35dd/64 Scope:Link
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:193344 errors:0 dropped:0 overruns:0 frame:0
          TX packets:301 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:25978737 (25.9 MB)  TX bytes:32000 (32.0 KB)

br-int    Link encap:Ethernet  HWaddr 62:6f:d2:ca:3b:43  
          inet6 addr: fe80::281b:41ff:fe46:3142/64 Scope:Link
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:24 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:1924 (1.9 KB)  TX bytes:648 (648.0 B)

br-tun    Link encap:Ethernet  HWaddr de:e0:7e:d9:2e:46  
          inet6 addr: fe80::b41f:94ff:fea4:ff87/64 Scope:Link
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:648 (648.0 B)

eth0      Link encap:Ethernet  HWaddr 74:d4:35:fa:0f:37  
          inet6 addr: fe80::76d4:35ff:fefa:f37/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:253969 errors:0 dropped:0 overruns:0 frame:0
          TX packets:285181 errors:0 dropped:0 overruns:0 carrier:1
          collisions:0 txqueuelen:1000 
          RX bytes:56960015 (56.9 MB)  TX bytes:66650148 (66.6 MB)

eth1      Link encap:Ethernet  HWaddr 68:05:ca:3b:e6:13  
          inet6 addr: fe80::6a05:caff:fe3b:e613/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:216435 errors:0 dropped:0 overruns:0 frame:0
          TX packets:358 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:28250255 (28.2 MB)  TX bytes:38270 (38.2 KB)
          Interrupt:16 Memory:fddc0000-fdde0000 

int-br-ex Link encap:Ethernet  HWaddr ea:28:33:73:cd:27  
          inet6 addr: fe80::e828:33ff:fe73:cd27/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:192051 errors:0 dropped:0 overruns:0 frame:0
          TX packets:31 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:23390016 (23.3 MB)  TX bytes:2482 (2.4 KB)

juju-br0  Link encap:Ethernet  HWaddr 74:d4:35:fa:0f:37  
          inet addr:10.0.0.12  Bcast:10.0.0.255  Mask:255.255.255.0
          inet6 addr: fe80::76d4:35ff:fefa:f37/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:253955 errors:0 dropped:0 overruns:0 frame:0
          TX packets:285175 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:53403609 (53.4 MB)  TX bytes:66642750 (66.6 MB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

phy-br-ex Link encap:Ethernet  HWaddr 82:73:1e:a1:49:de  
          inet6 addr: fe80::8073:1eff:fea1:49de/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:31 errors:0 dropped:0 overruns:0 frame:0
          TX packets:192051 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:2482 (2.4 KB)  TX bytes:23390016 (23.3 MB)

tap1c34ee72-81 Link encap:Ethernet  HWaddr 1e:35:21:f9:ba:8b  
          inet6 addr: fe80::1c35:21ff:fef9:ba8b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:50 errors:0 dropped:0 overruns:0 frame:0
          TX packets:192065 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:4184 (4.1 KB)  TX bytes:23391103 (23.3 MB)

tap3a6679cd-f3 Link encap:Ethernet  HWaddr de:15:c1:e0:4a:de  
          inet6 addr: fe80::dc15:c1ff:fee0:4ade/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:18 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:648 (648.0 B)  TX bytes:1428 (1.4 KB)

tap5f2f6ea9-a8 Link encap:Ethernet  HWaddr 76:9d:30:1a:f0:80  
          inet6 addr: fe80::749d:30ff:fe1a:f080/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:9 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:648 (648.0 B)  TX bytes:726 (726.0 B)

Thanks for your attention !