Ask Your Question
2

During Kolla deploy, when neutron comes up, networking goes down

asked 2016-06-12 22:17:59 -0600

brainbarker gravatar image

updated 2016-06-13 09:42:48 -0600

Greetings,

I've been trying to complete a Kolla deployment for about a week now, and I'm just plain stuck. I've gotten the boxes set up, and can get through the prechecks, but when I execute deploy, I only get as far as "Starting neutron-server container". At that point, all of the nodes lose network connectivity. Logging into the nodes from the console, I can see that everything appears normal on the box. The interfaces are up, and you can ping from each interface to the other, but no traffic can get into or out of the box. Stopping the docker containers does not fix the problem. Rebooting the box will bring the networking back, but only until you restart docker and the containers, at which point the network stops working again.

I'm assuming I've made some obvious noobie mistake in the configuration, which I hope someone here can spot for me.

All boxes have two NICs. The primary interface is on 192.168.60.0/24, and has internet connectivity. The secondary interface is on 192.168.65.0/24, and only has connectivity between the nodes.

The interface names are different on the control/network nodes, and the compute/storage nodes, and are specified in the multinode inventory file.

The interesting part of my multinode inventory file looks like:

[control]
smss02 neutron_external_interface=ens11 api_interface=ens11d1 storage_interface=ens11d1 tunnel_interface=ens11d1 network_interface=ens11d1 kolla_external_vip_interface=ens11
smss04 neutron_external_interface=ens11 api_interface=ens11d1 storage_interface=ens11d1 tunnel_interface=ens11d1 network_interface=ens11d1 kolla_external_vip_interface=ens11

[network]
smss03 neutron_external_interface=ens11 api_interface=ens11d1 storage_interface=ens11d1 tunnel_interface=ens11d1 network_interface=ens11d1 kolla_external_vip_interface=ens11
smss05 neutron_external_interface=ens11 api_interface=ens11d1 storage_interface=ens11d1 tunnel_interface=ens11d1 network_interface=ens11d1 kolla_external_vip_interface=ens11

[compute]
smst[01:16] neutron_external_interface=ens6 api_interface=ens5 storage_interface=ens5 tunnel_interface=ens5 network_interface=ens5 kolla_external_vip_interface=ens6

The only non-comment lines in my globals.yml are:

kolla_internal_vip_address: 192.168.65.254
kolla_external_vip_address: 192.168.60.254
docker_registry: "192.168.60.50:4000"
keepalived_virtual_router_id: "10"
openstack_logging_debug: "True"
enable_cinder: "yes"

Is there anything blatantly wrong with this setup? I can provide further info if it will help.

Any comments or suggestions will be greatly appreciated.

Many thanks, Brian.

EDIT 1: A few more system details:

Servers are running Centos7:
[root@smss01 ~]# uname -a
Linux smss01 3.10.0-327.18.2.el7.x86_64 #1 SMP Thu May 12 11:03:55 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

[root@smss01 ~]# ansible --version
ansible 1.9.6
  configured module search path = None

[root@smss01 ~]# docker --version
Docker version 1.11.2, build b9f10c9

Kolla up-to-date as of Wed Jun 8 16:57:49 2016 +0000

EDIT 2:

Here are the interfaces on one of the compute/storage nodes. The control/network nodes are similar, but with different interface names.

[root@smst01 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope ...
(more)
edit retag flag offensive close merge delete

Comments

Is there any other place where interfaces are defined? Maybe var precendece force to not use vars defined in the inventory. At ansible logs, can you see which interfaces use while deployment. Paste your globals file.

Regards

Eduardo Gonzalez gravatar imageEduardo Gonzalez ( 2016-06-13 00:28:16 -0600 )edit

2 answers

Sort by ยป oldest newest most voted
0

answered 2016-09-17 14:53:58 -0600

mm9745 gravatar image

brainbarker, were you ever able to resolve this issue? I'm seeing something very similar on my kolla multinode, two-nic-apiece setup. As soon as OVS is deployed by kolla-ansible, none of the nodes can ping each across one of the two interfaces. tcpdump shows ARP requests being sent and received, but no ARP replies get sent in response. ARP table entries show peer IP addresses on this interface's network with an "(incomplete)" HWaddress status.

Thanks, Matt

edit flag offensive delete link more
0

answered 2018-12-06 23:51:41 -0600

I have same issue and fixed it by deactive neutron_external_interface so it will not get DHCP ip.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2016-06-12 22:17:59 -0600

Seen: 709 times

Last updated: Jun 13 '16