Here is how I figured out the solution. Please note: I was running a trunk port into my openstack nodes as interface enp0s8 that I put into bond0.
vi /etc/kolla/config/neutron/ml2_conf.ini
[ml2_type_vlan]
network_vlan_ranges = physnet1
Modify the following lines in kolla-ansible/tools/init-runonce
[root@controller tools]# diff init-runonce init-runonce2
16,18c16,18
< EXT_NET_CIDR='10.0.2.0/24'
< EXT_NET_RANGE='start=10.0.2.150,end=10.0.2.199'
< EXT_NET_GATEWAY='10.0.2.1'
---
> EXT_NET_CIDR='20.0.0.0/24'
> EXT_NET_RANGE='start=20.0.0.150,end=20.0.0.199'
> EXT_NET_GATEWAY='20.0.0.1'
72,73c72,74
< openstack network create --external --provider-physical-network physnet1 \
< --provider-network-type flat public1
---
> #openstack network create --external --provider-physical-network physnet1 \
> # --provider-network-type flat public1
> openstack network create public1 --external --no-share --provider-physical-network physnet1 --provider-network-type vlan --provider-segment 20
79,80c80,81
< openstack subnet create --subnet-range 10.0.0.0/24 --network demo-net \
< --gateway 10.0.0.1 --dns-nameserver 8.8.8.8 demo-subnet
---
> openstack subnet create --subnet-range 30.0.0.0/24 --network demo-net \
> --gateway 30.0.0.1 --dns-nameserver 8.8.8.8 demo-subnet
[root@controller tools]#
You're essentially defining your external network with env variables:
EXT_NET_CIDR='20.0.0.0/24'
EXT_NET_RANGE='start=20.0.0.150,end=20.0.0.199'
EXT_NET_GATEWAY='20.0.0.1'
Then when you create the external provider network you specify the vlan (in my case 20).
openstack network create public1 --external --no-share --provider-physical-network physnet1 --provider-network-type vlan --provider-segment 20
You'd perform this procedure for each vlan that you want to use as an external provider network.
Now, if you're foolish/smart enough to use VirtualBox to test this deploy on CentOS here are a few notes that I had to do to get things to work.
In VirtualBox:
Enable Promiscuous Mode: "Allow All" on network interfaces.
In CentOS 7:
Make sure you nics have line:
NM_CONTROLLED=no
Without that line, everything would work in the VXLAN and public network, except connectivity to the EXT_NET_GATEWAY. I had to have NM_CONTROLLED=no for that portion to work!
E.g.,
[root@controller tools]# cat /etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
BONDING_MASTER=yes
TYPE=Bond
ONBOOT=yes
BOOTPROTO=none
BONDING_OPTS="mode=0 miimon=100"
NM_CONTROLLED=no
[root@controller tools]# cat /etc/sysconfig/network-scripts/ifcfg-enp0s8
HWADDR=08:00:27:8F:E0:FD
TYPE=Ethernet
BOOTPROTO="none"
NAME=enp0s8
UUID=64541e9c-2e4e-4226-b0bc-ba4fad2c5a07
DEVICE=enp0s8
ONBOOT=yes
MASTER=bond0
SLAVE=yes
NM_CONTROLLED=no
[root@controller tools]#
Also, it is worth noting that the second vlan currently has issues starting the second qrouter when the following commands are run to create the second external provider network:
#!/bin/bash
EXT_NET_CIDR='50.0.0.0/24'
EXT_NET_RANGE='start=50.0.0.150,end=50.0.0.199'
EXT_NET_GATEWAY='50.0.0.1'
openstack network create public5 --external --no-share --provider-physical-network physnet1 --provider-network-type vlan --provider-segment 50
openstack subnet create --no-dhcp \
--allocation-pool ${EXT_NET_RANGE} --network public5 \
--subnet-range ${EXT_NET_CIDR} --gateway ${EXT_NET_GATEWAY} public5-subnet
openstack network create --provider-network-type vxlan demo-net5
openstack subnet create --subnet-range 60.0.0.0/24 --network demo-net5 \
--gateway 60 ...
(more)