We are trying to deploy a simple system with Devstack Queens on two servers (baremetal). Our infra people have connected the two servers as displayed in the attached network diagram. (https://ibb.co/3vBkvHM) One of the servers will host the openstack controller and one compute node. The second server will be used to host another compute node. Our goal is to give the controller the possibility to choose where to deploy instances.
So far, we managed to install the controller and compute node in the first machine and compute in the second.
On the controller machine, apparently everything seems to be working. We are able to create instances, we can ping outside and are able to assign floating ips to them. Also if we create instances in the same compute node, we can ping from one to another both ways.
But then, on the other side of the coin, when we look at the second compute node, we can instantiate vms on that compute, but we can´t get them to communicate with the controller. We have noticed that if we deploy vms inside that compute, they can ping each other, but if we try to ping the gateway in that network, they can’t.
Note that when deploying compute node and discovering hosts on controller node, a patch between br-tun on both servers is created (see ovs-vsctl info appended). Though, no packet is using told path since br-tun always drops any input packet. Therefore, br-ex is the logical output preference to communicate between computes but as seen in figure, there is no connection between br-ex and Interface enp12s0.
Since we are deploying a provider network scenario, which packet flow between computes should we expect? Through br-ex(Compute External) -> Switch -> br-ex(Compute on Controller) or br-tun (Compute External) -> br-tun (Compute on Controller)
We are probably missing something somewhere. We are not trying to do anything complicated. So any help would be really appreciated.
Attached here some of the conf files:
local.conf from the controller
[[local|localrc]]
# PASSWORDS
ADMIN_PASSWORD=*******
DATABASE_PASSWORD=supersecret
RABBIT_PASSWORD=$DATABASE_PASSWORD
SERVICE_PASSWORD=$DATABASE_PASSWORD
# HOST_IP: API endpoint host, use own public IP so endpoint can be reached from outside.
HOST_IP=XX.YY.40.124
# SERVICE_HOST: Pointer to Controller @IP
SERVICE_HOST=$HOST_IP
MULTI_HOST=1
RECLONE=True
# Logging
LOGFILE=$DEST/logs/stack.sh.log
LOGDAYS=2
VERBOSE=True
LOG_COLOR=True
SCREEN_LOGDIR=$DEST/logs
## Neutron options
Q_USE_SECGROUP=True
PUBLIC_INTERFACE=enp132s0f1
NEUTRON_CREATE_INITIAL_NETWORKS=False
# Open vSwitch provider networking configuration
OVS_PHYSICAL_BRIDGE=br-ex
Q_ML2_TENANT_NETWORK_TYPE=vlan
Q_ML2_PLUGIN_TYPE_DRIVERS=flat,vlan,vxlan
PHYSICAL_NETWORK=default
ENABLE_TENANT_VLANS=True
TENANT_VLAN_RANGE=1200:1220
# Prevent long lockout
KEYSTONE_LOCKOUT_FAILURE_ATTEMPTS=10
KEYSTONE_LOCKOUT_DURATION=2
# Fix incompatibility: tunnel_types starting as gre by default
[[post-config|/$Q_PLUGIN_CONF_FILE]]
[agent]
tunnel_types=vxlan
Local.conf from the compute node
[[local|localrc]]
# PASSWORDS
ADMIN_PASSWORD=********
DATABASE_PASSWORD=supersecret
RABBIT_PASSWORD=$DATABASE_PASSWORD
SERVICE_PASSWORD=$DATABASE_PASSWORD
# HOST_IP: API endpoint host, use own public IP so endpoint can be reached from outside.
HOST_IP=XX.YY.40.75
# SERVICE_HOST: Pointer to Controller @IP
SERVICE_HOST=XX.YY.40.124
MULTI_HOST=1
RECLONE=True
# Logging
LOGFILE=$DEST/logs/stack.sh.log
LOGDAYS=2
VERBOSE=True
LOG_COLOR=True
SCREEN_LOGDIR=$DEST/logs
## Neutron options
Q_USE_SECGROUP=True
PUBLIC_INTERFACE=enp12s0
NEUTRON_CREATE_INITIAL_NETWORKS=False
# Open vSwitch provider networking configuration
OVS_PHYSICAL_BRIDGE=br-ex
Q_ML2_TENANT_NETWORK_TYPE=vlan
Q_ML2_PLUGIN_TYPE_DRIVERS=flat,vlan,vxlan
PHYSICAL_NETWORK=default
ENABLE_TENANT_VLANS=True
TENANT_VLAN_RANGE=1200:1220
ENABLED_SERVICES=n-cpu,q-meta,q-agt,q-dhcp,n-api-meta,placement-client
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
NOVA_VNC_ENABLED=True
NOVNCPROXY_URL="http://$SERVICE_HOST:6080/vnc_auto.html"
VNCSERVER_LISTEN=$HOST_IP
VNCSERVER_PROXYCLIENT_ADDRESS=$VNCSERVER_LISTEN
# Fix incompatibility: tunnel_types starting as gre by default
[[post-config|/$Q_PLUGIN_CONF_FILE]]
[agent]
tunnel_types=vxlan
Result from ovs-vsctl show in the controller node
stack@ias4:~/devstack$ sudo ovs-vsctl show
1890df78-aeeb-4b8f-b0ff-22fb50fa69bb
Manager "ptcp:6640:127.0.0.1"
is_connected: true
Bridge br-tun
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
Port br-tun
Interface br-tun
type: internal
Port "vxlan-5458284b"
Interface "vxlan-5458284b"
type: vxlan
options: {df_default="true", in_key=flow, local_ip="XX.YY.40.124", out_key=flow, remote_ip="XX.YY.40.75"}
Bridge br-ex
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port phy-br-ex
Interface phy-br-ex
type: patch
options: {peer=int-br-ex}
Port br-ex
Interface br-ex
type: internal
Port "enp132s0f1"
Interface "enp132s0f1"
Bridge br-int
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port "tap910b5d9d-75"
tag: 3
Interface "tap910b5d9d-75"
type: internal
Port "qvodff8c9ac-27"
tag: 1
Interface "qvodff8c9ac-27"
Port "qr-8abbf087-b9"
tag: 1
Interface "qr-8abbf087-b9"
type: internal
Port "tap61cc621f-d1"
tag: 1
Interface "tap61cc621f-d1"
type: internal
Port "qvo09afe2ed-a5"
tag: 1
Interface "qvo09afe2ed-a5"
Port br-int
Interface br-int
type: internal
Port "qr-86e90d03-a7"
tag: 3
Interface "qr-86e90d03-a7"
type: internal
Port "qg-1a4a061c-54"
tag: 2
Interface "qg-1a4a061c-54"
type: internal
Port "qg-670a33e3-b1"
tag: 2
Interface "qg-670a33e3-b1"
type: internal
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port int-br-ex
Interface int-br-ex
type: patch
options: {peer=phy-br-ex}
ovs_version: "2.8.4"
stack@ias4:~/devstack$
Result from ovs-vsctl show on the compute node
stack@papasseit:~/devstack$ sudo ovs-vsctl show
359558d4-2e33-44de-971c-13b24dac626e
Manager "ptcp:6640:127.0.0.1"
is_connected: true
Bridge br-int
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port "qvo19a2581c-b4"
tag: 1
Interface "qvo19a2581c-b4"
Port "tap760f1b89-1a"
tag: 1
Interface "tap760f1b89-1a"
type: internal
Port "tap3e532db9-2b"
tag: 2
Interface "tap3e532db9-2b"
type: internal
Port "qvo17e25008-ca"
tag: 1
Interface "qvo17e25008-ca"
Port br-int
Interface br-int
type: internal
Port "qvoce8817a8-cd"
tag: 2
Interface "qvoce8817a8-cd"
Port int-br-ex
Interface int-br-ex
type: patch
options: {peer=phy-br-ex}
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Bridge br-tun
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port br-tun
Interface br-tun
type: internal
Port "vxlan-5458287c"
Interface "vxlan-5458287c"
type: vxlan
options: {df_default="true", in_key=flow, local_ip="XX.YY.40.75", out_key=flow, remote_ip="XX.YY.40.124"}
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
Bridge br-ex
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port br-ex
Interface br-ex
type: internal
Port phy-br-ex
Interface phy-br-ex
type: patch
options: {peer=int-br-ex}
ovs_version: "2.8.4"
stack@papasseit:~/devstack$
result from running systemctl list-units | grep stack
stack@ias4:~/devstack$ systemctl list-units | grep stack
devstack@c-api.service loaded active running Devstack devstack@c-api.service
devstack@c-sch.service loaded active running Devstack devstack@c-sch.service
devstack@c-vol.service loaded active running Devstack devstack@c-vol.service
devstack@dstat.service loaded active running Devstack devstack@dstat.service
devstack@etcd.service loaded active running Devstack devstack@etcd.service
devstack@g-api.service loaded active running Devstack devstack@g-api.service
devstack@g-reg.service loaded active running Devstack devstack@g-reg.service
devstack@keystone.service loaded active running Devstack devstack@keystone.service
devstack@n-api-meta.service loaded active running Devstack devstack@n-api-meta.service
devstack@n-api.service loaded active running Devstack devstack@n-api.service
devstack@n-cauth.service loaded active running Devstack devstack@n-cauth.service
devstack@n-cond-cell1.service loaded active running Devstack devstack@n-cond-cell1.service
devstack@n-cpu.service loaded active running Devstack devstack@n-cpu.service
devstack@n-novnc.service loaded active running Devstack devstack@n-novnc.service
devstack@n-sch.service loaded active running Devstack devstack@n-sch.service
devstack@n-super-cond.service loaded active running Devstack devstack@n-super-cond.service
devstack@placement-api.service loaded active running Devstack devstack@placement-api.service
devstack@q-agt.service loaded active running Devstack devstack@q-agt.service
devstack@q-dhcp.service loaded active running Devstack devstack@q-dhcp.service
devstack@q-l3.service loaded active running Devstack devstack@q-l3.service
devstack@q-meta.service loaded active running Devstack devstack@q-meta.service
devstack@q-svc.service loaded active running Devstack devstack@q-svc.service
system-devstack.slice loaded active active system-devstack.slice
stack@ias4:~/devstack$
And last but not least results from running systemctl list-units | grep stack on the compute node
stack@papasseit:~/devstack$ systemctl list-units | grep stack
● devstack@c-sch.service not-found failed failed devstack@c-sch.service
● devstack@c-vol.service not-found failed failed devstack@c-vol.service
devstack@n-api-meta.service loaded active running Devstack devstack@n-api-meta.service
devstack@n-cpu.service loaded active running Devstack devstack@n-cpu.service
● devstack@n-sch.service not-found failed failed devstack@n-sch.service
devstack@q-agt.service loaded active running Devstack devstack@q-agt.service
devstack@q-dhcp.service loaded active running Devstack devstack@q-dhcp.service
devstack@q-meta.service loaded active running Devstack devstack@q-meta.service
● devstack@q-svc.service not-found failed failed devstack@q-svc.service
system-devstack.slice loaded active active system-devstack.slice
stack@papasseit:~/devstack$