Revision history [back]

click to hide/show revision 1
initial version

Dnsmasq failure to provide private IP to VM on fresh Juno install on TwoNode Controller (CentOS 7)

Seems to be a new issue . Never before experienced this problem
First on Controller&&Network node :

[root@juno1dev ~(keystone_admin)]# neutron net-list
+--------------------------------------+--------------+-----------------------------------------------------+
| id                                   | name         | subnets                                             |
+--------------------------------------+--------------+-----------------------------------------------------+
| 30e668d5-eb07-423d-aeb6-d408a715e9bb | demo_network | a9a78ba4-5cb1-4e7d-97c7-f1531ebb7fbe 50.0.0.0/24    |
| 2cb0f522-c91f-4ef1-80c3-c9ade696e3bf | public       | c3cac54d-acd9-4602-811f-8952c2fb94d8 192.168.1.0/24 |
+--------------------------------------+--------------+-----------------------------------------------------+
[root@juno1dev ~(keystone_admin)]# ip netns | grep 30e668d5-eb07-423d-aeb6-d408a715e9bb
qdhcp-30e668d5-eb07-423d-aeb6-d408a715e9bb
[root@juno1dev ~(keystone_admin)]# ip netns exec qdhcp-30e668d5-eb07-423d-aeb6-d408a715e9bb ifconfig
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 0  (Local Loopback)
        RX packets 25  bytes 2800 (2.7 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 25  bytes 2800 (2.7 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

tape0a57cc5-96: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 50.0.0.11  netmask 255.255.255.0  broadcast 50.0.0.255
        inet6 fe80::f816:3eff:fec2:fba9  prefixlen 64  scopeid 0x20<link>
        ether fa:16:3e:c2:fb:a9  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 29  bytes 1530 (1.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@juno1dev ~(keystone_admin)]# tcpdump -vv -i tape0a57cc5-96
tcpdump: tape0a57cc5-96: No such device exists
(SIOCGIFHWADDR: No such device)

Second /var/log/neutron/dnsmasq.log

Jan 25 10:47:08 dnsmasq[17647]: started, version 2.66 cachesize 150
Jan 25 10:47:08 dnsmasq[17647]: compile time options: IPv6 GNU-getopt DBus no-i18n IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth
Jan 25 10:47:08 dnsmasq[17647]: warning: no upstream servers configured
Jan 25 10:47:08 dnsmasq-dhcp[17647]: DHCP, static leases only on 50.0.0.0, lease time 1d
Jan 25 10:47:08 dnsmasq[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/addn_hosts - 2 addresses
Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/host
Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/opts
Jan 25 10:47:08 dnsmasq[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/addn_hosts - 2 addresses
Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/host
Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/opts
Jan 25 10:56:31 dnsmasq[19925]: started, version 2.66 cachesize 150
Jan 25 10:56:31 dnsmasq[19925]: compile time options: IPv6 GNU-getopt DBus no-i18n IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth
Jan 25 10:56:31 dnsmasq[19925]: warning: no upstream servers configured
Jan 25 10:56:31 dnsmasq-dhcp[19925]: DHCP, static leases only on 50.0.0.0, lease time 1d
Jan 25 10:56:31 dnsmasq[19925]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/addn_hosts - 3 addresses
Jan 25 10:56:31 dnsmasq-dhcp[19925]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/host
Jan 25 10:56:31 dnsmasq-dhcp[19925]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/opts

ovs-vsctl show looks good on both nodes :-

[root@juno1dev ~(keystone_admin)]# ovs-vsctl show
9734c6fb-448e-4052-94e9-15dce59b43f8
    Bridge br-ex
        Port "enp2s0"
            Interface "enp2s0"
        Port phy-br-ex
            Interface phy-br-ex
                type: patch
                options: {peer=int-br-ex}
        Port "qg-1e009472-ca"
            Interface "qg-1e009472-ca"
                type: internal
        Port br-ex
            Interface br-ex
                type: internal
    Bridge br-int
        fail_mode: secure
        Port "qr-bda23703-70"
            tag: 5
            Interface "qr-bda23703-70"
                type: internal
        Port int-br-ex
            Interface int-br-ex
                type: patch
                options: {peer=phy-br-ex}
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port "tape0a57cc5-96"
            tag: 5
            Interface "tape0a57cc5-96"
                type: internal
        Port br-int
            Interface br-int
                type: internal
    Bridge br-tun
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port br-tun
            Interface br-tun
                type: internal
        Port "vxlan-c0a80089"
            Interface "vxlan-c0a80089"
                type: vxlan
                options: {df_default="true", in_key=flow, local_ip="192.168.0.127", out_key=flow, remote_ip="192.168.0.137"}
    ovs_version: "2.1.3"
[root@juno1dev ~(keystone_admin)]#

Dnsmasq failure to provide private IP to VM on fresh Juno install on TwoNode Controller (CentOS 7)Controller&&Network,Compute CentOS 7 Cluster

Seems to be a new issue . Never before experienced this problem
First on Controller&&Network node :

[root@juno1dev ~(keystone_admin)]# neutron net-list
+--------------------------------------+--------------+-----------------------------------------------------+
| id                                   | name         | subnets                                             |
+--------------------------------------+--------------+-----------------------------------------------------+
| 30e668d5-eb07-423d-aeb6-d408a715e9bb | demo_network | a9a78ba4-5cb1-4e7d-97c7-f1531ebb7fbe 50.0.0.0/24    |
| 2cb0f522-c91f-4ef1-80c3-c9ade696e3bf | public       | c3cac54d-acd9-4602-811f-8952c2fb94d8 192.168.1.0/24 |
+--------------------------------------+--------------+-----------------------------------------------------+
[root@juno1dev ~(keystone_admin)]# ip netns | grep 30e668d5-eb07-423d-aeb6-d408a715e9bb
qdhcp-30e668d5-eb07-423d-aeb6-d408a715e9bb
[root@juno1dev ~(keystone_admin)]# ip netns exec qdhcp-30e668d5-eb07-423d-aeb6-d408a715e9bb ifconfig
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 0  (Local Loopback)
        RX packets 25  bytes 2800 (2.7 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 25  bytes 2800 (2.7 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

tape0a57cc5-96: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 50.0.0.11  netmask 255.255.255.0  broadcast 50.0.0.255
        inet6 fe80::f816:3eff:fec2:fba9  prefixlen 64  scopeid 0x20<link>
        ether fa:16:3e:c2:fb:a9  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 29  bytes 1530 (1.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@juno1dev ~(keystone_admin)]# tcpdump -vv -i tape0a57cc5-96
tcpdump: tape0a57cc5-96: No such device exists
(SIOCGIFHWADDR: No such device)

Second /var/log/neutron/dnsmasq.log

Jan 25 10:47:08 dnsmasq[17647]: started, version 2.66 cachesize 150
Jan 25 10:47:08 dnsmasq[17647]: compile time options: IPv6 GNU-getopt DBus no-i18n IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth
Jan 25 10:47:08 dnsmasq[17647]: warning: no upstream servers configured
Jan 25 10:47:08 dnsmasq-dhcp[17647]: DHCP, static leases only on 50.0.0.0, lease time 1d
Jan 25 10:47:08 dnsmasq[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/addn_hosts - 2 addresses
Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/host
Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/opts
Jan 25 10:47:08 dnsmasq[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/addn_hosts - 2 addresses
Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/host
Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/opts
Jan 25 10:56:31 dnsmasq[19925]: started, version 2.66 cachesize 150
Jan 25 10:56:31 dnsmasq[19925]: compile time options: IPv6 GNU-getopt DBus no-i18n IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth
Jan 25 10:56:31 dnsmasq[19925]: warning: no upstream servers configured
Jan 25 10:56:31 dnsmasq-dhcp[19925]: DHCP, static leases only on 50.0.0.0, lease time 1d
Jan 25 10:56:31 dnsmasq[19925]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/addn_hosts - 3 addresses
Jan 25 10:56:31 dnsmasq-dhcp[19925]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/host
Jan 25 10:56:31 dnsmasq-dhcp[19925]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/opts

ovs-vsctl show looks good on both nodes :-

[root@juno1dev ~(keystone_admin)]# ovs-vsctl show
9734c6fb-448e-4052-94e9-15dce59b43f8
    Bridge br-ex
        Port "enp2s0"
            Interface "enp2s0"
        Port phy-br-ex
            Interface phy-br-ex
                type: patch
                options: {peer=int-br-ex}
        Port "qg-1e009472-ca"
            Interface "qg-1e009472-ca"
                type: internal
        Port br-ex
            Interface br-ex
                type: internal
    Bridge br-int
        fail_mode: secure
        Port "qr-bda23703-70"
            tag: 5
            Interface "qr-bda23703-70"
                type: internal
        Port int-br-ex
            Interface int-br-ex
                type: patch
                options: {peer=phy-br-ex}
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port "tape0a57cc5-96"
            tag: 5
            Interface "tape0a57cc5-96"
                type: internal
        Port br-int
            Interface br-int
                type: internal
    Bridge br-tun
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port br-tun
            Interface br-tun
                type: internal
        Port "vxlan-c0a80089"
            Interface "vxlan-c0a80089"
                type: vxlan
                options: {df_default="true", in_key=flow, local_ip="192.168.0.127", out_key=flow, remote_ip="192.168.0.137"}
    ovs_version: "2.1.3"
[root@juno1dev ~(keystone_admin)]#

Dnsmasq failure to provide private IP to VM on fresh RDO Juno install on TwoNode Controller&&Network,Compute CentOS 7 Cluster

Seems to be a new issue . Never before experienced this problem
First on Controller&&Network node :

[root@juno1dev ~(keystone_admin)]# neutron net-list
+--------------------------------------+--------------+-----------------------------------------------------+
| id                                   | name         | subnets                                             |
+--------------------------------------+--------------+-----------------------------------------------------+
| 30e668d5-eb07-423d-aeb6-d408a715e9bb | demo_network | a9a78ba4-5cb1-4e7d-97c7-f1531ebb7fbe 50.0.0.0/24    |
| 2cb0f522-c91f-4ef1-80c3-c9ade696e3bf | public       | c3cac54d-acd9-4602-811f-8952c2fb94d8 192.168.1.0/24 |
+--------------------------------------+--------------+-----------------------------------------------------+
[root@juno1dev ~(keystone_admin)]# ip netns | grep 30e668d5-eb07-423d-aeb6-d408a715e9bb
qdhcp-30e668d5-eb07-423d-aeb6-d408a715e9bb
[root@juno1dev ~(keystone_admin)]# ip netns exec qdhcp-30e668d5-eb07-423d-aeb6-d408a715e9bb ifconfig
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 0  (Local Loopback)
        RX packets 25  bytes 2800 (2.7 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 25  bytes 2800 (2.7 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

tape0a57cc5-96: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 50.0.0.11  netmask 255.255.255.0  broadcast 50.0.0.255
        inet6 fe80::f816:3eff:fec2:fba9  prefixlen 64  scopeid 0x20<link>
        ether fa:16:3e:c2:fb:a9  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 29  bytes 1530 (1.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@juno1dev ~(keystone_admin)]# tcpdump -vv -i tape0a57cc5-96
tcpdump: tape0a57cc5-96: No such device exists
(SIOCGIFHWADDR: No such device)

Second /var/log/neutron/dnsmasq.log

Jan 25 10:47:08 dnsmasq[17647]: started, version 2.66 cachesize 150
Jan 25 10:47:08 dnsmasq[17647]: compile time options: IPv6 GNU-getopt DBus no-i18n IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth
Jan 25 10:47:08 dnsmasq[17647]: warning: no upstream servers configured
Jan 25 10:47:08 dnsmasq-dhcp[17647]: DHCP, static leases only on 50.0.0.0, lease time 1d
Jan 25 10:47:08 dnsmasq[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/addn_hosts - 2 addresses
Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/host
Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/opts
Jan 25 10:47:08 dnsmasq[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/addn_hosts - 2 addresses
Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/host
Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/opts
Jan 25 10:56:31 dnsmasq[19925]: started, version 2.66 cachesize 150
Jan 25 10:56:31 dnsmasq[19925]: compile time options: IPv6 GNU-getopt DBus no-i18n IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth
Jan 25 10:56:31 dnsmasq[19925]: warning: no upstream servers configured
Jan 25 10:56:31 dnsmasq-dhcp[19925]: DHCP, static leases only on 50.0.0.0, lease time 1d
Jan 25 10:56:31 dnsmasq[19925]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/addn_hosts - 3 addresses
Jan 25 10:56:31 dnsmasq-dhcp[19925]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/host
Jan 25 10:56:31 dnsmasq-dhcp[19925]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/opts

ovs-vsctl show looks good on both nodes :-

[root@juno1dev ~(keystone_admin)]# ovs-vsctl show
9734c6fb-448e-4052-94e9-15dce59b43f8
    Bridge br-ex
        Port "enp2s0"
            Interface "enp2s0"
        Port phy-br-ex
            Interface phy-br-ex
                type: patch
                options: {peer=int-br-ex}
        Port "qg-1e009472-ca"
            Interface "qg-1e009472-ca"
                type: internal
        Port br-ex
            Interface br-ex
                type: internal
    Bridge br-int
        fail_mode: secure
        Port "qr-bda23703-70"
            tag: 5
            Interface "qr-bda23703-70"
                type: internal
        Port int-br-ex
            Interface int-br-ex
                type: patch
                options: {peer=phy-br-ex}
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port "tape0a57cc5-96"
            tag: 5
            Interface "tape0a57cc5-96"
                type: internal
        Port br-int
            Interface br-int
                type: internal
    Bridge br-tun
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port br-tun
            Interface br-tun
                type: internal
        Port "vxlan-c0a80089"
            Interface "vxlan-c0a80089"
                type: vxlan
                options: {df_default="true", in_key=flow, local_ip="192.168.0.127", out_key=flow, remote_ip="192.168.0.137"}
    ovs_version: "2.1.3"
[root@juno1dev ~(keystone_admin)]#

Dnsmasq failure to provide private IP to VM on fresh RDO Juno install on TwoNode Controller&&Network,Compute CentOS 7 Cluster

Seems to be a new issue . Never before experienced this problem
First on Controller&&Network node :

[root@juno1dev ~(keystone_admin)]# neutron net-list
+--------------------------------------+--------------+-----------------------------------------------------+
| id                                   | name         | subnets                                             |
+--------------------------------------+--------------+-----------------------------------------------------+
| 30e668d5-eb07-423d-aeb6-d408a715e9bb | demo_network | a9a78ba4-5cb1-4e7d-97c7-f1531ebb7fbe 50.0.0.0/24    |
| 2cb0f522-c91f-4ef1-80c3-c9ade696e3bf | public       | c3cac54d-acd9-4602-811f-8952c2fb94d8 192.168.1.0/24 |
+--------------------------------------+--------------+-----------------------------------------------------+
[root@juno1dev ~(keystone_admin)]# ip netns | grep 30e668d5-eb07-423d-aeb6-d408a715e9bb
qdhcp-30e668d5-eb07-423d-aeb6-d408a715e9bb
[root@juno1dev ~(keystone_admin)]# ip netns exec qdhcp-30e668d5-eb07-423d-aeb6-d408a715e9bb ifconfig
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 0  (Local Loopback)
        RX packets 25  bytes 2800 (2.7 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 25  bytes 2800 (2.7 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

tape0a57cc5-96: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 50.0.0.11  netmask 255.255.255.0  broadcast 50.0.0.255
        inet6 fe80::f816:3eff:fec2:fba9  prefixlen 64  scopeid 0x20<link>
        ether fa:16:3e:c2:fb:a9  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 29  bytes 1530 (1.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@juno1dev ~(keystone_admin)]# tcpdump -vv -i tape0a57cc5-96
tcpdump: tape0a57cc5-96: No such device exists
(SIOCGIFHWADDR: No such device)

Second /var/log/neutron/dnsmasq.log

 Jan 25 10:47:08 dnsmasq[17647]: started, version 2.66 cachesize 150
 Jan 25 10:47:08 dnsmasq[17647]: compile time options: IPv6 GNU-getopt DBus no-i18n IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth
 Jan 25 10:47:08 dnsmasq[17647]: warning: no upstream servers configured
 Jan 25 10:47:08 dnsmasq-dhcp[17647]: DHCP, static leases only on 50.0.0.0, lease time 1d
 Jan 25 10:47:08 dnsmasq[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/addn_hosts - 2 addresses
 Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/host
 Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/opts
 Jan 25 10:47:08 dnsmasq[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/addn_hosts - 2 addresses
 Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/host
 Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/opts
 Jan 25 10:56:31 dnsmasq[19925]: started, version 2.66 cachesize 150
 Jan 25 10:56:31 dnsmasq[19925]: compile time options: IPv6 GNU-getopt DBus no-i18n IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth
 Jan 25 10:56:31 dnsmasq[19925]: warning: no upstream servers configured
 Jan 25 10:56:31 dnsmasq-dhcp[19925]: DHCP, static leases only on 50.0.0.0, lease time 1d
 Jan 25 10:56:31 dnsmasq[19925]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/addn_hosts - 3 addresses
 Jan 25 10:56:31 dnsmasq-dhcp[19925]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/host
 Jan 25 10:56:31 dnsmasq-dhcp[19925]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/opts

ovs-vsctl show looks good on both nodes :-

:-
 [root@juno1dev ~(keystone_admin)]# ovs-vsctl show
 9734c6fb-448e-4052-94e9-15dce59b43f8
     Bridge br-ex
         Port "enp2s0"
             Interface "enp2s0"
         Port phy-br-ex
             Interface phy-br-ex
                 type: patch
                 options: {peer=int-br-ex}
         Port "qg-1e009472-ca"
             Interface "qg-1e009472-ca"
                 type: internal
         Port br-ex
             Interface br-ex
                 type: internal
     Bridge br-int
         fail_mode: secure
         Port "qr-bda23703-70"
             tag: 5
             Interface "qr-bda23703-70"
                 type: internal
         Port int-br-ex
             Interface int-br-ex
                 type: patch
                 options: {peer=phy-br-ex}
         Port patch-tun
             Interface patch-tun
                 type: patch
                 options: {peer=patch-int}
         Port "tape0a57cc5-96"
             tag: 5
             Interface "tape0a57cc5-96"
                 type: internal
         Port br-int
             Interface br-int
                 type: internal
     Bridge br-tun
         Port patch-int
             Interface patch-int
                 type: patch
                 options: {peer=patch-tun}
         Port br-tun
             Interface br-tun
                 type: internal
         Port "vxlan-c0a80089"
             Interface "vxlan-c0a80089"
                 type: vxlan
                 options: {df_default="true", in_key=flow, local_ip="192.168.0.127", out_key=flow, remote_ip="192.168.0.137"}
     ovs_version: "2.1.3"
 [root@juno1dev ~(keystone_admin)]#

Dnsmasq failure to provide private IP to VM on fresh RDO Juno install on TwoNode Controller&&Network,Compute CentOS 7 Cluster

Seems to be a new issue . Never before experienced this problem
/var/log/neutron/dnsmasq.log

    Jan 25 10:47:08 dnsmasq[17647]: started, version 2.66 cachesize 150
    Jan 25 10:47:08 dnsmasq[17647]: compile time options: IPv6 GNU-getopt DBus no-i18n IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth
    Jan 25 10:47:08 dnsmasq[17647]: warning: no upstream servers configured
    Jan 25 10:47:08 dnsmasq-dhcp[17647]: DHCP, static leases only on 50.0.0.0, lease time 1d
    Jan 25 10:47:08 dnsmasq[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/addn_hosts - 2 addresses
    Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/host
    Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/opts
    Jan 25 10:47:08 dnsmasq[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/addn_hosts - 2 addresses
    Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/host
    Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/opts
    Jan 25 10:56:31 dnsmasq[19925]: started, version 2.66 cachesize 150
    Jan 25 10:56:31 dnsmasq[19925]: compile time options: IPv6 GNU-getopt DBus no-i18n IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth
    Jan 25 10:56:31 dnsmasq[19925]: warning: no upstream servers configured
    Jan 25 10:56:31 dnsmasq-dhcp[19925]: DHCP, static leases only on 50.0.0.0, lease time 1d
    Jan 25 10:56:31 dnsmasq[19925]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/addn_hosts - 3 addresses
    Jan 25 10:56:31 dnsmasq-dhcp[19925]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/host
    Jan 25 10:56:31 dnsmasq-dhcp[19925]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/opts

ovs-vsctl show looks good on both nodes :-

    [root@juno1dev ~(keystone_admin)]# ovs-vsctl show
    9734c6fb-448e-4052-94e9-15dce59b43f8
        Bridge br-ex
            Port "enp2s0"
                Interface "enp2s0"
            Port phy-br-ex
                Interface phy-br-ex
                    type: patch
                    options: {peer=int-br-ex}
            Port "qg-1e009472-ca"
                Interface "qg-1e009472-ca"
                    type: internal
            Port br-ex
                Interface br-ex
                    type: internal
        Bridge br-int
            fail_mode: secure
            Port "qr-bda23703-70"
                tag: 5
                Interface "qr-bda23703-70"
                    type: internal
            Port int-br-ex
                Interface int-br-ex
                    type: patch
                    options: {peer=phy-br-ex}
            Port patch-tun
                Interface patch-tun
                    type: patch
                    options: {peer=patch-int}
            Port "tape0a57cc5-96"
                tag: 5
                Interface "tape0a57cc5-96"
                    type: internal
            Port br-int
                Interface br-int
                    type: internal
        Bridge br-tun
            Port patch-int
                Interface patch-int
                    type: patch
                    options: {peer=patch-tun}
            Port br-tun
                Interface br-tun
                    type: internal
            Port "vxlan-c0a80089"
                Interface "vxlan-c0a80089"
                    type: vxlan
                    options: {df_default="true", in_key=flow, local_ip="192.168.0.127", out_key=flow, remote_ip="192.168.0.137"}
        ovs_version: "2.1.3"
    [root@juno1dev ~(keystone_admin)]#

VNC console gives code 1006 when attempt to open CirrOS login.

Dnsmasq failure to provide private IP to VM on fresh RDO Juno install on TwoNode Controller&&Network,Compute CentOS 7 Cluster

Seems to be a new issue . Never before experienced this problem
/var/log/neutron/dnsmasq.log

    Jan 25 10:47:08 dnsmasq[17647]: started, version 2.66 cachesize 150
    Jan 25 10:47:08 dnsmasq[17647]: compile time options: IPv6 GNU-getopt DBus no-i18n IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth
    Jan 25 10:47:08 dnsmasq[17647]: warning: no upstream servers configured
    Jan 25 10:47:08 dnsmasq-dhcp[17647]: DHCP, static leases only on 50.0.0.0, lease time 1d
    Jan 25 10:47:08 dnsmasq[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/addn_hosts - 2 addresses
    Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/host
    Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/opts
    Jan 25 10:47:08 dnsmasq[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/addn_hosts - 2 addresses
    Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/host
    Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/opts
    Jan 25 10:56:31 dnsmasq[19925]: started, version 2.66 cachesize 150
    Jan 25 10:56:31 dnsmasq[19925]: compile time options: IPv6 GNU-getopt DBus no-i18n IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth
    Jan 25 10:56:31 dnsmasq[19925]: warning: no upstream servers configured
    Jan 25 10:56:31 dnsmasq-dhcp[19925]: DHCP, static leases only on 50.0.0.0, lease time 1d
    Jan 25 10:56:31 dnsmasq[19925]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/addn_hosts - 3 addresses
    Jan 25 10:56:31 dnsmasq-dhcp[19925]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/host
    Jan 25 10:56:31 dnsmasq-dhcp[19925]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/opts
 

Create new private subnet && new router, set up gateway && interface same result

Jan 25 13:26:54 dnsmasq[9732]: started, version 2.66 cachesize 150
Jan 25 13:26:54 dnsmasq[9732]: compile time options: IPv6 GNU-getopt DBus no-i18n IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth
Jan 25 13:26:54 dnsmasq[9732]: warning: no upstream servers configured
Jan 25 13:26:54 dnsmasq-dhcp[9732]: DHCP, static leases only on 75.0.0.0, lease time 1d
Jan 25 13:26:54 dnsmasq[9732]: read /var/lib/neutron/dhcp/29a85223-6727-457c-bb13-2fff5719d4b8/addn_hosts - 2 addresses
Jan 25 13:26:54 dnsmasq-dhcp[9732]: read /var/lib/neutron/dhcp/29a85223-6727-457c-bb13-2fff5719d4b8/host
Jan 25 13:26:54 dnsmasq-dhcp[9732]: read /var/lib/neutron/dhcp/29a85223-6727-457c-bb13-2fff5719d4b8/opts
Jan 25 13:26:54 dnsmasq[9732]: read /var/lib/neutron/dhcp/29a85223-6727-457c-bb13-2fff5719d4b8/addn_hosts - 2 addresses
Jan 25 13:26:54 dnsmasq-dhcp[9732]: read /var/lib/neutron/dhcp/29a85223-6727-457c-bb13-2fff5719d4b8/host
Jan 25 13:26:54 dnsmasq-dhcp[9732]: read /var/lib/neutron/dhcp/29a85223-6727-457c-bb13-2fff5719d4b8/opts
Jan 25 13:35:10 dnsmasq-dhcp[9732]: 0 available DHCP subnet: 75.0.0.0/255.255.255.0
Jan 25 13:35:10 dnsmasq-dhcp[9732]: 0 DHCPRELEASE(tap6f93b6cb-1a) 75.0.0.10 fa:16:3e:22:36:07 unknown lease
Jan 25 13:35:10 dnsmasq[9732]: read /var/lib/neutron/dhcp/29a85223-6727-457c-bb13-2fff5719d4b8/addn_hosts - 1 addresses
Jan 25 13:35:10 dnsmasq-dhcp[9732]: read /var/lib/neutron/dhcp/29a85223-6727-457c-bb13-2fff5719d4b8/host
Jan 25 13:35:10 dnsmasq-dhcp[9732]: read /var/lib/neutron/dhcp/29a85223-

6727-457c-bb13-2fff5719d4b8/opts

ovs-vsctl show looks good on both nodes :-

    [root@juno1dev ~(keystone_admin)]# ovs-vsctl show
    9734c6fb-448e-4052-94e9-15dce59b43f8
        Bridge br-ex
            Port "enp2s0"
                Interface "enp2s0"
            Port phy-br-ex
                Interface phy-br-ex
                    type: patch
                    options: {peer=int-br-ex}
            Port "qg-1e009472-ca"
                Interface "qg-1e009472-ca"
                    type: internal
            Port br-ex
                Interface br-ex
                    type: internal
        Bridge br-int
            fail_mode: secure
            Port "qr-bda23703-70"
                tag: 5
                Interface "qr-bda23703-70"
                    type: internal
            Port int-br-ex
                Interface int-br-ex
                    type: patch
                    options: {peer=phy-br-ex}
            Port patch-tun
                Interface patch-tun
                    type: patch
                    options: {peer=patch-int}
            Port "tape0a57cc5-96"
                tag: 5
                Interface "tape0a57cc5-96"
                    type: internal
            Port br-int
                Interface br-int
                    type: internal
        Bridge br-tun
            Port patch-int
                Interface patch-int
                    type: patch
                    options: {peer=patch-tun}
            Port br-tun
                Interface br-tun
                    type: internal
            Port "vxlan-c0a80089"
                Interface "vxlan-c0a80089"
                    type: vxlan
                    options: {df_default="true", in_key=flow, local_ip="192.168.0.127", out_key=flow, remote_ip="192.168.0.137"}
        ovs_version: "2.1.3"
    [root@juno1dev ~(keystone_admin)]#

VNC console gives code 1006 when attempt to open CirrOS login.

Dnsmasq failure to provide private IP to VM on fresh RDO Juno install on TwoNode Controller&&Network,Compute CentOS 7 Cluster

It is ML2&OVS&VXLAN set up per http://bderzhavets.blogspot.com/2014/11/rdo-setup-two-real-node.html
Similar instances Conroller&Network and Compute on the same two boxes (different drives) , dual booting with affected ones,
work fine just now using the same VXLAN tunnel , same VXLAN endpoints enp5s1 on Controller and Compute Nodes, but not yum updated for a week or so. I will try run tcldump -vv -i enp5s1 on affected Compute instance right now

Seems to be a new issue . Never before experienced this problem
/var/log/neutron/dnsmasq.log

    Jan 25 10:47:08 dnsmasq[17647]: started, version 2.66 cachesize 150
    Jan 25 10:47:08 dnsmasq[17647]: compile time options: IPv6 GNU-getopt DBus no-i18n IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth
    Jan 25 10:47:08 dnsmasq[17647]: warning: no upstream servers configured
    Jan 25 10:47:08 dnsmasq-dhcp[17647]: DHCP, static leases only on 50.0.0.0, lease time 1d
    Jan 25 10:47:08 dnsmasq[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/addn_hosts - 2 addresses
    Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/host
    Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/opts
    Jan 25 10:47:08 dnsmasq[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/addn_hosts - 2 addresses
    Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/host
    Jan 25 10:47:08 dnsmasq-dhcp[17647]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/opts
    Jan 25 10:56:31 dnsmasq[19925]: started, version 2.66 cachesize 150
    Jan 25 10:56:31 dnsmasq[19925]: compile time options: IPv6 GNU-getopt DBus no-i18n IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth
    Jan 25 10:56:31 dnsmasq[19925]: warning: no upstream servers configured
    Jan 25 10:56:31 dnsmasq-dhcp[19925]: DHCP, static leases only on 50.0.0.0, lease time 1d
    Jan 25 10:56:31 dnsmasq[19925]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/addn_hosts - 3 addresses
    Jan 25 10:56:31 dnsmasq-dhcp[19925]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/host
    Jan 25 10:56:31 dnsmasq-dhcp[19925]: read /var/lib/neutron/dhcp/30e668d5-eb07-423d-aeb6-d408a715e9bb/opts

Create new private subnet && new router, set up gateway && interface same result

Jan 25 13:26:54 dnsmasq[9732]: started, version 2.66 cachesize 150
Jan 25 13:26:54 dnsmasq[9732]: compile time options: IPv6 GNU-getopt DBus no-i18n IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth
Jan 25 13:26:54 dnsmasq[9732]: warning: no upstream servers configured
Jan 25 13:26:54 dnsmasq-dhcp[9732]: DHCP, static leases only on 75.0.0.0, lease time 1d
Jan 25 13:26:54 dnsmasq[9732]: read /var/lib/neutron/dhcp/29a85223-6727-457c-bb13-2fff5719d4b8/addn_hosts - 2 addresses
Jan 25 13:26:54 dnsmasq-dhcp[9732]: read /var/lib/neutron/dhcp/29a85223-6727-457c-bb13-2fff5719d4b8/host
Jan 25 13:26:54 dnsmasq-dhcp[9732]: read /var/lib/neutron/dhcp/29a85223-6727-457c-bb13-2fff5719d4b8/opts
Jan 25 13:26:54 dnsmasq[9732]: read /var/lib/neutron/dhcp/29a85223-6727-457c-bb13-2fff5719d4b8/addn_hosts - 2 addresses
Jan 25 13:26:54 dnsmasq-dhcp[9732]: read /var/lib/neutron/dhcp/29a85223-6727-457c-bb13-2fff5719d4b8/host
Jan 25 13:26:54 dnsmasq-dhcp[9732]: read /var/lib/neutron/dhcp/29a85223-6727-457c-bb13-2fff5719d4b8/opts
Jan 25 13:35:10 dnsmasq-dhcp[9732]: 0 available DHCP subnet: 75.0.0.0/255.255.255.0
Jan 25 13:35:10 dnsmasq-dhcp[9732]: 0 DHCPRELEASE(tap6f93b6cb-1a) 75.0.0.10 fa:16:3e:22:36:07 unknown lease
Jan 25 13:35:10 dnsmasq[9732]: read /var/lib/neutron/dhcp/29a85223-6727-457c-bb13-2fff5719d4b8/addn_hosts - 1 addresses
Jan 25 13:35:10 dnsmasq-dhcp[9732]: read /var/lib/neutron/dhcp/29a85223-6727-457c-bb13-2fff5719d4b8/host
Jan 25 13:35:10 dnsmasq-dhcp[9732]: read /var/lib/neutron/dhcp/29a85223-

6727-457c-bb13-2fff5719d4b8/opts

ovs-vsctl show looks good on both nodes :-

    [root@juno1dev ~(keystone_admin)]# ovs-vsctl show
    9734c6fb-448e-4052-94e9-15dce59b43f8
        Bridge br-ex
            Port "enp2s0"
                Interface "enp2s0"
            Port phy-br-ex
                Interface phy-br-ex
                    type: patch
                    options: {peer=int-br-ex}
            Port "qg-1e009472-ca"
                Interface "qg-1e009472-ca"
                    type: internal
            Port br-ex
                Interface br-ex
                    type: internal
        Bridge br-int
            fail_mode: secure
            Port "qr-bda23703-70"
                tag: 5
                Interface "qr-bda23703-70"
                    type: internal
            Port int-br-ex
                Interface int-br-ex
                    type: patch
                    options: {peer=phy-br-ex}
            Port patch-tun
                Interface patch-tun
                    type: patch
                    options: {peer=patch-int}
            Port "tape0a57cc5-96"
                tag: 5
                Interface "tape0a57cc5-96"
                    type: internal
            Port br-int
                Interface br-int
                    type: internal
        Bridge br-tun
            Port patch-int
                Interface patch-int
                    type: patch
                    options: {peer=patch-tun}
            Port br-tun
                Interface br-tun
                    type: internal
            Port "vxlan-c0a80089"
                Interface "vxlan-c0a80089"
                    type: vxlan
                    options: {df_default="true", in_key=flow, local_ip="192.168.0.127", out_key=flow, remote_ip="192.168.0.137"}
        ovs_version: "2.1.3"
    [root@juno1dev ~(keystone_admin)]#

VNC console gives code 1006 when attempt to open CirrOS login.