Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Neutron releases 3 IPs after minor update in Mitaka

Hi, I just did an minor update on Mitaka to "8.3.0-0ubuntu1.1~cloud0" on Ubuntu 14.04. During the upgrade nothing special happened. Only rabbitmq was upgraded to 3.6.6 (which caused an upgrade in Rabbitmq), but this is working fine.

The problem that I have now that new VMs get three IPs assigned when booting. Why? Good question? The VM has only one interface and one MAC. The associated mac address gets a ip address and is reachable over this. The two other IPs have different MAC address, but they are still assigned to the VM.

E.g:

root@gc-controller01:/var/lib/neutron/dhcp/14594365-3c61-4228-a70f-68cc37b67f84# grep 10.10.105.149 *
addn_hosts:10.10.105.149    host-10-10-105-149.openstacklocal host-10-10-105-149
host:fa:16:3e:59:14:ac,host-10-10-105-149.openstacklocal,10.10.105.149
root@gc-controller01:/var/lib/neutron/dhcp/14594365-3c61-4228-a70f-68cc37b67f84# grep 10.10.105.148 *
addn_hosts:10.10.105.148    host-10-10-105-148.openstacklocal host-10-10-105-148
host:fa:16:3e:76:91:13,host-10-10-105-148.openstacklocal,10.10.105.148
root@gc-controller01:/var/lib/neutron/dhcp/14594365-3c61-4228-a70f-68cc37b67f84# grep 10.10.105.147 *
addn_hosts:10.10.105.147    host-10-10-105-147.openstacklocal host-10-10-105-147
host:fa:16:3e:0e:8b:5e,host-10-10-105-147.openstacklocal,10.10.105.147
root@gc-controller01:/var/lib/neutron/dhcp/14594365-3c61-4228-a70f-68cc37b67f84#

Within the VM:

Jan  2 14:07:22 monero-1 dhclient[954]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 3 (xid=0xf6ca6917)
Jan  2 14:07:22 monero-1 dhclient[954]: DHCPREQUEST of 10.10.105.149 on eth0 to 255.255.255.255 port 67 (xid=0x1769caf6)
Jan  2 14:07:22 monero-1 sh[939]: DHCPREQUEST of 10.10.105.149 on eth0 to 255.255.255.255 port 67 (xid=0x1769caf6)
Jan  2 14:07:22 monero-1 sh[939]: DHCPOFFER of 10.10.105.149 from 10.10.104.11
Jan  2 14:07:22 monero-1 dhclient[954]: DHCPOFFER of 10.10.105.149 from 10.10.104.11
Jan  2 14:07:22 monero-1 dhclient[954]: DHCPACK of 10.10.105.149 from 10.10.104.11
Jan  2 14:07:22 monero-1 sh[939]: DHCPACK of 10.10.105.149 from 10.10.104.11
Jan  2 14:07:22 monero-1 dhclient[954]: bound to 10.10.105.149 -- renewal in 155961 seconds.
Jan  2 14:07:22 monero-1 sh[939]: bound to 10.10.105.149 -- renewal in 155961 seconds.

Any idea why this is happening? Rabbit looks good, the VM is running and reachable. Nothing suspicious in the logs except this, but I doubt that this has something to do with the problem:

2017-01-02 14:55:58.244 12136 ERROR nova.scheduler.utils [req-bc03ee37-35b0-45f1-97c8-972ef8a36b20 c38bf6f97e134c26aba5041afcd17a68 b33954142af1438eadbd159b3801db7f - - -] [instance: 93cb46fa-4585-4090-9565-cb41f65a5937] Error from last host: gc-hv04.example.com (node gc-hv04.example.com): [u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1926, in _do_build_and_run_instance\n    filter_properties)\n', u'  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2116, in _build_and_run_instance\n    instance_uuid=instance.uuid, reason=six.text_type(e))\n', u"RescheduledException: Build of instance 93cb46fa-4585-4090-9565-cb41f65a5937 was re-scheduled: internal error: Process exited prior to exec: libvirt:  error : unable to set AppArmor profile 'libvirt-93cb46fa-4585-4090-9565-cb41f65a5937' for '/usr/bin/qemu-system-x86_64': No such file or directory\n"]

I removed apparmor on the hypervisor and didn't restart qemu yet.

ANY idea or hin?

Thanks

bert

Neutron releases 3 IPs after minor update in Mitaka

Hi, I just did an minor update on Mitaka to "8.3.0-0ubuntu1.1~cloud0" on Ubuntu 14.04. During the upgrade nothing special happened. Only rabbitmq was upgraded to 3.6.6 (which caused an upgrade in Rabbitmq), but this is working fine.

The problem that I have now that new VMs get three IPs assigned when booting. Why? Good question? The VM has only one interface and one MAC. The associated mac address gets a ip address and is reachable over this. The two other IPs have different MAC address, but they are still assigned to the VM.

E.g:

root@gc-controller01:/var/lib/neutron/dhcp/14594365-3c61-4228-a70f-68cc37b67f84# grep 10.10.105.149 *
addn_hosts:10.10.105.149    host-10-10-105-149.openstacklocal host-10-10-105-149
host:fa:16:3e:59:14:ac,host-10-10-105-149.openstacklocal,10.10.105.149
host:fa:16:3e:59:14:ac,host-10-10-105-149.openstacklocal,10.10.105.149  <-------- this is the real MAC
root@gc-controller01:/var/lib/neutron/dhcp/14594365-3c61-4228-a70f-68cc37b67f84# grep 10.10.105.148 *
addn_hosts:10.10.105.148    host-10-10-105-148.openstacklocal host-10-10-105-148
host:fa:16:3e:76:91:13,host-10-10-105-148.openstacklocal,10.10.105.148
root@gc-controller01:/var/lib/neutron/dhcp/14594365-3c61-4228-a70f-68cc37b67f84# grep 10.10.105.147 *
addn_hosts:10.10.105.147    host-10-10-105-147.openstacklocal host-10-10-105-147
host:fa:16:3e:0e:8b:5e,host-10-10-105-147.openstacklocal,10.10.105.147
root@gc-controller01:/var/lib/neutron/dhcp/14594365-3c61-4228-a70f-68cc37b67f84#

Within the VM:

Jan  2 14:07:22 monero-1 dhclient[954]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 3 (xid=0xf6ca6917)
Jan  2 14:07:22 monero-1 dhclient[954]: DHCPREQUEST of 10.10.105.149 on eth0 to 255.255.255.255 port 67 (xid=0x1769caf6)
Jan  2 14:07:22 monero-1 sh[939]: DHCPREQUEST of 10.10.105.149 on eth0 to 255.255.255.255 port 67 (xid=0x1769caf6)
Jan  2 14:07:22 monero-1 sh[939]: DHCPOFFER of 10.10.105.149 from 10.10.104.11
Jan  2 14:07:22 monero-1 dhclient[954]: DHCPOFFER of 10.10.105.149 from 10.10.104.11
Jan  2 14:07:22 monero-1 dhclient[954]: DHCPACK of 10.10.105.149 from 10.10.104.11
Jan  2 14:07:22 monero-1 sh[939]: DHCPACK of 10.10.105.149 from 10.10.104.11
Jan  2 14:07:22 monero-1 dhclient[954]: bound to 10.10.105.149 -- renewal in 155961 seconds.
Jan  2 14:07:22 monero-1 sh[939]: bound to 10.10.105.149 -- renewal in 155961 seconds.

Any idea why this is happening? Rabbit looks good, the VM is running and reachable. Nothing suspicious in the logs except this, but I doubt that this has something to do with the problem:

2017-01-02 14:55:58.244 12136 ERROR nova.scheduler.utils [req-bc03ee37-35b0-45f1-97c8-972ef8a36b20 c38bf6f97e134c26aba5041afcd17a68 b33954142af1438eadbd159b3801db7f - - -] [instance: 93cb46fa-4585-4090-9565-cb41f65a5937] Error from last host: gc-hv04.example.com (node gc-hv04.example.com): [u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1926, in _do_build_and_run_instance\n    filter_properties)\n', u'  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2116, in _build_and_run_instance\n    instance_uuid=instance.uuid, reason=six.text_type(e))\n', u"RescheduledException: Build of instance 93cb46fa-4585-4090-9565-cb41f65a5937 was re-scheduled: internal error: Process exited prior to exec: libvirt:  error : unable to set AppArmor profile 'libvirt-93cb46fa-4585-4090-9565-cb41f65a5937' for '/usr/bin/qemu-system-x86_64': No such file or directory\n"]

I removed apparmor on the hypervisor and didn't restart qemu yet.

ANY idea or hin?

Thanks

bert