live migration failing with "Unexpected API Error"

asked 2017-08-06 17:07:19 -0500

pacman gravatar image

updated 2017-08-08 09:10:10 -0500

Hi

I installed openstack according to the instructions in the official getting started guide. The Stack is working mostly except for some hickups. One of them are failing live migrations.

Everytime I try to live migrate an instance it fails with "Unexpected API Error". Here is my /var/log/nova/nova-api.log:

2017-08-06 23:27:21.011 9550 ERROR nova.api.openstack.extensions [req-585db0ac-b3ef-4e07-afb5-914ad9005ae5 
735e4dd64a0e451bbb5254950af6e6ad f9873284dd3b40edbf9564703c0903ad - default default] Unexpected exception in API method
2017-08-06 23:27:21.011 9550 ERROR nova.api.openstack.extensions Traceback (most recent call last):
2017-08-06 23:27:21.011 9550 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/openstack/extensions.py", line 338, in wrapped
2017-08-06 23:27:21.011 9550 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)
2017-08-06 23:27:21.011 9550 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 108, in wrapper
2017-08-06 23:27:21.011 9550 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)
2017-08-06 23:27:21.011 9550 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 108, in wrapper
2017-08-06 23:27:21.011 9550 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)
2017-08-06 23:27:21.011 9550 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 108, in wrapper
2017-08-06 23:27:21.011 9550 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)
2017-08-06 23:27:21.011 9550 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/migrate_server.py", line 100, in _migrate_live
2017-08-06 23:27:21.011 9550 ERROR nova.api.openstack.extensions     disk_over_commit, host, force, async)
2017-08-06 23:27:21.011 9550 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 170, in inner
2017-08-06 23:27:21.011 9550 ERROR nova.api.openstack.extensions     return function(self, context, instance, *args, **kwargs)
2017-08-06 23:27:21.011 9550 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 178, in _wrapped
2017-08-06 23:27:21.011 9550 ERROR nova.api.openstack.extensions     return fn(self, context, instance, *args, **kwargs)
2017-08-06 23:27:21.011 9550 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 151, in inner
2017-08-06 23:27:21.011 9550 ERROR nova.api.openstack.extensions     return f(self, context, instance, *args, **kw)
2017-08-06 23:27:21.011 9550 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 3903, in live_migrate
2017-08-06 23:27:21.011 9550 ERROR nova.api.openstack.extensions     messaging_timeout)
2017-08-06 23:27:21.011 9550 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-08-06 23:27:21.011 9550 ERROR nova.api.openstack.extensions     self.force_reraise()
2017-08-06 23 ...
(more)
edit retag flag offensive close merge delete

Comments

Hi, how many compute nodes you have? what are the configuration changes you did to setup vm live migration ? BTW check the status of firewall/ iptables. share the output of cat /etc/hosts and getenforce commands

Praveen N gravatar imagePraveen N ( 2017-08-08 04:34:16 -0500 )edit

Hi, i got 5 absolutely identical compute nodes. honestly at first i did not do any modifications - i just expected it to work when following the official setup.

pacman gravatar imagepacman ( 2017-08-08 09:06:15 -0500 )edit

meanwhile i found some older manuals (2013) stating that some libvirt settings have to be adjusted: https://kimizhang.wordpress.com/2013/08/26/openstack-vm-live-migration/ (https://kimizhang.wordpress.com/2013/...) however i did not try it yet since i found out that neither cold migration nor resize are working as well (working on that now)

pacman gravatar imagepacman ( 2017-08-08 09:07:30 -0500 )edit

i am just about to reinstall the cluster but i can say for sure that selinux is off - i added the hosts file in my original post as the comment size is to limited

pacman gravatar imagepacman ( 2017-08-08 09:10:40 -0500 )edit

Have you find a way to solve this problem? I'm facing the exact same issue!

angelvargas gravatar imageangelvargas ( 2019-07-01 08:11:32 -0500 )edit