Revision history [back]

click to hide/show revision 1
initial version

UnsupportedVersion: Endpoint does not support RPC version

In the process of a Kilo to Liberty Upgrade and cannot live migrate VMs between Kilo and Liberty compute hosts. The Control nodes and a single Compute node have been upgraded to Liberty. We get the following error are in compute nodes log when we attempt to live migrate a instance from the Kilo node to Liberty;

Traceback (most recent call last):
   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 5314, in live_migration
     block_migration, disk, dest, migrate_data)
   File "/usr/lib/python2.7/site-packages/nova/compute/rpcapi.py", line 623, in pre_live_migration
     disk=disk, migrate_data=migrate_data)
   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 156, in call
    retry=self.retry)
   File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 90, in _send
    timeout=timeout, retry=retry)
   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 350, in send
     retry=retry)
   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 341, in _send
     raise result
 RemoteError: Remote error: UnsupportedVersion Endpoint does not support RPC version 3.19. Attempted met
hod: pre_live_migration



Traceback (most recent call last):
   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
     executor_callback))
   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
     executor_callback)
   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch
     result = func(ctxt, **new_args)
   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 6810, in live_migration
     migrate_data=migrate_data)
   File "/usr/lib/python2.7/site-packages/nova/exception.py", line 88, in wrapped
     payload)
   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
     six.reraise(self.type_, self.value, self.tb)
   File "/usr/lib/python2.7/site-packages/nova/exception.py", line 71, in wrapped
     return f(self, context, *args, **kw)
   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 361, in decorated_function
     kwargs['instance'], e, sys.exc_info())
   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
     six.reraise(self.type_, self.value, self.tb)
   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 349, in decorated_function
     return function(self, context, *args, **kwargs)
   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 5322, in live_migration
     block_migration, migrate_data)
   File "/usr/lib/python2.7/site-packages/nova/exception.py", line 88, in wrapped
     payload)
   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
     six.reraise(self.type_, self.value, self.tb)
   File "/usr/lib/python2.7/site-packages/nova/exception.py", line 71, in wrapped
     return f(self, context, *args, **kw)
   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 361, in decorated_function
     kwargs['instance'], e, sys.exc_info())
   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
     six.reraise(self.type_, self.value, self.tb)
   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 349, in decorated_function
     return function(self, context, *args, **kwargs)
   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 5564, in _rollback_live_migration
     context, instance, bdm.volume_id, dest)
   File "/usr/lib/python2.7/site-packages/nova/compute/rpcapi.py", line 715, in remove_volume_connection
     instance=instance, volume_id=volume_id)
   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 156, in call
     retry=self.retry)
   File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 90, in _send
     timeout=timeout, retry=retry)
   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 350, in send
     retry=retry)
   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 341, in _send
     raise result
 RemoteError: Remote error: UnsupportedVersion Endpoint does not support RPC version 3.30. Attempted method: remove_volume_connection

Upgrade level has been set to "kilo" on all Nova Control and Compute nodes in /etc/nova/nova.conf;

[upgrade_levels]
compute = kilo

Live migration worked correctly with Liberty Control nodes and Kilo Compute nodes. Instances can be booted and run on both Liberty and Kilo Compute nodes. Any idea whats going on here?