Ask Your Question
0

Pre live migration failed with "oslo_messaging.rpc.dispatcher MessagingTimeout: Timed out waiting for a reply to message ID ef3a0ab639804d38a7791282bc819ac0" [closed]

asked 2015-09-08 07:32:33 -0500

jamieliu gravatar image
Please kindly help to look into this blocking issue during live migration. The log is from nova-compute.log.
Kilo, oslo.messaging (1.8.3), RabbitMQ(3.3.5)

 2015-09-08 19:23:12.460 27618 WARNING nova.virt.libvirt.driver [req-cb326d29-206f-4008-94eb-4ba1b14ab631 32f6c0cfa5574f8b85739f6dbc00983d 247eb626619b483b9f6d1750b1a8b7cb - - -] [instance: a8007dc6-32e2-48cc-9f48-4780c88abd73] File injection into a boot from volume instance is not supported
    2015-09-08 19:23:29.503 27618 WARNING nova.virt.libvirt.driver [req-cb326d29-206f-4008-94eb-4ba1b14ab631 32f6c0cfa5574f8b85739f6dbc00983d 247eb626619b483b9f6d1750b1a8b7cb - - -] Ignoring error check os_type image (Error mounting /mnt/testpool/instances/a8007dc6-32e2-48cc-9f48-4780c88abd73/disk with libguestfs (/mnt/testpool/instances/a8007dc6-32e2-48cc-9f48-4780c88abd73/disk: No such file or directory))
    2015-09-08 19:23:29.516 27618 WARNING nova.virt.libvirt.driver [req-cb326d29-206f-4008-94eb-4ba1b14ab631 32f6c0cfa5574f8b85739f6dbc00983d 247eb626619b483b9f6d1750b1a8b7cb - - -] [instance: a8007dc6-32e2-48cc-9f48-4780c88abd73] Image /mnt/testpool/instances/a8007dc6-32e2-48cc-9f48-4780c88abd73/disk not found on disk storage. Continue without injecting data
    **2015-09-08 19:27:26.667 27618 ERROR nova.compute.manager [req-f78da5cf-535f-46a2-aed4-437f3b71897d 32f6c0cfa5574f8b85739f6dbc00983d 247eb626619b483b9f6d1750b1a8b7cb - - -] [instance: a8007dc6-32e2-48cc-9f48-4780c88abd73] Pre live migration failed at 10.221.72.106**
    2015-09-08 19:27:26.667 27618 TRACE nova.compute.manager [instance: a8007dc6-32e2-48cc-9f48-4780c88abd73] Traceback (most recent call last):
    2015-09-08 19:27:26.667 27618 TRACE nova.compute.manager [instance: a8007dc6-32e2-48cc-9f48-4780c88abd73]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 5276, in live_migration
    2015-09-08 19:27:26.667 27618 TRACE nova.compute.manager [instance: a8007dc6-32e2-48cc-9f48-4780c88abd73]     block_migration, disk, dest, migrate_data)
    2015-09-08 19:27:26.667 27618 TRACE nova.compute.manager [instance: a8007dc6-32e2-48cc-9f48-4780c88abd73]   File "/usr/lib/python2.7/site-packages/nova/compute/rpcapi.py", line 623, in pre_live_migration
    2015-09-08 19:27:26.667 27618 TRACE nova.compute.manager [instance: a8007dc6-32e2-48cc-9f48-4780c88abd73]     disk=disk, migrate_data=migrate_data)
    2015-09-08 19:27:26.667 27618 TRACE nova.compute.manager [instance: a8007dc6-32e2-48cc-9f48-4780c88abd73]   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 156, in call
    2015-09-08 19:27:26.667 27618 TRACE nova.compute.manager [instance: a8007dc6-32e2-48cc-9f48-4780c88abd73]     retry=self.retry)
    2015-09-08 19:27:26.667 27618 TRACE nova.compute.manager [instance: a8007dc6-32e2-48cc-9f48-4780c88abd73]   File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 90, in _send
    2015-09-08 19:27:26.667 27618 TRACE nova.compute.manager [instance: a8007dc6-32e2-48cc-9f48-4780c88abd73]     timeout=timeout, retry=retry)
    2015-09-08 19:27:26.667 27618 TRACE nova.compute.manager [instance: a8007dc6-32e2-48cc-9f48-4780c88abd73]   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 350, in send
    2015-09-08 19:27:26.667 27618 TRACE nova.compute.manager [instance: a8007dc6-32e2-48cc-9f48-4780c88abd73]     retry=retry)
    2015-09-08 19:27:26.667 27618 TRACE nova.compute.manager [instance: a8007dc6-32e2-48cc-9f48-4780c88abd73]   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 339, in _send
    2015-09-08 19:27:26.667 27618 TRACE nova.compute.manager [instance: a8007dc6-32e2-48cc-9f48-4780c88abd73]     result = self._waiter.wait(msg_id, timeout)
    2015-09-08 19:27:26.667 27618 TRACE nova.compute.manager [instance: a8007dc6-32e2-48cc-9f48-4780c88abd73]   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 243, in wait
    2015-09-08 19:27:26.667 27618 TRACE nova.compute.manager [instance: a8007dc6-32e2-48cc-9f48-4780c88abd73]     message = self.waiters.get(msg_id, timeout=timeout)
    2015-09-08 19:27:26.667 27618 TRACE nova.compute.manager [instance: a8007dc6-32e2-48cc-9f48-4780c88abd73]   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 149, in get
    2015-09-08 19:27:26.667 27618 TRACE nova.compute.manager [instance: a8007dc6-32e2-48cc-9f48-4780c88abd73]     'to message ...
(more)
edit retag flag offensive reopen merge delete

Closed for the following reason the question is answered, right answer was accepted by jamieliu
close date 2015-09-08 22:46:26.682171

1 answer

Sort by ยป oldest newest most voted
0

answered 2015-09-08 22:46:03 -0500

jamieliu gravatar image

It's because of the nova-network doesn't consume the message and only one node has nova-network installed. After fix the problem in nova-network and scale out the nova-network deployment, the live migration is working now.

edit flag offensive delete link more

Get to know Ask OpenStack

Resources for moderators

Question Tools

Stats

Asked: 2015-09-08 07:32:33 -0500

Seen: 756 times

Last updated: Sep 08 '15