Why is cinder volume migration failing?
The context is that I have an RDO openstack Kilo cluster running on Centos 7 composed of one controller and five compute nodes. Each of the compute nodes is also running cinder-volume and manages ~2Tb of space. One node has a disk which is behaving as if failure is imminent. I am trying to migrate volumes off of this node to other nodes.
There is absolutely no output from the cinder migrate command.
The volume.log on the destination node contains the following. The top hit when googling the warning message is the python source code. I assume that means this is not a showstopper, but in any case google did not return anything I understood as a description of the problem or indicators of possible solutions. As far as I can tell, cinder takes about 20 seconds to change its mind from "creating a new volume on the target" to "deleting". I don't understand the reason for the change. Can you help me understand whether the warning is relevant or really what the first step in debugging this problem is?
2015-11-03 16:45:35.614 2259 INFO cinder.volume.flows.manager.create_volume [req-9407ea97-a0a3-4c6d-b72c-4f0edbf19d75 530392db20334fb49460c40facf9f6b1 0ed69c3d01094c18b8c0eea04060e916 - - -] Volume ead174e4-e7d0-45e9-804d-47b915a48492: being created as raw with specification: {'status': u'creating', 'volume_size': 200, 'volume_name': u'volume-ead174e4-e7d0-45e9-804d-47b915a48492'}
2015-11-03 16:45:36.090 2259 INFO cinder.volume.flows.manager.create_volume [req-9407ea97-a0a3-4c6d-b72c-4f0edbf19d75 530392db20334fb49460c40facf9f6b1 0ed69c3d01094c18b8c0eea04060e916 - - -] Volume volume-ead174e4-e7d0-45e9-804d-47b915a48492 (ead174e4-e7d0-45e9-804d-47b915a48492): created successfully
2015-11-03 16:45:43.154 2259 INFO cinder.volume.targets.lio [req-9407ea97-a0a3-4c6d-b72c-4f0edbf19d75 530392db20334fb49460c40facf9f6b1 0ed69c3d01094c18b8c0eea04060e916 - - -] Creating iscsi_target for volume: volume-ead174e4-e7d0-45e9-804d-47b915a48492
2015-11-03 16:45:43.611 2259 WARNING cinder.volume.targets.lio [req-9407ea97-a0a3-4c6d-b72c-4f0edbf19d75 530392db20334fb49460c40facf9f6b1 0ed69c3d01094c18b8c0eea04060e916 - - -] Failed to save iscsi LIO configuration when modifying volume id: volume-ead174e4-e7d0-45e9-804d-47b915a48492.
2015-11-03 16:45:43.963 2259 WARNING cinder.volume.targets.lio [req-9407ea97-a0a3-4c6d-b72c-4f0edbf19d75 530392db20334fb49460c40facf9f6b1 0ed69c3d01094c18b8c0eea04060e916 - - -] Failed to save iscsi LIO configuration when modifying volume id: ead174e4-e7d0-45e9-804d-47b915a48492.
2015-11-03 16:46:07.817 2259 WARNING cinder.volume.targets.lio [req-9407ea97-a0a3-4c6d-b72c-4f0edbf19d75 530392db20334fb49460c40facf9f6b1 0ed69c3d01094c18b8c0eea04060e916 - - -] Failed to save iscsi LIO configuration when modifying volume id: ead174e4-e7d0-45e9-804d-47b915a48492.
2015-11-03 16:46:08.087 2259 INFO cinder.volume.manager [req-9407ea97-a0a3-4c6d-b72c-4f0edbf19d75 530392db20334fb49460c40facf9f6b1 0ed69c3d01094c18b8c0eea04060e916 - - -] volume ead174e4-e7d0-45e9-804d-47b915a48492: deleting
2015-11-03 16:46:08.234 2259 INFO cinder.volume.targets.lio [req-9407ea97-a0a3-4c6d-b72c-4f0edbf19d75 530392db20334fb49460c40facf9f6b1 0ed69c3d01094c18b8c0eea04060e916 - - -] Removing iscsi_target: ead174e4-e7d0-45e9-804d-47b915a48492
2015-11-03 16:46:08.537 2259 WARNING cinder.volume.targets.lio [req-9407ea97-a0a3-4c6d-b72c-4f0edbf19d75 530392db20334fb49460c40facf9f6b1 0ed69c3d01094c18b8c0eea04060e916 - - -] Failed to save iscsi LIO configuration when modifying volume id: ead174e4-e7d0-45e9-804d-47b915a48492.
2015-11-03 16:46:08.733 2259 INFO cinder.volume.utils [req-9407ea97-a0a3-4c6d-b72c-4f0edbf19d75 530392db20334fb49460c40facf9f6b1 0ed69c3d01094c18b8c0eea04060e916 - - -] Performing secure delete on volume: /dev/mapper/cinder--volumes-volume--ead174e4--e7d0--45e9--804d--47b915a48492