openstack volume migrate stuck

asked 2018-09-28 08:49:46 -0500

schegi gravatar image

updated 2018-09-28 09:02:08 -0500

Hi,

i am currently trying to debug a strange problem in our openstack deployment. When issuing a 'openstack volume migrate --host <host> <volume id=""> or the equivalent cinder command cinder migrate <volume id=""> <destination host="">, to migrate an attached volume. Everything works fine, only the migration process never completes. A new volume is created, the content of the old volume is copied to the new one and the volume attachment of the Instance is swapped from the old to the new volume. But then it stops, the old volume is never deleted and both volume stay in migration state. The old volume id is 07f12bf7-f32b-4680-8d11-04616476f3e8 and the new one is c4c6e07a-5b26-46c1-8d09-d3c7af5730e0. This is the state in which the volumes are stuck after issuing a migration. I am aware that migration might take time but it never finishes and all volumes i tested the process with are just 1G volumes.

Old Volume:

| id | 07f12bf7-f32b-4680-8d11-04616476f3e8

| migration_status | migrating

| name | mig-test-vol1

| os-vol-host-attr:host | server1@iscsi0#LVM_iSCSI |

| os-vol-mig-status-attr:migstat | migrating

| os-vol-mig-status-attr:name_id | None

| os-vol-tenant-attr:tenant_id | edc68db95ed04818bfd9dbaad8f4553d

New Volume:

| id | c4c6e07a-5b26-46c1-8d09-d3c7af5730e0

| migration_status | target:07f12bf7-f32b-4680-8d11-04616476f3e8

| name | mig-test-vol1

| os-vol-host-attr:host | server2@iscsi0#LVM_iSCSI

| os-vol-mig-status-attr:migstat | target:07f12bf7-f32b-4680-8d11-04616476f3e8

| os-vol-mig-status-attr:name_id | None

| os-vol-tenant-attr:tenant_id | edc68db95ed04818bfd9dbaad8f4553d

I am able to finish the process on my own, but reseting the migration status with 'cinder reset-state --reset-migration-status <volume_id> and deleting the old volume, the Instance works fine with the newly created volume.

Anybody encountered a similar issue or any advices?

Stefan

edit retag flag offensive close merge delete

Comments

Have you(are you able) to start your cinder service with debug enabled, to see if there are any low-level complaints? I cant see which version you are on?

theque42 gravatar imagetheque42 ( 2018-10-02 00:53:15 -0500 )edit

I too faced the same issue. I am running OSP-12 in my environment. Had to try the "cinder reset-state --reset-migration-state <vol-id>" and then tried the migration again and that worked fine. Not sure if its a bug or high Latency issues. If any long term fix available please do share. Thanks

ramp85 gravatar imageramp85 ( 2019-03-28 00:05:33 -0500 )edit