Hello, I would like to ask about "Evacuate Failure".

Although I tried to do evacuate a instance from failed host to specified one, it was failed. The instance data is placed on shared storage(Ceph).

Environment

• Openstack: IceHouse
• Ceph: Firefly
• OS: CentOS 6.5

Procedure

Initial State:

# nova hypervisor-list
+----+-----------------------+
| ID | Hypervisor hostname   |
+----+-----------------------+
| 9  | node-28.mng.kvh.ne.jp |
| 12 | node-29.mng.kvh.ne.jp |
+----+-----------------------+

# nova list --field name,status,,OS-EXT-STS:vm_state,OS-EXT-AZ:availability_zone,OS-EXT-SRV-ATTR:hypervisor_hostname | egrep "ID|admin_vm02"
| ID                                   | Name       | Status |  | OS-EXT-STS: Vm State | OS-EXT-AZ: Availability Zone | OS-EXT-SRV-ATTR: Hypervisor Hostname |
| 496e2e97-e167-42e9-a0ce-394eb4fd72db | admin_vm02 | ACTIVE |  | active               | nova2                        | node-29.mng.kvh.ne.jp                |

# nova availability-zone-list | egrep -A 10 "^\| nova"
| nova                     | available                              |
| |- node-28.mng.kvh.ne.jp |                                        |
| | |- nova-compute        | enabled :-) 2015-02-18T04:39:15.000000 |
| nova2                    | available                              |
| |- node-29.mng.kvh.ne.jp |                                        |
| | |- nova-compute        | enabled :-) 2015-02-18T04:39:15.000000 |
+--------------------------+----------------------------------------+

Stop nova-compute(node-29):

# service openstack-nova-compute stop

# nova service-list | egrep "Host|nova-compute"
| Binary           | Host                  | Zone     | Status  | State | Updated_at                 | Disabled Reason |
| nova-compute     | node-28.mng.kvh.ne.jp | nova     | enabled | up    | 2015-02-18T05:37:55.000000 | -               |
| nova-compute     | node-29.mng.kvh.ne.jp | nova2    | enabled | down  | 2015-02-18T05:36:15.000000 | -               |

Evacuate (from node-29 to node-28):

# nova evacuate --on-shared-storage 496e2e97-e167-42e9-a0ce-394eb4fd72db node-28.mng.kvh.ne.jp

Compute Log (node-28):

<182>Feb 18 05:42:48 node-28 nova-compute 2015-02-18 14:42:48.014 7240 INFO nova.compute.resource_tracker [-] Compute_service record updated for node-28.mng.kvh.ne.jp:node-28.mng.kvh.ne.jp
<180>Feb 18 05:43:02 node-28 nova-compute 2015-02-18 14:43:02.251 7240 AUDIT nova.compute.manager [req-85f022af-cf7a-46c8-a95d-6e885055fcf7 None] [instance: 496e2e97-e167-42e9-a0ce-394eb4fd72db] Rebuilding instance
<179>Feb 18 05:43:02 node-28 nova-compute 2015-02-18 14:43:02.253 7240 ERROR nova.compute.manager [req-85f022af-cf7a-46c8-a95d-6e885055fcf7 None] [instance: 496e2e97-e167-42e9-a0ce-394eb4fd72db] Setting instance vm_state to ERROR
2015-02-18 14:43:02.253 7240 TRACE nova.compute.manager [instance: 496e2e97-e167-42e9-a0ce-394eb4fd72db] Traceback (most recent call last):
2015-02-18 14:43:02.253 7240 TRACE nova.compute.manager [instance: 496e2e97-e167-42e9-a0ce-394eb4fd72db]   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 5564, in _error_out_instance_on_exception
2015-02-18 14:43:02.253 7240 TRACE nova.compute.manager [instance: 496e2e97-e167-42e9-a0ce-394eb4fd72db]     yield
2015-02-18 14:43:02.253 7240 TRACE nova.compute.manager [instance: 496e2e97-e167-42e9-a0ce-394eb4fd72db]   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 2444, in rebuild_instance
2015-02-18 14:43:02.253 7240 TRACE nova.compute.manager [instance: 496e2e97-e167-42e9-a0ce-394eb4fd72db]     _("Invalid state of instance files on shared"
2015-02-18 14:43:02.253 7240 TRACE nova.compute.manager [instance: 496e2e97-e167-42e9-a0ce-394eb4fd72db] InvalidSharedStorage: Invalid state of instance files on shared storage
2015-02-18 14:43:02.253 7240 TRACE nova.compute.manager [instance: 496e2e97-e167-42e9-a0ce-394eb4fd72db]
<179>Feb 18 05:43:02 node-28 nova-compute 2015-02-18 14:43:02.538 7240 ERROR oslo.messaging.rpc.dispatcher [req-85f022af-cf7a-46c8-a95d-6e885055fcf7 ] Exception during message handling: Invalid state of instance files on shared storage
2015-02-18 14:43:02.538 7240 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
2015-02-18 14:43:02.538 7240 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.6/site-packages/oslo/messaging/rpc/dispatcher.py", line 137, in _dispatch_and_reply
2015-02-18 14:43:02.538 7240 TRACE oslo.messaging.rpc.dispatcher     incoming.message))
2015-02-18 ...