Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Instance creation fails (Block Device Mapping)

I'm using Openstack 1.0.1 on Centos 7 64bits, with 3 controllers, 2 compute nodes and storage mounted through nfs.

On Horizon i'm trying to launch a new instance (Boot from image - creates a new volume), and i keep getting the error:

Error: Failed to launch instance "windows_test2": Please try again later [Error: Build of instance e541f467-9a49-4fbf-b8ae-a792c389aa98 aborted: Failure prepping block device.].

  1. The new volume is created and it's OK
  2. An IP is assigned to the instance.
  3. Instance creation stops and shows status ERROR

The status displays Block device mapping for a long time, until it stops and gives the error presented above.

I've tried launching the instance through the command line but the error remains.

On the controller node cinder logs (volume and api) reports no error (logs are with debug=True).

On the compute node nova-compute.log shows:

2015-03-10 16:51:21.938 72172 ERROR nova.compute.manager [-] [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98] Instance failed block device setup
2015-03-10 16:51:21.938 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98] Traceback (most recent call last):
2015-03-10 16:51:21.938 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1819, in _prep_block_device
2015-03-10 16:51:21.938 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]     do_check_attach=do_check_attach) +
2015-03-10 16:51:21.938 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]   File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 407, in attach_block_devices
2015-03-10 16:51:21.938 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]     map(_log_and_attach, block_device_mapping)
2015-03-10 16:51:21.938 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]   File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 405, in _log_and_attach
2015-03-10 16:51:21.938 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]     bdm.attach(*attach_args, **attach_kwargs)
2015-03-10 16:51:21.938 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]   File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 333, in attach
2015-03-10 16:51:21.938 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]     wait_func(context, vol['id'])
2015-03-10 16:51:21.938 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1263, in _await_block_device_map_created
2015-03-10 16:51:21.938 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]     attempts=attempts)
2015-03-10 16:51:21.938 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98] VolumeNotCreated: Volume 31d43244-7f63-4088-82de-6a521c384404 did not finish being created even after we waited 254 seconds or 61 attempts.
2015-03-10 16:51:21.938 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98] 
2015-03-10 16:51:21.939 72172 ERROR nova.compute.manager [-] [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98] Failure prepping block device
2015-03-10 16:51:21.939 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98] Traceback (most recent call last):
2015-03-10 16:51:21.939 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2218, in _build_resources
2015-03-10 16:51:21.939 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]     block_device_mapping)
2015-03-10 16:51:21.939 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1847, in _prep_block_device
2015-03-10 16:51:21.939 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]     raise exception.InvalidBDM()
2015-03-10 16:51:21.939 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98] InvalidBDM: Block Device Mapping is Invalid.
2015-03-10 16:51:21.939 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98] 
2015-03-10 16:51:21.939 72172 DEBUG nova.compute.claims [-] [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98] Aborting claim: [Claim: 2048 MB memory, 20 GB disk] abort /usr/lib/python2.7/site-packages/nova/compute/claims.py:128

Also relevant

2015-03-10 16:51:21.982 72172 ERROR nova.compute.manager [-] [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98] Build of instance e541f467-9a49-4fbf-b8ae-a792c389aa98 aborted: Failure prepping block device.
2015-03-10 16:51:21.982 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98] Traceback (most recent call last):
2015-03-10 16:51:21.982 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2030, in _do_build_and_run_instance
2015-03-10 16:51:21.982 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]     filter_properties)
2015-03-10 16:51:21.982 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2129, in _build_and_run_instance
2015-03-10 16:51:21.982 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]     'create.error', fault=e)
2015-03-10 16:51:21.982 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]   File "/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line 82, in __exit__
2015-03-10 16:51:21.982 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]     six.reraise(self.type_, self.value, self.tb)
2015-03-10 16:51:21.982 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2102, in _build_and_run_instance
2015-03-10 16:51:21.982 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]     block_device_mapping) as resources:
2015-03-10 16:51:21.982 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]   File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
2015-03-10 16:51:21.982 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]     return self.gen.next()
2015-03-10 16:51:21.982 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2240, in _build_resources
2015-03-10 16:51:21.982 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98]     reason=msg)
2015-03-10 16:51:21.982 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98] BuildAbortException: Build of instance e541f467-9a49-4fbf-b8ae-a792c389aa98 aborted: Failure prepping block device.
2015-03-10 16:51:21.982 72172 TRACE nova.compute.manager [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98] 
2015-03-10 16:51:21.982 72172 DEBUG nova.compute.manager [-] [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98] Deallocating network for instance _deallocate_network /usr/lib/python2.7/site-packages/nova/compute/manager.py:1916

And

2015-03-10 16:51:22.087 72172 DEBUG neutronclient.client [-] 
REQ: curl -i http://192.168.253.117:9696/v2.0/ports/None.json -X PUT -H "X-Auth-Token: 58edf80bb7b84eb6bff279a3f3289524" -H "User-Agent: python-neutronclient" -d '{"port": {"device_owner": "", "device_id": ""}}'
 http_log_req /usr/lib/python2.7/site-packages/neutronclient/common/utils.py:140
2015-03-10 16:51:22.095 72172 DEBUG neutronclient.client [-] RESP:404 {'date': 'Tue, 10 Mar 2015 16:51:22 GMT', 'content-length': '52', 'content-type': 'text/plain; charset=UTF-8', 'x-openstack-request-id': 'req-262fa0eb-cded-481d-9d5b-958034be845f'} 404 Not Found

The resource could not be found.


 http_log_resp /usr/lib/python2.7/site-packages/neutronclient/common/utils.py:149
2015-03-10 16:51:22.095 72172 DEBUG neutronclient.v2_0.client [-] Error message: 404 Not Found

The resource could not be found.

    _handle_fault_response /usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py:1207
2015-03-10 16:51:22.096 72172 INFO nova.network.neutronv2.api [-] [instance: e541f467-9a49-4fbf-b8ae-a792c389aa98] Unable to reset device ID for port None
2015-03-10 16:51:22.096 72172 DEBUG neutronclient.client [-] 
REQ: curl -i http://192.168.253.117:9696/v2.0/ports/cc708f5c-e1cd-4864-9937-ef7fb0b37429.json -X DELETE -H "X-Auth-Token: 58edf80bb7b84eb6bff279a3f3289524" -H "User-Agent: python-neutronclient"

If a launch an instance with option Boot from Image or Boot from Volume, it works.