Instance stuck in build status

asked 2016-05-25 05:10:06 -0600

bertly gravatar image

Hi, just building a openstack environment with liberty on ubuntu 14.04 and ceph. Creating a volume and uploading a image with glance/cinder is working without any problem, but I cant start any instance. It get stuck in the Build status. hte problem is, I dont see any errors at all in the logs (debug enabled)

nova list

+--------------------------------------+------+--------+------------+-------------+----------+
| ID                                   | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------+--------+------------+-------------+----------+
| e4cf8f29-9a20-4171-b80f-d316cd6deec0 | 3    | BUILD  | -          | NOSTATE     |          |
+--------------------------------------+------+--------+------------+-------------+----------+

compute node logs for compute-node

grep "e4cf8f29-9a20-4171-b80f-d316cd6deec0" /var/log/nova/nova-compute.log
2016-05-25 10:50:18.204 17078 DEBUG oslo_concurrency.lockutils [req-7211746b-c2be-469e-acd4-70e6293f2c0a f9e96bf3296e49e3b68637c5e7419559 72d3e55789444e98acb42a0cf267c449 - - -] Lock "e4cf8f29-9a20-4171-b80f-d316cd6deec0" acquired by "nova.compute.manager._locked_do_build_and_run_instance" :: waited 0.000s inner /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:253
2016-05-25 10:50:18.251 17078 INFO nova.compute.manager [req-7211746b-c2be-469e-acd4-70e6293f2c0a f9e96bf3296e49e3b68637c5e7419559 72d3e55789444e98acb42a0cf267c449 - - -] [instance: e4cf8f29-9a20-4171-b80f-d316cd6deec0] Starting instance...
2016-05-25 10:50:18.489 17078 INFO nova.compute.claims [req-7211746b-c2be-469e-acd4-70e6293f2c0a f9e96bf3296e49e3b68637c5e7419559 72d3e55789444e98acb42a0cf267c449 - - -] [instance: e4cf8f29-9a20-4171-b80f-d316cd6deec0] Attempting claim: memory 512 MB, disk 1 GB
2016-05-25 10:50:18.490 17078 INFO nova.compute.claims [req-7211746b-c2be-469e-acd4-70e6293f2c0a f9e96bf3296e49e3b68637c5e7419559 72d3e55789444e98acb42a0cf267c449 - - -] [instance: e4cf8f29-9a20-4171-b80f-d316cd6deec0] Total memory: 15934 MB, used: 1024.00 MB
2016-05-25 10:50:18.491 17078 INFO nova.compute.claims [req-7211746b-c2be-469e-acd4-70e6293f2c0a f9e96bf3296e49e3b68637c5e7419559 72d3e55789444e98acb42a0cf267c449 - - -] [instance: e4cf8f29-9a20-4171-b80f-d316cd6deec0] memory limit: 23901.00 MB, free: 22877.00 MB
2016-05-25 10:50:18.492 17078 INFO nova.compute.claims [req-7211746b-c2be-469e-acd4-70e6293f2c0a f9e96bf3296e49e3b68637c5e7419559 72d3e55789444e98acb42a0cf267c449 - - -] [instance: e4cf8f29-9a20-4171-b80f-d316cd6deec0] Total disk: 9178 GB, used: 1.00 GB
2016-05-25 10:50:18.493 17078 INFO nova.compute.claims [req-7211746b-c2be-469e-acd4-70e6293f2c0a f9e96bf3296e49e3b68637c5e7419559 72d3e55789444e98acb42a0cf267c449 - - -] [instance: e4cf8f29-9a20-4171-b80f-d316cd6deec0] disk limit: 9178.00 GB, free: 9177.00 GB
2016-05-25 10:50:18.532 17078 INFO nova.compute.claims [req-7211746b-c2be-469e-acd4-70e6293f2c0a f9e96bf3296e49e3b68637c5e7419559 72d3e55789444e98acb42a0cf267c449 - - -] [instance: e4cf8f29-9a20-4171-b80f-d316cd6deec0] Claim successful
2016-05-25 10:50:18.576 17078 DEBUG nova.compute.utils [req-7211746b-c2be-469e-acd4-70e6293f2c0a f9e96bf3296e49e3b68637c5e7419559 72d3e55789444e98acb42a0cf267c449 - - -] [instance: e4cf8f29-9a20-4171-b80f-d316cd6deec0] Instance e4cf8f29-9a20-4171-b80f-d316cd6deec0 could not be found. notify_about_instance_usage /usr/lib/python2.7/dist-packages/nova/compute/utils.py:284
2016-05-25 10:50:18.577 17078 DEBUG nova.compute.manager [req-7211746b-c2be-469e-acd4-70e6293f2c0a f9e96bf3296e49e3b68637c5e7419559 72d3e55789444e98acb42a0cf267c449 - - -] [instance: e4cf8f29-9a20-4171-b80f-d316cd6deec0] Instance disappeared during build. _do_build_and_run_instance /usr/lib/python2.7/dist-packages/nova/compute/manager.py:1948
2016-05-25 10:50:18.578 17078 DEBUG nova.compute.manager [req-7211746b-c2be-469e-acd4-70e6293f2c0a f9e96bf3296e49e3b68637c5e7419559 72d3e55789444e98acb42a0cf267c449 - - -] [instance: e4cf8f29-9a20-4171-b80f-d316cd6deec0] Deallocating network for instance _deallocate_network /usr/lib/python2.7/dist-packages/nova/compute/manager.py:1808
2016-05-25 10:50:18.579 17078 DEBUG nova.network.neutronv2.api [req-7211746b-c2be-469e-acd4-70e6293f2c0a f9e96bf3296e49e3b68637c5e7419559 72d3e55789444e98acb42a0cf267c449 - - -] [instance: e4cf8f29-9a20-4171-b80f-d316cd6deec0] deallocate_for_instance() deallocate_for_instance /usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py:832
2016-05-25 10:50:18.581 17078 DEBUG keystoneclient.session [req-7211746b-c2be-469e-acd4-70e6293f2c0a f9e96bf3296e49e3b68637c5e7419559 72d3e55789444e98acb42a0cf267c449 - - -] REQ: curl -g -i -X GET http://gc-controller01-k58.int.glispa.com:9696/v2.0/ports.json?device_id=e4cf8f29-9a20-4171-b80f-d316cd6deec0 -H "User-Agent: python-neutronclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}dfbea371338aa75edba567f413e07bbd612484b3" _http_log_request /usr/lib/python2.7/dist-packages/keystoneclient/session.py:198
2016-05-25 10:50:18.623 17078 DEBUG nova.network.neutronv2.api [req-7211746b-c2be-469e-acd4-70e6293f2c0a f9e96bf3296e49e3b68637c5e7419559 72d3e55789444e98acb42a0cf267c449 - - -] [instance: e4cf8f29-9a20-4171-b80f-d316cd6deec0] Instance cache missing network info. _get_preexisting_port_ids /usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py:1672
2016-05-25 10:50:18.720 17078 DEBUG nova.network.base_api [req-7211746b-c2be-469e-acd4-70e6293f2c0a f9e96bf3296e49e3b68637c5e7419559 72d3e55789444e98acb42a0cf267c449 - - -] [instance: e4cf8f29-9a20-4171-b80f-d316cd6deec0] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info /usr/lib/python2.7/dist-packages/nova/network/base_api ...
(more)
edit retag flag offensive close merge delete

Comments

I think you need to look into this Skipping network cache update for instance because it is Building. _heal_instance_info_cache

rduncan-t gravatar imagerduncan-t ( 2016-05-25 06:28:15 -0600 )edit

it looks like that the glance-registry was not working as expected. restarted it and it works now.

bertly gravatar imagebertly ( 2016-05-26 03:23:34 -0600 )edit