Icehouse Glance 2014.1.5 with RBD backend hang while downloading image
Hi all,
I have an issue with my glance service.
I run icehouse on ubuntu 14.04.2 LTS in icehouse version 2014.1.5. Glance is setup with RDB backend.
I can :
- image-create
- image-delete
- image-list
- image-show
but i can not glance image-download i got no error just glance client hang. The impact is i can not create new instance out of the same ceph cluster.
I run 2 nova availability zone. One with instance attached to cinder volume in the same ceph cluster and i allow Copy-on-write functionality and another one with only ephemeral instances.
The creation of cinder volume on the same ceph cluster with Copy-on-write works. The creation of new instances where i have to locally download the image on the compute node doesn't work as the glance image-download command stay hang.
After i attempted to create an instance the whole glance-api on port 9292 stay hang. I have to restart the glance-api service to get back to "normal service".
I see no error in the debug log : glance-api.log
2015-07-20 17:35:50.402 19974 DEBUG glance.api.middleware.version_negotiation [-] Determining version of request: HEAD /v1/images/8ce3f00e-79ef-43a5-b44c-54648f4d02bd Accept: process_request /usr/lib/python2.7/dist-packages/glance/api/middleware/version_negotiation.py:44
2015-07-20 17:35:50.402 19974 DEBUG glance.api.middleware.version_negotiation [-] Using url versioning process_request /usr/lib/python2.7/dist-packages/glance/api/middleware/version_negotiation.py:57
2015-07-20 17:35:50.403 19974 DEBUG glance.api.middleware.version_negotiation [-] Matched version: v1 process_request /usr/lib/python2.7/dist-packages/glance/api/middleware/version_negotiation.py:69
2015-07-20 17:35:50.403 19974 DEBUG glance.api.middleware.version_negotiation [-] new path /v1/images/8ce3f00e-79ef-43a5-b44c-54648f4d02bd process_request /usr/lib/python2.7/dist-packages/glance/api/middleware/version_negotiation.py:70
2015-07-20 17:35:50.403 19974 DEBUG glance.api.policy [-] Loading policy from /etc/glance/policy.json _read_policy_file /usr/lib/python2.7/dist-packages/glance/api/policy.py:106
2015-07-20 17:35:50.404 19974 DEBUG glance.api.policy [-] Loaded policy rules: {u'get_task': '@', u'get_image_location': '@', u'add_image': '@', u'modify_image': '@', u'manage_image_cache': 'role:admin', u'delete_member': '@', u'get_images': '@', u'delete_image': '@', u'publicize_image': '@', u'get_member': '@', u'add_member': '@', u'set_image_location': '@', u'get_image': '@', u'modify_member': '@', u'context_is_admin': 'role:admin', u'upload_image': '@', u'modify_task': '@', u'get_members': '@', u'get_tasks': '@', u'add_task': '@', u'default': '@', u'delete_image_location': '@', u'copy_from': '@', u'download_image': '@'} load_rules /usr/lib/python2.7/dist-packages/glance/api/policy.py:85
2015-07-20 17:35:50.404 19974 DEBUG routes.middleware [d94ef9ef-ee84-438e-abcd-1d02eaf047d4 - - - - -] Matched HEAD /images/8ce3f00e-79ef-43a5-b44c-54648f4d02bd __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:100
2015-07-20 17:35:50.404 19974 DEBUG routes.middleware [d94ef9ef-ee84-438e-abcd-1d02eaf047d4 - - - - -] Route path: '/images/{id}', defaults: {'action': u'meta', 'controller': <glance.common.wsgi.Resource object at 0x7f207bf9e1d0>} __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:102
2015-07-20 17:35:50.404 19974 DEBUG routes.middleware [d94ef9ef-ee84-438e-abcd-1d02eaf047d4 - - - - -] Match dict: {'action': u'meta', 'controller': <glance.common.wsgi.Resource object at 0x7f207bf9e1d0>, 'id': u'8ce3f00e-79ef-43a5-b44c-54648f4d02bd'} __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:103
2015-07-20 17:35:50.405 19974 DEBUG glance.common.client [d94ef9ef-ee84-438e-abcd-1d02eaf047d4 - - - - -] Constructed URL: http ...