Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Invalid image identifier or unable to access requested image.

Hi, I did an upgrade from Mitaka to Newton and running into a problem to start a virtual machine. As storage we are using ceph. It seems that cinder cannot access glance, but the glance api responds with 200er and I see a different error in the nova-compute logs.

cinder-api.log

2017-05-02 14:35:52.845 3769 INFO cinder.api.openstack.wsgi [req-ce751821-c230-49d6-a5f3-4e003043607a 217ce8c543b3495fa23b48a99cf9e4c5 3723cf7355e946efbf40f4f29ff845f6 - default default] POST http://controller:8776/v2/3723cf7355e946efbf40f4f29ff845f6/volumes
2017-05-02 14:35:52.847 3769 INFO cinder.api.v2.volumes [req-ce751821-c230-49d6-a5f3-4e003043607a 217ce8c543b3495fa23b48a99cf9e4c5 3723cf7355e946efbf40f4f29ff845f6 - default default] Create volume of 1 GB
2017-05-02 14:35:52.865 3769 INFO cinder.api.openstack.wsgi [req-ce751821-c230-49d6-a5f3-4e003043607a 217ce8c543b3495fa23b48a99cf9e4c5 3723cf7355e946efbf40f4f29ff845f6 - default default] HTTP exception thrown: Invalid image identifier or unable to access requested image. <-------------------
2017-05-02 14:35:52.867 3769 INFO cinder.api.openstack.wsgi [req-ce751821-c230-49d6-a5f3-4e003043607a 217ce8c543b3495fa23b48a99cf9e4c5 3723cf7355e946efbf40f4f29ff845f6 - default default] http://controller:8776/v2/3723cf7355e946efbf40f4f29ff845f6/volumes returned with HTTP 400
2017-05-02 14:35:52.869 3769 INFO eventlet.wsgi.server [req-ce751821-c230-49d6-a5f3-4e003043607a 217ce8c543b3495fa23b48a99cf9e4c5 3723cf7355e946efbf40f4f29ff845f6 - default default] 172.16.102.99 "POST /v2/3723cf7355e946efbf40f4f29ff845f6/volumes HTTP/1.1" status: 400  len: 392 time: 0.0301850

glance-api.log

2017-05-02 14:35:50.490 3678 INFO eventlet.wsgi.server [req-701563e7-c5cc-4c24-afeb-48696ee01d03 217ce8c543b3495fa23b48a99cf9e4c5 3723cf7355e946efbf40f4f29ff845f6 - default default] 172.16.102.99 - - [02/May/2017 14:35:50] "GET /v2/images/d5dcdfa0-1638-4c53-9d2f-27781ff5c76b HTTP/1.1" 200 907 0.026464
2017-05-02 14:35:50.500 3678 INFO eventlet.wsgi.server [req-e5b3420a-741c-44e7-afb2-6e88c11957f9 217ce8c543b3495fa23b48a99cf9e4c5 3723cf7355e946efbf40f4f29ff845f6 - default default] 172.16.102.99 - - [02/May/2017 14:35:50] "GET /v2/schemas/image HTTP/1.1" 200 4364 0.006064
2017-05-02 14:35:51.004 3679 INFO eventlet.wsgi.server [-] 172.16.102.99 - - [02/May/2017 14:35:51] "OPTIONS / HTTP/1.0" 200 94 0.002375
2017-05-02 14:35:51.017 3679 INFO eventlet.wsgi.server [-] 172.16.102.99 - - [02/May/2017 14:35:51] "OPTIONS / HTTP/1.0" 200 94 0.001645
2017-05-02 14:35:51.062 3679 INFO eventlet.wsgi.server [req-4cb01749-2138-4e09-a15b-3c25628b919b 217ce8c543b3495fa23b48a99cf9e4c5 3723cf7355e946efbf40f4f29ff845f6 - default default] 172.16.102.99 - - [02/May/2017 14:35:51] "GET /v2/images/d5dcdfa0-1638-4c53-9d2f-27781ff5c76b HTTP/1.1" 200 907 0.043746
2017-05-02 14:35:51.076 3679 INFO eventlet.wsgi.server [req-87b96c28-b3ba-4dd6-a507-16a7c5bd13ea 217ce8c543b3495fa23b48a99cf9e4c5 3723cf7355e946efbf40f4f29ff845f6 - default default] 172.16.102.99 - - [02/May/2017 14:35:51] "GET /v2/schemas/image HTTP/1.1" 200 4364 0.007997
2017-05-02 14:35:51.945 3680 INFO eventlet.wsgi.server [req-e108571a-2775-4c74-9399-f0141ba61c86 217ce8c543b3495fa23b48a99cf9e4c5 3723cf7355e946efbf40f4f29ff845f6 - default default] 172.16.102.99 - - [02/May/2017 14:35:51] "GET /v2/images?limit=1000&sort_key=created_at&sort_dir=desc HTTP/1.1" 200 2661 0.056409
2017-05-02 14:35:51.958 3680 INFO eventlet.wsgi.server [req-52ed314c-921d-427f-bdcd-9e002e9ee6f2 217ce8c543b3495fa23b48a99cf9e4c5 3723cf7355e946efbf40f4f29ff845f6 - default default] 172.16.102.99 - - [02/May/2017 14:35:51] "GET /v2/schemas/image HTTP/1.1" 200 4364 0.007294

nova-compute.log

2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [req-3ebef2dd-6c88-45de-91c3-89c44d2e0411 217ce8c543b3495fa23b48a99cf9e4c5 3723cf7355e946efbf40f4f29ff845f6 - - -] [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca] Build of instance 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca aborted: Block Device Mapping is Invalid.
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca] Traceback (most recent call last):
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1783, in _do_build_and_run_instance
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]     filter_properties)
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1943, in _build_and_run_instance
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]     'create.error', fault=e)
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]     self.force_reraise()
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]     six.reraise(self.type_, self.value, self.tb)
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1908, in _build_and_run_instance
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]     block_device_mapping) as resources:
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]   File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]     return self.gen.next()
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2070, in _build_resources
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]     reason=e.format_message())
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca] BuildAbortException: Build of instance 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca aborted: Block Device Mapping is Invalid. <-------------------
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]

I can't find any directions and why it should not work. All services are registered and online. Its possible to access rbd with the user that cinder/glance are using. Image creating is working. Also creating volumes is no problem.

Any idea or hint whats going on here?

Thanks

bert

Invalid image identifier or unable to access requested image.

Hi, I did an upgrade from Mitaka to Newton and running into a problem to start a virtual machine. As storage we are using ceph. It seems that cinder cannot access glance, but the glance api responds with 200er and I see a different error in the nova-compute logs.

UPDATE: When I enable glance api v1 in glance-api then it is working again. Any ideas why version 2 is not working?

glance-api.conf ... enable_v1_api=True enable_v2_api=True ...

cinder-api.log

2017-05-02 14:35:52.845 3769 INFO cinder.api.openstack.wsgi [req-ce751821-c230-49d6-a5f3-4e003043607a 217ce8c543b3495fa23b48a99cf9e4c5 3723cf7355e946efbf40f4f29ff845f6 - default default] POST http://controller:8776/v2/3723cf7355e946efbf40f4f29ff845f6/volumes
2017-05-02 14:35:52.847 3769 INFO cinder.api.v2.volumes [req-ce751821-c230-49d6-a5f3-4e003043607a 217ce8c543b3495fa23b48a99cf9e4c5 3723cf7355e946efbf40f4f29ff845f6 - default default] Create volume of 1 GB
2017-05-02 14:35:52.865 3769 INFO cinder.api.openstack.wsgi [req-ce751821-c230-49d6-a5f3-4e003043607a 217ce8c543b3495fa23b48a99cf9e4c5 3723cf7355e946efbf40f4f29ff845f6 - default default] HTTP exception thrown: Invalid image identifier or unable to access requested image. <-------------------
2017-05-02 14:35:52.867 3769 INFO cinder.api.openstack.wsgi [req-ce751821-c230-49d6-a5f3-4e003043607a 217ce8c543b3495fa23b48a99cf9e4c5 3723cf7355e946efbf40f4f29ff845f6 - default default] http://controller:8776/v2/3723cf7355e946efbf40f4f29ff845f6/volumes returned with HTTP 400
2017-05-02 14:35:52.869 3769 INFO eventlet.wsgi.server [req-ce751821-c230-49d6-a5f3-4e003043607a 217ce8c543b3495fa23b48a99cf9e4c5 3723cf7355e946efbf40f4f29ff845f6 - default default] 172.16.102.99 "POST /v2/3723cf7355e946efbf40f4f29ff845f6/volumes HTTP/1.1" status: 400  len: 392 time: 0.0301850

glance-api.log

2017-05-02 14:35:50.490 3678 INFO eventlet.wsgi.server [req-701563e7-c5cc-4c24-afeb-48696ee01d03 217ce8c543b3495fa23b48a99cf9e4c5 3723cf7355e946efbf40f4f29ff845f6 - default default] 172.16.102.99 - - [02/May/2017 14:35:50] "GET /v2/images/d5dcdfa0-1638-4c53-9d2f-27781ff5c76b HTTP/1.1" 200 907 0.026464
2017-05-02 14:35:50.500 3678 INFO eventlet.wsgi.server [req-e5b3420a-741c-44e7-afb2-6e88c11957f9 217ce8c543b3495fa23b48a99cf9e4c5 3723cf7355e946efbf40f4f29ff845f6 - default default] 172.16.102.99 - - [02/May/2017 14:35:50] "GET /v2/schemas/image HTTP/1.1" 200 4364 0.006064
2017-05-02 14:35:51.004 3679 INFO eventlet.wsgi.server [-] 172.16.102.99 - - [02/May/2017 14:35:51] "OPTIONS / HTTP/1.0" 200 94 0.002375
2017-05-02 14:35:51.017 3679 INFO eventlet.wsgi.server [-] 172.16.102.99 - - [02/May/2017 14:35:51] "OPTIONS / HTTP/1.0" 200 94 0.001645
2017-05-02 14:35:51.062 3679 INFO eventlet.wsgi.server [req-4cb01749-2138-4e09-a15b-3c25628b919b 217ce8c543b3495fa23b48a99cf9e4c5 3723cf7355e946efbf40f4f29ff845f6 - default default] 172.16.102.99 - - [02/May/2017 14:35:51] "GET /v2/images/d5dcdfa0-1638-4c53-9d2f-27781ff5c76b HTTP/1.1" 200 907 0.043746
2017-05-02 14:35:51.076 3679 INFO eventlet.wsgi.server [req-87b96c28-b3ba-4dd6-a507-16a7c5bd13ea 217ce8c543b3495fa23b48a99cf9e4c5 3723cf7355e946efbf40f4f29ff845f6 - default default] 172.16.102.99 - - [02/May/2017 14:35:51] "GET /v2/schemas/image HTTP/1.1" 200 4364 0.007997
2017-05-02 14:35:51.945 3680 INFO eventlet.wsgi.server [req-e108571a-2775-4c74-9399-f0141ba61c86 217ce8c543b3495fa23b48a99cf9e4c5 3723cf7355e946efbf40f4f29ff845f6 - default default] 172.16.102.99 - - [02/May/2017 14:35:51] "GET /v2/images?limit=1000&sort_key=created_at&sort_dir=desc HTTP/1.1" 200 2661 0.056409
2017-05-02 14:35:51.958 3680 INFO eventlet.wsgi.server [req-52ed314c-921d-427f-bdcd-9e002e9ee6f2 217ce8c543b3495fa23b48a99cf9e4c5 3723cf7355e946efbf40f4f29ff845f6 - default default] 172.16.102.99 - - [02/May/2017 14:35:51] "GET /v2/schemas/image HTTP/1.1" 200 4364 0.007294

nova-compute.log

2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [req-3ebef2dd-6c88-45de-91c3-89c44d2e0411 217ce8c543b3495fa23b48a99cf9e4c5 3723cf7355e946efbf40f4f29ff845f6 - - -] [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca] Build of instance 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca aborted: Block Device Mapping is Invalid.
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca] Traceback (most recent call last):
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1783, in _do_build_and_run_instance
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]     filter_properties)
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1943, in _build_and_run_instance
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]     'create.error', fault=e)
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]     self.force_reraise()
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]     six.reraise(self.type_, self.value, self.tb)
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1908, in _build_and_run_instance
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]     block_device_mapping) as resources:
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]   File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]     return self.gen.next()
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2070, in _build_resources
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]     reason=e.format_message())
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca] BuildAbortException: Build of instance 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca aborted: Block Device Mapping is Invalid. <-------------------
2017-05-02 14:35:56.575 30849 ERROR nova.compute.manager [instance: 55ec6f65-a7cc-474e-8bb6-74d74dfb2cca]

I can't find any directions and why it should not work. All services are registered and online. Its possible to access rbd with the user that cinder/glance are using. Image creating is working. Also creating volumes is no problem.

Any idea or hint whats going on here?

Thanks

bert