Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

asked 2014-12-08 05:49:13 -0500

iffyshot gravatar image

Cannot run instance

Hi all

I wonder if you can give me a hand as i am new to openstack

I have followed the instructions on the install guide to install 3 nodes, I have got the dashboard installed ok and can add networks, and routers but when i try to run an instance I get the error No Valid hosts

here is the output from this command tail -f /var/log/nova/nova-scheduler.log | grep ERROR

2014-12-08 11:36:13.762 2673 ERROR nova.scheduler.filter_scheduler [req-d77194d4 -a5b2-4185-ab6e-e9a12a5024d5 0b188de927e34893acd94a489fb1e167 d0d944054e504a7eb2 ab114c0de98d63] [instance: 61db67f6-88bb-4c2d-924e-da82452c04c4] Error from last host: compute1 (node compute1): [u'Traceback (most recent call last):\n', u' F ile "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1333, in _b uild_instance\n set_access_ip=set_access_ip)\n', u' File "/usr/lib/python2.7 /dist-packages/nova/compute/manager.py", line 401, in decorated_function\n re turn function(self, context, args, *kwargs)\n', u' File "/usr/lib/python2.7/d ist-packages/nova/compute/manager.py", line 1745, in _spawn\n LOG.exception(_ (\'Instance failed to spawn\'), instance=instance)\n', u' File "/usr/lib/python 2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/di st-packages/nova/compute/manager.py", line 1742, in _spawn\n block_device_inf o)\n', u' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2288, in spawn\n admin_pass=admin_password)\n', u' File "/usr/lib/pytho n2.7/dist-packages/nova/virt/libvirt/driver.py", line 2657, in _create_image\n project_id=instance[\'project_id\'])\n', u' File "/usr/lib/python2.7/dist-pac kages/nova/virt/libvirt/imagebackend.py", line 192, in cache\n args, *kwarg s)\n', u' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend .py", line 383, in create_image\n prepare_template(target=base, max_size=size , args, *kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/nova/openstack /common/lockutils.py", line 249, in inner\n return f(args, *kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 182, in fetch_func_sync\n fetch_func(target=target, args, *kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/utils.py", line 653, i n fetch_image\n max_size=max_size)\n', u' File "/usr/lib/python2.7/dist-pack ages/nova/virt/images.py", line 78, in fetch_to_raw\n max_size=max_size)\n', u' File "/usr/lib/python2.7/dist-packages/nova/virt/images.py", line 72, in fet ch\n image_service.download(context, image_id, dst_path=path)\n', u' File "/ usr/lib/python2.7/dist-packages/nova/image/glance.py", line 331, in download\n _reraise_translated_image_exception(image_id)\n', u' File "/usr/lib/python2.7 /dist-packages/nova/image/glance.py", line 329, in download\n image_chunks = self._client.call(context, 1, \'data\', image_id)\n', u' File "/usr/lib/python2 .7/dist-packages/nova/image/glance.py", line 209, in call\n return getattr(cl ient.images, method)(args, *kwargs)\n', u' File "/usr/lib/python2.7/dist-pack ages/glanceclient/v1/images.py", line 127, in data\n % urllib.quote(str(image _id)))\n', u' File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.p y", line 289, in raw_request\n return self._http_request(url, method, **kwarg s)\n', u' File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line 249, in _http_request\n raise exc.from_response(resp, body_str)\n', u'Im ageNotAuthorized: Not authorized for image 844227bc-f161-4af6-92da-b8c7a4dc9fbd. \n'] 2014-12-08 11:36:13.769 2673 WARNING nova.scheduler.utils [req-d77194d4-a5b2-418 5-ab6e-e9a12a5024d5 0b188de927e34893acd94a489fb1e167 d0d944054e504a7eb2ab114c0de 98d63] [instance: 61db67f6-88bb-4c2d-924e-da82452c04c4] Setting instance to ERRO R state.

i also get a lot of debug messages appear on the screen when i run nova-compute service list nova-manage service list 2014-12-08 11:48:06.976 12821 DEBUG nova.servicegroup.api [-] ServiceGroup driver defined as an instance of db __new__ /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:65 2014-12-08 11:48:07.081 12821 DEBUG stevedore.extension [-] found extension EntryPoint.parse('file = nova.image.download.file') _load_plugins /usr/lib/python2.7/dist-packages/stevedore/extension.py:156 2014-12-08 11:48:07.109 12821 DEBUG stevedore.extension [-] found extension EntryPoint.parse('file = nova.image.download.file') _load_plugins /usr/lib/python2.7/dist-packages/stevedore/extension.py:156 Binary Host Zone Status State Updated_At 2014-12-08 11:48:07.849 12821 DEBUG nova.servicegroup.api [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] Check if the given member [{'binary': u'nova-cert', 'availability_zone': 'internal', 'deleted': 0L, 'created_at': datetime.datetime(2014, 12, 4, 10, 15, 26), 'updated_at': datetime.datetime(2014, 12, 8, 11, 48, 7), 'report_count': 4810L, 'topic': u'cert', 'host': u'controller1', 'disabled': False, 'deleted_at': None, 'disabled_reason': None, 'id': 1L}] is part of the ServiceGroup, is up service_is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:116 2014-12-08 11:48:07.850 12821 DEBUG nova.servicegroup.drivers.db [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] DB_Driver.is_up last_heartbeat = 2014-12-08 11:48:07 elapsed = 0.850147 is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/db.py:71 nova-cert controller1 internal enabled :-) 2014-12-08 11:48:07 2014-12-08 11:48:07.850 12821 DEBUG nova.servicegroup.api [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] Check if the given member [{'binary': u'nova-consoleauth', 'availability_zone': 'internal', 'deleted': 0L, 'created_at': datetime.datetime(2014, 12, 4, 10, 15, 53), 'updated_at': datetime.datetime(2014, 12, 8, 11, 48, 7), 'report_count': 4808L, 'topic': u'consoleauth', 'host': u'controller1', 'disabled': False, 'deleted_at': None, 'disabled_reason': None, 'id': 2L}] is part of the ServiceGroup, is up service_is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:116 2014-12-08 11:48:07.851 12821 DEBUG nova.servicegroup.drivers.db [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] DB_Driver.is_up last_heartbeat = 2014-12-08 11:48:07 elapsed = 0.851013 is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/db.py:71 nova-consoleauth controller1 internal enabled :-) 2014-12-08 11:48:07 2014-12-08 11:48:07.851 12821 DEBUG nova.servicegroup.api [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] Check if the given member [{'binary': u'nova-scheduler', 'availability_zone': 'internal', 'deleted': 0L, 'created_at': datetime.datetime(2014, 12, 4, 10, 16, 11), 'updated_at': datetime.datetime(2014, 12, 8, 11, 48, 7), 'report_count': 4807L, 'topic': u'scheduler', 'host': u'controller1', 'disabled': False, 'deleted_at': None, 'disabled_reason': None, 'id': 3L}] is part of the ServiceGroup, is up service_is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:116 2014-12-08 11:48:07.851 12821 DEBUG nova.servicegroup.drivers.db [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] DB_Driver.is_up last_heartbeat = 2014-12-08 11:48:07 elapsed = 0.851661 is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/db.py:71 nova-scheduler controller1 internal enabled :-) 2014-12-08 11:48:07 2014-12-08 11:48:07.852 12821 DEBUG nova.servicegroup.api [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] Check if the given member [{'binary': u'nova-conductor', 'availability_zone': 'internal', 'deleted': 0L, 'created_at': datetime.datetime(2014, 12, 4, 10, 16, 21), 'updated_at': datetime.datetime(2014, 12, 8, 11, 48, 7), 'report_count': 4808L, 'topic': u'conductor', 'host': u'controller1', 'disabled': False, 'deleted_at': None, 'disabled_reason': None, 'id': 4L}] is part of the ServiceGroup, is up service_is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:116 2014-12-08 11:48:07.852 12821 DEBUG nova.servicegroup.drivers.db [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] DB_Driver.is_up last_heartbeat = 2014-12-08 11:48:07 elapsed = 0.852286 is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/db.py:71 nova-conductor controller1 internal enabled :-) 2014-12-08 11:48:07 2014-12-08 11:48:07.852 12821 DEBUG nova.servicegroup.api [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] Check if the given member [{'binary': u'nova-compute', 'availability_zone': 'nova', 'deleted': 0L, 'created_at': datetime.datetime(2014, 12, 4, 10, 39, 54), 'updated_at': datetime.datetime(2014, 12, 8, 11, 48, 6), 'report_count': 3040L, 'topic': u'compute', 'host': u'compute1', 'disabled': False, 'deleted_at': None, 'disabled_reason': None, 'id': 5L}] is part of the ServiceGroup, is up service_is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:116 2014-12-08 11:48:07.853 12821 DEBUG nova.servicegroup.drivers.db [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] DB_Driver.is_up last_heartbeat = 2014-12-08 11:48:06 elapsed = 1.853084 is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/db.py:71 nova-compute compute1 nova enabled :-) 2014-12-08 11:48:06

any help in troubleshooting this would be greatly received thanks

Cannot run instance

Hi all

I wonder if you can give me a hand as i am new to openstack

I have followed the instructions on the install guide to install 3 nodes, I have got the dashboard installed ok and can add networks, and routers but when i try to run an instance I get the error No Valid hosts

here is the output from this command tail -f /var/log/nova/nova-scheduler.log | grep ERROR

2014-12-08 11:36:13.762 2673 ERROR nova.scheduler.filter_scheduler [req-d77194d4 -a5b2-4185-ab6e-e9a12a5024d5 0b188de927e34893acd94a489fb1e167 d0d944054e504a7eb2 ab114c0de98d63] [instance: 61db67f6-88bb-4c2d-924e-da82452c04c4] Error from last host: compute1 (node compute1): [u'Traceback (most recent call last):\n', u' F ile "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1333, in _b uild_instance\n set_access_ip=set_access_ip)\n', u' File "/usr/lib/python2.7 /dist-packages/nova/compute/manager.py", line 401, in decorated_function\n re turn function(self, context, args, *kwargs)\n', u' File "/usr/lib/python2.7/d ist-packages/nova/compute/manager.py", line 1745, in _spawn\n LOG.exception(_ (\'Instance failed to spawn\'), instance=instance)\n', u' File "/usr/lib/python 2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/di st-packages/nova/compute/manager.py", line 1742, in _spawn\n block_device_inf o)\n', u' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2288, in spawn\n admin_pass=admin_password)\n', u' File "/usr/lib/pytho n2.7/dist-packages/nova/virt/libvirt/driver.py", line 2657, in _create_image\n project_id=instance[\'project_id\'])\n', u' File "/usr/lib/python2.7/dist-pac kages/nova/virt/libvirt/imagebackend.py", line 192, in cache\n args, *kwarg s)\n', u' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend .py", line 383, in create_image\n prepare_template(target=base, max_size=size , args, *kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/nova/openstack /common/lockutils.py", line 249, in inner\n return f(args, *kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 182, in fetch_func_sync\n fetch_func(target=target, args, *kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/utils.py", line 653, i n fetch_image\n max_size=max_size)\n', u' File "/usr/lib/python2.7/dist-pack ages/nova/virt/images.py", line 78, in fetch_to_raw\n max_size=max_size)\n', u' File "/usr/lib/python2.7/dist-packages/nova/virt/images.py", line 72, in fet ch\n image_service.download(context, image_id, dst_path=path)\n', u' File "/ usr/lib/python2.7/dist-packages/nova/image/glance.py", line 331, in download\n _reraise_translated_image_exception(image_id)\n', u' File "/usr/lib/python2.7 /dist-packages/nova/image/glance.py", line 329, in download\n image_chunks = self._client.call(context, 1, \'data\', image_id)\n', u' File "/usr/lib/python2 .7/dist-packages/nova/image/glance.py", line 209, in call\n return getattr(cl ient.images, method)(args, *kwargs)\n', u' File "/usr/lib/python2.7/dist-pack ages/glanceclient/v1/images.py", line 127, in data\n % urllib.quote(str(image _id)))\n', u' File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.p y", line 289, in raw_request\n return self._http_request(url, method, **kwarg s)\n', u' File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line 249, in _http_request\n raise exc.from_response(resp, body_str)\n', u'Im ageNotAuthorized: Not authorized for image 844227bc-f161-4af6-92da-b8c7a4dc9fbd. \n'] 2014-12-08 11:36:13.769 2673 WARNING nova.scheduler.utils [req-d77194d4-a5b2-418 5-ab6e-e9a12a5024d5 0b188de927e34893acd94a489fb1e167 d0d944054e504a7eb2ab114c0de 98d63] [instance: 61db67f6-88bb-4c2d-924e-da82452c04c4] Setting instance to ERRO R state.

i also get a lot of debug messages appear on the screen when i run nova-compute service list nova-manage service list list

2014-12-08 11:48:06.976 12821 DEBUG nova.servicegroup.api [-] ServiceGroup driver defined as an instance of db __new__ /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:65 2014-12-08 11:48:07.081 12821 DEBUG stevedore.extension [-] found extension EntryPoint.parse('file = nova.image.download.file') _load_plugins /usr/lib/python2.7/dist-packages/stevedore/extension.py:156 2014-12-08 11:48:07.109 12821 DEBUG stevedore.extension [-] found extension EntryPoint.parse('file = nova.image.download.file') _load_plugins /usr/lib/python2.7/dist-packages/stevedore/extension.py:156 Binary Host Zone Status State Updated_At 2014-12-08 11:48:07.849 12821 DEBUG nova.servicegroup.api [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] Check if the given member [{'binary': u'nova-cert', 'availability_zone': 'internal', 'deleted': 0L, 'created_at': datetime.datetime(2014, 12, 4, 10, 15, 26), 'updated_at': datetime.datetime(2014, 12, 8, 11, 48, 7), 'report_count': 4810L, 'topic': u'cert', 'host': u'controller1', 'disabled': False, 'deleted_at': None, 'disabled_reason': None, 'id': 1L}] is part of the ServiceGroup, is up service_is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:116 2014-12-08 11:48:07.850 12821 DEBUG nova.servicegroup.drivers.db [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] DB_Driver.is_up last_heartbeat = 2014-12-08 11:48:07 elapsed = 0.850147 is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/db.py:71 nova-cert controller1 internal enabled :-) 2014-12-08 11:48:07 2014-12-08 11:48:07.850 12821 DEBUG nova.servicegroup.api [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] Check if the given member [{'binary': u'nova-consoleauth', 'availability_zone': 'internal', 'deleted': 0L, 'created_at': datetime.datetime(2014, 12, 4, 10, 15, 53), 'updated_at': datetime.datetime(2014, 12, 8, 11, 48, 7), 'report_count': 4808L, 'topic': u'consoleauth', 'host': u'controller1', 'disabled': False, 'deleted_at': None, 'disabled_reason': None, 'id': 2L}] is part of the ServiceGroup, is up service_is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:116 2014-12-08 11:48:07.851 12821 DEBUG nova.servicegroup.drivers.db [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] DB_Driver.is_up last_heartbeat = 2014-12-08 11:48:07 elapsed = 0.851013 is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/db.py:71 nova-consoleauth controller1 internal enabled :-) 2014-12-08 11:48:07 2014-12-08 11:48:07.851 12821 DEBUG nova.servicegroup.api [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] Check if the given member [{'binary': u'nova-scheduler', 'availability_zone': 'internal', 'deleted': 0L, 'created_at': datetime.datetime(2014, 12, 4, 10, 16, 11), 'updated_at': datetime.datetime(2014, 12, 8, 11, 48, 7), 'report_count': 4807L, 'topic': u'scheduler', 'host': u'controller1', 'disabled': False, 'deleted_at': None, 'disabled_reason': None, 'id': 3L}] is part of the ServiceGroup, is up service_is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:116 2014-12-08 11:48:07.851 12821 DEBUG nova.servicegroup.drivers.db [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] DB_Driver.is_up last_heartbeat = 2014-12-08 11:48:07 elapsed = 0.851661 is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/db.py:71 nova-scheduler controller1 internal enabled :-) 2014-12-08 11:48:07 2014-12-08 11:48:07.852 12821 DEBUG nova.servicegroup.api [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] Check if the given member [{'binary': u'nova-conductor', 'availability_zone': 'internal', 'deleted': 0L, 'created_at': datetime.datetime(2014, 12, 4, 10, 16, 21), 'updated_at': datetime.datetime(2014, 12, 8, 11, 48, 7), 'report_count': 4808L, 'topic': u'conductor', 'host': u'controller1', 'disabled': False, 'deleted_at': None, 'disabled_reason': None, 'id': 4L}] is part of the ServiceGroup, is up service_is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:116 2014-12-08 11:48:07.852 12821 DEBUG nova.servicegroup.drivers.db [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] DB_Driver.is_up last_heartbeat = 2014-12-08 11:48:07 elapsed = 0.852286 is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/db.py:71 nova-conductor controller1 internal enabled :-) 2014-12-08 11:48:07 2014-12-08 11:48:07.852 12821 DEBUG nova.servicegroup.api [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] Check if the given member [{'binary': u'nova-compute', 'availability_zone': 'nova', 'deleted': 0L, 'created_at': datetime.datetime(2014, 12, 4, 10, 39, 54), 'updated_at': datetime.datetime(2014, 12, 8, 11, 48, 6), 'report_count': 3040L, 'topic': u'compute', 'host': u'compute1', 'disabled': False, 'deleted_at': None, 'disabled_reason': None, 'id': 5L}] is part of the ServiceGroup, is up service_is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:116 2014-12-08 11:48:07.853 12821 DEBUG nova.servicegroup.drivers.db [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] DB_Driver.is_up last_heartbeat = 2014-12-08 11:48:06 elapsed = 1.853084 is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/db.py:71 nova-compute compute1 nova enabled :-) 2014-12-08 11:48:06

any help in troubleshooting this would be greatly received thanks

click to hide/show revision 3
No.3 Revision

updated 2014-12-08 10:50:19 -0500

mpetason gravatar image

Cannot run instance

Hi all

I wonder if you can give me a hand as i am new to openstack

I have followed the instructions on the install guide to install 3 nodes, I have got the dashboard installed ok and can add networks, and routers but when i try to run an instance I get the error No Valid hosts

here is the output from this command tail -f /var/log/nova/nova-scheduler.log | grep ERROR

2014-12-08 11:36:13.762 2673 ERROR nova.scheduler.filter_scheduler [req-d77194d4                                   -a5b2-4185-ab6e-e9a12a5024d5 0b188de927e34893acd94a489fb1e167 d0d944054e504a7eb2                                   ab114c0de98d63] [instance: 61db67f6-88bb-4c2d-924e-da82452c04c4] Error from last                                    host: compute1 (node compute1): [u'Traceback (most recent call last):\n', u'  F                                   ile "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1333, in _b                                   uild_instance\n    set_access_ip=set_access_ip)\n', u'  File "/usr/lib/python2.7                                   /dist-packages/nova/compute/manager.py", line 401, in decorated_function\n    re                                   turn function(self, context, args, *kwargs)\n', *args, **kwargs)\n', u'  File "/usr/lib/python2.7/d                                   ist-packages/nova/compute/manager.py", line 1745, in _spawn\n    LOG.exception(_                                   (\'Instance failed to spawn\'), instance=instance)\n', u'  File "/usr/lib/python                                   2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__\n                                       six.reraise(self.type_, self.value, self.tb)\n', u'  File "/usr/lib/python2.7/di                                   st-packages/nova/compute/manager.py", line 1742, in _spawn\n    block_device_inf                                   o)\n', u'  File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py",                                    line 2288, in spawn\n    admin_pass=admin_password)\n', u'  File "/usr/lib/pytho                                   n2.7/dist-packages/nova/virt/libvirt/driver.py", line 2657, in _create_image\n                                       project_id=instance[\'project_id\'])\n', u'  File "/usr/lib/python2.7/dist-pac                                   kages/nova/virt/libvirt/imagebackend.py", line 192, in cache\n    args, *kwarg *args, **kwarg                                   s)\n', u'  File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend                                   .py", line 383, in create_image\n    prepare_template(target=base, max_size=size                                   , args, *kwargs)\n', *args, **kwargs)\n', u'  File "/usr/lib/python2.7/dist-packages/nova/openstack                                   /common/lockutils.py", line 249, in inner\n    return f(args, *kwargs)\n', f(*args, **kwargs)\n', u'                                     File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line                                    182, in fetch_func_sync\n    fetch_func(target=target, args, *kwargs)\n', *args, **kwargs)\n', u'                                     File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/utils.py", line 653, i                                   n fetch_image\n    max_size=max_size)\n', u'  File "/usr/lib/python2.7/dist-pack                                   ages/nova/virt/images.py", line 78, in fetch_to_raw\n    max_size=max_size)\n',                                    u'  File "/usr/lib/python2.7/dist-packages/nova/virt/images.py", line 72, in fet                                   ch\n    image_service.download(context, image_id, dst_path=path)\n', u'  File "/                                   usr/lib/python2.7/dist-packages/nova/image/glance.py", line 331, in download\n                                       _reraise_translated_image_exception(image_id)\n', u'  File "/usr/lib/python2.7                                   /dist-packages/nova/image/glance.py", line 329, in download\n    image_chunks =                                    self._client.call(context, 1, \'data\', image_id)\n', u'  File "/usr/lib/python2                                   .7/dist-packages/nova/image/glance.py", line 209, in call\n    return getattr(cl                                   ient.images, method)(args, *kwargs)\n', method)(*args, **kwargs)\n', u'  File "/usr/lib/python2.7/dist-pack                                   ages/glanceclient/v1/images.py", line 127, in data\n    % urllib.quote(str(image                                   _id)))\n', u'  File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.p                                   y", line 289, in raw_request\n    return self._http_request(url, method, **kwarg                                   s)\n', u'  File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py",                                    line 249, in _http_request\n    raise exc.from_response(resp, body_str)\n', u'Im                                   ageNotAuthorized: Not authorized for image 844227bc-f161-4af6-92da-b8c7a4dc9fbd.                                   \n']
2014-12-08 11:36:13.769 2673 WARNING nova.scheduler.utils [req-d77194d4-a5b2-418                                   5-ab6e-e9a12a5024d5 0b188de927e34893acd94a489fb1e167 d0d944054e504a7eb2ab114c0de                                   98d63] [instance: 61db67f6-88bb-4c2d-924e-da82452c04c4] Setting instance to ERRO                                   R state.
state.

i also get a lot of debug messages appear on the screen when i run nova-compute service list nova-manage service list

2014-12-08 11:48:06.976 12821 DEBUG nova.servicegroup.api [-] ServiceGroup driver defined as an instance of db __new__ /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:65
2014-12-08 11:48:07.081 12821 DEBUG stevedore.extension [-] found extension EntryPoint.parse('file = nova.image.download.file') _load_plugins /usr/lib/python2.7/dist-packages/stevedore/extension.py:156
2014-12-08 11:48:07.109 12821 DEBUG stevedore.extension [-] found extension EntryPoint.parse('file = nova.image.download.file') _load_plugins /usr/lib/python2.7/dist-packages/stevedore/extension.py:156
Binary           Host                                 Zone             Status     State Updated_At
2014-12-08 11:48:07.849 12821 DEBUG nova.servicegroup.api [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] Check if the given member [{'binary': u'nova-cert', 'availability_zone': 'internal', 'deleted': 0L, 'created_at': datetime.datetime(2014, 12, 4, 10, 15, 26), 'updated_at': datetime.datetime(2014, 12, 8, 11, 48, 7), 'report_count': 4810L, 'topic': u'cert', 'host': u'controller1', 'disabled': False, 'deleted_at': None, 'disabled_reason': None, 'id': 1L}] is part of the ServiceGroup, is up service_is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:116
2014-12-08 11:48:07.850 12821 DEBUG nova.servicegroup.drivers.db [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] DB_Driver.is_up last_heartbeat = 2014-12-08 11:48:07 elapsed = 0.850147 is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/db.py:71
nova-cert        controller1                          internal         enabled    :-)   2014-12-08 11:48:07
2014-12-08 11:48:07.850 12821 DEBUG nova.servicegroup.api [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] Check if the given member [{'binary': u'nova-consoleauth', 'availability_zone': 'internal', 'deleted': 0L, 'created_at': datetime.datetime(2014, 12, 4, 10, 15, 53), 'updated_at': datetime.datetime(2014, 12, 8, 11, 48, 7), 'report_count': 4808L, 'topic': u'consoleauth', 'host': u'controller1', 'disabled': False, 'deleted_at': None, 'disabled_reason': None, 'id': 2L}] is part of the ServiceGroup, is up service_is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:116
2014-12-08 11:48:07.851 12821 DEBUG nova.servicegroup.drivers.db [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] DB_Driver.is_up last_heartbeat = 2014-12-08 11:48:07 elapsed = 0.851013 is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/db.py:71
nova-consoleauth controller1                          internal         enabled    :-)   2014-12-08 11:48:07
2014-12-08 11:48:07.851 12821 DEBUG nova.servicegroup.api [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] Check if the given member [{'binary': u'nova-scheduler', 'availability_zone': 'internal', 'deleted': 0L, 'created_at': datetime.datetime(2014, 12, 4, 10, 16, 11), 'updated_at': datetime.datetime(2014, 12, 8, 11, 48, 7), 'report_count': 4807L, 'topic': u'scheduler', 'host': u'controller1', 'disabled': False, 'deleted_at': None, 'disabled_reason': None, 'id': 3L}] is part of the ServiceGroup, is up service_is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:116
2014-12-08 11:48:07.851 12821 DEBUG nova.servicegroup.drivers.db [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] DB_Driver.is_up last_heartbeat = 2014-12-08 11:48:07 elapsed = 0.851661 is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/db.py:71
nova-scheduler   controller1                          internal         enabled    :-)   2014-12-08 11:48:07
2014-12-08 11:48:07.852 12821 DEBUG nova.servicegroup.api [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] Check if the given member [{'binary': u'nova-conductor', 'availability_zone': 'internal', 'deleted': 0L, 'created_at': datetime.datetime(2014, 12, 4, 10, 16, 21), 'updated_at': datetime.datetime(2014, 12, 8, 11, 48, 7), 'report_count': 4808L, 'topic': u'conductor', 'host': u'controller1', 'disabled': False, 'deleted_at': None, 'disabled_reason': None, 'id': 4L}] is part of the ServiceGroup, is up service_is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:116
2014-12-08 11:48:07.852 12821 DEBUG nova.servicegroup.drivers.db [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] DB_Driver.is_up last_heartbeat = 2014-12-08 11:48:07 elapsed = 0.852286 is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/db.py:71
nova-conductor   controller1                          internal         enabled    :-)   2014-12-08 11:48:07
2014-12-08 11:48:07.852 12821 DEBUG nova.servicegroup.api [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] Check if the given member [{'binary': u'nova-compute', 'availability_zone': 'nova', 'deleted': 0L, 'created_at': datetime.datetime(2014, 12, 4, 10, 39, 54), 'updated_at': datetime.datetime(2014, 12, 8, 11, 48, 6), 'report_count': 3040L, 'topic': u'compute', 'host': u'compute1', 'disabled': False, 'deleted_at': None, 'disabled_reason': None, 'id': 5L}] is part of the ServiceGroup, is up service_is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:116
2014-12-08 11:48:07.853 12821 DEBUG nova.servicegroup.drivers.db [req-70f5ce0b-a6ed-4071-ac11-d63d79b0ae73 None None] DB_Driver.is_up last_heartbeat = 2014-12-08 11:48:06 elapsed = 1.853084 is_up /usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/db.py:71
nova-compute     compute1                             nova             enabled    :-)   2014-12-08 11:48:06
11:48:06

any help in troubleshooting this would be greatly received thanks

Update:

i checked the nova-compute.log and there are some errors in it

2014-12-08 14:22:23.423 26914 ERROR nova.virt.libvirt.driver [-] [instance: 3a6e02e3-65e9-4cc3-bf82-8cad375ad725] During wait destroy, instance disappeared. 2014-12-08 14:22:23.864 26914 ERROR nova.compute.manager [req-964ed70c-5d18-4bb3-a0d9-c4c7f9fe1d8e 0b188de927e34893acd94a489fb1e167 d0d944054e504a7eb2ab114c0de98d63] [instance: 3a6e02e3-65e9-4cc3-bf82-8cad375ad725] Error: Not authorized for image 278cf386-0222-48b1-be3d-b2b32a3e523b. 2014-12-08 14:22:24.600 26914 ERROR glanceclient.common.http [-] Request returned failure status. 2014-12-08 14:22:24.601 26914 ERROR nova.compute.manager [req-964ed70c-5d18-4bb3-a0d9-c4c7f9fe1d8e 0b188de927e34893acd94a489fb1e167 d0d944054e504a7eb2ab114c0de98d63] [instance: 3a6e02e3-65e9-4cc3-bf82-8cad375ad725] Instance failed to spawn 2014-12-08 14:22:24.776 26914 ERROR nova.virt.libvirt.driver [-] [instance: 3a6e02e3-65e9-4cc3-bf82-8cad375ad725] During wait destroy, instance disappeared. 2014-12-08 14:22:25.060 26914 ERROR nova.compute.manager [req-964ed70c-5d18-4bb3-a0d9-c4c7f9fe1d8e 0b188de927e34893acd94a489fb1e167 d0d944054e504a7eb2ab114c0de98d63] [instance: 3a6e02e3-65e9-4cc3-bf82-8cad375ad725] Error: Not authorized for image 278cf386-0222-48b1-be3d-b2b32a3e523b. 2014-12-08 14:22:25.711 26914 ERROR glanceclient.common.http [-] Request returned failure status. 2014-12-08 14:22:25.712 26914 ERROR nova.compute.manager [req-964ed70c-5d18-4bb3-a0d9-c4c7f9fe1d8e 0b188de927e34893acd94a489fb1e167 d0d944054e504a7eb2ab114c0de98d63] [instance: 3a6e02e3-65e9-4cc3-bf82-8cad375ad725] Instance failed to spawn 2014-12-08 14:22:25.936 26914 ERROR nova.virt.libvirt.driver [-] [instance: 3a6e02e3-65e9-4cc3-bf82-8cad375ad725] During wait destroy, instance disappeared. 2014-12-08 14:22:26.395 26914 ERROR nova.compute.manager [req-964ed70c-5d18-4bb3-a0d9-c4c7f9fe1d8e 0b188de927e34893acd94a489fb1e167 d0d944054e504a7eb2ab114c0de98d63] [instance: 3a6e02e3-65e9-4cc3-bf82-8cad375ad725] Error: Not authorized for image 278cf386-0222-48b1-be3d-b2b32a3e523b.

so i downloaded a new image, it is the cirrus one from the icehouse install guide so it should be qemu compatible

here is my nova.conf from compute 

[DEFAULT] debug=true dhcpbridge_flagfile=/etc/nova/nova.conf dhcpbridge=/usr/bin/nova-dhcpbridge logdir=/var/log/nova state_path=/var/lib/nova lock_path=/var/lock/nova force_dhcp_release=True iscsi_helper=tgtadm libvirt_use_virtio_for_bridges=True connection_type=libvirt root_helper=sudo nova-rootwrap /etc/nova/rootwrap.conf verbose=True ec2_private_dns_show_ip=True api_paste_config=/etc/nova/api-paste.ini volumes_path=/var/lib/nova/volumes enabled_apis=ec2,osapi_compute,metadata auth_stratergy = keystone rpc_backend = rabbit rabbit_host = controller1 rabbit_password = rabbitpasss my_ip = controllermgmtip vnc_enabled = True vncserver_listen = 0.0.0.0 vncserver_proxyclient_address = controllermgmtip novncproxy_base_url = http://controller1:6080/vnc_auto.html network_api_class = nova.network.neutronv2.api.API neutron_url = http://controller1:9696 neutron_auth_strategy = keystone neutron_admin_tenant_name = service neutron_admin_username = neutron neutron_admin_password = neutronpass neutron_admin_auth_url = http://controller1:35357/v2.0 linuxnet_interface_driver = nova.network.linux_net.LinuxOVSInterfaceDriver firewall_driver = nova.virt.firewall.NoopFirewallDriver security_group_api = neutron glance_host=controller1 scheduler_default_filters=AllHostsFilter

[database]
The SQLAlchemy connection string used to connect to the database

connection = mysql://nova:novadbpass@controller/nova

[keystone_authtoken] auth_uri = http://controller1:5000 auth_host = controller1 auth_port = 35357 auth_protocol = http admin_tenant_name = service admin_user = nova admin_password = novapass

and here is the one from the controller node 

[DEFAULT] debug=true dhcpbridge_flagfile=/etc/nova/nova.conf dhcpbridge=/usr/bin/nova-dhcpbridge logdir=/var/log/nova state_path=/var/lib/nova lock_path=/var/lock/nova force_dhcp_release=True iscsi_helper=tgtadm libvirt_use_virtio_for_bridges=True connection_type=libvirt root_helper=sudo nova-rootwrap /etc/nova/rootwrap.conf verbose=True ec2_private_dns_show_ip=True api_paste_config=/etc/nova/api-paste.ini volumes_path=/var/lib/nova/volumes enabled_apis=ec2,osapi_compute,metadata rpc_backend = rabbit rabbit_host = controller1 rabbit_password = rabbitpass auth_strategy = keystone my_ip = controllermgmtip vncserver_listen = controllermgmtip vncserver_proxyclient_address = controllermgmtip network_api_class = nova.network.neutronv2.api.API neutron_url = http://controller1:9696 neutron_auth_strategy = keystone neutron_admin_tenant_name = service neutron_admin_username = neutron neutron_admin_password = neutronpass neutron_admin_auth_url = http://controller1:35357/v2.0 linuxnet_interface_driver = nova.network.linux_net.LinuxOVSInterfaceDriver firewall_driver = nova.virt.firewall.NoopFirewallDriver security_group_api = neutron service_neutron_metadata_proxy = true neutron_metadata_proxy_shared_secret = sharedsecret scheduler_default_filters=AllHostsFilter

host = controller1

[database] connection = mysql://nova:novadbpass@controller1/nova

[keystone_authtoken] auth_uri = http://controller1:5000 auth_host = controller1 auth_port = 35357 auth_protocol = http admin_tenant_name = service admin_user = nova admin_password = novapass

i have restarted the services and also checked the libvirtd.log on both the controller and compute nodes

i see this error from compute 

2014-12-08 14:22:00.223+0000: 1839: error : virNetSocketReadWire:1454 : End of file while reading data: Input/output error

and this from controller 

2014-12-08 14:33:22.290+0000: 5353: error : virNetSocketReadWire:1454 : End of file while reading data: Input/output error

thanks in advance for any help