Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

I added some resources in my flavor. Now, baremetal nodes are power on but the provisioning state is "wait call-back" and stack failed:

2019-02-21 14:50:04.263 8 DEBUG nova.virt.ironic.driver [-] [instance: 4b053b9b-d379-426f-89ba-b4c46c02aecf] Still waiting for ironic node 0fc5aab5-5d66-4b07-8452-2dd75e0a8dba to become ACTIVE: power_state="power on", target_power_state= None, provision_state="wait call-back", target_provision_state="active" _log_ironic_polling /usr/lib/python2.7/site-packages/nova/virt/ironic/driver.py:131 2019-02-21 14:50:05.493 8 ERROR oslo.service.loopingcall [-] Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to provision instance 8bc4c696-f63b-4983-a6f6-66cf9271 fbcf: Timeout reached while waiting for callback for node e940f5da-6512-449b-a1ab-4191029041fb 2019-02-21 14:50:05.493 8 ERROR oslo.service.loopingcall Traceback (most recent call last): 2019-02-21 14:50:05.493 8 ERROR oslo.service.loopingcall File "/usr/lib/python2.7/site-packages/oslo_service/loopingcall.py", line 187, in _run_loop 2019-02-21 14:50:05.493 8 ERROR oslo.service.loopingcall result = func(self.args, *self.kw) 2019-02-21 14:50:05.493 8 ERROR oslo.service.loopingcall File "/usr/lib/python2.7/site-packages/nova/virt/ironic/driver.py", line 509, in _wait_for_active 2019-02-21 14:50:05.493 8 ERROR oslo.service.loopingcall raise exception.InstanceDeployFailure(msg) 2019-02-21 14:50:05.493 8 ERROR oslo.service.loopingcall InstanceDeployFailure: Failed to provision instance 8bc4c696-f63b-4983-a6f6-66cf9271fbcf: Timeout reached while waiting for callback for node e940f5da-6512-449b-a1ab-4191029041fb 2019-02-21 14:50:05.493 8 ERROR oslo.service.loopingcall 2019-02-21 14:50:05.495 8 ERROR nova.virt.ironic.driver [req-db44df35-092e-441e-a98f-3262ff5cea53 930bb6fe0aba42d0bc757d5ab0c8af26 ec351b6a59a246d6932fc973e0535339 - default default] Error deploying instance 8bc4c696-f63b-4983-a6f6-66cf9271fbcf on baremetal node e940f5da-6512-449b-a1ab-4191029041fb.: InstanceDeployFailure: Failed to provision instance 8bc4c696-f63b-4983-a6f6-66cf9271fbcf: Timeout reached while waiting for callback for node e940f5da-6512-449b-a1ab-4191029041fb 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [req-db44df35-092e-441e-a98f-3262ff5cea53 930bb6fe0aba42d0bc757d5ab0c8af26 ec351b6a59a246d6932fc973e0535339 - default default] [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] Instance failed to spawn: InstanceDeployFailure: Failed to provision instance 8bc4c696-f63b-4983-a6f6-66cf9271fbcf: Timeout reached while waiting for callback for node e940f5da-6512-449b-a1ab-4191029041fb 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] Traceback (most recent call last): 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2368, in _build_resources 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] yield resources 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2132, in _build_and_run_instance 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] block_device_info=block_device_info) 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] File "/usr/lib/python2.7/site-packages/nova/virt/ironic/driver.py", line 1158, in spawn 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] 'node': node_uuid}) 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] self.force_reraise() 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] six.reraise(self.type_, self.value, self.tb) 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] File "/usr/lib/python2.7/site-packages/nova/virt/ironic/driver.py", line 1150, in spawn 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] timer.start(interval=CONF.ironic.api_retry_interval).wait() 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] return hubs.get_hub().switch() 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 294, in switch 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] return self.greenlet.switch() 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] File "/usr/lib/python2.7/site-packages/oslo_service/loopingcall.py", line 187, in _run_loop 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] result = func(self.args, *self.kw) 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] File "/usr/lib/python2.7/site-packages/nova/virt/ironic/driver.py", line 509, in _wait_for_active 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] raise exception.InstanceDeployFailure(msg) 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] InstanceDeployFailure: Failed to provision instance 8bc4c696-f63b-4983-a6f6-66cf9271fbcf: Timeout reached while waiting for callback for node e940f5da-6512-449b-a1ab-4191029041fb 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] 2019-02-21 14:50:05.497 8 INFO nova.compute.manager [req-db44df35-092e-441e-a98f-3262ff5cea53 930bb6fe0aba42d0bc757d5ab0c8af26 ec351b6a59a246d6932fc973e0535339 - default default] [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] Terminating instance 2019-02-21 14:50:05.514 8 DEBUG nova.compute.manager [req-db44df35-092e-441e-a98f-3262ff5cea53 930bb6fe0aba42d0bc757d5ab0c8af26 ec351b6a59a246d6932fc973e0535339 - default default] [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] Start destroying the instance on the hypervisor. _shutdown_instance /usr/lib/python2.7/site-packages/nova/compute/manager.py:2500 2019-02-21 14:50:05.515 8 DEBUG nova.virt.ironic.driver [req-db44df35-092e-441e-a98f-3262ff5cea53 930bb6fe0aba42d0bc757d5ab0c8af26 ec351b6a59a246d6932fc973e0535339 - default default] [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] Destroy called for instance destroy /usr/lib/python2.7/site-packages/nova/virt/ironic/driver.py:1240 2019-02-21 14:50:05.854 8 DEBUG nova.virt.ironic.driver [-] [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] Still waiting for ironic node e940f5da-6512-449b-a1ab-4191029041fb to unprovision: power_state="power on", target_power_state=None, provision_state="deleting", target_provision_state="available" _log_ironic_polling /usr/lib/python2.7/site-packages/nova/virt/ironic/driver.py:131 2019-02-21 14:50:06.258 8 ERROR oslo.service.loopingcall [-] Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to provision instance 4b053b9b-d379-426f-89ba-b4c46c02aecf: Timeout reached while waiting for callback for node 0fc5aab5-5d66-4b07-8452-2dd75e0a8dba