Ask Your Question
0

TripleO Overcloud installation failed

asked 2018-12-28 03:43:25 -0500

asrmarco13 gravatar image

Hello, I use TripleO (Rocky version) to deploy 2 baremetal nodes. I followed the instructions as per https://docs.openstack.org/tripleo-docs/latest/install/basic_deployment/basic_deployment_cli.html (https://docs.openstack.org/tripleo-do...) The heat stack-list shows CREATE_FAILED.

openstack stack list -c 'Stack Name' -c 'Stack Status'         

| Stack Name | Stack Status  |
| overcloud  | CREATE_FAILED |

Controller e Compute node creation fails

openstack stack resource list overcloud -c resource_name -c resource_status | grep 'CREATE_FAILED'

| Controller                                | CREATE_FAILED   |
| Compute                                   | CREATE_FAILED   |

The status reason for Controller and Compute reported the following message: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fc185e0f-c5ef-433d-a234-8b8a5f6daeb1., Code: 500

openstack stack resource show overcloud Controller
+------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field                  | Value                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
+------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| attributes             | {u'attributes': None, u'refs': None, u'refs_map': None, u'removed_rsrc_list': []}                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| creation_time          | 2018-12-27T18:11:55Z                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| description            |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| links                  | [{u'href': u'http://192.168.51.28:8004/v1/8f6ef97fe47149fbbd37c9ab8fb3b22e/stacks/overcloud/f85239bb-671f-4e5f-b15c-2475e48b2205/resources/Controller', u'rel': u'self'}, {u'href': u'http://192.168.51.28:8004/v1/8f6ef97fe47149fbbd37c9ab8fb3b22e/stacks/overcloud/f85239bb-671f-4e5f-b15c-2475e48b2205', u'rel': u'stack'}, {u'href': u'http://192.168.51.28:8004/v1/8f6ef97fe47149fbbd37c9ab8fb3b22e/stacks/overcloud-Controller-jinynqgg5ksd/545e4c39-888b-43ae-8375-6eedb6c6ae5a', u'rel': u'nested'}]                                 |
| logical_resource_id    | Controller                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| physical_resource_id   | 545e4c39-888b-43ae-8375-6eedb6c6ae5a                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| required_by            | [u'allNodesConfig', u'SshKnownHostsHostnames', u'BlockStorageAllNodesValidationConfig', u'ControllerAllNodesDeployment', u'SshKnownHostsConfig', u'ServerOsCollectConfigData', u'BlacklistedHostnames', u'ControllerIpListMap', u'DeployedServerEnvironment', u'ControllerAllNodesValidationConfig', u'hostsConfig', u'ControllerServers', u'ServerIdMap', u'CephStorageAllNodesValidationConfig', u'BlacklistedIpAddresses', u'ComputeAllNodesValidationConfig', u'ControllerNetworkHostnameMap', u'ObjectStorageAllNodesValidationConfig'] |
| resource_name          | Controller                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| resource_status        | CREATE_FAILED                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| resource_status_reason | resources.Controller: Resource CREATE failed: ResourceInError: resources[0].resources.Controller: Went to status ERROR due to "Message: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fc185e0f-c5ef-433d-a234-8b8a5f6daeb1., Code: 500"                                                                                                                                                                                                                                         |
| resource_type          | OS::Heat::ResourceGroup                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| updated_time           | 2018-12-27T18:11:55Z                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
+------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Compute resource

openstack stack resource show overcloud Compute
+------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field                  | Value                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
+------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| attributes             | {u'attributes': None, u'refs': None, u'refs_map': None, u'removed_rsrc_list': []}                                                                                                                                                                                                                                                                                                                                                                                                                      |
| creation_time          | 2018-12-27T18:11:55Z                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| description            |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| links                  | [{u'href': u'http://192.168.51.28:8004/v1/8f6ef97fe47149fbbd37c9ab8fb3b22e/stacks/overcloud/f85239bb-671f-4e5f-b15c-2475e48b2205/resources/Compute', u'rel': u'self'}, {u'href': u'http://192.168.51.28:8004/v1/8f6ef97fe47149fbbd37c9ab8fb3b22e/stacks/overcloud/f85239bb-671f-4e5f-b15c-2475e48b2205', u'rel': u'stack'}, {u'href': u'http://192.168.51.28:8004/v1/8f6ef97fe47149fbbd37c9ab8fb3b22e/stacks/overcloud-Compute-33mhizavm4gp/be8a7ff2-798f-4204-b833-411c525e67e6', u'rel': u'nested'}] |
| logical_resource_id    | Compute                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| physical_resource_id   | be8a7ff2-798f-4204-b833-411c525e67e6                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| required_by            | [u'DeployedServerEnvironment', u'SshKnownHostsConfig', u'ComputeIpListMap', u'ComputeNetworkHostnameMap', u'allNodesConfig', u'BlacklistedHostnames', u'BlacklistedIpAddresses', u'ComputeServers', u'SshKnownHostsHostnames', u'ServerOsCollectConfigData', u'ServerIdMap', u'ComputeAllNodesDeployment', u'hostsConfig']                                                                                                                                                                             |
| resource_name          | Compute                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| resource_status        | CREATE_FAILED                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| resource_status_reason | resources.Compute: Resource CREATE failed: ResourceInError: resources[0].resources.NovaCompute: Went to status ERROR due to "Message: Exceeded maximum number of retries. Exceeded max scheduling attempts 2 for instance d7313939-6465-4018-9cde-f06dbec6dde6. Last exception: Failed to provision instance d7313939-6465-4018-9cde-f06dbec6dde6: Timeout reached while waiting for callback for nod, Code: 500"                                                                                      |
| resource_type          | OS::Heat::ResourceGroup                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| updated_time           | 2018-12-27T18:11:55Z                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
+------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

The PXE credentials are correct but the overcloud creation fails. Following the heat and nova log. Heat log

2018-12-27 22:10:53.968 39 DEBUG novaclient.v2.client [req-fe975a84-0ecb-4513-840a-32c277b7d644 - admin - default default] RESP BODY: {"server": {"OS-EXT-STS:task_state": null, "addresses": {}, "links": [{"href": "http://192.168.51.28:8774/v2.1/servers/7a2b0efd-7ff5-4067-8338-c75d479c0c9b", "rel ...
(more)
edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted
0

answered 2019-01-05 08:12:08 -0500

amkgi gravatar image

updated 2019-01-05 14:14:02 -0500

I had this problem. My overcloud has compute on bare metal and a controller on a VM. Failed to deploy the controller. I added some resources to the CPU, RAM and HDD, and the problem disappeared. Maybe you have too small resources on your bare metal nodes too. Take a look "openstack flavor list".

edit flag offensive delete link more

Comments

Hi amkgi, my baremetal nodes have much more resources then required. Anyway, now i have a different problem: when barematel nodes up the Provisioning State is "wait call-back".

asrmarco13 gravatar imageasrmarco13 ( 2019-02-21 08:41:21 -0500 )edit
0

answered 2019-02-21 08:48:39 -0500

asrmarco13 gravatar image

I added some resources in my flavor. Now, baremetal nodes are power on but the provisioning state is "wait call-back" and stack failed:

2019-02-21 14:50:04.263 8 DEBUG nova.virt.ironic.driver [-] [instance: 4b053b9b-d379-426f-89ba-b4c46c02aecf] Still waiting for ironic node 0fc5aab5-5d66-4b07-8452-2dd75e0a8dba to become ACTIVE: power_state="power on", target_power_state= None, provision_state="wait call-back", target_provision_state="active" _log_ironic_polling /usr/lib/python2.7/site-packages/nova/virt/ironic/driver.py:131 2019-02-21 14:50:05.493 8 ERROR oslo.service.loopingcall [-] Fixed interval looping call 'nova.virt.ironic.driver.IronicDriver._wait_for_active' failed: InstanceDeployFailure: Failed to provision instance 8bc4c696-f63b-4983-a6f6-66cf9271 fbcf: Timeout reached while waiting for callback for node e940f5da-6512-449b-a1ab-4191029041fb 2019-02-21 14:50:05.493 8 ERROR oslo.service.loopingcall Traceback (most recent call last): 2019-02-21 14:50:05.493 8 ERROR oslo.service.loopingcall File "/usr/lib/python2.7/site-packages/oslo_service/loopingcall.py", line 187, in _run_loop 2019-02-21 14:50:05.493 8 ERROR oslo.service.loopingcall result = func(self.args, *self.kw) 2019-02-21 14:50:05.493 8 ERROR oslo.service.loopingcall File "/usr/lib/python2.7/site-packages/nova/virt/ironic/driver.py", line 509, in _wait_for_active 2019-02-21 14:50:05.493 8 ERROR oslo.service.loopingcall raise exception.InstanceDeployFailure(msg) 2019-02-21 14:50:05.493 8 ERROR oslo.service.loopingcall InstanceDeployFailure: Failed to provision instance 8bc4c696-f63b-4983-a6f6-66cf9271fbcf: Timeout reached while waiting for callback for node e940f5da-6512-449b-a1ab-4191029041fb 2019-02-21 14:50:05.493 8 ERROR oslo.service.loopingcall 2019-02-21 14:50:05.495 8 ERROR nova.virt.ironic.driver [req-db44df35-092e-441e-a98f-3262ff5cea53 930bb6fe0aba42d0bc757d5ab0c8af26 ec351b6a59a246d6932fc973e0535339 - default default] Error deploying instance 8bc4c696-f63b-4983-a6f6-66cf9271fbcf on baremetal node e940f5da-6512-449b-a1ab-4191029041fb.: InstanceDeployFailure: Failed to provision instance 8bc4c696-f63b-4983-a6f6-66cf9271fbcf: Timeout reached while waiting for callback for node e940f5da-6512-449b-a1ab-4191029041fb 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [req-db44df35-092e-441e-a98f-3262ff5cea53 930bb6fe0aba42d0bc757d5ab0c8af26 ec351b6a59a246d6932fc973e0535339 - default default] [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] Instance failed to spawn: InstanceDeployFailure: Failed to provision instance 8bc4c696-f63b-4983-a6f6-66cf9271fbcf: Timeout reached while waiting for callback for node e940f5da-6512-449b-a1ab-4191029041fb 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] Traceback (most recent call last): 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2368, in _build_resources 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] yield resources 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2132, in _build_and_run_instance 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] block_device_info=block_device_info) 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] File "/usr/lib/python2.7/site-packages/nova/virt/ironic/driver.py", line 1158, in spawn 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] 'node': node_uuid}) 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] self.force_reraise() 2019-02-21 14:50:05.495 8 ERROR nova.compute.manager [instance: 8bc4c696-f63b-4983-a6f6-66cf9271fbcf] File "/usr/lib/python2 ... (more)

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2018-12-28 03:43:25 -0500

Seen: 145 times

Last updated: Jan 05