Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

TripleO Overcloud installation failed

Hello, I use TripleO (Rocky version) to deploy 2 baremetal nodes. I followed the instructions as per https://docs.openstack.org/tripleo-docs/latest/install/basic_deployment/basic_deployment_cli.html The heat stack-list shows CREATE_FAILED.

openstack stack list -c 'Stack Name' -c 'Stack Status'         

| Stack Name | Stack Status  |
| overcloud  | CREATE_FAILED |

Controller e Compute node creation fails

openstack stack resource list overcloud -c resource_name -c resource_status | grep 'CREATE_FAILED'

| Controller                                | CREATE_FAILED   |
| Compute                                   | CREATE_FAILED   |

The status reason for Controller and Compute reported the following message: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fc185e0f-c5ef-433d-a234-8b8a5f6daeb1., Code: 500

openstack stack resource show overcloud Controller
+------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field                  | Value                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
+------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| attributes             | {u'attributes': None, u'refs': None, u'refs_map': None, u'removed_rsrc_list': []}                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| creation_time          | 2018-12-27T18:11:55Z                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| description            |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| links                  | [{u'href': u'http://192.168.51.28:8004/v1/8f6ef97fe47149fbbd37c9ab8fb3b22e/stacks/overcloud/f85239bb-671f-4e5f-b15c-2475e48b2205/resources/Controller', u'rel': u'self'}, {u'href': u'http://192.168.51.28:8004/v1/8f6ef97fe47149fbbd37c9ab8fb3b22e/stacks/overcloud/f85239bb-671f-4e5f-b15c-2475e48b2205', u'rel': u'stack'}, {u'href': u'http://192.168.51.28:8004/v1/8f6ef97fe47149fbbd37c9ab8fb3b22e/stacks/overcloud-Controller-jinynqgg5ksd/545e4c39-888b-43ae-8375-6eedb6c6ae5a', u'rel': u'nested'}]                                 |
| logical_resource_id    | Controller                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| physical_resource_id   | 545e4c39-888b-43ae-8375-6eedb6c6ae5a                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| required_by            | [u'allNodesConfig', u'SshKnownHostsHostnames', u'BlockStorageAllNodesValidationConfig', u'ControllerAllNodesDeployment', u'SshKnownHostsConfig', u'ServerOsCollectConfigData', u'BlacklistedHostnames', u'ControllerIpListMap', u'DeployedServerEnvironment', u'ControllerAllNodesValidationConfig', u'hostsConfig', u'ControllerServers', u'ServerIdMap', u'CephStorageAllNodesValidationConfig', u'BlacklistedIpAddresses', u'ComputeAllNodesValidationConfig', u'ControllerNetworkHostnameMap', u'ObjectStorageAllNodesValidationConfig'] |
| resource_name          | Controller                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| resource_status        | CREATE_FAILED                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| resource_status_reason | resources.Controller: Resource CREATE failed: ResourceInError: resources[0].resources.Controller: Went to status ERROR due to "Message: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fc185e0f-c5ef-433d-a234-8b8a5f6daeb1., Code: 500"                                                                                                                                                                                                                                         |
| resource_type          | OS::Heat::ResourceGroup                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| updated_time           | 2018-12-27T18:11:55Z                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
+------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Compute resource

openstack stack resource show overcloud Compute
+------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field                  | Value                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
+------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| attributes             | {u'attributes': None, u'refs': None, u'refs_map': None, u'removed_rsrc_list': []}                                                                                                                                                                                                                                                                                                                                                                                                                      |
| creation_time          | 2018-12-27T18:11:55Z                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| description            |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| links                  | [{u'href': u'http://192.168.51.28:8004/v1/8f6ef97fe47149fbbd37c9ab8fb3b22e/stacks/overcloud/f85239bb-671f-4e5f-b15c-2475e48b2205/resources/Compute', u'rel': u'self'}, {u'href': u'http://192.168.51.28:8004/v1/8f6ef97fe47149fbbd37c9ab8fb3b22e/stacks/overcloud/f85239bb-671f-4e5f-b15c-2475e48b2205', u'rel': u'stack'}, {u'href': u'http://192.168.51.28:8004/v1/8f6ef97fe47149fbbd37c9ab8fb3b22e/stacks/overcloud-Compute-33mhizavm4gp/be8a7ff2-798f-4204-b833-411c525e67e6', u'rel': u'nested'}] |
| logical_resource_id    | Compute                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| physical_resource_id   | be8a7ff2-798f-4204-b833-411c525e67e6                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| required_by            | [u'DeployedServerEnvironment', u'SshKnownHostsConfig', u'ComputeIpListMap', u'ComputeNetworkHostnameMap', u'allNodesConfig', u'BlacklistedHostnames', u'BlacklistedIpAddresses', u'ComputeServers', u'SshKnownHostsHostnames', u'ServerOsCollectConfigData', u'ServerIdMap', u'ComputeAllNodesDeployment', u'hostsConfig']                                                                                                                                                                             |
| resource_name          | Compute                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| resource_status        | CREATE_FAILED                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| resource_status_reason | resources.Compute: Resource CREATE failed: ResourceInError: resources[0].resources.NovaCompute: Went to status ERROR due to "Message: Exceeded maximum number of retries. Exceeded max scheduling attempts 2 for instance d7313939-6465-4018-9cde-f06dbec6dde6. Last exception: Failed to provision instance d7313939-6465-4018-9cde-f06dbec6dde6: Timeout reached while waiting for callback for nod, Code: 500"                                                                                      |
| resource_type          | OS::Heat::ResourceGroup                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| updated_time           | 2018-12-27T18:11:55Z                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
+------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

The PXE credentials are correct but the overcloud creation fails. Following the heat and nova log. Heat log

2018-12-27 22:10:53.968 39 DEBUG novaclient.v2.client [req-fe975a84-0ecb-4513-840a-32c277b7d644 - admin - default default] RESP BODY: {"server": {"OS-EXT-STS:task_state": null, "addresses": {}, "links": [{"href": "http://192.168.51.28:8774/v2.1/servers/7a2b0efd-7ff5-4067-8338-c75d479c0c9b", "rel": "self"}, {"href": "http://192.168.51.28:8774/servers/7a2b0efd-7ff5-4067-8338-c75d479c0c9b", "rel": "bookmark"}], "image": {"id": "7b844d78-ee48-41cc-aaaf-35850d5858eb", "links": [{"href": "http://192.168.51.28:8774/images/7b844d78-ee48-41cc-aaaf-35850d5858eb", "rel": "bookmark"}]}, "OS-EXT-SRV-ATTR:user_data": "Q29udGVudC1UeXBlOiBtdWx0aXBhcnQvbWl4ZWQ7IGJvdW5kYXJ5PSI9PT09PT09PT09PT09PT0yNTUyNzMxMDYyOTE2MDQ1MDQ3PT0iCk1JTUUtVmVyc2lvbjogMS4wCgotLT09PT09PT09PT09PT09PTI1NTI3MzEwNjI5MTYwNDUwNDc9PQpDb250ZW50LVR5cGU6IHRleHQvY2xvdWQtY29uZmlnOyBjaGFyc2V0PSJ1cy1hc2NpaSIKTUlNRS1WZXJzaW9uOiAxLjAKQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzogN2JpdApDb250ZW50LURpc3Bvc2l0aW9uOiBhdHRhY2htZW50OyBmaWxlbmFtZT0iY2xvdWQtY29uZmlnIgoKCgojIENhcHR1cmUgYWxsIHN1YnByb2Nlc3Mgb3V0cHV0IGludG8gYSBsb2dmaWxlCiMgVXNlZnVsIGZvciB0cm91Ymxlc2hvb3RpbmcgY2xvdWQtaW5pdCBpc3N1ZXMKb3V0cHV0OiB7YWxsOiAnfCB0ZWUgLWEgL3Zhci9sb2cvY2xvdWQtaW5pdC1vdXRwdXQubG9nJ30KCi0tPT09PT09PT09PT09PT09MjU1MjczMTA2MjkxNjA0NTA0Nz09CkNvbnRlbnQtVHlwZTogdGV4dC9jbG91ZC1ib290aG9vazsgY2hhcnNldD0idXMtYXNjaWkiCk1JTUUtVmVyc2lvbjogMS4wCkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IDdiaXQKQ29udGVudC1EaXNwb3NpdGlvbjogYXR0YWNobWVudDsgZmlsZW5hbWU9ImJvb3Rob29rLnNoIgoKIyEvYmluL2Jhc2gKCiMgRklYTUUoc2hhZG93ZXIpIHRoaXMgaXMgYSB3b3JrYXJvdW5kIGZvciBjbG91ZC1pbml0IDAuNi4zIHByZXNlbnQgaW4gVWJ1bnR1CiMgMTIuMDQgTFRTOgojIGh0dHBzOi8vYnVncy5sYXVuY2hwYWQubmV0L2hlYXQvK2J1Zy8xMjU3NDEwCiMKIyBUaGUgb2xkIGNsb3VkLWluaXQgZG9lc24ndCBjcmVhdGUgdGhlIHVzZXJzIGRpcmVjdGx5IHNvIHRoZSBjb21tYW5kcyB0byBkbwojIHRoaXMgYXJlIGluamVjdGVkIHRob3VnaCBub3ZhX3V0aWxzLnB5LgojCiMgT25jZSB3ZSBkcm9wIHN1cHBvcnQgZm9yIDAuNi4zLCB3ZSBjYW4gc2FmZWx5IHJlbW92ZSB0aGlzLgoKCiMgaW4gY2FzZSBoZWF0LWNmbnRvb2xzIGhhcyBiZWVuIGluc3RhbGxlZCBmcm9tIHBhY2thZ2UgYnV0IG5vIHN5bWxpbmtzCiMgYXJlIHlldCBpbiAvb3B0L2F3cy9iaW4vCmNmbi1jcmVhdGUtYXdzLXN5bWxpbmtzCgojIERvIG5vdCByZW1vdmUgLSB0aGUgY2xvdWQgYm9vdGhvb2sgc2hvdWxkIGFsd2F5cyByZXR1cm4gc3VjY2VzcwpleGl0IDAKCi0tPT09PT09PT09PT09PT09MjU1MjczMTA2MjkxNjA0NTA0Nz09CkNvbnRlbnQtVHlwZTogdGV4dC9wYXJ0LWhhbmRsZXI7IGNoYXJzZXQ9InVzLWFzY2lpIgpNSU1FLVZlcnNpb246IDEuMApDb250ZW50LVRyYW5zZmVyLUVuY29kaW5nOiA3Yml0CkNvbnRlbnQtRGlzcG9zaXRpb246IGF0dGFjaG1lbnQ7IGZpbGVuYW1lPSJwYXJ0LWhhbmRsZXIucHkiCgojIHBhcnQtaGFuZGxlcgojCiMgICAgTGljZW5zZWQgdW5kZXIgdGhlIEFwYWNoZSBMaWNlbnNlLCBWZXJzaW9uIDIuMCAodGhlICJMaWNlbnNlIik7IHlvdSBtYXkKIyAgICBub3QgdXNlIHRoaXMgZmlsZSBleGNlcHQgaW4gY29tcGxpYW5jZSB3aXRoIHRoZSBMaWNlbnNlLiBZb3UgbWF5IG9idGFpbgojICAgIGEgY29weSBvZiB0aGUgTGljZW5zZSBhdAojCiMgICAgICAgICBodHRwOi8vd3d3LmFwYWNoZS5vcmcvbGljZW5zZXMvTElDRU5TRS0yLjAKIwojICAgIFVubGVzcyByZXF1aXJlZCBieSBhcHBsaWNhYmxlIGxhdyBvciBhZ3JlZWQgdG8gaW4gd3JpdGluZywgc29mdHdhcmUKIyAgICBkaXN0cmlidXRlZCB1bmRlciB0aGUgTGljZW5zZSBpcyBkaXN0cmlidXRlZCBvbiBhbiAiQVMgSVMiIEJBU0lTLCBXSVRIT1VUCiMgICAgV0FSUkFOVElFUyBPUiBDT05ESVRJT05TIE9GIEFOWSBLSU5ELCBlaXRoZXIgZXhwcmVzcyBvciBpbXBsaWVkLiBTZWUgdGhlCiMgICAgTGljZW5zZSBmb3IgdGhlIHNwZWNpZmljIGxhbmd1YWdlIGdvdmVybmluZyBwZXJtaXNzaW9ucyBhbmQgbGltaXRhdGlvbnMKIyAgICB1bmRlciB0aGUgTGljZW5zZS4KCmltcG9ydCBkYXRldGltZQppbXBvcnQgZXJybm8KaW1wb3J0IG9zCmltcG9ydCBzeXMKCgpkZWYgbGlzdF90eXBlcygpOgogICAgcmV0dXJuKFsidGV4dC94LWNmbmluaXRkYXRhIl0pCgoKZGVmIGhhbmRsZV9wYXJ0KGRhdGEsIGN0eXBlLCBmaWxlbmFtZSwgcGF5bG9hZCk6CiAgICBpZiBjdHlwZSA9PSAiX19iZWdpbl9fIjoKICAgICAgICB0cnk6CiAgICAgICAgICAgIG9zLm1ha2VkaXJzKCcvdmFyL2xpYi9oZWF0LWNmbnRvb2xzJywgaW50KCI3MDAiLCA4KSkKICAgICAgICBleGNlcHQgT1NFcnJvcjoKICAgICAgICAgICAgZXhfdHlwZSwgZSwgdGIgPSBzeXMuZXhjX2luZm8oKQogICAgICAgICAgICBpZiBlLmVycm5vICE9IGVycm5vLkVFWElTVDoKICAgICAgICAgICAgICAgIHJhaXNlCiAgICAgICAgcmV0dXJuCgogICAgaWYgY3R5cGUgPT0gIl9fZW5kX18iOgogICAgICAgIHJldHVybgoKICAgIHRpbWVzdGFtcCA9IGRhdGV0aW1lLmRhdGV0aW1lLm5vdygpCiAgICB3aXRoIG9wZW4oJy92YXIvbG9nL3BhcnQtaGFuZGxlci5sb2cnLCAnYScpIGFzIGxvZzoKICAgICAgICBsb2cud3JpdGUoJyVzIGZpbGVuYW1lOiVzLCBjdHlwZTolc1xuJyAlICh0aW1lc3RhbXAsIGZpbGVuYW1lLCBjdHlwZSkpCgogICAgaWYgY3R5cGUgPT0gJ3RleHQveC1jZm5pbml0ZGF0YSc6CiAgICAgICAgd2l0aCBvcGVuKCcvdmFyL2xpYi9oZWF0LWNmbnRvb2xzLyVzJyAlIGZpbGVuYW1lLCAndycpIGFzIGY6CiAgICAgICAgICAgIGYud3JpdGUocGF5bG9hZCkKCiAgICAgICAgIyBUT0RPKHNkYWtlKSBob3BlZnVsbHkgdGVtcG9yYXJ5IHVudGlsIHVzZXJzIG1vdmUgdG8gaGVhdC1jZm50b29scy0xLjMKICAgICAgICB3aXRoIG9wZW4oJy92YXIvbGliL2Nsb3VkL2RhdGEvJXMnICUgZmlsZW5hbWUsICd3JykgYXMgZjoKICAgICAgICAgICAgZi53cml0ZShwYXlsb2FkKQoKLS09PT09PT09PT09PT09PT0yNTUyNzMxMDYyOTE2MDQ1MDQ3PT0KQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFyc2V0PSJ1cy1hc2NpaSIKTUlNRS1WZXJzaW9uOiAxLjAKQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzogN2JpdApDb250ZW50LURpc3Bvc2l0aW9uOiBhdHRhY2htZW50OyBmaWxlbmFtZQoKI2Nsb3VkLWNvbmZpZwpzc2hfYXV0aG9yaXplZF9rZXlzOiBbc3NoLXJzYSBBQUFBQjNOemFDMXljMkVBQUFBREFRQUJBQUFCQVFEV1ZYTHhudXpnbGZRc1JmVVdqT0xUZEF4UUlRNk1sTFJaSmlKWWlUb0lmcE0zaERBS2FnbklDejBnY2JWRWVHMWJQTEtGZ3NIcXlTOW15c2p5RVEwYzJzcE9JUFpxTDE0QlAyS25VSTVYZXZUUmhBUVBXYVV6VXVxZExSeFVxT1Z2cFQ4KzBkWXRpd2NVd0RrcU9td2ljRWRDdVlVR2lIUzRmWHFjTnNvZmVYTkJUUzl1TytUZ2Y2cnRCRXh6MzF1cktNTVFGTUVzbFlYOCtXa21GRHFMZnVIQUZPbUswMHowd0JnN0tnUGdORUovWlJpLzhGb1NNL2FvUmVUWE5ORjhnUWpPTEpwOXg5MnRUT3luSmhVOWNLOFNVS1VRWnYycGFJSEtIQ3ltM1pGT1dUMGViZzcrWVFIdWJqUlZmcFVDM05TRnllZW84WkhGOXpVRGYxTGIKICAgIEdlbmVyYXRlZCBieSBUcmlwbGVPXQp1c2VyOiBoZWF0LWFkbWluCgotLT09PT09PT09PT09PT09PTI1NTI3MzEwNjI5MTYwNDUwNDc9PQpDb250ZW50LVR5cGU6IHRleHQvcGxhaW47IGNoYXJzZXQ9InVzLWFzY2lpIgpNSU1FLVZlcnNpb246IDEuMApDb250ZW50LVRyYW5zZmVyLUVuY29kaW5nOiA3Yml0CkNvbnRlbnQtRGlzcG9zaXRpb246IGF0dGFjaG1lbnQ7IGZpbGVuYW1lCgoKLS09PT09PT09PT09PT09PT0yNTUyNzMxMDYyOTE2MDQ1MDQ3PT0KQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFyc2V0PSJ1cy1hc2NpaSIKTUlNRS1WZXJzaW9uOiAxLjAKQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzogN2JpdApDb250ZW50LURpc3Bvc2l0aW9uOiBhdHRhY2htZW50OyBmaWxlbmFtZQoKCi0tPT09PT09PT09PT09PT09MjU1MjczMTA2MjkxNjA0NTA0Nz09CkNvbnRlbnQtVHlwZTogdGV4dC94LWNmbmluaXRkYXRhOyBjaGFyc2V0PSJ1cy1hc2NpaSIKTUlNRS1WZXJzaW9uOiAxLjAKQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzogN2JpdApDb250ZW50LURpc3Bvc2l0aW9uOiBhdHRhY2htZW50OyBmaWxlbmFtZT0iY2ZuLWluaXQtZGF0YSIKCnsib3MtY29sbGVjdC1jb25maWciOiB7InNwbGF5IjogMzAsICJjb21tYW5kIjogIm9zLXJlZnJlc2gtY29uZmlnIC0tdGltZW91dCAxNDQwMCIsICJyZXF1ZXN0IjogeyJtZXRhZGF0YV91cmwiOiAiaHR0cDovLzE5Mi4xNjguNTEuMjg6ODA4MC92MS9BVVRIXzhmNmVmOTdmZTQ3MTQ5ZmJiZDM3YzlhYjhmYjNiMjJlL292LW1oaXphdm00Z3AtMC1kc2Y1Y2VjdHhmaDUtTm92YUNvbXB1dGUteGhucGZ4Nm8zb3JsL2U5N2IyMDM5LWY5NDgtNDFkNC05ZmMwLWQzNWMxNTIyN2M1YT90ZW1wX3VybF9zaWc9N2MwNjUwZGIxNWMxNjY0Mzc0OGZlOTZiODgyZjljMzYwMWNjZWQxNiZ0ZW1wX3VybF9leHBpcmVzPTIxNDc0ODM1ODYifSwgImNvbGxlY3RvcnMiOiBbImVjMiIsICJyZXF1ZXN0IiwgImxvY2FsIl19LCAiZGVwbG95bWVudHMiOiBbXX0KLS09PT09PT09PT09PT09PT0yNTUyNzMxMDYyOTE2MDQ1MDQ3PT0tLQ==", "OS-EXT-STS:vm_state": "error", "OS-EXT-SRV-ATTR:instance_name": "instance-00000038", "OS-EXT-SRV-ATTR:root_device_name": "/dev/sda", "OS-SRV-USG:launched_at": null, "flavor": {"ephemeral": 0, "ram": 4096, "original_name": "baremetal", "vcpus": 1, "extra_specs": {"capabilities:boot_option": "local", "resources:CUSTOM_BAREMETAL": "1", "resources:MEMORY_MB": "0", "resources:VCPU": "0", "resources:DISK_GB": "0"}, "swap": 0, "disk": 40}, "id": "7a2b0efd-7ff5-4067-8338-c75d479c0c9b", "OS-SRV-USG:terminated_at": null, "os-extended-volumes:volumes_attached": [], "user_id": "960e4c0367064fb59e46b4d20e3510a7", "OS-EXT-SRV-ATTR:hostname": "overcloud-novacompute-0", "OS-DCF:diskConfig": "MANUAL", "accessIPv4": "", "accessIPv6": "", "OS-EXT-SRV-ATTR:reservation_id": "r-wqlrhomo", "OS-EXT-STS:power_state": 0, "OS-EXT-AZ:availability_zone": "", "config_drive": "", "status": "ERROR", "OS-EXT-SRV-ATTR:ramdisk_id": "cf4226ce-c1f9-4dd2-a78e-eaa0d7377ff2", "updated": "2018-12-27T22:10:53Z", "hostId": "", "OS-EXT-SRV-ATTR:host": null, "description": null, "tags": [], "key_name": "default", "OS-EXT-SRV-ATTR:kernel_id": "aec48798-7b92-42b0-9330-1fff89b7b333", "OS-EXT-SRV-ATTR:hypervisor_hostname": null, "locked": false, "name": "overcloud-novacompute-0", "OS-EXT-SRV-ATTR:launch_index": 0, "created": "2018-12-27T21:38:37Z", "tenant_id": "8f6ef97fe47149fbbd37c9ab8fb3b22e", "host_status": "", "trusted_image_certificates": null, "fault": {"message": "Exceeded maximum number of retries. Exceeded max scheduling attempts 2 for instance 7a2b0efd-7ff5-4067-8338-c75d479c0c9b. Last exception: Failed to provision instance 7a2b0efd-7ff5-4067-8338-c75d479c0c9b: Failed to deploy: {u'priority': 100, u'interface':", "code": 500, "details": "  File \"/usr/lib/python2.7/site-packages/nova/conductor/manager.py\", line 587, in build_instances\n    filter_properties, instances[0].uuid)\n  File \"/usr/lib/python2.7/site-packages/nova/scheduler/utils.py\", line 673, in populate_retry\n    raise exception.MaxRetriesExceeded(reason=msg)\n", "created": "2018-12-27T22:10:53Z"}, "metadata": {}}} _http_log_response /usr/lib/python2.7/site-packages/keystoneauth1/session.py:511
2018-12-27 22:10:53.970 39 ERROR heat.engine.resource Traceback (most recent call last):
2018-12-27 22:10:53.970 39 ERROR heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 924, in _action_recorder
2018-12-27 22:10:53.970 39 ERROR heat.engine.resource     yield
2018-12-27 22:10:53.970 39 ERROR heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 1037, in _do_action
2018-12-27 22:10:53.970 39 ERROR heat.engine.resource     yield self.action_handler_task(action, args=handler_args)
2018-12-27 22:10:53.970 39 ERROR heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/scheduler.py", line 346, in wrapper
2018-12-27 22:10:53.970 39 ERROR heat.engine.resource     step = next(subtask)
2018-12-27 22:10:53.970 39 ERROR heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 986, in action_handler_task
2018-12-27 22:10:53.970 39 ERROR heat.engine.resource     done = check(handler_data)
2018-12-27 22:10:53.970 39 ERROR heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resources/openstack/nova/server.py", line 879, in check_create_complete
2018-12-27 22:10:53.970 39 ERROR heat.engine.resource     check = self.client_plugin()._check_active(server_id)
2018-12-27 22:10:53.970 39 ERROR heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/clients/os/nova.py", line 246, in _check_active
2018-12-27 22:10:53.970 39 ERROR heat.engine.resource     'code': fault.get('code', _('Unknown'))
2018-12-27 22:10:53.970 39 ERROR heat.engine.resource ResourceInError: Went to status ERROR due to "Message: Exceeded maximum number of retries. Exceeded max scheduling attempts 2 for instance 7a2b0efd-7ff5-4067-8338-c75d479c0c9b. Last exception: Failed to provision instance 7a2b0efd-7ff5-4067-8338-c75d479c0c9b: Failed to deploy: {u'priority': 100, u'interface':, Code: 500"
2018-12-27 22:10:53.970 39 ERROR heat.engine.resource 
2018-12-27 22:10:54.024 39 INFO heat.engine.stack [req-fe975a84-0ecb-4513-840a-32c277b7d644 - admin - default default] Stack CREATE FAILED (overcloud-Compute-33mhizavm4gp-0-dsf5cectxfh5): Resource CREATE failed: ResourceInError: resources.NovaCompute: Went to status ERROR due to "Message: Exceeded maximum number of retries. Exceeded max scheduling attempts 2 for instance d7313939-6465-4018-9cde-f06dbec6dde6. Last exception: Failed to provision instance d7313939-6465-4018-9cde-f06dbec6dde6: Timeout reached while waiting for callback for nod, Code: 500"
2018-12-27 22:10:54.835 37 ERROR heat.engine.resource Traceback (most recent call last):
2018-12-27 22:10:54.835 37 ERROR heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 924, in _action_recorder
2018-12-27 22:10:54.835 37 ERROR heat.engine.resource     yield
2018-12-27 22:10:54.835 37 ERROR heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 1037, in _do_action
2018-12-27 22:10:54.835 37 ERROR heat.engine.resource     yield self.action_handler_task(action, args=handler_args)
2018-12-27 22:10:54.835 37 ERROR heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/scheduler.py", line 346, in wrapper
2018-12-27 22:10:54.835 37 ERROR heat.engine.resource     step = next(subtask)
2018-12-27 22:10:54.835 37 ERROR heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 986, in action_handler_task
2018-12-27 22:10:54.835 37 ERROR heat.engine.resource     done = check(handler_data)
2018-12-27 22:10:54.835 37 ERROR heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", line 409, in check_create_complete
2018-12-27 22:10:54.835 37 ERROR heat.engine.resource     return self._check_status_complete(self.CREATE)
2018-12-27 22:10:54.835 37 ERROR heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", line 463, in _check_status_complete
2018-12-27 22:10:54.835 37 ERROR heat.engine.resource     action=action)
2018-12-27 22:10:54.835 37 ERROR heat.engine.resource ResourceFailure: ResourceInError: resources[0].resources.NovaCompute: Went to status ERROR due to "Message: Exceeded maximum number of retries. Exceeded max scheduling attempts 2 for instance d7313939-6465-4018-9cde-f06dbec6dde6. Last exception: Failed to provision instance d7313939-6465-4018-9cde-f06dbec6dde6: Timeout reached while waiting for callback for nod, Code: 500"
2018-12-27 22:10:54.835 37 ERROR heat.engine.resource 
2018-12-27 22:10:54.917 37 INFO heat.engine.stack [req-fe975a84-0ecb-4513-840a-32c277b7d644 - admin - default default] Stack UPDATE FAILED (overcloud-Compute-33mhizavm4gp): Resource CREATE failed: ResourceInError: resources[0].resources.NovaCompute: Went to status ERROR due to "Message: Exceeded maximum number of retries. Exceeded max scheduling attempts 2 for instance d7313939-6465-4018-9cde-f06dbec6dde6. Last exception: Failed to provision instance d7313939-6465-4018-9cde-f06dbec6dde6: Timeout reached while waiting for callback for nod, Code: 500"
2018-12-27 22:10:55.226 39 ERROR heat.engine.resource Traceback (most recent call last):
2018-12-27 22:10:55.226 39 ERROR heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 924, in _action_recorder
2018-12-27 22:10:55.226 39 ERROR heat.engine.resource     yield
2018-12-27 22:10:55.226 39 ERROR heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 1037, in _do_action
2018-12-27 22:10:55.226 39 ERROR heat.engine.resource     yield self.action_handler_task(action, args=handler_args)
2018-12-27 22:10:55.226 39 ERROR heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/scheduler.py", line 346, in wrapper
2018-12-27 22:10:55.226 39 ERROR heat.engine.resource     step = next(subtask)
2018-12-27 22:10:55.226 39 ERROR heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 986, in action_handler_task
2018-12-27 22:10:55.226 39 ERROR heat.engine.resource     done = check(handler_data)
2018-12-27 22:10:55.226 39 ERROR heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resources/openstack/heat/resource_group.py", line 433, in check_create_complete
2018-12-27 22:10:55.226 39 ERROR heat.engine.resource     if not checker.step():
2018-12-27 22:10:55.226 39 ERROR heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/scheduler.py", line 212, in step
2018-12-27 22:10:55.226 39 ERROR heat.engine.resource     poll_period = next(self._runner)
2018-12-27 22:10:55.226 39 ERROR heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resources/openstack/heat/resource_group.py", line 442, in _run_to_completion
2018-12-27 22:10:55.226 39 ERROR heat.engine.resource     self).check_update_complete(updater):
2018-12-27 22:10:55.226 39 ERROR heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", line 562, in check_update_complete
2018-12-27 22:10:55.226 39 ERROR heat.engine.resource     cookie=cookie)
2018-12-27 22:10:55.226 39 ERROR heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", line 463, in _check_status_complete
2018-12-27 22:10:55.226 39 ERROR heat.engine.resource     action=action)
2018-12-27 22:10:55.226 39 ERROR heat.engine.resource ResourceFailure: resources.Compute: Resource CREATE failed: ResourceInError: resources[0].resources.NovaCompute: Went to status ERROR due to "Message: Exceeded maximum number of retries. Exceeded max scheduling attempts 2 for instance d7313939-6465-4018-9cde-f06dbec6dde6. Last exception: Failed to provision instance d7313939-6465-4018-9cde-f06dbec6dde6: Timeout reached while waiting for callback for nod, Code: 500"
2018-12-27 22:10:55.226 39 ERROR heat.engine.resource 
2018-12-27 22:10:55.284 39 INFO heat.engine.stack [req-3a47992b-0308-4d9c-ab61-3d6c61e43b89 - - - - -] Stack CREATE FAILED (overcloud): Resource CREATE failed: resources.Compute: Resource CREATE failed: ResourceInError: resources[0].resources.NovaCompute: Went to status ERROR due to "Message: Exceeded maximum number of retries. Exceeded max scheduling attempts 2 for instance d7313939-6465-4018-9cde-f06dbec6dde6. Last exception: Failed to provision instance d7313939-6465-4018-9cde-f06dbec6dde6: Timeout reached while waiting for callback for nod, Code: 500"
2018-12-27 22:10:55.329 39 DEBUG heat.db.sqlalchemy.api [req-3a47992b-0308-4d9c-ab61-3d6c61e43b89 - - - - -] Did not set stack state with values {'action': u'CREATE', 'status': 'FAILED', 'status_reason': u'Resource CREATE failed: resources.Compute: Resource CREATE failed: ResourceInError: resources[0].resources.NovaCompute: Went to status ERROR due to "Message: Exceeded maximum number of retries. Exceeded max scheduling attempts 2 for instance d7313939-6465-4018-9cde-f06dbec6dde6. Last exception: Failed to provision instance d7313939-6465-4018-9cde-f06dbec6dde6: Timeout reached while waiting for callback for nod, Code: 500"'}, stack id: f85239bb-671f-4e5f-b15c-2475e48b2205 with expected traversal: 99bc19be-c55c-4b67-8a70-f3f0bcd8798e stack_update /usr/lib/python2.7/site-packages/heat/db/sqlalchemy/api.py:784

Nova Conductor log

2018-12-27 19:26:01.237 24 ERROR nova.scheduler.utils [req-953acf28-762a-4ba1-aff8-658f7369ca76 960e4c0367064fb59e46b4d20e3510a7 8f6ef97fe47149fbbd37c9ab8fb3b22e - default default] [instance: d7313939-6465-4018-9cde-f06dbec6dde6] Error from last host: iad099-undercloud.localdomain (node 6eb42c8f-0fc0-4d0b-bddb-5098ddcfa7a2): [u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1940, in _do_build_and_run_instance\n    filter_properties, request_spec)\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2229, in _build_and_run_instance\n    instance_uuid=instance.uuid, reason=six.text_type(e))\n', u'RescheduledException: Build of instance d7313939-6465-4018-9cde-f06dbec6dde6 was re-scheduled: Failed to provision instance d7313939-6465-4018-9cde-f06dbec6dde6: Timeout reached while waiting for callback for node 6eb42c8f-0fc0-4d0b-bddb-5098ddcfa7a2\n']
2018-12-27 19:26:01.239 24 WARNING nova.scheduler.utils [req-953acf28-762a-4ba1-aff8-658f7369ca76 960e4c0367064fb59e46b4d20e3510a7 8f6ef97fe47149fbbd37c9ab8fb3b22e - default default] [instance: d7313939-6465-4018-9cde-f06dbec6dde6] Setting instance to ERROR state.: MaxRetriesExceeded: Exceeded maximum number of retries. Exceeded max scheduling attempts 2 for instance d7313939-6465-4018-9cde-f06dbec6dde6. Last exception: Failed to provision instance d7313939-6465-4018-9cde-f06dbec6dde6: Timeout reached while waiting for callback for node 6eb42c8f-0fc0-4d0b-bddb-5098ddcfa7a2
2018-12-27 19:58:01.318 25 ERROR nova.scheduler.utils [req-6407a921-2b9d-419d-adf8-82439eb5bacb 960e4c0367064fb59e46b4d20e3510a7 8f6ef97fe47149fbbd37c9ab8fb3b22e - default default] [instance: 772a874b-b999-45f5-aa29-6396c763663b] Error from last host: iad099-undercloud.localdomain (node 6eb42c8f-0fc0-4d0b-bddb-5098ddcfa7a2): [u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1940, in _do_build_and_run_instance\n    filter_properties, request_spec)\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2229, in _build_and_run_instance\n    instance_uuid=instance.uuid, reason=six.text_type(e))\n', u'RescheduledException: Build of instance 772a874b-b999-45f5-aa29-6396c763663b was re-scheduled: Failed to provision instance 772a874b-b999-45f5-aa29-6396c763663b: Timeout reached while waiting for callback for node 6eb42c8f-0fc0-4d0b-bddb-5098ddcfa7a2\n']
2018-12-27 19:59:01.924 24 ERROR nova.scheduler.utils [req-6407a921-2b9d-419d-adf8-82439eb5bacb 960e4c0367064fb59e46b4d20e3510a7 8f6ef97fe47149fbbd37c9ab8fb3b22e - default default] [instance: 772a874b-b999-45f5-aa29-6396c763663b] Error from last host: iad099-undercloud.localdomain (node c9abb9b5-40e4-4e29-8542-608ec82b0ee5): [u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1940, in _do_build_and_run_instance\n    filter_properties, request_spec)\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2229, in _build_and_run_instance\n    instance_uuid=instance.uuid, reason=six.text_type(e))\n', u"RescheduledException: Build of instance 772a874b-b999-45f5-aa29-6396c763663b was re-scheduled: Failed to provision instance 772a874b-b999-45f5-aa29-6396c763663b: Failed to deploy: {u'priority': 100, u'interface': u'deploy', u'step': u'deploy', u'argsinfo': None}\n"]
2018-12-27 19:59:01.925 24 WARNING nova.scheduler.utils [req-6407a921-2b9d-419d-adf8-82439eb5bacb 960e4c0367064fb59e46b4d20e3510a7 8f6ef97fe47149fbbd37c9ab8fb3b22e - default default] [instance: 772a874b-b999-45f5-aa29-6396c763663b] Setting instance to ERROR state.: MaxRetriesExceeded: Exceeded maximum number of retries. Exceeded max scheduling attempts 2 for instance 772a874b-b999-45f5-aa29-6396c763663b. Last exception: Failed to provision instance 772a874b-b999-45f5-aa29-6396c763663b: Failed to deploy: {u'priority': 100, u'interface': u'deploy', u'step': u'deploy', u'argsinfo': None}
2018-12-27 20:31:01.481 25 ERROR nova.scheduler.utils [req-2c824917-e6b7-44ce-a1f1-804872d303f1 960e4c0367064fb59e46b4d20e3510a7 8f6ef97fe47149fbbd37c9ab8fb3b22e - default default] [instance: e87c9b32-f55f-4109-84a0-e12c6309076c] Error from last host: iad099-undercloud.localdomain (node 6eb42c8f-0fc0-4d0b-bddb-5098ddcfa7a2): [u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1940, in _do_build_and_run_instance\n    filter_properties, request_spec)\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2229, in _build_and_run_instance\n    instance_uuid=instance.uuid, reason=six.text_type(e))\n', u'RescheduledException: Build of instance e87c9b32-f55f-4109-84a0-e12c6309076c was re-scheduled: Failed to provision instance e87c9b32-f55f-4109-84a0-e12c6309076c: None\n']
2018-12-27 20:31:53.404 26 ERROR nova.scheduler.utils [req-2c824917-e6b7-44ce-a1f1-804872d303f1 960e4c0367064fb59e46b4d20e3510a7 8f6ef97fe47149fbbd37c9ab8fb3b22e - default default] [instance: e87c9b32-f55f-4109-84a0-e12c6309076c] Error from last host: iad099-undercloud.localdomain (node c9abb9b5-40e4-4e29-8542-608ec82b0ee5): [u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1940, in _do_build_and_run_instance\n    filter_properties, request_spec)\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2229, in _build_and_run_instance\n    instance_uuid=instance.uuid, reason=six.text_type(e))\n', u"RescheduledException: Build of instance e87c9b32-f55f-4109-84a0-e12c6309076c was re-scheduled: Failed to provision instance e87c9b32-f55f-4109-84a0-e12c6309076c: Failed to deploy: {u'priority': 100, u'interface': u'deploy', u'step': u'deploy', u'argsinfo': None}\n"]
2018-12-27 20:31:53.406 26 WARNING nova.scheduler.utils [req-2c824917-e6b7-44ce-a1f1-804872d303f1 960e4c0367064fb59e46b4d20e3510a7 8f6ef97fe47149fbbd37c9ab8fb3b22e - default default] [instance: e87c9b32-f55f-4109-84a0-e12c6309076c] Setting instance to ERROR state.: MaxRetriesExceeded: Exceeded maximum number of retries. Exceeded max scheduling attempts 2 for instance e87c9b32-f55f-4109-84a0-e12c6309076c. Last exception: Failed to provision instance e87c9b32-f55f-4109-84a0-e12c6309076c: Failed to deploy: {u'priority': 100, u'interface': u'deploy', u'step': u'deploy', u'argsinfo': None}
2018-12-27 21:04:00.285 24 ERROR nova.scheduler.utils [req-bf4c28ee-05e5-4729-a599-c570b05e9796 960e4c0367064fb59e46b4d20e3510a7 8f6ef97fe47149fbbd37c9ab8fb3b22e - default default] [instance: 6484548f-1eae-4868-baa3-b8647b248a89] Error from last host: iad099-undercloud.localdomain (node 6eb42c8f-0fc0-4d0b-bddb-5098ddcfa7a2): [u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1940, in _do_build_and_run_instance\n    filter_properties, request_spec)\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2229, in _build_and_run_instance\n    instance_uuid=instance.uuid, reason=six.text_type(e))\n', u'RescheduledException: Build of instance 6484548f-1eae-4868-baa3-b8647b248a89 was re-scheduled: Failed to provision instance 6484548f-1eae-4868-baa3-b8647b248a89: Timeout reached while waiting for callback for node 6eb42c8f-0fc0-4d0b-bddb-5098ddcfa7a2\n']
2018-12-27 21:04:53.460 25 ERROR nova.scheduler.utils [req-bf4c28ee-05e5-4729-a599-c570b05e9796 960e4c0367064fb59e46b4d20e3510a7 8f6ef97fe47149fbbd37c9ab8fb3b22e - default default] [instance: 6484548f-1eae-4868-baa3-b8647b248a89] Error from last host: iad099-undercloud.localdomain (node c9abb9b5-40e4-4e29-8542-608ec82b0ee5): [u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1940, in _do_build_and_run_instance\n    filter_properties, request_spec)\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2229, in _build_and_run_instance\n    instance_uuid=instance.uuid, reason=six.text_type(e))\n', u"RescheduledException: Build of instance 6484548f-1eae-4868-baa3-b8647b248a89 was re-scheduled: Failed to provision instance 6484548f-1eae-4868-baa3-b8647b248a89: Failed to deploy: {u'priority': 100, u'interface': u'deploy', u'step': u'deploy', u'argsinfo': None}\n"]
2018-12-27 21:04:53.461 25 WARNING nova.scheduler.utils [req-bf4c28ee-05e5-4729-a599-c570b05e9796 960e4c0367064fb59e46b4d20e3510a7 8f6ef97fe47149fbbd37c9ab8fb3b22e - default default] [instance: 6484548f-1eae-4868-baa3-b8647b248a89] Setting instance to ERROR state.: MaxRetriesExceeded: Exceeded maximum number of retries. Exceeded max scheduling attempts 2 for instance 6484548f-1eae-4868-baa3-b8647b248a89. Last exception: Failed to provision instance 6484548f-1eae-4868-baa3-b8647b248a89: Failed to deploy: {u'priority': 100, u'interface': u'deploy', u'step': u'deploy', u'argsinfo': None}
2018-12-27 21:37:00.739 24 ERROR nova.scheduler.utils [req-48499ae8-fe83-41bc-b7b1-bb3020280947 960e4c0367064fb59e46b4d20e3510a7 8f6ef97fe47149fbbd37c9ab8fb3b22e - default default] [instance: 2bf78dd0-0ac2-46cc-b771-1bc2a498e2a0] Error from last host: iad099-undercloud.localdomain (node 6eb42c8f-0fc0-4d0b-bddb-5098ddcfa7a2): [u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1940, in _do_build_and_run_instance\n    filter_properties, request_spec)\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2229, in _build_and_run_instance\n    instance_uuid=instance.uuid, reason=six.text_type(e))\n', u'RescheduledException: Build of instance 2bf78dd0-0ac2-46cc-b771-1bc2a498e2a0 was re-scheduled: Failed to provision instance 2bf78dd0-0ac2-46cc-b771-1bc2a498e2a0: Timeout reached while waiting for callback for node 6eb42c8f-0fc0-4d0b-bddb-5098ddcfa7a2\n']
2018-12-27 21:37:53.166 27 ERROR nova.scheduler.utils [req-48499ae8-fe83-41bc-b7b1-bb3020280947 960e4c0367064fb59e46b4d20e3510a7 8f6ef97fe47149fbbd37c9ab8fb3b22e - default default] [instance: 2bf78dd0-0ac2-46cc-b771-1bc2a498e2a0] Error from last host: iad099-undercloud.localdomain (node c9abb9b5-40e4-4e29-8542-608ec82b0ee5): [u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1940, in _do_build_and_run_instance\n    filter_properties, request_spec)\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2229, in _build_and_run_instance\n    instance_uuid=instance.uuid, reason=six.text_type(e))\n', u"RescheduledException: Build of instance 2bf78dd0-0ac2-46cc-b771-1bc2a498e2a0 was re-scheduled: Failed to provision instance 2bf78dd0-0ac2-46cc-b771-1bc2a498e2a0: Failed to deploy: {u'priority': 100, u'interface': u'deploy', u'step': u'deploy', u'argsinfo': None}\n"]
2018-12-27 21:37:53.168 27 WARNING nova.scheduler.utils [req-48499ae8-fe83-41bc-b7b1-bb3020280947 960e4c0367064fb59e46b4d20e3510a7 8f6ef97fe47149fbbd37c9ab8fb3b22e - default default] [instance: 2bf78dd0-0ac2-46cc-b771-1bc2a498e2a0] Setting instance to ERROR state.: MaxRetriesExceeded: Exceeded maximum number of retries. Exceeded max scheduling attempts 2 for instance 2bf78dd0-0ac2-46cc-b771-1bc2a498e2a0. Last exception: Failed to provision instance 2bf78dd0-0ac2-46cc-b771-1bc2a498e2a0: Failed to deploy: {u'priority': 100, u'interface': u'deploy', u'step': u'deploy', u'argsinfo': None}
2018-12-27 22:10:00.346 27 ERROR nova.scheduler.utils [req-392552c0-803d-4e5e-bec8-026e4edebbd3 960e4c0367064fb59e46b4d20e3510a7 8f6ef97fe47149fbbd37c9ab8fb3b22e - default default] [instance: 7a2b0efd-7ff5-4067-8338-c75d479c0c9b] Error from last host: iad099-undercloud.localdomain (node 6eb42c8f-0fc0-4d0b-bddb-5098ddcfa7a2): [u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1940, in _do_build_and_run_instance\n    filter_properties, request_spec)\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2229, in _build_and_run_instance\n    instance_uuid=instance.uuid, reason=six.text_type(e))\n', u'RescheduledException: Build of instance 7a2b0efd-7ff5-4067-8338-c75d479c0c9b was re-scheduled: Failed to provision instance 7a2b0efd-7ff5-4067-8338-c75d479c0c9b: Timeout reached while waiting for callback for node 6eb42c8f-0fc0-4d0b-bddb-5098ddcfa7a2\n']
2018-12-27 22:10:52.944 26 ERROR nova.scheduler.utils [req-392552c0-803d-4e5e-bec8-026e4edebbd3 960e4c0367064fb59e46b4d20e3510a7 8f6ef97fe47149fbbd37c9ab8fb3b22e - default default] [instance: 7a2b0efd-7ff5-4067-8338-c75d479c0c9b] Error from last host: iad099-undercloud.localdomain (node c9abb9b5-40e4-4e29-8542-608ec82b0ee5): [u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1940, in _do_build_and_run_instance\n    filter_properties, request_spec)\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2229, in _build_and_run_instance\n    instance_uuid=instance.uuid, reason=six.text_type(e))\n', u"RescheduledException: Build of instance 7a2b0efd-7ff5-4067-8338-c75d479c0c9b was re-scheduled: Failed to provision instance 7a2b0efd-7ff5-4067-8338-c75d479c0c9b: Failed to deploy: {u'priority': 100, u'interface': u'deploy', u'step': u'deploy', u'argsinfo': None}\n"]
2018-12-27 22:10:52.946 26 WARNING nova.scheduler.utils [req-392552c0-803d-4e5e-bec8-026e4edebbd3 960e4c0367064fb59e46b4d20e3510a7 8f6ef97fe47149fbbd37c9ab8fb3b22e - default default] [instance: 7a2b0efd-7ff5-4067-8338-c75d479c0c9b] Setting instance to ERROR state.: MaxRetriesExceeded: Exceeded maximum number of retries. Exceeded max scheduling attempts 2 for instance 7a2b0efd-7ff5-4067-8338-c75d479c0c9b. Last exception: Failed to provision instance 7a2b0efd-7ff5-4067-8338-c75d479c0c9b: Failed to deploy: {u'priority': 100, u'interface': u'deploy', u'step': u'deploy', u'argsinfo': None}

Can someone help me to solve the problem and deploy the overcloud successfully?