Ask Your Question
0

Swarm cluster-create fails

asked 2016-11-24 07:39:30 -0600

Mutty Putty gravatar image

updated 2017-01-10 08:09:43 -0600

Hi Everyone,

             I have installed and configured the Container Infrastructure Management service, code-named magnum of Newton OpenStack on CentOS 7. But I am unable to create  Docker Swarm cluster.

Here is my all observation.

 [hpchost1@controller ~]$ magnum cluster-show swarm-cluster_test
+---------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property            | Value                                                                                                                                                                                                                      |
+---------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| status              | CREATE_FAILED                                                                                                                                                                                                              |
| cluster_template_id | 15bdcb55-a334-4699-ad86-f98ffa338589                                                                                                                                                                                       |
| uuid                | 3e6c1f12-e9e9-4fbd-8262-d32d89aa7c3e                                                                                                                                                                                       |
| stack_id            | 89003fb7-a094-4ed8-b744-00121e8cd577                                                                                                                                                                                       |
| status_reason       | Timed out                                                                                                                                                                                                                  |
| created_at          | 2017-01-10T10:48:11+00:00                                                                                                                                                                                                  |
| name                | swarm-cluster_test                                                                                                                                                                                                         |
| updated_at          | 2017-01-10T11:48:19+00:00                                                                                                                                                                                                  |
| discovery_url       | https://discovery.etcd.io/77b79dca8dbece069155914d61f77d9b                                                                                                                                                                 |
| faults              | {'swarm_masters': 'CREATE aborted (Task create from ResourceGroup "swarm_masters" Stack "swarm-cluster_test-nftqb3b77za6" [89003fb7-a094-4ed8-b744-00121e8cd577] Timed out)', '0': 'resources[0]: Stack CREATE cancelled'} |
| api_address         | -                                                                                                                                                                                                                          |
| coe_version         | -                                                                                                                                                                                                                          |
| master_addresses    | []                                                                                                                                                                                                                         |
| create_timeout      | 60                                                                                                                                                                                                                         |
| node_addresses      | []                                                                                                                                                                                                                         |
| master_count        | 1                                                                                                                                                                                                                          |
| container_version   | -                                                                                                                                                                                                                          |
| node_count          | 1                                                                                                                                                                                                                          |
+---------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

next...

 [hpchost1@controller ~]$ heat stack-list -n | grep swarm-cluster_test
WARNING (shell) "heat stack-list" is deprecated, please use "openstack stack list" instead
| 89003fb7-a094-4ed8-b744-00121e8cd577 | swarm-cluster_test-nftqb3b77za6                                                                            | CREATE_FAILED   | 2017-01-10T10:48:09Z | None                 | None                                 |
| 10655a90-845e-4b7b-8fce-2f7e7a97ae15 | swarm-cluster_test-nftqb3b77za6-swarm_masters-ukvpffafljws                                                 | CREATE_FAILED   | 2017-01-10T10:48:29Z | None                 | 89003fb7-a094-4ed8-b744-00121e8cd577 |
| eeebab48-bfe8-449c-a39f-341d99e88074 | swarm-cluster_test-nftqb3b77za6-swarm_masters-ukvpffafljws-0-v2vaqnbd3h52                                  | CREATE_FAILED   | 2017-01-10T10:48:38Z | None                 | 10655a90-845e-4b7b-8fce-2f7e7a97ae15 |
| dd19704c-9261-4a6c-8ad2-6f667c78697e | swarm-cluster_test-nftqb3b77za6-swarm_masters-ukvpffafljws-0-v2vaqnbd3h52-etcd_address_switch-hzykf3thzdha | CREATE_COMPLETE | 2017-01-10T10:48:44Z | None                 | eeebab48-bfe8-449c-a39f-341d99e88074 |
| 03c01386-71e2-41c6-ad28-6d63c66ee110 | swarm-cluster_test-nftqb3b77za6-swarm_masters-ukvpffafljws-0-v2vaqnbd3h52-api_address_switch-lprrlk3rh6p6  | CREATE_COMPLETE | 2017-01-10T10:48:48Z | None                 | eeebab48-bfe8-449c-a39f-341d99e88074 |

next...

[hpchost1@controller ~]$ heat resource-list swarm-cluster_test-nftqb3b77za6-swarm_masters-ukvpffafljws-0-v2vaqnbd3h52 | grep "FAILED"
    WARNING (shell) "heat resource-list" is deprecated, please use "openstack stack resource list" instead
    | master_wait_condition               |                                      | OS::Heat::WaitCondition                      | CREATE_FAILED   | 2017-01-10T10:48:39Z |

    [hpchost1@controller ~]$ heat resource-list swarm-cluster_test-nftqb3b77za6-swarm_masters-ukvpffafljws | grep "FAILED"
    WARNING (shell) "heat resource-list" is deprecated, please use "openstack stack resource list" instead
    | 0             | eeebab48-bfe8-449c-a39f-341d99e88074 | file:///usr/lib/python2.7/site-packages/magnum/drivers/swarm_fedora_atomic_v1/templates/swarmmaster.yaml | CREATE_FAILED   | 2017-01-10T10:48:29Z |

    [hpchost1@controller ~]$ heat resource-list swarm-cluster_test-nftqb3b77za6 | grep "FAILED"
    WARNING (shell) "heat resource-list" is deprecated, please use "openstack stack resource list" instead
    | swarm_masters       | 10655a90-845e-4b7b-8fce-2f7e7a97ae15                                                | OS::Heat::ResourceGroup                         | CREATE_FAILED   | 2017-01-10T10:48:11Z |
    [hpchost1@controller ~]$

next...

 [hpchost1@controller ~]$ heat resource-show swarm-cluster_test-nftqb3b77za6-swarm_masters-ukvpffafljws-0-v2vaqnbd3h52 OS::Heat::WaitCondition
WARNING (shell) "heat resource-show" is deprecated, please use "openstack stack resource show" instead
Stack or resource not found: swarm-cluster_test-nftqb3b77za6-swarm_masters-ukvpffafljws-0-v2vaqnbd3h52 OS::Heat::WaitCondition
[hpchost1@controller ~]$ heat resource-show swarm-cluster_test-nftqb3b77za6 OS::Heat::ResourceGroup
WARNING (shell) "heat resource-show" is deprecated, please use "openstack stack resource show" instead
Stack or resource not found: swarm-cluster_test-nftqb3b77za6 OS::Heat::ResourceGroup
[hpchost1@controller ~]$

[hpchost1@controller ~]$ heat resource-show swarm-cluster_test-nftqb3b77za6 swarm_masters
WARNING (shell) "heat resource-show" is deprecated, please use "openstack stack resource show" instead
+------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property               | Value                                                                                                                                                                      |
+------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| attributes             | {                                                                                                                                                                          |
|                        |   "attributes": null,                                                                                                                                                      |
|                        |   "refs": null,                                                                                                                                                            |
|                        |   "refs_map": null,                                                                                                                                                        |
|                        |   "removed_rsrc_list": []                                                                                                                                                  |
|                        | }                                                                                                                                                                          |
| creation_time          | 2017-01-10T10:48:11Z                                                                                                                                                       |
| description            |                                                                                                                                                                            |
| links                  | http://controller:8004/v1/282838c02c784f7ab8c89dd37ccfa87c/stacks/swarm-cluster_test-nftqb3b77za6/89003fb7-a094-4ed8-b744-00121e8cd577/resources/swarm_masters (self)      |
|                        | http://controller:8004/v1/282838c02c784f7ab8c89dd37ccfa87c/stacks/swarm-cluster_test-nftqb3b77za6/89003fb7-a094-4ed8-b744-00121e8cd577 (stack)                             |
|                        | http://controller:8004/v1/282838c02c784f7ab8c89dd37ccfa87c/stacks/swarm-cluster_test-nftqb3b77za6-swarm_masters-ukvpffafljws/10655a90-845e-4b7b-8fce-2f7e7a97ae15 (nested) |
| logical_resource_id    | swarm_masters                                                                                                                                                              |
| physical_resource_id   | 10655a90-845e-4b7b-8fce-2f7e7a97ae15                                                                                                                                       |
| required_by            | etcd_address_switch                                                                                                                                                        |
|                        | api_address_switch                                                                                                                                                         |
| resource_name          | swarm_masters                                                                                                                                                              |
| resource_status        | CREATE_FAILED                                                                                                                                                              |
| resource_status_reason | CREATE aborted (Task create from ResourceGroup "swarm_masters" Stack "swarm-cluster_test-nftqb3b77za6" [89003fb7-a094-4ed8-b744-00121e8cd577] Timed out)                   |
| resource_type          | OS::Heat::ResourceGroup                                                                                                                                                    |
| updated_time           | 2017-01-10T10:48:11Z                                                                                                                                                       |
+------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

heat-engine.log file on controller node..

2017-01-10 11:48:09.256 4105 INFO heat.engine.scheduler [req-d44de78e-2835-43ad-8669-bfeaa96ccb20 - - - - -] Task create from ResourceGroup "swarm_masters" Stack "swarm-cluster_test-nftqb3b77za6" [89003fb7-a094-4ed8-b744-00121e8cd577] timed out
2017-01-10 11:48:09.286 4102 INFO heat.engine.service [req-d0e8b866-3e6e-44f0-a9f3-b2be5c40a981 641b42e296d544fdbf9fe7e8bfc65c57 282838c02c784f7ab8c89dd37ccfa87c - - -] Starting cancel of updating stack swarm-cluster_test-nftqb3b77za6-swarm_masters-ukvpffafljws
2017-01-10 11:48:09.336 4102 INFO heat.engine.stack [req-d0e8b866-3e6e-44f0-a9f3-b2be5c40a981 641b42e296d544fdbf9fe7e8bfc65c57 282838c02c784f7ab8c89dd37ccfa87c - - -] Stack CREATE FAILED (swarm-cluster_test-nftqb3b77za6-swarm_masters-ukvpffafljws): Stack CREATE cancelled
2017-01-10 11:48:09.383 4105 INFO heat.engine.stack [req-d44de78e-2835-43ad-8669-bfeaa96ccb20 - - - - -] Stack CREATE FAILED ...
(more)
edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted
0

answered 2017-01-10 16:19:22 -0600

Resource CREATE failed: WaitConditionTimeout: resources.master_wait_condition: 0 of 1 received indicates that the agent on the server was unable to signal back to Heat that the server had booted successfully. This could be due to a problem with the image, a problem with the networking between the server and Heat, or some other problem along this path.

edit flag offensive delete link more

Comments

@ Zaneb, Thank you so much for your reply. Just to make sure we are on the same page of understanding, Which agent are you taking about(is it agent inside swarm-cluster master)? I am using fedora-atomic-latest image following RDO centos 7 openstack newton guide.

Mutty Putty gravatar imageMutty Putty ( 2017-01-10 23:07:47 -0600 )edit

Me to think the same, agent is unable to send the signal back to heat that stack successfully created. could you please tell me troubleshooting procedure in details? I have noticed that out of 5 stacks of the cluster, 2 stacks getting successfully completed quickly and after log time rest failed.

Mutty Putty gravatar imageMutty Putty ( 2017-01-10 23:10:45 -0600 )edit
-1

answered 2016-11-25 05:23:17 -0600

ashish235 gravatar image

Same issue.

2016-11-25 16:21:24.062 13858 ERROR heat.engine.resource [req-51007baf-2b3c-45b0-bfad-613a542b12de 16bfd673590e48069db821c9128a7c72 527da6b91b9c49afb97a2c286fa5d2f9 - - -] Resource type OS::Neutron::RouterInterface unavailable 2016-11-25 16:21:24.062 13858 ERROR heat.engine.resource Traceback (most recent call last):

edit flag offensive delete link more

Comments

@ashish, did you get any answer?

Mutty Putty gravatar imageMutty Putty ( 2016-12-05 03:34:07 -0600 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2016-11-24 07:39:30 -0600

Seen: 114 times

Last updated: Jan 10