Heat : Build of instance aborted and cannot cleanup.

asked 2015-05-04 16:52:38 -0500

tkekan gravatar image

updated 2015-05-06 20:24:09 -0500

Hi,

I'm running some tests using heat where some users create a stack, each of 16 cirros vm followed by deletion of it. When I increase the number of users to say 4 users at once, there is at least one vm of the four users that goes into error state. Going through the logs I figured that the create server failed because the port status remained "DOWN". Hence the create server fails. Now, I'm not sure why does the port status always remains "DOWN". So following this, the heat shall do a stack rollback wherein it will delete the VM's which where allocated and delete the ports that where part of this stack. Here too it is found that some ports are deleted prior to heat issuing a request for delete the port. Hence I can see the below errors as well:

2015-05-04 13:26:42.837 27748 DEBUG neutronclient.client [-] RESP:404 {'date': 'Mon, 04 May 2015 20:26:42 GMT', 'content-length': '131', 'content-type': 'application/json; charset=UTF-8', 'x-openstack-request-id': 'req-99fa2334-8c79-40b5-b80c-023f3a8980d2'} {"NeutronError": {"message": "Port a4a3e27a-186a-46f7-99ed-a5c037415f1f could not be found", "type": "PortNotFound", "detail": ""}}
Subsequently, some vm's goes into error state while deletion as well since they are not cleaned up properly. 


Main error:
2015-05-04 13:26:21.775 27748 INFO heat.engine.resource [-] CREATE: Server "r13" [79445c03-a84d-463e-8163-e5fe05a82053] Stack "heat_test-02.yml" [c6b0b5fb-a5b6-4fdf-87a3-f71a86667680]
2015-05-04 13:26:21.775 27748 TRACE heat.engine.resource Traceback (most recent call last):
114794 2015-05-04 13:26:21.775 27748 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/resource.py",        line 439, in _action_recorder
114795 2015-05-04 13:26:21.775 27748 TRACE heat.engine.resource     yield
114796 2015-05-04 13:26:21.775 27748 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/resource.py",        line 509, in _do_action
114797 2015-05-04 13:26:21.775 27748 TRACE heat.engine.resource     yield self.action_handler_task(action, args=handler_args)
114798 2015-05-04 13:26:21.775 27748 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/scheduler.py"       , line 303, in wrapper
114799 2015-05-04 13:26:21.775 27748 TRACE heat.engine.resource     step = next(subtask)
114800 2015-05-04 13:26:21.775 27748 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/resource.py",        line 483, in action_handler_task
114801 2015-05-04 13:26:21.775 27748 TRACE heat.engine.resource     while not check(handler_data):
114802 2015-05-04 13:26:21.775 27748 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/resources/ser       ver.py", line 556, in check_create_complete
114803 2015-05-04 13:26:21.775 27748 TRACE heat.engine.resource     return self._check_active(server)
114804 2015-05-04 13:26:21.775 27748 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/resources/ser       ver.py", line 575, in _check_active
114805 2015-05-04 13:26:21.775 27748 TRACE heat.engine.resource     'code': fault.get('code', _('Unknown'))
114806 2015-05-04 13:26:21.775 27748 TRACE heat.engine.resource ResourceInError: Went to status ERROR ...
(more)
edit retag flag offensive close merge delete

Comments

Can you edit the subject, so it will help others.

SGPJ gravatar imageSGPJ ( 2015-05-05 04:56:57 -0500 )edit