HEAT HA: crashed VM instances are deleted?

asked 2014-01-09 01:36:45 -0600

chmishra gravatar image

I tried the HA tempates for HEAT. I am using a Fedora 17 image for instance as for creating a heat stack. The fedora instance keeps on crashing, and as a result HEAT recreates another instance whenever the VM crashes. Interesingly, the crashed image is deleted by HEAT before a new instance is created. Is this expected behaviour? Is there any documentation which documents this behaviour?

My concern is, if the instance is automatically deleted, we don't get a chance to peek into the kernel and figure out what is causing the VM's OS to crash. Is there a way HEAT can be configured NOT to delete the instances?

edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted

answered 2014-02-26 20:50:02 -0600

Qiming gravatar image

updated 2014-02-26 23:23:47 -0600

In Heat's design (maybe the Amazon CloudFormation as well), a stack is to be restarted when a service inside the VM has failed certain times during a specified interval. When the stack is restarted, the old instance is deleted. That is the expected behavior.

My understanding is that you are trying to debug the guest OS crash. If that is the case, you need to tweak the HA template a little bit.

For example, in the template, you may find that a file named '/etc/cfn/notify-httpd-restarted'. The content in that file was a script that calls 'cfn-push-stats --watch '. You can delete that line and the following two lines:

delete this

          - '/opt/aws/bin/cfn-push-stats --watch '
          - {Ref: HttpFailureAlarm}
          - ' --service-failure

delete the lines above

Then you can use nova to capture a image of the instance when the failures kept showing up.

BTW, I don't think the problem of your instance lies in the kernel. It could be that the httpd service was not installed or configured correctly.

edit flag offensive delete link more

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower


Asked: 2014-01-09 01:36:45 -0600

Seen: 556 times

Last updated: Feb 26 '14