Ask Your Question
0

How do I recreate every resource in an AutoScalingGroup automatically during rolling_updates?

asked 2017-04-10 17:28:56 -0500

philsttr gravatar image

My end goal is to have an OS::Heat::AutoScalingGroup with rolling updates, where the rollout process will wait for each server to complete its cloud-init before moving onto the next (to prevent downtime).

I'd also like each server to have some random data provided by a OS::Heat::RandomString.

To accomplish this, I have tried creating an OS::Heat::AutoScalingGroup with the following resources:

  • OS::Nova::Server
  • OS::Neutron::LBaaS::PoolMember (adds the server to a OS::Neutron::LBaaS::Pool)
  • OS::Heat::SoftwareConfig (used be the server, does some stuff, and signals the WaitConditionHandle)
  • OS::Heat::RandomString (used by the software config)
  • OS::Heat::WaitCondition
  • OS::Heat::WaitConditionHandle (signaled by the server after cloud-init finishes)

The AutoScalingGroup has

  rolling_updates:
    max_batch_size: 1
    min_in_service: 1

When I first create the stack, everything works great. Each server gets a new/different RandomString, the WaitConditions are waited on appropriately, etc.

However, I've noticed that when I update the stack after modifying the SoftwareConfig, only the SoftwareConfig, Server, and PoolMember resources are recreated. The RandomString, WaitCondition, and WaitConditionHandle are not recreated.

This has a couple problems:

  1. The RandomString for the new servers are the same as the old servers.
  2. The WaitCondition doesn't work. Since they aren't recreated, and were successfully triggered when they were first created, the rolling update doesn't "wait" for the condition again before moving onto the next set. This effectively renders the rolling_update useless, since it takes down all the servers before the new ones have completed initializing, which causes downtime.

Is there a way to tell the AutoScalingGroup to recreate all resources in a set when any of the resources are changed? i.e. for each "set" of those 6 resources I mentioned above, I want all 6 of those resources created, instead of only recreating 3 of them, and reusing the previous 3.

If not, is there any other way to accomplish rolling_updates in an ASG while waiting on cloud-init to finish before moving onto the next server? (to prevent any downtime)

edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted
0

answered 2017-04-17 08:56:41 -0500

zaneb gravatar image

I believe the WaitCondition problem can be resolved using the OS::Heat::UpdateWaitConditionHandle resource instead of OS::Heat::WaitConditionHandle.

For the RandomString, you probably want to change the salt each time you update.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

2 followers

Stats

Asked: 2017-04-10 17:28:56 -0500

Seen: 138 times

Last updated: Apr 17 '17