How do I recreate every resource in an AutoScalingGroup automatically during rolling_updates?
My end goal is to have an OS::Heat::AutoScalingGroup with rolling updates, where the rollout process will wait for each server to complete its cloud-init before moving onto the next (to prevent downtime).
I'd also like each server to have some random data provided by a OS::Heat::RandomString.
To accomplish this, I have tried creating an OS::Heat::AutoScalingGroup with the following resources:
OS::Nova::ServerOS::Neutron::LBaaS::PoolMember(adds the server to a OS::Neutron::LBaaS::Pool)OS::Heat::SoftwareConfig(used be the server, does some stuff, and signals theWaitConditionHandle)OS::Heat::RandomString(used by the software config)OS::Heat::WaitConditionOS::Heat::WaitConditionHandle(signaled by the server after cloud-init finishes)
The AutoScalingGroup has
rolling_updates:
max_batch_size: 1
min_in_service: 1
When I first create the stack, everything works great. Each server gets a new/different RandomString, the WaitConditions are waited on appropriately, etc.
However, I've noticed that when I update the stack after modifying the SoftwareConfig, only the SoftwareConfig, Server, and PoolMember resources are recreated. The RandomString, WaitCondition, and WaitConditionHandle are not recreated.
This has a couple problems:
- The
RandomStringfor the new servers are the same as the old servers. - The
WaitConditiondoesn't work. Since they aren't recreated, and were successfully triggered when they were first created, the rolling update doesn't "wait" for the condition again before moving onto the next set. This effectively renders the rolling_update useless, since it takes down all the servers before the new ones have completed initializing, which causes downtime.
Is there a way to tell the AutoScalingGroup to recreate all resources in a set when any of the resources are changed? i.e. for each "set" of those 6 resources I mentioned above, I want all 6 of those resources created, instead of only recreating 3 of them, and reusing the previous 3.
If not, is there any other way to accomplish rolling_updates in an ASG while waiting on cloud-init to finish before moving onto the next server? (to prevent any downtime)
