How do I recreate every resource in an AutoScalingGroup automatically during rolling_updates?
My end goal is to have an OS::Heat::AutoScalingGroup
with rolling updates, where the rollout process will wait for each server to complete its cloud-init before moving onto the next (to prevent downtime).
I'd also like each server to have some random data provided by a OS::Heat::RandomString
.
To accomplish this, I have tried creating an OS::Heat::AutoScalingGroup
with the following resources:
OS::Nova::Server
OS::Neutron::LBaaS::PoolMember
(adds the server to a OS::Neutron::LBaaS::Pool)OS::Heat::SoftwareConfig
(used be the server, does some stuff, and signals theWaitConditionHandle
)OS::Heat::RandomString
(used by the software config)OS::Heat::WaitCondition
OS::Heat::WaitConditionHandle
(signaled by the server after cloud-init finishes)
The AutoScalingGroup
has
rolling_updates:
max_batch_size: 1
min_in_service: 1
When I first create the stack, everything works great. Each server gets a new/different RandomString
, the WaitCondition
s are waited on appropriately, etc.
However, I've noticed that when I update the stack after modifying the SoftwareConfig
, only the SoftwareConfig
, Server
, and PoolMember
resources are recreated. The RandomString
, WaitCondition
, and WaitConditionHandle
are not recreated.
This has a couple problems:
- The
RandomString
for the new servers are the same as the old servers. - The
WaitCondition
doesn't work. Since they aren't recreated, and were successfully triggered when they were first created, the rolling update doesn't "wait" for the condition again before moving onto the next set. This effectively renders the rolling_update useless, since it takes down all the servers before the new ones have completed initializing, which causes downtime.
Is there a way to tell the AutoScalingGroup
to recreate all resources in a set when any of the resources are changed? i.e. for each "set" of those 6 resources I mentioned above, I want all 6 of those resources created, instead of only recreating 3 of them, and reusing the previous 3.
If not, is there any other way to accomplish rolling_updates in an ASG while waiting on cloud-init to finish before moving onto the next server? (to prevent any downtime)