Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Heat autoscale StackValidation failed for authentication

Hi, I'm trying to define a simple autoscaling template for heat. This is how it looks like:

heat_template_version: 2013-05-23

description: Test Template

parameters:
  image:
    type: string
    description: Image use to boot a server
    default: "ubuntu-trusty"
  flavor:
    type: string
    description: Flavor for the image
    constraints:
      - allowed_values: [ m1.tiny, m1.small, m1.medium, m1.large, m1.xlarge ]
    default: "m1.large"
  network:
    type: string
    description: Network ID for the server
    default: "d7f8f8c4-3ab3-4643-978c-b98bf94567d7"
  key:
    type: string
    description: Public key to be used

resources:
  server_group:
    type: OS::Heat::AutoScalingGroup
    properties:
      min_size: 1
      max_size: 3
      resource:
        type: OS::Nova::Server
        properties:
          flavor: {get_param: flavor}
          image: {get_param: image}
          key_name: {get_param: key}
          networks:
            - network: { get_param: network }
          metadata: {"metering.stack": {get_param: "OS::stack_id"}}

  server_scaleup_policy:
    type: OS::Heat::ScalingPolicy
    properties:
      adjustment_type: change_in_capacity
      auto_scaling_group_id: {get_resource: server_group}
      cooldown: 30
      scaling_adjustment: 1

  server_scaledown_policy:
    type: OS::Heat::ScalingPolicy
    properties:
      adjustment_type: change_in_capacity
      auto_scaling_group_id: {get_resource: server_group}
      cooldown: 30
      scaling_adjustment: -1

  cpu_alarm_high:
    type: OS::Ceilometer::Alarm
    properties:
      description: Scale-up if the average CPU > 15% for 30 seconds
      meter_name: cpu_util
      statistic: avg
      period: 30
      evaluation_periods: 1
      threshold: 15
      alarm_actions:
        - {get_attr: [server_scaleup_policy, alarm_url]}
      matching_metadata:
        'metadata.user_metadata.stack': {get_param: "OS::stack_id"}
      comparison_operator: gt

  cpu_alarm_low:
    type: OS::Ceilometer::Alarm
    properties:
      description: Scale-down if the average CPU < 10% for 30 seconds
      meter_name: cpu_util
      statistic: avg
      period: 30
      evaluation_periods: 1
      threshold: 10
      alarm_actions:
        - {get_attr: [server_scaledown_policy, alarm_url]}
      matching_metadata:
        'metadata.user_metadata.stack': {get_param: "OS::stack_id"}
      comparison_operator: lt

It can successfully create the instance but doesn't scale up when the cpu goes over the threshold. The alarm is triggered but no acton is taken:

2014-12-03 15:42:31.049 21214 INFO heat.engine.resources.openstack.scaling_policy [req-7521a3f0-9873-4dd8-8a87-a898c5669fe5 None] Alarm server_scaleup_policy, new state alarm
2014-12-03 15:42:31.050 21214 INFO heat.engine.resources.openstack.scaling_policy [req-7521a3f0-9873-4dd8-8a87-a898c5669fe5 None] server_scaleup_policy Alarm, adjusting Group server_group with id test-server_group-apysatvxf34k by 1
2014-12-03 15:42:31.055 21214 INFO heat.engine.environment [req-7521a3f0-9873-4dd8-8a87-a898c5669fe5 None] Registering OS::Heat::ScaledResource -> AWS::EC2::Instance
2014-12-03 15:42:31.064 21214 INFO heat.engine.environment [req-7521a3f0-9873-4dd8-8a87-a898c5669fe5 None] Registering OS::Heat::ScaledResource -> AWS::EC2::Instance
2014-12-03 15:42:31.067 21214 INFO heat.engine.resource [req-7521a3f0-9873-4dd8-8a87-a898c5669fe5 None] Validating Server "uxivdelzdf3o"
2014-12-03 15:42:31.115 21214 WARNING heat.common.keystoneclient [req-7521a3f0-9873-4dd8-8a87-a898c5669fe5 None] stack_user_domain_id or stack_user_domain_name not set in heat.conf falling back to using default
2014-12-03 15:42:31.117 21214 INFO urllib3.connectionpool [req-7521a3f0-9873-4dd8-8a87-a898c5669fe5 ] Starting new HTTP connection (1): controller01
2014-12-03 15:42:31.172 21214 INFO heat.engine.stack [req-7521a3f0-9873-4dd8-8a87-a898c5669fe5 None] Property error : uxivdelzdf3o: key_name The request you have made requires authentication. (HTTP 401)
2014-12-03 15:42:31.173 21214 ERROR heat.engine.resource [req-7521a3f0-9873-4dd8-8a87-a898c5669fe5 None] signal AutoScalingPolicy "server_scaleup_policy" [7955b841fdcc4b198abdc61d7968ed0f] Stack "test" [ef131d8d-bfdd-427a-9bba-893b7fe05680] : Property error : uxivdelzdf3o: key_name The request you have made requires authentication. (HTTP 401)
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource Traceback (most recent call last):
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/resource.py", line 1032, in signal
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource     signal_result = self.handle_signal(details)
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/resources/openstack/scaling_policy.py", line 150, in handle_signal
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource     group.adjust(self.properties[self.SCALING_ADJUSTMENT], adjustment_type)
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/resources/autoscaling.py", line 655, in adjust
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource     LOG.exception(_('Failed sending error notification'))
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/oslo/utils/excutils.py", line 82, in __exit__
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource     six.reraise(self.type_, self.value, self.tb)
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/resources/autoscaling.py", line 646, in adjust
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource     self.resize(new_capacity)
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/resources/autoscaling.py", line 382, in resize
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource     self._environment())
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/stack_resource.py", line 228, in update_with_template
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource     timeout_mins)
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/stack_resource.py", line 156, in _parse_nested_stack
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource     nested.validate()
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/stack.py", line 458, in validate
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource     raise ex
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource StackValidationFailed: Property error : uxivdelzdf3o: key_name The request you have made requires authentication. (HTTP 401)

Here's the triggered alarm:

+--------------------------------------+----------------------------------+-------+---------+------------+--------------------------------+------------------+
| Alarm ID                             | Name                             | State | Enabled | Continuous | Alarm condition                | Time constraints |
+--------------------------------------+----------------------------------+-------+---------+------------+--------------------------------+------------------+
| 12911f9f-4fc5-493f-b84d-0b888d077fa8 | test-cpu_alarm_low-xdci44lu6iai  | ok    | True    | False      | cpu_util < 10.0 during 1 x 30s | None             |
| af8f0fe6-2800-4898-a6c3-e24fe8b446be | test-cpu_alarm_high-wajeqevh5cgr | alarm | True    | False      | cpu_util > 15.0 during 1 x 30s | None             |
+--------------------------------------+----------------------------------+-------+---------+------------+--------------------------------+------------------+

Anybody knows if anything's missing from the template?

Thank you

Heat autoscale StackValidation failed for authentication

Hi, I'm trying to define a simple autoscaling template for heat. This is how it looks like:

heat_template_version: 2013-05-23

description: Test Template

parameters:
  image:
    type: string
    description: Image use to boot a server
    default: "ubuntu-trusty"
  flavor:
    type: string
    description: Flavor for the image
    constraints:
      - allowed_values: [ m1.tiny, m1.small, m1.medium, m1.large, m1.xlarge ]
    default: "m1.large"
  network:
    type: string
    description: Network ID for the server
    default: "d7f8f8c4-3ab3-4643-978c-b98bf94567d7"
  key:
    type: string
    description: Public key to be used

resources:
  server_group:
    type: OS::Heat::AutoScalingGroup
    properties:
      min_size: 1
      max_size: 3
      resource:
        type: OS::Nova::Server
        properties:
          flavor: {get_param: flavor}
          image: {get_param: image}
          key_name: {get_param: key}
          networks:
            - network: { get_param: network }
          metadata: {"metering.stack": {get_param: "OS::stack_id"}}

  server_scaleup_policy:
    type: OS::Heat::ScalingPolicy
    properties:
      adjustment_type: change_in_capacity
      auto_scaling_group_id: {get_resource: server_group}
      cooldown: 30
      scaling_adjustment: 1

  server_scaledown_policy:
    type: OS::Heat::ScalingPolicy
    properties:
      adjustment_type: change_in_capacity
      auto_scaling_group_id: {get_resource: server_group}
      cooldown: 30
      scaling_adjustment: -1

  cpu_alarm_high:
    type: OS::Ceilometer::Alarm
    properties:
      description: Scale-up if the average CPU > 15% for 30 seconds
      meter_name: cpu_util
      statistic: avg
      period: 30
      evaluation_periods: 1
      threshold: 15
      alarm_actions:
        - {get_attr: [server_scaleup_policy, alarm_url]}
      matching_metadata:
        'metadata.user_metadata.stack': {get_param: "OS::stack_id"}
      comparison_operator: gt

  cpu_alarm_low:
    type: OS::Ceilometer::Alarm
    properties:
      description: Scale-down if the average CPU < 10% for 30 seconds
      meter_name: cpu_util
      statistic: avg
      period: 30
      evaluation_periods: 1
      threshold: 10
      alarm_actions:
        - {get_attr: [server_scaledown_policy, alarm_url]}
      matching_metadata:
        'metadata.user_metadata.stack': {get_param: "OS::stack_id"}
      comparison_operator: lt

It can successfully create the instance but doesn't scale up when the cpu goes over the threshold. The alarm is triggered but no acton is taken:

2014-12-03 15:42:31.049 21214 INFO heat.engine.resources.openstack.scaling_policy [req-7521a3f0-9873-4dd8-8a87-a898c5669fe5 None] Alarm server_scaleup_policy, new state alarm
2014-12-03 15:42:31.050 21214 INFO heat.engine.resources.openstack.scaling_policy [req-7521a3f0-9873-4dd8-8a87-a898c5669fe5 None] server_scaleup_policy Alarm, adjusting Group server_group with id test-server_group-apysatvxf34k by 1
2014-12-03 15:42:31.055 21214 INFO heat.engine.environment [req-7521a3f0-9873-4dd8-8a87-a898c5669fe5 None] Registering OS::Heat::ScaledResource -> AWS::EC2::Instance
2014-12-03 15:42:31.064 21214 INFO heat.engine.environment [req-7521a3f0-9873-4dd8-8a87-a898c5669fe5 None] Registering OS::Heat::ScaledResource -> AWS::EC2::Instance
2014-12-03 15:42:31.067 21214 INFO heat.engine.resource [req-7521a3f0-9873-4dd8-8a87-a898c5669fe5 None] Validating Server "uxivdelzdf3o"
2014-12-03 15:42:31.115 21214 WARNING heat.common.keystoneclient [req-7521a3f0-9873-4dd8-8a87-a898c5669fe5 None] stack_user_domain_id or stack_user_domain_name not set in heat.conf falling back to using default
2014-12-03 15:42:31.117 21214 INFO urllib3.connectionpool [req-7521a3f0-9873-4dd8-8a87-a898c5669fe5 ] Starting new HTTP connection (1): controller01
2014-12-03 15:42:31.172 21214 INFO heat.engine.stack [req-7521a3f0-9873-4dd8-8a87-a898c5669fe5 None] Property error : uxivdelzdf3o: key_name The request you have made requires authentication. (HTTP 401)
2014-12-03 15:42:31.173 21214 ERROR heat.engine.resource [req-7521a3f0-9873-4dd8-8a87-a898c5669fe5 None] signal AutoScalingPolicy "server_scaleup_policy" [7955b841fdcc4b198abdc61d7968ed0f] Stack "test" [ef131d8d-bfdd-427a-9bba-893b7fe05680] : Property error : uxivdelzdf3o: key_name The request you have made requires authentication. (HTTP 401)
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource Traceback (most recent call last):
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/resource.py", line 1032, in signal
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource     signal_result = self.handle_signal(details)
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/resources/openstack/scaling_policy.py", line 150, in handle_signal
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource     group.adjust(self.properties[self.SCALING_ADJUSTMENT], adjustment_type)
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/resources/autoscaling.py", line 655, in adjust
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource     LOG.exception(_('Failed sending error notification'))
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/oslo/utils/excutils.py", line 82, in __exit__
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource     six.reraise(self.type_, self.value, self.tb)
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/resources/autoscaling.py", line 646, in adjust
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource     self.resize(new_capacity)
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/resources/autoscaling.py", line 382, in resize
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource     self._environment())
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/stack_resource.py", line 228, in update_with_template
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource     timeout_mins)
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/stack_resource.py", line 156, in _parse_nested_stack
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource     nested.validate()
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/stack.py", line 458, in validate
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource     raise ex
2014-12-03 15:42:31.173 21214 TRACE heat.engine.resource StackValidationFailed: Property error : uxivdelzdf3o: key_name The request you have made requires authentication. (HTTP 401)

Here's the triggered alarm:

+--------------------------------------+----------------------------------+-------+---------+------------+--------------------------------+------------------+
| Alarm ID                             | Name                             | State | Enabled | Continuous | Alarm condition                | Time constraints |
+--------------------------------------+----------------------------------+-------+---------+------------+--------------------------------+------------------+
| 12911f9f-4fc5-493f-b84d-0b888d077fa8 | test-cpu_alarm_low-xdci44lu6iai  | ok    | True    | False      | cpu_util < 10.0 during 1 x 30s | None             |
| af8f0fe6-2800-4898-a6c3-e24fe8b446be | test-cpu_alarm_high-wajeqevh5cgr | alarm | True    | False      | cpu_util > 15.0 during 1 x 30s | None             |
+--------------------------------------+----------------------------------+-------+---------+------------+--------------------------------+------------------+

Anybody knows if anything's missing from the template?

This is my /etc/heat/heat.conf file with no comments. I just followed the Juno doc for the installation so I changed only the values specified there.

[DEFAULT]
rpc_backend = rabbit
rabbit_host = controller01
rabbit_password = password 
heat_metadata_server_url = http://controller01:8000
heat_waitcondition_server_url = http://controller01:8000/v1/waitcondition
verbose = True
log_dir=/var/log/heat
[auth_password]
[clients]
[clients_ceilometer]
[clients_cinder]
[clients_glance]
[clients_heat]
[clients_keystone]
[clients_neutron]
[clients_nova]
[clients_swift]
[clients_trove]
[database]
connection = mysql://heat:password@controller01/heat
[ec2authtoken]
auth_uri = http://controller01:5000/v2.0
[heat_api]
[heat_api_cfn]
[heat_api_cloudwatch]
[keystone_authtoken]
auth_uri = http://controller01:5000/v2.0
identity_uri = http://controller01:35357
admin_tenant_name = service
admin_user = heat
admin_password = password
[matchmaker_redis]
[matchmaker_ring]
[paste_deploy]
[revision]

Thank you