Ask Your Question
0

Autoscaling not working

asked 2019-03-02 01:17:20 -0500

JeffBannister gravatar image

Hi All, I’m trying a very simple autoscaling exercise with my Openstack (Ocata) using Aodh for alarms. My cluster has 3 nodes (controller, compute and network) and was set up using Packstack on Centos7. I’ve tried multiple combinations and permutations based on advice online and regardless my alarm evaluation always comes out as “insufficient data” even though I can see the alarm being collected. I am not sure what log file would show me the problem as the Aodh & Gnocchi logs look fine. I’ve included my simple stack & shown some outputs below. I’m hoping it is something very simple I’m missing but I’ve been struggling with this problem for a while. Thanks in advance for your help. Warm regards, Jeff Bannister

heat_template_version: 2016-04-08
description: A simple auto scaling group
resources:
  group:
    type: OS::Heat::AutoScalingGroup 
    properties:
      cooldown: 300 
      desired_capacity: 2 
      max_size: 5 
      min_size: 1 
      resource:
        type: OS::Nova::Server 
        properties: 
          image: 'cirros' 
          flavor: m1.tiny
          networks:
            - network: orbitage

  scaleup_policy: 
    type: OS::Heat::ScalingPolicy
    properties:
      adjustment_type: change_in_capacity 
      auto_scaling_group_id: {get_resource: group} 
      cooldown: 300 
      scaling_adjustment: 1 

  cpu_alarm_high: 
    type: OS::Aodh::GnocchiAggregationByResourcesAlarm
    properties:
        metric: cpu_util  
        aggregation_method: mean        
        granularity: 300           
        evaluation_periods: 1 
        threshold: 50  
        resource_type: instance
        comparison_operator: gt      
        alarm_actions: 
            - str_replace:
                template: trust+url
                params:
                    url: {get_attr: [scaleup_policy, signal_url]}      
        query:
            str_replace:
                template: '{"=": {"server_group": "stack_id"}}'
                params:
                    stack_id: {get_param: "OS::stack_id"}
outputs:
    scaleup_policy_signal_url:
        value: {get_attr: [scaleup_policy, signal_url]}

Outputs:

[root@controller ~(keystone_gnocchi)]# openstack stack list
+--------------------------------------+------------+----------------------------------+-----------------+----------------------+--------------+
| ID                                   | Stack Name | Project                          | Stack Status    | Creation Time        | Updated Time |
+--------------------------------------+------------+----------------------------------+-----------------+----------------------+--------------+
| e9e1d007-6a25-442b-b614-abdacbcba0f7 | scaler8    | a0a75e4b57dc47ef8ce0105a8db1b434 | CREATE_COMPLETE | 2019-02-25T02:54:17Z | None         |
+--------------------------------------+------------+----------------------------------+-----------------+----------------------+--------------+
[root@controller ~(keystone_gnocchi)]# openstack server list
+--------------------------------------+------------------------------------------------------+--------+--------------------------+--------+---------+
| ID                                   | Name                                                 | Status | Networks                 | Image  | Flavor  |
+--------------------------------------+------------------------------------------------------+--------+--------------------------+--------+---------+
| 16808f88-92db-488b-baa2-3d4c93cf297e | scaler8-group-wtcwtetsglid-wklxhd6xrzae-ejmaiybxdosd | ACTIVE | orbitage=192.168.101.113 | cirros | m1.tiny |
| 3fe4e62c-ad72-4928-b9c7-a41fc4074a9d | scaler8-group-wtcwtetsglid-plgwojxq2f5i-xvh3oa7yefvc | ACTIVE | orbitage=192.168.101.118 | cirros | m1.tiny |
+--------------------------------------+------------------------------------------------------+--------+--------------------------+--------+---------+
[root@controller ~(keystone_gnocchi)]# openstack alarm list
+--------------------------------------+--------------------------------------------+-------------------------------------+-------------------+----------+---------+
| alarm_id                             | type                                       | name                                | state             | severity | enabled |
+--------------------------------------+--------------------------------------------+-------------------------------------+-------------------+----------+---------+
| 59a0bbff-8960-4df0-8b25-f10f4fa42ff1 | gnocchi_aggregation_by_resources_threshold | scaler8-cpu_alarm_high-okczotx6ck7h | insufficient data | low      | True    |
+--------------------------------------+--------------------------------------------+-------------------------------------+-------------------+----------+---------+
[root@controller ~(keystone_gnocchi)]# gnocchi measures show --resource-id 16808f88-92db-488b-baa2-3d4c93cf297e cpu_util
+---------------------------+-------------+---------------+
| timestamp                 | granularity |        value  |
+---------------------------+-------------+---------------+
| 2019-02-24T22:00:00-05:00 |       300.0 |  0.0667749275 |
| 2019-02-24T22:05:00-05:00 |       300.0 |  0.0699992097 |
| 2019-02-24T22:10:00-05:00 |       300.0 |  0.0633377277 |
| 2019-02-24T22:15:00-05:00 |       300.0 |  0.063312082  |
| 2019-02-24T22:20:00-05:00 |       300.0 |  0.0633542018 |
| 2019-02-24T22:25:00-05:00 |       300.0 | 99.1123780114 |
+---------------------------+-------------+---------------+
[root@controller ~(keystone_gnocchi)]# openstack alarm show scaler8-cpu_alarm_high-okczotx6ck7h
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+
| Field                     | Value                                                                                                                                              |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+
| aggregation_method        | mean                                                                                                                                               |
| alarm_actions             | [u'http://10.0.1.11:8004/v1/a0a75e4b57dc47ef8ce0105a8db1b434/stacks/scaler8/e9e1d007-6a25-442b-b614-abdacbcba0f7/resources/scaleup_policy/signal'] |
| alarm_id                  | 59a0bbff-8960-4df0-8b25-f10f4fa42ff1                                                                                                               |
| comparison_operator       | gt                                                                                                                                                 |
| description               | gnocchi_aggregation_by_resources_threshold alarm rule                                                                                              |
| enabled                   | True                                                                                                                                               |
| evaluation_periods        | 1                                                                                                                                                  |
| granularity               | 300                                                                                                                                                |
| insufficient_data_actions | []                                                                                                                                                 |
| metric                    | cpu_util                                                                                                                                           |
| name                      | scaler8-cpu_alarm_high-okczotx6ck7h                                                                                                                |
| ok_actions                | []                                                                                                                                                 |
| project_id                | a0a75e4b57dc47ef8ce0105a8db1b434                                                                                                                   |
| query                     | {"=": {"server_group": "e9e1d007-6a25-442b-b614-abdacbcba0f7"}}                                                                                    |
| repeat_actions            | True                                                                                                                                               |
| resource_type             | instance                                                                                                                                           |
| severity                  | low                                                                                                                                                |
| state                     | insufficient data                                                                                                                                  |
| state_reason              | Not evaluated yet                                                                                                                                  |
| state_timestamp           | 2019-02-25T02:54:40.271809                                                                                                                         |
| threshold                 | 50.0                                                                                                                                               |
| time_constraints          | []                                                                                                                                                 |
| timestamp                 | 2019-02-25T02:54:40.271809                                                                                                                         |
| type                      | gnocchi_aggregation_by_resources_threshold                                                                                                         |
| user_id                   | c4160a661d854645952d5024aa1ec881                                                                                                                   |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+
edit retag flag offensive close merge delete

2 answers

Sort by » oldest newest most voted
1

answered 2019-03-13 05:05:50 -0500

JeffBannister gravatar image

I solved my problem - I discovered that I needed the following line in my server properties:

metadata: {"metering.server_group": {get_param: "OS::stack_id"}}
edit flag offensive delete link more
0

answered 2019-03-04 12:57:40 -0500

zaneb gravatar image

I suspect that the server_group query in Aodh refers to an OS::Nova::ServerGroup, not to the ID of the stack. (Nova doesn't know that the servers are part of a Heat stack, and Nova events are where Gnocchi gets its measurements from.) Since there is no Server Group with that ID in Nova, the alarm is not seeing any measurements.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2019-03-02 01:17:20 -0500

Seen: 102 times

Last updated: Mar 13