How to fill context for rpc call when calling heat engine

asked 2013-06-25 05:56:41 -0500

samuli-silvius gravatar image

I have created heat extension which listens nova notifications "" and does actions for instances in stacks accordingly (e.g. instance stopped could restart instance in stack). After received the notification I'm calling heat engine through RPC mechanism (i.e. using heat/heat/rpc/ and EngineClient class from there).

The problem is how to fill the context parameter correctly when calling heat engine? E.g. when I call set_watch_state(ctxt, watch_name, state) i.e. want to set alarm (whihc has restarter policy in my template). How to fill ctxt parameter correctly concerning passwords?

I can get stack related context from heat db with

        rs = db_api.resource_get_by_physical_resource_id(None, instance_id)
        if not rs:
            LOG.debug("No stack found for stopped instance '%s'. Do nothing." % instance_id)

        user_creds = db_api.user_creds_get(rs.stack.user_creds_id)
        stack_context = context.RequestContext.from_dict(user_creds)

with instanceId and then use stack_context successfully in any further db reads etc. But when using the same context in rpc call it fails to utf8 encoding as "password" and "service_password" are in some odd format when read from db.

Everything works if I in the code assign stack_context.password = "root" #my admin password happens to be root stack_context.service_password = "" # just set it empty to avoid utf8 encoding error

So how I get the password correctly?

edit retag flag offensive close merge delete

7 answers

Sort by ยป oldest newest most voted

answered 2013-07-01 09:16:47 -0500

samuli-silvius gravatar image


I got it working like you suggested, putting my code running inside the heat-engine. Then there is no problem with passwords read from database with " db_api.user_creds_get(rs.stack.user_creds_id)".

But we will be working on to better solution (ceilometer integration) as suggested by you. Hope we could contribute at some point.

edit flag offensive delete link more

answered 2013-06-27 10:05:52 -0500

shardy gravatar image


The use case you describe is already supported, but not yet via nova notifications, see these templates (one as previously linked by Angus);

The first template restarts the httpd service if it is not running, then rebuilds the instance if this fails more than a defined number of times.

The second template sends a heartbeat message from the instance, such that it is rebuilt if the heartbeat timeout is exceeded.

Clearly using nova notifications to do this will be better long-term, but this is the solution we have today (in grizzly).

For havana, as previously outlined, things are set to change considerably - the watch-ceilometer BP is tracking the integration with the new-for-havana ceilometer alarms functionality. This work includes the "webhook" mechanism described, it does not yet exist, we're still working out how to implement it.

I would suggest not pursuing the option of implementing this all in heat-engine, as we plan to remove the periodic/watchrule functionality when the ceilometer integration is completed.

IMO your effort is likely to be better spent ensuring that the new ceilometer alarms functionality is capable of alarming in the scenarios where you wish to trigger heat actions from, then when we complete the watch-ceilometer work you should have a solution that works for you in havana.

edit flag offensive delete link more

answered 2013-06-25 09:33:11 -0500

shardy gravatar image

Basically, you need to either do this inside heat-engine (where we store encrypted credentials in the database for exactly this kind of scenario), or store credentials in your code.

Is there a reason why you aren't implementing this either in the main heat-engine code, or via a custom resource plugin?

See: and

We would love to have improved support for nova notifications in heat, so it would be great if you could work with us and contribute to heat, instead of doing something standalone and accessing via the RPC API.

Note the RPC API is intended to be private, and may change at any time (we aren't planning to declare the RPC version final until the havana release), so if you go ahead with this approach, your code is likely to get broken, it would be much better to add this to heat-engine or use a custom resource plugin IMO, we do not support usage of the heat RPC interfaces in this way.

edit flag offensive delete link more

answered 2013-06-25 12:21:36 -0500

Is it true, that resource plugins are kind of "passive", i.e. they do not "run" constantly (for example in a dedicated thread, or part of event-loop), but are instead passively waiting to be invoked in order to do their time-constrained task?

If that's the case, it sounds that resource plugins wouldn't be as such suitable for continuous monitoring of nova-/ceilometer-/whatever notifications?

Would it be feasible to provide some sort of "listener plug-in" -support for heat-engine? Any preliminary ideas how to build one?

edit flag offensive delete link more

answered 2013-06-26 00:00:36 -0500

asalkeld gravatar image

We want to remove _all_ constantly running jobs! I think you are doing this in the wrong way. Everything should be described in the template, and a resource backed by an external service should provide this functionality.

have a look at this: (it provides server resarting)

We are working on using Ceilometer for alarms. Ceilometer consumes all the notifcations from the openstack projects. We can then create alarm resources within the template (which sets up an alarm in ceilometer) then you need a policy resource that will handle actioning the alarm.

I encourage you get involved in the upstream development, or to at least convey your use case so we can build that functionality in.

edit flag offensive delete link more

answered 2013-06-26 08:19:44 -0500

shardy gravatar image

As Angus has outlined, several aspects of this may become easier when the ceilometer integration is completed. You can track this work via this BP:

Part of this work will be to implement a "webhook" style interface to allow notifications to heat on alarm state changes.

It seems like nova notifications should be something we can receive via the same/similar mechanism, or perhaps ceilometer will allow us to set alarms on specific nova notifications directly.

The other aspect of nova notifications which we would like to investigate is using notifications for state-changes (e.g BUILD to ACTIVE), to avoid the polling of nova instance state that we currently do:

Please do come and get involved with the community so we can more fully understand your use-case! :)

edit flag offensive delete link more

answered 2013-06-27 09:31:54 -0500

samuli-silvius gravatar image

Thanks for your replies so far!

For a short time solution I could try to integrate the whole functionality inside heat-engine as proposed. But this future solution sounds better as later we would also like to consume ceilometer meters concerning stack instances.

Our current use case is to get immediate info (nova notification) about instance stopped (e.g. killed with kill -9) and restart it if certain MetricName/Namespace defined in Alarm for that instance (having restarter policy also).

Is there already some ready code + examples templates about "webhook" way to get notifications to heat? Does that solution use ceilometer current (grizzly) api or does that need changes to ceilometer also?

This area heat-ceilometer and getting "fault management" and "performance management" related online information could be the area where to contribute if getting familiar enough to these plans.

edit flag offensive delete link more

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower


Asked: 2013-06-25 05:56:41 -0500

Seen: 161 times

Last updated: Jul 01 '13