shardy's profile - activity

2017-10-16 08:05:55 -0600 received badge  Necromancer (source)
2016-01-13 02:55:13 -0600 received badge  Enthusiast
2016-01-12 05:08:05 -0600 answered a question Heat stack create failed

Check out

http://docs.openstack.org/developer/t...

and

http://hardysteven.blogspot.co.uk/201...

In this case, it appears puppet failed to apply via an OS::Heat::SoftwareDeployment resource, so the steps to debug are:

  1. heat resource-list -n5 overcloud | grep FAILED | grep "SoftwareDeployment "
  2. Take the ID of the failed deployment and do heat deployment-show <id>

This should show you the stderr from puppet, and provide more clues why things failed. Work is currently underway to expose this sort of debug data more automatically via tripleoclient.

The root cause of this is often mismatched images/templates, e.g if you run the same images in glance for a while, eventually a change will be made in tripleo-heat-templates which requires e.g newer puppet modules in the image, so you'll have to rebuild the images and update them in glance. Check out tripleo-docs for info on this:

http://docs.openstack.org/developer/t...

You can pass --update-existing to openstack overcloud image upload to update the existing images in glance.

2014-11-10 05:50:15 -0600 commented answer Heats webhook respond 'incomplete-signature'-error (not conform to AWS standards)

See https://ask.openstack.org/en/question... for the resolution to this problem

2014-11-10 05:03:55 -0600 answered a question Heat Autoscaling Webhook authorization failure

I believe this is the same issue as https://bugs.launchpad.net/heat/+bug/... , where the reporter had a spurious ec2tokens in their auth_uri in heat.conf.

We can see the evidence in your engine.log:

http://xxx.xxx.xxx.xxx:5000/v2.0/ec2t...

Changing the ec2tokens auth_uri in your heat.conf to remove the ec2tokens path suffix should resolve the issue.

Note we added logic to tolerate this configuration to Juno heat, perhaps this should be backported to Icehouse, and definitely if there are still puppet manifiests and/or docs which configure heat this way, they should be fixed.

Here's the heat patch:

https://review.openstack.org/#/c/98827/

So, to clarify, change: auth_uri=http://xxx.xxx.xxx.xxx:5000/v2.0/ec2tokens to: auth_uri=http://xxx.xxx.xxx.xxx:5000/v2.0

and it should work fine.

2014-11-06 03:27:58 -0600 answered a question Heats webhook respond 'incomplete-signature'-error (not conform to AWS standards)

Hi, please can you try the following:

curl -XPOST -i "http://xxx.xxx.xxx.xxx:8000/v1/signal/arn%3Aopenstack%3Aheat%3A%3A9036a06bb9f648beb8b4d4592e693735%3Astacks%2FTestStack%2F3d0054f4-401a-4578-b9fd-4b6374bc69b2%2Fresources%2Fweb_server_scaleup_policy?Timestamp=2014-11-06T02%3A47%3A19Z&SignatureMethod=HmacSHA256&AWSAccessKeyId=e4f6cefc18a34695b1aba1bd5e0bbb0a&SignatureVersion=2&Signature=dOjsDdfO37Ym0%2F3t5A1qzxNmrjfYHdohJZ%2FQ3HBJafU%3D"

If you don't quote the URL, then the & character before the first query-string argument backgrounds the curl and fails to pass the arguments via the request

2014-09-04 02:24:28 -0600 received badge  Nice Answer (source)
2013-12-20 12:18:49 -0600 answered a question How to recover from failed stack updates

Hi,

This is a known problem, which we're working on fixing. See https://bugs.launchpad.net/heat/+bug/... which I've linked.

Also related, see summit etherpad:

https://etherpad.openstack.org/p/iceh...

And related blueprints:

https://blueprints.launchpad.net/heat... https://blueprints.launchpad.net/heat...

There is a patch posted for retry-failed-update, which when merged, I think may solve your problem: https://review.openstack.org/#/c/62936/

2013-12-17 18:39:49 -0600 answered a question for loop in HOT

This idea has been discussed, there was even a PoC patch which would allow this sort of templating using jinja2 ( https://review.openstack.org/#/c/41858/ ), but currently the answer is no, this is not possible in HOT, and there are no immediate plans to add advanced templating or program-control primitives to HOT like for loops.

The issue is that we have to weigh up the advantages of adding functionality to the HOT DSL, over the complexity and ovehead involved in maintaining it (and as mentioned in the review above, security concerns in the case of some feature-rich templating engines)

For now, I would suggest you have a simple wrapper script which mechanically generates your template, this should be very simple, e.g in python or shell script.

2013-10-23 09:11:15 -0600 answered a question Missing Orchestration v1 API documentation for Develop applications for OpenStack clouds

Note there is also documentation on resource interfaces and template syntax here:

http://docs.openstack.org/developer/h...

And some documentation for the python API provided by python-heatclient:

http://docs.openstack.org/developer/p...

The latter is in need of improvement however.

2013-10-23 09:07:44 -0600 answered a question Missing Orchestration v1 API documentation for Develop applications for OpenStack clouds

Hi,

We have API documentation here:

http://api.openstack.org/api-ref-orch...

Does that meet your requirements?

Steve

2013-10-17 09:46:49 -0600 answered a question How to get floating IP of an instance in a template

Raised bug #1240893 to add the missing attributes_schema

2013-10-17 09:21:59 -0600 answered a question loadbalancer.py hard-coded for Fedora17?

Unfortunately this is a known issue, we need to update the nested AWS::ElasticLoadBalancing::LoadBalancer resource template to use a newer version of Fedora, as F17 is EOL.

Your options are: - Download the F17 image which is currently required, and upload it to glance: http://fedorapeople.org/groups/heat/p...

I've linked the related bug

2013-10-17 04:06:50 -0600 received badge  Editor (source)
2013-10-17 04:05:29 -0600 commented answer heat security group id

Edits added to my initial answer

2013-10-16 14:56:51 -0600 answered a question heat security group id
  • Can you use OS::Neutron::Firewall instead of AWS::EC2::SecurityGroup? In general we expect users to use either the native Neutron resources (recommended), or the AWS-compatible VPC resources if portability when CFN is a concern

Edit: Yes, OS::Neutron::Firewall provides perimeter firewall functionality, but I was thinking it could possibly solve your use-case as follows (disclaimer, I'm not a Neutron expert):

  • Modify the default security group for the project/tenant to allow the traffic required

  • Connect all instances to a Neutron subnet, which routes via a Neutron router to your external network

  • Configure the Neutron FWaaS on the router via OS::Neutron::Firewall to enforce the rules required outside the private subnet between instances/stacks

  • If you define the VpcId property of the AWS::EC2::SecurityGroup, Ref returns the security group ID, not name, does this solve your problem?

Edit: Re VpcId, you could use AWS::EC2::VPC to create the network/router and pass that into the security group, but just setting the VpcId property to AWS::EC2::SecurityGroup makes it use the neutron security group API instead of the nova one, which is probably what you want in this case.

I agree, it's not a clean interface - what we probably need is a heat-native OS::Neutron::SecurityGroup resource.

2013-07-31 12:02:13 -0600 answered a question Authorization problem when using "WaitCondition" by non-admin tenant user
2013-07-23 10:32:31 -0600 answered a question How do I solve error NotAuthenticated when trying to upload image to Glance

In addition to the suggestions above, this can happen if the glance service is misconfigured; check that /etc/glance/glance-api-paste.ini and /etc/glance/glance-registry-paste.ini have admin_tenant_name, admin_user and admin_password set correctly

2013-07-22 09:08:02 -0600 answered a question ElasticLoadBalancing

Also check that you have enough RAM to spin up the instance - nova fails in this way (with a not-that-obvious error output) when the hypervisor has insufficient resources for the requested nova instance flavor, a common problem in resource-constrained test environments.

2013-07-06 02:32:49 -0600 received badge  Nice Answer (source)
2013-07-01 09:56:02 -0600 answered a question Port is removed in instance delete and causes instance start failure with HARestarter

This looks like it's probably a bug, raised 1196479

2013-06-27 10:05:52 -0600 answered a question How to fill context for rpc call when calling heat engine

Hi,

The use case you describe is already supported, but not yet via nova notifications, see these templates (one as previously linked by Angus);

https://github.com/openstack/heat-tem... https://github.com/openstack/heat-tem...

The first template restarts the httpd service if it is not running, then rebuilds the instance if this fails more than a defined number of times.

The second template sends a heartbeat message from the instance, such that it is rebuilt if the heartbeat timeout is exceeded.

Clearly using nova notifications to do this will be better long-term, but this is the solution we have today (in grizzly).

For havana, as previously outlined, things are set to change considerably - the watch-ceilometer BP is tracking the integration with the new-for-havana ceilometer alarms functionality. This work includes the "webhook" mechanism described, it does not yet exist, we're still working out how to implement it.

I would suggest not pursuing the option of implementing this all in heat-engine, as we plan to remove the periodic/watchrule functionality when the ceilometer integration is completed.

IMO your effort is likely to be better spent ensuring that the new ceilometer alarms functionality is capable of alarming in the scenarios where you wish to trigger heat actions from, then when we complete the watch-ceilometer work you should have a solution that works for you in havana.

2013-06-26 08:19:44 -0600 answered a question How to fill context for rpc call when calling heat engine

As Angus has outlined, several aspects of this may become easier when the ceilometer integration is completed. You can track this work via this BP:

https://blueprints.launchpad.net/heat...

Part of this work will be to implement a "webhook" style interface to allow notifications to heat on alarm state changes.

It seems like nova notifications should be something we can receive via the same/similar mechanism, or perhaps ceilometer will allow us to set alarms on specific nova notifications directly.

The other aspect of nova notifications which we would like to investigate is using notifications for state-changes (e.g BUILD to ACTIVE), to avoid the polling of nova instance state that we currently do:

https://github.com/openstack/heat/blo...

Please do come and get involved with the community so we can more fully understand your use-case! :)

2013-06-25 09:33:11 -0600 answered a question How to fill context for rpc call when calling heat engine

Basically, you need to either do this inside heat-engine (where we store encrypted credentials in the database for exactly this kind of scenario), or store credentials in your code.

Is there a reason why you aren't implementing this either in the main heat-engine code, or via a custom resource plugin?

See:

https://wiki.openstack.org/wiki/Heat/... and https://github.com/openstack/heat/blo...

We would love to have improved support for nova notifications in heat, so it would be great if you could work with us and contribute to heat, instead of doing something standalone and accessing via the RPC API.

Note the RPC API is intended to be private, and may change at any time (we aren't planning to declare the RPC version final until the havana release), so if you go ahead with this approach, your code is likely to get broken, it would be much better to add this to heat-engine or use a custom resource plugin IMO, we do not support usage of the heat RPC interfaces in this way.

2013-06-17 04:53:05 -0600 answered a question Why HAproxy fails with CFN Auto-Scaling Template (Heat / Grizzly) ?

This is probably happening for one of two reasons:

1 - The autoscaling group instance has not yet launched - until the first instance in the WebServerGroup AutoScalingGroup is launched, the /etc/haproxy/haproxy.cfg will be empty, which will cause the haproxy error you describe to appear in the logs.

When the first WebServerGroup instance is built, the haproxy.cfg is updated (via cfn-hup, which is configured to poll for metadata updates, which contain the haproxy config to be applied, which changes after the initial WebServerGroup instance is built, and when any scaling adjustment happens resulting in modification of the list of instances owned by the AutoScalingGroup.

2 - The haproxy instance cannot access the metadata. This can happen if you've not configured your firewall to allow cfn-hup inside the LoadBalancer instance access to the heat-api-cfn service (tcp port 8000), or if you've forgotten to update heat_metadata_server_url in /etc/heat/heat-engine.conf (which must be an IP accessible to the instance, so either the bridge device IP, or the public IP of the heat-api-cfn service depending on your environment.)

# echo "Do this inside the LoadBalancer instance"
# yum install nmap
# cat /var/lib/cloud/data/cfn-metadata-server 
http://192.168.251.1:8000
# nmap 192.168.251.1
Starting Nmap 6.01 ( http://nmap.org ) at 2013-06-17 05:32 EDT
Nmap scan report for 192.168.251.1
Host is up (0.00037s latency).
Not shown: 998 filtered ports
PORT     STATE SERVICE
53/tcp   open  domain
8000/tcp open  http-alt

So in the test above, if /var/lib/cloud/data/cfn-metadata-server is 127.0.0.1, you've forgotten to update heat-engine.conf, and if you can't see 8000/tcp in the nmap output, you're probably missing some firewall rules.

See http://docs.openstack.org/developer/heat/getting_started/on_fedora.html#update-heat-engine-configuration-file and http://docs.openstack.org/developer/heat/getting_started/on_fedora.html#iptables-rules

2013-06-15 04:06:38 -0600 received badge  Teacher (source)
2013-06-14 03:22:43 -0600 answered a question heat stack-list returns Invalid OpenStack Identity credentials

The password 'secrete' may well be OK, since it's the default used in the keystone sample data script (/usr/share/openstack-keystone/sample_data.sh)

The problem will be easier to diagnose with the corresponding tail of tail /var/log/heat/api.log, it may be that some of the auth* values in /etc/heat/heat-api.conf are wrong (this needs to be configured to authenticate users with keystone, did you remember to set the host/tenant/user/password?)

Note if you're running grizzly the auth* settings are in api-paste.ini not heat-api.conf

2013-05-09 10:02:38 -0600 answered a question What are the differences among different pre-built images

Hi the images are built using the process documented in: http://docs.openstack.org/developer/h...

Using the oz templates in: https://github.com/openstack/heat-tem...

The "gold" templates do not contain the additional heat-cfntools scripts, which are required for some heat functionality, however we no longer maintain the "gold" tdls, so these images should probably be removed

Likewise the "-pkg" images look to be old images which should probably be removed - I'm not sure exactly what differences there are from the non-pkg images, but we won't be supporting their usage.

If you want to check the image contents, you can use libguestfs, then diff the results for two images: virt-ls -Rl -a F17-x86_64-cfntools.qcow2 /

I suggest using the F17-x86_64-cfntools.qcow2 image. I'll follow up and get the old images removed soon, and hopefully get some newer images for F18 and newer Ubuntu version uploaded.

2013-05-09 08:01:52 -0600 answered a question Would the VMs created by Heat download packages from internet?

Also note there are some details in the wiki re local mirroring of Fedora Yum repos, which can be used if you want to avoid the instances downloading from the internet repos every time you launch a stack:

https://wiki.openstack.org/wiki/Heat/...

2013-03-26 10:22:36 -0600 answered a question Heat checks version each time setup.py is executed

Heat uses oslo setup to check the current version, in a similar way to other openstack projects:

https://github.com/openstack/oslo-inc...

This version info is used to create the python egg directory name - hence we need either a tag name or a git unique SHA version (for development versions) to create an appropriately named egg to install.

Please drop in to #heat on Freenode if you need further information, as I'm not sure I understand your use-case based on the "I cant push heat submodule into my repository" comment, thanks!

2013-02-13 11:13:13 -0600 answered a question Windows Server support

Heat does not currently have any support for Windows instances - although it could probably launch a windows instance using an image from glance

We do not have support for the in-instance tools (heat-cfntools) on windows, nor have we done any testing of the windows cloud-init which would be required to allow customization of the instance once launched.

There is a blueprint to investigate adding support for windows instances, but there is not anyone currently assigned to it, and we are not working on delivering it for the Grizzly release cycle:

https://blueprints.launchpad.net/heat...

If this feature proves important to enough users (or potential users), I guess we may decide to look at it during the H development cycle.