Ask Your Question
0

Inconsistencies - Failure to launch instances

asked 2015-07-09 13:15:43 -0500

Wendy gravatar image

Caveat -- I'm an Openstack newbie.

We have built an openstack cloud for research purposes. It was launched using Mirantis Fuel 6.0.

I was trying to get an idea of how many VMS we could run and started 4 XL instances and 4 Large instances before running out of VCPUs. The numbers add up according to the specs for the instances and the resources shown available. I terminated all instances using the dashboard. Then I tried again -- however, this time I can create only 1 XL instance and when I try to launch a second XL instance, I get a message saying:

Failed to launch instance "xltwo": Please try again later [Error: No valid host was found. ].

The dashboard shows tons of free resources. (Well, "tons" is an exaggeration since our cloud is pretty small).

I don't even know what to do to troubleshoot -- any suggestions?

edit retag flag offensive close merge delete

Comments

check the nova log files for clues. Also the nova compute log files on the compute node(s). If you find nothing, set the debug flags in the config files and try again. The dashboard shows the quota you have left over, not the amount of memory (most likely the limiting factor) on the compute nodes.

Bernd Bausch gravatar imageBernd Bausch ( 2015-07-10 05:33:25 -0500 )edit

2 answers

Sort by ยป oldest newest most voted
0

answered 2015-07-10 01:55:21 -0500

Did you wait a bit for the old deleted instances to really free the resources that were used ? It's probably not instantaneous.

edit flag offensive delete link more

Comments

I tried again this morning --- same issue.

Wendy gravatar imageWendy ( 2015-07-10 10:35:41 -0500 )edit

Ok, so I've left it over the weekend and tried again to create instances. I was able to. Does it really take that long to reclaim resources??

Wendy gravatar imageWendy ( 2015-07-13 12:12:12 -0500 )edit
0

answered 2015-07-10 10:35:11 -0500

Wendy gravatar image

The only "issues" I see in the logs are the following: Nova-conductor.log <182>Jul 10 13:32:14 node-3 nova-api 192.168.0.2 "POST /v2/b298d3ac41d84b3ebef5a29442a2d15d/servers HTTP/1.1" status: 202 len: 750 time: 0.5268559 <180>Jul 10 13:32:14 node-3 nova-scheduler Host has more disk space than database expected (89gb > 9gb) <182>Jul 10 13:32:14 node-3 nova-scheduler Filter RamFilter returned 0 hosts <180>Jul 10 13:32:14 node-3 nova-conductor NoValidHost exception with message: 'No valid host was found.' <180>Jul 10 13:32:14 node-3 nova-conductor Setting instance to ERROR state.

Nova-scheduler.log 2015-07-10 13:32:14.467 21106 WARNING nova.scheduler.host_manager [req-ecd07914-99fa-43c4-a5b8-835e8da8fc2e None] Host has more disk space than database expected (89gb > 9gb) 2015-07-10 13:32:14.489 21106 INFO nova.filters [req-ecd07914-99fa-43c4-a5b8-835e8da8fc2e None] Filter RamFilter returned 0 hosts

Node1-nova-compute.log 2015-07-08 18:00:13.194 17536 ERROR oslo.messaging._drivers.impl_rabbit [req-01c15be9-ed98-44b9-8d8f-c82058c2a2fa ] AMQP server on localhost:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 11 seconds. <#my note -- it connects eventually

edit flag offensive delete link more

Comments

Scheduler log "Filter RamFilter returned 0 hosts" indicates that there is not enough free memory on the compute hosts (or at least that's what the scheduler thinks). Could well be a temporary condition - the instances you created earlier still occupy some memory.

Bernd Bausch gravatar imageBernd Bausch ( 2015-07-11 07:29:17 -0500 )edit

Compute log "AMQP server on localhost:5672 is unreachable: [Errno 111] ECONNREFUSED" - are you running a single node cloud? If not, the AMQP server should be on the controller, not localhost. Does this message occur frequently? Perhaps the scheduler simply doesn't find any compute node at all.

Bernd Bausch gravatar imageBernd Bausch ( 2015-07-11 07:31:19 -0500 )edit

AMQP server is running on the controller. There are 3 compute nodes and the controller.

Wendy gravatar imageWendy ( 2015-07-13 12:06:46 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2015-07-09 13:15:43 -0500

Seen: 380 times

Last updated: Jul 10 '15