(Queens) (Ubuntu 16.04) Cannot start any instances: they never move beyond scheduling

asked 2019-01-15 15:24:55 -0500

njohnson gravatar image

Hi all,

All new instances I create get stuck in the scheduling phase. I can create new volumes, networks, routers and images just fine. A little bit about my setup: two load balancers running haproxy and keepalived, two controller nodes, two compute nodes and single storage node running Cinder. The database is a Galera cluster replicated across three nodes. Also I am using rabbitmq as my message queue.

I have been for the openstack install guide for queens and trying to start an instance using the cirros image and m1.nano flavor described in the guide. My compute nodes have 16 cores, 128 GB of RAM, and 78 GB of local HDD space. The m1.nano uses 1 CPU and 64 MB of RAM so I should have plenty of resources on either node. There are no other instances running on the nodes.

https://docs.openstack.org/install-guide/launch-instance.html (link text)

Logs: nova-api.log http://paste.openstack.org/show/742685/ (link text) nova-scheduler.log http://paste.openstack.org/show/742686/ (link text) nova-conductor.log http://paste.openstack.org/show/742687/ (link text) rabbitmq log http://paste.openstack.org/show/742688/ (link text)

The only errors I see are dropped connections when oslo_db does it's connection check and as far as I can tell that shouldn't have any negative effect. I can manually access the databases as the openstack service users without issue.

I've been banging my head against this issue for far too long now so any help would be much appreciated. I'm pretty new to OpenStack so don't be afraid to point out obvious mistakes or things I've overlooked.

edit retag flag offensive close merge delete

Comments

Your log files don't contain any signs of instance launches or scheduling activities. For example, the nova-api log only contains HTTP GETs, but the API to launch a server requires a POST. Are you sure the logs come from the correct controller and the correct time frame?

Bernd Bausch gravatar imageBernd Bausch ( 2019-01-15 18:32:13 -0500 )edit

I must have grabbed the logs from the wrong controller. Started another new instance this morning and searched the logs on both controllers for any POST commands. This was only one:

2019-01-16 10:07:43 "POST /v2.1/os-volumes_boot HTTP/1.1" status: 202 len: 890 time: 1.0537820

njohnson gravatar imagenjohnson ( 2019-01-16 09:30:11 -0500 )edit