Ask Your Question

cinder volume remains stuck in creating

asked 2014-06-10 01:30:46 -0600

Krist gravatar image

updated 2014-06-10 02:16:56 -0600

Hi, When I try to create a cinder volume it remains "stuck" in the creating state. When looking at the logs I find the following error message in the scheduler.log:

2014-06-10 05:16:39.328 21506 ERROR cinder.openstack.common.rpc.impl_qpid [req-5da3e23e-540a-4bc5-a8a1-14c3324ed72d 1380322e96eb4759a4a1488af6a1b
017 3e4609c9647948c79b887b19242ccf23] Failed to publish message to topic 'cinder-volume:store-g9s1202': connection aborted

On the cinder-volume host (store-g9s1202) no messages related to this request are found. Both the cinder-scheduler and cinder-volume processes are connected to the amqp service. I verified this with lsof, and there are no messages in the logs indicating conneciton problems. On the amqp server (we're using qpid) i verified that the queu for topic cinder-volume:store-g9s1202' exists.

This looks like cinder-scheduler trying to tell cinder-volume to do something, and failing. But why?

We are running Havana on RHEL 6.5

edit retag flag offensive close merge delete


With what release of Openstack (Havana or IceHouse) you are using QPID ?

dbaxps gravatar imagedbaxps ( 2014-06-10 01:52:46 -0600 )edit

Havana on RHEL 6.5

Krist gravatar imageKrist ( 2014-06-10 02:17:44 -0600 )edit

Please, double check AMPQ Queues (qpid in your case) running and well configured in your Openstack components
and the Centos Firewall opened to accept connectios to QPID Port.
View cinder-scheduler,cinder-volume log files. Error or warning should be somewhere.

dbaxps gravatar imagedbaxps ( 2014-06-10 02:25:54 -0600 )edit

I have verified that both cinder-scheduler and cinder-volume can connect to qpidd. I have verified that they are connected. I am not seeing any errors except for the one I posted.

Krist gravatar imageKrist ( 2014-06-10 02:33:33 -0600 )edit

Not sure, it would be acceptable for you (production env and etc .. )
1 cinder force-delete volume-id
2. service qpidd restart

dbaxps gravatar imagedbaxps ( 2014-06-10 02:47:44 -0600 )edit

cindr force-delete results in the volume now being stuck in a "deleting" state. starting qpid did not change a thing.

Krist gravatar imageKrist ( 2014-06-10 06:34:01 -0600 )edit

Would try to create a small cinder volume via CLI and see what happens after qpidd restart ?

dbaxps gravatar imagedbaxps ( 2014-06-10 06:38:37 -0600 )edit

All the actions I tried result in the same connection aborted errors in scheduler.log. So for some reason qpid is refusing connections. This is odd, as the qpid config is identical for nova-sechduler (which runs on the same host) which has no problems communicating with qpid...

Krist gravatar imageKrist ( 2014-06-10 07:02:32 -0600 )edit

It' s firewall issue.
$ iptables-save | grep QPIDD_PORT (5672 I guess)
$ netstat -lnpt | grep qpidd shoud give pid of QPIDD process

dbaxps gravatar imagedbaxps ( 2014-06-10 07:07:29 -0600 )edit

Please , check port status :-
$ iptables-save | grep QPIDD_PORT (5672 I guess)
$ netstat -lnpt | grep qpidd shoud give pid of QPIDD process
To verify daemon itself, not nova logs

dbaxps gravatar imagedbaxps ( 2014-06-10 07:28:11 -0600 )edit

No it is not a firewall issue. I can see cinder-volume and cinder-scheduler connected to qpid. I verified that using netstat, and lsof. The firewall is ok as well.

Krist gravatar imageKrist ( 2014-06-10 09:17:48 -0600 )edit

2 answers

Sort by ยป oldest newest most voted

answered 2014-07-01 04:31:29 -0600

Eduard Barrera gravatar image

I'm experiencing the same issue,

iptables flushed selinux in permissive

Any idea ?

edit flag offensive delete link more

answered 2015-03-29 00:27:36 -0600

ethode gravatar image

I am having the same issue on Ubuntu 14.04 - no firewall even installed

edit flag offensive delete link more

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower


Asked: 2014-06-10 01:30:46 -0600

Seen: 1,472 times

Last updated: Jul 01 '14