Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

volume service is down

All of a sudden I cannot perform any action on cinder (like create for example)

I ran cinder-manage service list and got Binary Host Zone Status State Updated At

cinder-backup localhost nova enabled :-) 2015-12-31 06:30:04

cinder-scheduler localhost nova enabled :-) 2015-12-31 06:30:05

cinder-volume localhost nova enabled XXX 2015-12-13 21:47:37

cinder-volume localhost nova enabled :-) 2015-12-31 06:30:05

and also service openstack-cinder-volume status

Redirecting to /bin/systemctl status openstack-cinder-volume.service

openstack-cinder-volume.service - OpenStack Cinder Volume Server

Loaded: loaded (/usr/lib/systemd/system/openstack-cinder-volume.service; enabled)

Active: active (running) since Thu 2015-12-31 01:03:56 EST; 26min ago

I tried to create a volume just in case and saw under cinder’s scheduler.log : 2015-12-31 01:25:41.098 1052 ERROR cinder.scheduler.flows.create_volume [req-9e5a2373-b06b-4320-9293-05d3f7d0638d 34f31b50e2554f68b55419d6de5a8e25 5aa43c3e3e3c4558a982c99542b44248 - - -] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid host was found. No weighed hosts available

Other logs that indicates the same:

-98dbce07a7b9 - - - - -] volume service is down. (host: localhost.localdomain@lvm) 2015-12-31 01:04:43.721 1052 WARNING cinder.scheduler.host_manager [req-edb5abbd-9405-4511-821b-98dbce07a7b9 - - - - -] volume service is down. (host: localhost.localdomain@scaleio) 2015-12-31 01:04:43.791 1052 WARNING oslo_config.cfg [req-edb5abbd-9405-4511-821b-98dbce07a7b9 - - - - -] Option "amqp_durable_queues" from group "DEFAULT" is deprecated. Use option "amqp_durable_queues" from group "oslo_messaging_rabbit". 2015-12-31 01:04:43.794 1052 WARNING oslo_config.cfg [req-edb5abbd-9405-4511-821b-98dbce07a7b9 - - - - -] Option "lock_path" from group "DEFAULT" is deprecated. Use option "lock_path" from group "oslo_concurrency". 2015-12-31 01:04:43.795 1052 INFO cinder.service [-] Starting cinder-scheduler node (version 7.0.0) 2015-12-31 01:04:45.215 1052 INFO oslo.messaging._drivers.impl_rabbit [req-bfa51c3d-471b-4a5c-a001-b1d2fce90567 - - - - -] Connecting to AMQP server on 10.136.242.238:5672 2015-12-31 01:11:55.414 1052 WARNING cinder.scheduler.host_manager [req-bb076884-9d67-40d9-bbd8-49eaf9b97fa6 34f31b50e2554f68b55419d6de5a8e25 5aa43c3e3e3c4558a982c99542b44248 - - -] volume service is down. (host: localhost.localdomain@lvm) 2015-12-31 01:11:55.416 1052 INFO cinder.openstack.common.scheduler.base_filter [req-bb076884-9d67-40d9-bbd8-49eaf9b97fa6 34f31b50e2554f68b55419d6de5a8e25 5aa43c3e3e3c4558a982c99542b44248 - - -] Filter CapabilitiesFilter returned 0 host(s) 2015-12-31 01:11:55.417 1052 ERROR cinder.scheduler.manager [req-bb076884-9d67-40d9-bbd8-49eaf9b97fa6 34f31b50e2554f68b55419d6de5a8e25 5aa43c3e3e3c4558a982c99542b44248 - - -] Could

volume service is down

All of a sudden I cannot perform any action on cinder (like create for example)

I ran cinder-manage service list and got got

Binary           Host                                 Zone             Status     State Updated At

At cinder-backup localhost nova enabled :-) 2015-12-31 06:30:04

06:30:04 cinder-scheduler localhost nova enabled :-) 2015-12-31 06:30:05

06:30:05 cinder-volume localhost nova enabled XXX 2015-12-13 21:47:37

21:47:37 cinder-volume localhost nova enabled :-) 2015-12-31 06:30:05

06:30:05

and also service openstack-cinder-volume status

Redirecting to /bin/systemctl status  openstack-cinder-volume.service

openstack-cinder-volume.service openstack-cinder-volume.service - OpenStack Cinder Volume Server

Server Loaded: loaded (/usr/lib/systemd/system/openstack-cinder-volume.service; enabled)

enabled) Active: active (running) since Thu 2015-12-31 01:03:56 EST; 26min ago

ago

I tried to create a volume just in case and saw under cinder’s scheduler.log : :

2015-12-31 01:25:41.098 1052 ERROR cinder.scheduler.flows.create_volume [req-9e5a2373-b06b-4320-9293-05d3f7d0638d 34f31b50e2554f68b55419d6de5a8e25 5aa43c3e3e3c4558a982c99542b44248 - - -] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid host was found. No weighed hosts available

available

Other logs that indicates the same:

-98dbce07a7b9 - - - - -] volume service is down. (host: localhost.localdomain@lvm)
2015-12-31 01:04:43.721 1052 WARNING cinder.scheduler.host_manager [req-edb5abbd-9405-4511-821b-98dbce07a7b9 - - - - -] volume service is down. (host: localhost.localdomain@scaleio)
2015-12-31 01:04:43.791 1052 WARNING oslo_config.cfg [req-edb5abbd-9405-4511-821b-98dbce07a7b9 - - - - -] Option "amqp_durable_queues" from group "DEFAULT" is deprecated. Use option "amqp_durable_queues" from group "oslo_messaging_rabbit".
2015-12-31 01:04:43.794 1052 WARNING oslo_config.cfg [req-edb5abbd-9405-4511-821b-98dbce07a7b9 - - - - -] Option "lock_path" from group "DEFAULT" is deprecated. Use option "lock_path" from group "oslo_concurrency".
2015-12-31 01:04:43.795 1052 INFO cinder.service [-] Starting cinder-scheduler node (version 7.0.0)
2015-12-31 01:04:45.215 1052 INFO oslo.messaging._drivers.impl_rabbit [req-bfa51c3d-471b-4a5c-a001-b1d2fce90567 - - - - -] Connecting to AMQP server on 10.136.242.238:5672
2015-12-31 01:11:55.414 1052 WARNING cinder.scheduler.host_manager [req-bb076884-9d67-40d9-bbd8-49eaf9b97fa6 34f31b50e2554f68b55419d6de5a8e25 5aa43c3e3e3c4558a982c99542b44248 - - -] volume service is down. (host: localhost.localdomain@lvm)
2015-12-31 01:11:55.416 1052 INFO cinder.openstack.common.scheduler.base_filter [req-bb076884-9d67-40d9-bbd8-49eaf9b97fa6 34f31b50e2554f68b55419d6de5a8e25 5aa43c3e3e3c4558a982c99542b44248 - - -] Filter CapabilitiesFilter returned 0 host(s)
2015-12-31 01:11:55.417 1052 ERROR cinder.scheduler.manager [req-bb076884-9d67-40d9-bbd8-49eaf9b97fa6 34f31b50e2554f68b55419d6de5a8e25 5aa43c3e3e3c4558a982c99542b44248 - - -] Could 

Could