Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

cinder-volume service is down

cinder-volume service is down. I have been setting up my openstack environment for the Newton release following the documentation. My challenge is with getting the cinder-volume service to stay up and running. NTP is running (using chrony) and the cinder host is configured to synchronize with the controller. I have two disks on the node, sda and sdb. sda has normal partitioning that runs the underlying os (Ubuntu 16.04 LTS), sdb is configured as LVM with the volume group "cinder-volumes" with a size of 100GB. Any attempt to create a volume returns an error status (which is expected since cinder-volume service is not up).

Here's my cinder-volume log: /var/log/cinder/cinder-volume.log

   432  2016-11-23 17:38:31.199 1431 WARNING oslo_reports.guru_meditation_report [-] Guru mediation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports.
   433  2016-11-23 17:38:31.384 1431 INFO root [-] Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
   434  2016-11-23 17:38:31.477 1431 INFO root [-] Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
   435  2016-11-23 17:38:31.553 1431 WARNING py.warnings [req-0a43a43f-e810-4699-814f-fef6019b5a29 - - - - -] /usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py:241: NotSupportedWarning: Configuration option(s) ['use_tpool'] not supported
   436    exception.NotSupportedWarning
   437
   438  2016-11-23 17:38:31.759 1431 INFO cinder.rpc [req-0a43a43f-e810-4699-814f-fef6019b5a29 - - - - -] Automatically selected cinder-scheduler objects version 1.11 as minimum service version.
   439  2016-11-23 17:38:31.760 1431 ERROR cinder.cmd.volume [req-0a43a43f-e810-4699-814f-fef6019b5a29 - - - - -] Volume service cinder@lvm failed to start.
   440  2016-11-23 17:38:31.760 1431 ERROR cinder.cmd.volume Traceback (most recent call last):
   441  2016-11-23 17:38:31.760 1431 ERROR cinder.cmd.volume   File "/usr/lib/python2.7/dist-packages/cinder/cmd/volume.py", line 81, in main
   442  2016-11-23 17:38:31.760 1431 ERROR cinder.cmd.volume     binary='cinder-volume')
   443  2016-11-23 17:38:31.760 1431 ERROR cinder.cmd.volume   File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 268, in create
   444  2016-11-23 17:38:31.760 1431 ERROR cinder.cmd.volume     service_name=service_name)
   445  2016-11-23 17:38:31.760 1431 ERROR cinder.cmd.volume   File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 150, in __init__
   446  2016-11-23 17:38:31.760 1431 ERROR cinder.cmd.volume     *args, **kwargs)
   447  2016-11-23 17:38:31.760 1431 ERROR cinder.cmd.volume   File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 235, in __init__
   448  2016-11-23 17:38:31.760 1431 ERROR cinder.cmd.volume     *args, **kwargs)
   449  2016-11-23 17:38:31.760 1431 ERROR cinder.cmd.volume   File "/usr/lib/python2.7/dist-packages/cinder/manager.py", line 156, in __init__
   450  2016-11-23 17:38:31.760 1431 ERROR cinder.cmd.volume     self.scheduler_rpcapi = scheduler_rpcapi.SchedulerAPI()
   451  2016-11-23 17:38:31.760 1431 ERROR cinder.cmd.volume   File "/usr/lib/python2.7/dist-packages/cinder/rpc.py", line 188, in __init__
   452  2016-11-23 17:38:31.760 1431 ERROR cinder.cmd.volume     serializer = base.CinderObjectSerializer(obj_version_cap)
   453  2016-11-23 17:38:31.760 1431 ERROR cinder.cmd.volume   File "/usr/lib/python2.7/dist-packages/cinder/objects/base.py", line 412, in __init__
   454  2016-11-23 17:38:31.760 1431 ERROR cinder.cmd.volume     raise exception.CappedVersionUnknown(version=version_cap)
   455  2016-11-23 17:38:31.760 1431 ERROR cinder.cmd.volume CappedVersionUnknown: Unrecoverable Error: Versioned Objects in DB are capped to unknown version 1.11.
   456  2016-11-23 17:38:31.760 1431 ERROR cinder.cmd.volume
   457  2016-11-23 17:38:31.762 1431 ERROR cinder.cmd.volume [req-0a43a43f-e810-4699-814f-fef6019b5a29 - - - - -] No volume service(s) started successfully, terminating.

NTP settings:

root@cinder:~# chronyc sources

210 Number of sources = 1
MS Name/IP address         Stratum Poll Reach LastRx Last sample
===============================================================================
^* controller                    3   6   377     6  +1094us[+1146us] +/-  111ms

Here is the output from running the command: openstack volume service list

root@controller:~# openstack volume service list
+------------------+------------+------+---------+-------+----------------------------+
| Binary           | Host       | Zone | Status  | State | Updated At                 |
+------------------+------------+------+---------+-------+----------------------------+
| cinder-scheduler | controller | nova | enabled | up    | 2016-11-23T16:47:14.000000 |
| cinder-volume    | cinder@lvm | nova | enabled | down  | None                       |
+------------------+------------+------+---------+-------+----------------------------+

Here is the my Volume group settings:

root@cinder:~# vgdisplay
  --- Volume group ---
  VG Name               cinder-volumes
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  1
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                0
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               100.00 GiB
  PE Size               4.00 MiB
  Total PE              25599
  Alloc PE / Size       0 / 0
  Free  PE / Size       25599 / 100.00 GiB
  VG UUID               AfX4QW-BreS-7jg0-B5Bq-2oQO-AfjF-P59Ire

Here's my attempt to create a 4GB volume:

root@controller:~# cinder list
+--------------------------------------+--------+---------+------+-------------+----------+-------------+
| ID                                   | Status | Name    | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+--------+---------+------+-------------+----------+-------------+
| cb221f6f-c805-4799-a8b2-aa44a7552393 | error  | volume1 | 4    | -           | false    |             |
+--------------------------------------+--------+---------+------+-------------+----------+-------------+