Cinder Vol error when creating volumes larger than 200G

Question

1

Cinder Vol error when creating volumes larger than 200G

asked 2014-02-05 06:41:00 -0600

mili
93 ●10 ●15 ●21

updated 2014-02-07 11:58:47 -0600

Hi,

My cinder was working just fine till now, I have 9TB available on this Volume but when i try to create something bigger than 200G the vol goes in error state. here is the cinder-volume log:

2014-02-05 12:35:37.765 2022 WARNING cinder.taskflow.utils [-] Activating 5 rollbacks due to <cinder.taskflow.utils.FlowFailure object at 0x39c6950>.
2014-02-05 12:35:37.787 2022 ERROR cinder.volume.flows.create_volume [req-1fe0dd36-129e-4fe7-b65f-932804fb4cd6 99d3e2d97d2c430782d4a32e4a7e1e40 a99a7cc02a7a40f9aeea5efa0b397740] Volume 14629ea8-272f-45d9-b7ae-ce2320b00f3a: create failed
2014-02-05 12:35:37.787 2022 ERROR cinder.volume.flows.create_volume [req-1fe0dd36-129e-4fe7-b65f-932804fb4cd6 99d3e2d97d2c430782d4a32e4a7e1e40 a99a7cc02a7a40f9aeea5efa0b397740] Unexpected build error:
2014-02-05 12:35:37.787 2022 TRACE cinder.volume.flows.create_volume Traceback (most recent call last):
2014-02-05 12:35:37.787 2022 TRACE cinder.volume.flows.create_volume   File "/usr/lib/python2.7/dist-packages/cinder/taskflow/patterns/linear_flow.py", line 172, in run_it
2014-02-05 12:35:37.787 2022 TRACE cinder.volume.flows.create_volume     result = runner(context, *args, **kwargs)
2014-02-05 12:35:37.787 2022 TRACE cinder.volume.flows.create_volume   File "/usr/lib/python2.7/dist-packages/cinder/taskflow/utils.py", line 260, in __call__
2014-02-05 12:35:37.787 2022 TRACE cinder.volume.flows.create_volume     self.result = self.task(*args, **kwargs)
2014-02-05 12:35:37.787 2022 TRACE cinder.volume.flows.create_volume   File "/usr/lib/python2.7/dist-packages/cinder/volume/flows/create_volume/__init__.py", line 1499, in __call__
2014-02-05 12:35:37.787 2022 TRACE cinder.volume.flows.create_volume     **volume_spec)
2014-02-05 12:35:37.787 2022 TRACE cinder.volume.flows.create_volume   File "/usr/lib/python2.7/dist-packages/cinder/volume/flows/create_volume/__init__.py", line 1470, in _create_raw_volume
2014-02-05 12:35:37.787 2022 TRACE cinder.volume.flows.create_volume     return self.driver.create_volume(volume_ref)
2014-02-05 12:35:37.787 2022 TRACE cinder.volume.flows.create_volume   File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/lvm.py", line 165, in create_volume
2014-02-05 12:35:37.787 2022 TRACE cinder.volume.flows.create_volume     mirror_count)
2014-02-05 12:35:37.787 2022 TRACE cinder.volume.flows.create_volume   File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/lvm.py", line 154, in _create_volume
2014-02-05 12:35:37.787 2022 TRACE cinder.volume.flows.create_volume     vg_ref.create_volume(name, size, lvm_type, mirror_count)
2014-02-05 12:35:37.787 2022 TRACE cinder.volume.flows.create_volume   File "/usr/lib/python2.7/dist-packages/cinder/brick/local_dev/lvm.py", line 370, in create_volume
2014-02-05 12:35:37.787 2022 TRACE cinder.volume.flows.create_volume     run_as_root=True)
2014-02-05 12:35:37.787 2022 TRACE cinder.volume.flows.create_volume   File "/usr/lib/python2.7/dist-packages/cinder/utils.py", line 142, in execute
2014-02-05 12:35:37.787 2022 TRACE cinder.volume.flows.create_volume     return processutils.execute(*cmd, **kwargs)
2014-02-05 12:35:37.787 2022 TRACE cinder.volume.flows.create_volume   File "/usr/lib/python2.7/dist-packages/cinder/openstack/common/processutils.py", line 173, in execute
2014-02-05 12:35:37.787 2022 TRACE cinder.volume.flows.create_volume     cmd=' '.join(cmd))
2014-02-05 12:35:37.787 2022 TRACE cinder.volume.flows.create_volume ProcessExecutionError: Unexpected error while running command.
2014-02-05 12:35:37.787 2022 TRACE ...

(more)

edit retag flag offensive close delete

Comments

Important information missing here; this is a SAN attached Raid with LVM on top of it. Also the missing part of the trace: 2014-02-07 11:12:23.586 2022 TRACE cinder.openstack.common.rpc.amqp Stderr: ' device-mapper: resume ioctl failed: Invalid argument\n Unable to resume cinder--volumes1-volume--a0297a8d--0a0e--4244--bc64--d882454fa0ce (252:3)\n Internal error: Maps lock 14544896 < unlock 14548992\n Aborting. Failed to activate new LV to wipe the start of it.\n' Suspect your backend device isn't matching actual capacity to what you setup in the VG. Regardless you should check the system logs for info about your backend device here.

jgriffith ( 2014-02-07 12:27:41 -0600 )edit

Important information missing here; this is a SAN attached Raid with LVM on top of it. Also the missing part of the trace: 2014-02-07 11:12:23.586 2022 TRACE cinder.openstack.common.rpc.amqp Stderr: ' device-mapper: resume ioctl failed: Invalid argument\n Unable to resume cinder--volumes1-volume--a0297a8d--0a0e--4244--bc64--d882454fa0ce (252:3)\n Internal error: Maps lock 14544896 < unlock 14548992\n Aborting. Failed to activate new LV to wipe the start of it.\n' Suspect your backend device isn't matching actual capacity to what you setup in the VG. Regardless you should check the system logs for info about your backend device here.
jgriffith ( 2014-02-07 12:27:41 -0600 )edit

Answer 1

1

answered 2014-02-07 12:36:27 -0600

jgriffith
146 ●1 ●5

updated 2014-02-07 12:42:42 -0600

So after chatting in IRC, this is an issue with your SAN device that you're using. The kernel logs included the message stating you were out of space on the device and couldn't allocate as much as you requested.

Given that you're running an external attached device and laying the Cinder LVM driver over top of it and that the kernel is not agreeing with what you see as "free-space" I don't think this is a Cinder problem at all but more of an error with your backend device, it's reporting and your configuration of the Cinder Volume Group on top of it.

I'd suggest removing cinder from the equation. Since you've attached the SAN Raid device and created the VG, just try lvcreate manually and see if you have the same issue.

edit flag offensive delete publish link

Comments

Thank you very much jgriffith, I'm still investigating this and will update the post as soon as i have more, but at this point it looks like a a strange SAN issue

mili ( 2014-02-13 06:28:13 -0600 )edit

Cinder Vol error when creating volumes larger than 200G

Comments

1 answer

Comments

Your Answer

Question Tools

Stats

Related questions

Feedback About This Page

OpenStack

Community

Documentation

Branding & Legal