Ask Your Question
0

nova detach iscsi error

asked 2015-06-26 03:29:25 -0500

foexle gravatar image

Hi guys,

i get a strange problem on all my compute hosts with volume attachments by Cinder. All volumes are attached with multipath iSCSI.

  • Ubuntu 12.04
  • Icehouse
  • KVM

how to reproduce: * Attach volume * mount volume and fill with files * umount * detach volume

root@comp8:~# multipath -ll
36006016013503c00a91d1364f910e511 dm-1 DGC,VRAID
size=3.1T features='1 queue_if_no_path' hwhandler='1 emc' wp=rw
|-+- policy='round-robin 0' prio=1 status=active
| `- 13:0:0:7 sdg 8:96 active ready  running
`-+- policy='round-robin 0' prio=0 status=enabled
  `- 12:0:0:7 sdf 8:80 active ready  running

Nova Compute log:

2015-06-26 08:17:01.994 13059 TRACE oslo.messaging.rpc.dispatcher   File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py",
line 68, in __exit__
2015-06-26 08:17:01.994 13059 TRACE oslo.messaging.rpc.dispatcher    
six.reraise(self.type_, self.value, self.tb)
2015-06-26 08:17:01.994 13059 TRACE oslo.messaging.rpc.dispatcher   File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 307, in
decorated_function
2015-06-26 08:17:01.994 13059 TRACE oslo.messaging.rpc.dispatcher    
return function(self, context, *args, **kwargs)
2015-06-26 08:17:01.994 13059 TRACE oslo.messaging.rpc.dispatcher   File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4296,
in detach_volume
2015-06-26 08:17:01.994 13059 TRACE oslo.messaging.rpc.dispatcher    
self._detach_volume(context, instance, bdm)
2015-06-26 08:17:01.994 13059 TRACE oslo.messaging.rpc.dispatcher   File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4243,
in _detach_volume
2015-06-26 08:17:01.994 13059 TRACE oslo.messaging.rpc.dispatcher    
connection_info = jsonutils.loads(bdm.connection_info)
2015-06-26 08:17:01.994 13059 TRACE oslo.messaging.rpc.dispatcher   File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/jsonutils.py",
line 164, in loads
2015-06-26 08:17:01.994 13059 TRACE oslo.messaging.rpc.dispatcher    
return json.loads(s)
2015-06-26 08:17:01.994 13059 TRACE oslo.messaging.rpc.dispatcher   File
"/usr/lib/python2.7/json/__init__.py", line 326, in loads
2015-06-26 08:17:01.994 13059 TRACE oslo.messaging.rpc.dispatcher    
return _default_decoder.decode(s)
2015-06-26 08:17:01.994 13059 TRACE oslo.messaging.rpc.dispatcher   File
"/usr/lib/python2.7/json/decoder.py", line 366, in decode
2015-06-26 08:17:01.994 13059 TRACE oslo.messaging.rpc.dispatcher    
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
2015-06-26 08:17:01.994 13059 TRACE oslo.messaging.rpc.dispatcher
TypeError: expected string or buffer
2015-06-26 08:17:01.994 13059 TRACE oslo.messaging.rpc.dispatcher
2015-06-26 08:17:01.999 13059 ERROR oslo.messaging._drivers.common [-]
Returning exception expected string or buffer to caller
2015-06-26 08:17:02.000 13059 ERROR oslo.messaging._drivers.common [-]
['Traceback (most recent call last):\n', '  File
"/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py",
line 133, in _dispatch_and_reply\n    incoming.message))\n', '  File
"/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py",
line 176, in _dispatch\n    return self._do_dispatch(endpoint, method,
ctxt, args)\n', '  File
"/usr/lib ...
(more)
edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted
0

answered 2015-06-26 06:14:42 -0500

foexle gravatar image

Hi again,

found the problem. This issue occurs only in combination with iscsi and multipath because the connection_info string is only set for one target in the database. There is a bug description https://bugs.launchpad.net/nova/+bug/1327218

It can be used in Icehouse too!

But in addition there is another problem with detaching. The detach process is now working but after that only one device (of two if you're using multipath) is marked as deleted. So you can't attach this volume on the same instance on the same device name.

Cheers Heiko

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2015-06-26 03:29:25 -0500

Seen: 249 times

Last updated: Jun 26 '15