Ask Your Question
0

Cinder volume is not attaching with error "iscsiadm: No session found"

asked 2016-11-08 08:43:22 -0500

vathanlal gravatar image

updated 2016-11-08 08:46:47 -0500

Hai,

I tried to attach volume to my instance test1 as below

# nova volume-attach test1 4b2f4ac6-3994-4163-a896-ecb009f6628a
+----------+--------------------------------------+
| Property | Value                                |
+----------+--------------------------------------+
| device   | /dev/vdb                             |
| id       | 4b2f4ac6-3994-4163-a896-ecb009f6628a |
| serverId | 3bde7c56-f845-4fb2-b190-a068202cc2c9 |
| volumeId | 4b2f4ac6-3994-4163-a896-ecb009f6628a |
+----------+--------------------------------------+

# nova volume-list
+--------------------------------------+-----------+--------------+------+-------------+-------------+
| ID                                   | Status    | Display Name | Size | Volume Type | Attached to |
+--------------------------------------+-----------+--------------+------+-------------+-------------+
| 4b2f4ac6-3994-4163-a896-ecb009f6628a | attaching | test1        | 1    | None        |             |
| d402e3ef-eea5-4cb6-b368-3cc8a8f30813 | available | test-volume1 | 15   | None        |             |

+--------------------------------------+-----------+--------------+------+-------------+-------------+

As seen above it is showing attaching for long time and nothing is happening. After sometime it will again show as available without anything attached to the instance. In my /var/log/nova/nova-compute.log iam getting an error like this.

2016-11-08 13:23:28.906 49554 TRACE nova.virt.block_device [instance: 3bde7c56-f845-4fb2-b190-a068202cc2c9] ProcessExecutionError: Unexpected error while running command.



  2016-11-08 13:23:28.906 49554 TRACE nova.virt.block_device [instance: 3bde7c56-f845-4fb2-b190-a068202cc2c9] Command: sudo nova-rootwrap /etc/nova/rootwrap.conf iscsiadm -m node -T iqn.2010-10.org.openstack:volume-4b2f4ac6-3994-4163-a896-ecb009f6628a -p 172.30.210.190:3260 --rescan

2016-11-08 13:23:28.906 49554 TRACE nova.virt.block_device [instance: 3bde7c56-f845-4fb2-b190-a068202cc2c9] Exit code: 21

2016-11-08 13:23:28.906 49554 TRACE nova.virt.block_device [instance: 3bde7c56-f845-4fb2-b190-a068202cc2c9] Stdout: u''

2016-11-08 13:23:28.906 49554 TRACE nova.virt.block_device [instance: 3bde7c56-f845-4fb2-b190-a068202cc2c9] Stderr: u'iscsiadm: No session found.\n'

2016-11-08 13:23:28.906 49554 TRACE nova.virt.block_device [instance: 3bde7c56-f845-4fb2-b190-a068202cc2c9]

2016-11-08 13:23:28.909 49554 DEBUG nova.volume.cinder [req-164d070f-466b-4422-84a2-3ecbb007ad33 None] Cinderclient connection created using URL: http://os_cinder:8776/v1/15fa6e1ce28d446fabf8dba757802093 get_cinder_client_version /usr/lib/python2.7/dist-packages/nova/volume/cinder.py:255

2016-11-08 13:23:29.990 49554 ERROR nova.compute.manager [req-164d070f-466b-4422-84a2-3ecbb007ad33 None] [instance: 3bde7c56-f845-4fb2-b190-a068202cc2c9] Failed to attach 4b2f4ac6-3994-4163-a896-ecb009f6628a at /dev/vdb

Iam also getting an error in my /var/log/cinder/cinder-api.log iam also getting an error like this.

2016-11-08 14:54:45.263 22416 TRACE cinder Traceback (most recent call last):
2016-11-08 14:54:45.263 22416 TRACE cinder   File "/usr/bin/cinder-api", line 60, in <module>
2016-11-08 14:54:45.263 22416 TRACE cinder     server = service.WSGIService('osapi_volume')
2016-11-08 14:54:45.263 22416 TRACE cinder   File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 345, in __init__
2016-11-08 14:54:45.263 22416 TRACE cinder     port=self.port)
2016-11-08 14:54:45.263 22416 TRACE cinder   File "/usr/lib/python2.7/dist-packages/cinder/wsgi.py", line 186, in __init__
2016-11-08 14:54:45.263 22416 TRACE cinder     {'host': host, 'port': port})
2016-11-08 14:54:45.263 22416 TRACE cinder RuntimeError: Could not bind to 0.0.0.0:8776 after trying for 30 seconds

I dont know whether it is caused by the cinder-api error or something else. The strange thing is everything was working fine couple of weeks ago. I can create volume and attach it to the instances. But now I can only create volumes but attaching to instance is not working. Any help regarding this is highly appreciated.

edit retag flag offensive close merge delete

Comments

1

Was there any network changes/firewall?

bishoy gravatar imagebishoy ( 2016-11-08 09:14:29 -0500 )edit

No there is no network changes..

vathanlal gravatar imagevathanlal ( 2016-11-08 09:41:06 -0500 )edit

Something wrong with the setup around iSCSI or the iSCSI target? Check if there is anything in the system log, and try running iscsiadm commands to see what it might be. More detailed instructions here: https://ask.openstack.org/en/question...

Bernd Bausch gravatar imageBernd Bausch ( 2016-11-08 16:35:31 -0500 )edit

Then the Cinder API error - did you run cinder service-list? Is Cinder functional at all? I note however that the time stamps of the two messages are far apart; are you sure they are related?

Bernd Bausch gravatar imageBernd Bausch ( 2016-11-08 16:43:39 -0500 )edit

ya I run the cinder service-list and I can see the cinder-scheduler and cinder-volume up. But when I stopped the cinder services still one process is running in port 8776. I tried to kill it but it keeps on running with new process id. I dont know whether that is causing the issue..

vathanlal gravatar imagevathanlal ( 2016-11-09 02:45:56 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
0

answered 2016-11-10 04:35:39 -0500

vathanlal gravatar image

updated 2016-11-10 04:36:39 -0500

at last I found the solution..

cinder RuntimeError: Could not bind to 0.0.0.0:8776 after trying for 30 seconds

the above error is due to the fact that another process is running in the cluster with port 8776 and you need to kill that mannually.. use netstat to check whether another process is running with port 8776. And kill that process and restart cinder service will solve this issue

netstat -tupln | grep 8776 // identify the proc-id
kill -9 proc-id  // proc-id using 8776

Then the next error

Stderr: u'iscsiadm: No session found.\n'

For me this error was caused by the miss configuration someone did with my management IP's in the cinder /etc/cinder/cinder.conf file in controller node.. I changed that to my proper management IP and now I can attach volumes..

my_ip = your proper management IP
edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2016-11-08 08:43:22 -0500

Seen: 1,879 times

Last updated: Nov 10 '16