Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Cannot work any instance aftrer target.service restart on Cinder

Dear all Hi.

I'm building Juno system.

Some instances was working.

However they couldn't work ever since then I restarted target.service on Cinder.

And that doesn't happen rare occurrence.(I wonder that...)

Log messages on Compute at that time are as follows.

after target.service restart
[Compute]
$ tail -f /var/log/message
Dec 16 10:53:08 localhost kernel: connection10:0: detected conn error (1020)
Dec 16 10:53:09 localhost iscsid: conn 0 login rejected: target error (03/01)
Dec 16 10:53:11 localhost kernel: connection10:0: detected conn error (1020)
Dec 16 10:53:12 localhost iscsid: conn 0 login rejected: target error (03/01)
Dec 16 10:53:14 localhost kernel: connection10:0: detected conn error (1020)

Somehow Compute connected to any volumes on Cinder.

And that connection couldn't log out.

$ iscsiadm -m session
tcp: [10] 172.16.4.145:3260,1 iqn.2010-10.org.openstack:volume-8e604962-49d7-4895-919f-1864f3e54af5 (non-flash)
tcp: [11] 172.16.4.145:3260,1 iqn.2010-10.org.openstack:volume-c6bf3ff1-fcee-449d-a578-5f660f10c6b7 (non-flash)

$ iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e604962-49d7-4895-919f-1864f3e54af5 -u
Logging out of session [sid: 10, target: iqn.2010-10.org.openstack:volume-8e604962-49d7-4895-919f-1864f3e54af5, portal: 172.16.4.145,3260]
iscsiadm: Could not logout of [sid: 10, target: iqn.2010-10.org.openstack:volume-8e604962-49d7-4895-919f-1864f3e54af5, portal: 172.16.4.145,3260].
iscsiadm: initiator reported error (9 - internal error)
iscsiadm: Could not logout of all requested sessions

Iface settings is clear.

[Compute]
$ iscsiadm -m session -P 3
        Iface IPaddress: <empty>

Do you have any ideas?

Regard.

Cannot work any instance aftrer target.service restart on Cinder

Dear all Hi.

I'm building Juno system.

Some instances was working.

However they couldn't work ever since then I restarted target.service on Cinder.

And that doesn't happen rare occurrence.(I wonder that...)

Log messages on Compute at that time are as follows.

after target.service restart
[Compute]
$ tail -f /var/log/message
Dec 16 10:53:08 14:15:24 localhost kernel: connection10:0: detected conn error (1020)
iscsid: connect to 172.16.4.145:3260 failed (Connection refused)
Dec 16 10:53:09 14:15:28 localhost iscsid: conn 0 login rejected: target connect to 172.16.4.145:3260 failed (Connection refused)

However output error (03/01) in/var/log/message on Compute when I start instance.

Dec 16 10:53:11 localhost kernel: connection10:0: detected conn error (1020)
Dec 16 10:53:12 localhost iscsid: conn 0 login rejected: target error (03/01)
Dec 16 10:53:14 localhost kernel: connection10:0: detected conn error (1020)

Somehow Compute connected to any volumes on Cinder.

And that 14:16:43 dev1128 journal: Failed to open file '/dev/disk/by-path/ip-172.16.4.145:3260-iscsi-iqn.2010-10.org.openstack:volume-f46978a5-0515-48ad-b69f-89f489842ce6-lun-0': No such device or address

That connection couldn't could log out.out, but not be able to log in.

$ iscsiadm -m session
tcp: [10] [2] 172.16.4.145:3260,1 iqn.2010-10.org.openstack:volume-8e604962-49d7-4895-919f-1864f3e54af5 iqn.2010-10.org.openstack:volume-49874b58-c8fd-4614-a39c-34d53d4fe750 (non-flash)
tcp: [11] [3] 172.16.4.145:3260,1 iqn.2010-10.org.openstack:volume-c6bf3ff1-fcee-449d-a578-5f660f10c6b7 iqn.2010-10.org.openstack:volume-f46978a5-0515-48ad-b69f-89f489842ce6 (non-flash)

$ iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e604962-49d7-4895-919f-1864f3e54af5 iqn.2010-10.org.openstack:volume-49874b58-c8fd-4614-a39c-34d53d4fe750 -u
Logging out of session [sid: 10, 2, target: iqn.2010-10.org.openstack:volume-8e604962-49d7-4895-919f-1864f3e54af5, iqn.2010-10.org.openstack:volume-49874b58-c8fd-4614-a39c-34d53d4fe750, portal: 172.16.4.145,3260]
iscsiadm: Could not logout Logout of [sid: 10, 2, target: iqn.2010-10.org.openstack:volume-8e604962-49d7-4895-919f-1864f3e54af5, iqn.2010-10.org.openstack:volume-49874b58-c8fd-4614-a39c-34d53d4fe750, portal: 172.16.4.145,3260].
iscsiadm: initiator reported error (9 - internal error)
iscsiadm: Could not logout of all requested sessions
172.16.4.145,3260] successful.

$ iscsiadm -m node -T iqn.2010-10.org.openstack:volume-49874b58-c8fd-4614-a39c-34d53d4fe750 -l
Logging in to [iface: default, target: iqn.2010-10.org.openstack:volume-49874b58-c8fd-4614-a39c-34d53d4fe750, portal: 172.16.4.145,3260] (multiple)
.
.
.
I press "Ctrl + C" to escape the program because couldn't finish.

Iface settings is clear.

[Compute]
$ iscsiadm -m session -P 3
        Iface IPaddress: <empty>

Do you have any ideas?

Regard.

Cannot work any instance aftrer target.service restart on Cinder

Dear all Hi.

I'm building Juno system.

Some instances was working.

However they couldn't work ever since then I restarted target.service on Cinder.

And that doesn't happen rare occurrence.(I wonder that...)

Log messages on Compute at that time are as follows.

after target.service restart
[Compute]
$ tail -f /var/log/message
Dec 16 14:15:24 localhost iscsid: connect to 172.16.4.145:3260 failed (Connection refused)
Dec 16 14:15:28 localhost iscsid: connect to 172.16.4.145:3260 failed (Connection refused)

However output error in/var/log/message on Compute when I start instance.

Dec 16 14:16:43 dev1128 journal: Failed to open file '/dev/disk/by-path/ip-172.16.4.145:3260-iscsi-iqn.2010-10.org.openstack:volume-f46978a5-0515-48ad-b69f-89f489842ce6-lun-0': No such device or address

That connection could log out, but not be able to log in.

$ iscsiadm -m session
tcp: [2] 172.16.4.145:3260,1 iqn.2010-10.org.openstack:volume-49874b58-c8fd-4614-a39c-34d53d4fe750 (non-flash)
tcp: [3] 172.16.4.145:3260,1 iqn.2010-10.org.openstack:volume-f46978a5-0515-48ad-b69f-89f489842ce6 (non-flash)

$ iscsiadm -m node -T iqn.2010-10.org.openstack:volume-49874b58-c8fd-4614-a39c-34d53d4fe750 -u
Logging out of session [sid: 2, target: iqn.2010-10.org.openstack:volume-49874b58-c8fd-4614-a39c-34d53d4fe750, portal: 172.16.4.145,3260]
Logout of [sid: 2, target: iqn.2010-10.org.openstack:volume-49874b58-c8fd-4614-a39c-34d53d4fe750, portal: 172.16.4.145,3260] successful.

$ iscsiadm -m node -T iqn.2010-10.org.openstack:volume-49874b58-c8fd-4614-a39c-34d53d4fe750 -l
Logging in to [iface: default, target: iqn.2010-10.org.openstack:volume-49874b58-c8fd-4614-a39c-34d53d4fe750, portal: 172.16.4.145,3260] (multiple)
.
.
.
I press "Ctrl + C" to escape the program because couldn't finish.

Iface settings is clear.

[Compute]
$ iscsiadm -m session -P 3
        Iface IPaddress: <empty>
[Cinder]
$ targetcli ls
o- / ......................................................................................................... [...]
  o- backstores .............................................................................................. [...]
  | o- block .................................................................................. [Storage Objects: 0]
  | o- fileio ................................................................................. [Storage Objects: 0]
  | o- pscsi .................................................................................. [Storage Objects: 0]
  | o- ramdisk ................................................................................ [Storage Objects: 0]
  o- iscsi ............................................................................................ [Targets: 0]
  o- loopback ......................................................................................... [Targets: 0

Do you have any ideas?

Regard.

Cannot work any instance aftrer target.service restart on Cinder

Dear all Hi.

I'm building Juno system.

Some instances was working.

However they couldn't work ever since then I restarted target.service on Cinder.

And that doesn't happen rare occurrence.(I wonder that...)

Log messages on Compute at that time are as follows.

after target.service restart
[Compute]
$ tail -f /var/log/message
Dec 16 14:15:24 localhost iscsid: connect to 172.16.4.145:3260 failed (Connection refused)
Dec 16 14:15:28 localhost iscsid: connect to 172.16.4.145:3260 failed (Connection refused)

However output error in/var/log/message on Compute when I start instance.

Dec 16 14:16:43 dev1128 journal: Failed to open file '/dev/disk/by-path/ip-172.16.4.145:3260-iscsi-iqn.2010-10.org.openstack:volume-f46978a5-0515-48ad-b69f-89f489842ce6-lun-0': No such device or address

That connection could log out, but not be able to log in.

$ iscsiadm -m session
tcp: [2] 172.16.4.145:3260,1 iqn.2010-10.org.openstack:volume-49874b58-c8fd-4614-a39c-34d53d4fe750 (non-flash)
tcp: [3] 172.16.4.145:3260,1 iqn.2010-10.org.openstack:volume-f46978a5-0515-48ad-b69f-89f489842ce6 (non-flash)

$ iscsiadm -m node -T iqn.2010-10.org.openstack:volume-49874b58-c8fd-4614-a39c-34d53d4fe750 -u
Logging out of session [sid: 2, target: iqn.2010-10.org.openstack:volume-49874b58-c8fd-4614-a39c-34d53d4fe750, portal: 172.16.4.145,3260]
Logout of [sid: 2, target: iqn.2010-10.org.openstack:volume-49874b58-c8fd-4614-a39c-34d53d4fe750, portal: 172.16.4.145,3260] successful.

$ iscsiadm -m node -T iqn.2010-10.org.openstack:volume-49874b58-c8fd-4614-a39c-34d53d4fe750 -l
Logging in to [iface: default, target: iqn.2010-10.org.openstack:volume-49874b58-c8fd-4614-a39c-34d53d4fe750, portal: 172.16.4.145,3260] (multiple)
.
.
.
I press "Ctrl + C" to escape the program because couldn't finish.

Iface settings is clear.

[Compute]
$ iscsiadm -m session -P 3
        Iface IPaddress: <empty>

$ less /etc/iscsi/initiatorname.iscsi
InitiatorName=iqn.1994-05.com.redhat:68da7f5ad5
[Cinder]
$ targetcli ls
o- / ......................................................................................................... [...]
  o- backstores .............................................................................................. [...]
  | o- block .................................................................................. [Storage Objects: 0]
  | o- fileio ................................................................................. [Storage Objects: 0]
  | o- pscsi .................................................................................. [Storage Objects: 0]
  | o- ramdisk ................................................................................ [Storage Objects: 0]
  o- iscsi ............................................................................................ [Targets: 0]
  o- loopback ......................................................................................... [Targets: 0

Do you have any ideas?

Regard.