swift Account HEAD 503 Internal Server Error
Hallo!
Can't verify swift stat:
[root@swift-proxy01 swift]# swift -V 2.0 -A http://192.168.1.29:5000/v2.0 -U admin:admin -K 12345As stat
Account HEAD failed: http://192.168.1.30/v1/AUTH_18c35fb509b84966aa7e5734eebdf8fb 503 Internal Server Error
In proxy log I see:
May 7 13:04:55 swift-proxy01 proxy-server: Account HEAD returning 503 for [] (txn: tx5fedde22118b469b8e753-005369f737) (client_ip: 192.168.1.30)
May 7 13:04:59 swift-proxy01 proxy-server: ERROR with Account server 10.10.2.35:6002/sdb1 re: Trying to HEAD /v1/AUTH_18c35fb509b84966aa7e5734eebdf8fb: Connection refused (txn: tx3faaa6270e414611895e2-005369f73b) (client_ip : 192.168.1.30)
May 7 13:04:59 swift-proxy01 proxy-server: ERROR with Account server 10.10.2.34:6002/sdb1 re: Trying to HEAD /v1/AUTH_18c35fb509b84966aa7e5734eebdf8fb: Connection refused (txn: tx3faaa6270e414611895e2-005369f73b) (client_ip : 192.168.1.30)
May 7 13:04:59 swift-proxy01 proxy-server: ERROR with Account server 10.10.2.31:6002/sdb1 re: Trying to HEAD /v1/AUTH_18c35fb509b84966aa7e5734eebdf8fb: Connection refused (txn: tx3faaa6270e414611895e2-005369f73b) (client_ip : 192.168.1.30)
May 7 13:04:59 swift-proxy01 proxy-server: Handoff requested (1) (txn: tx3faaa6270e414611895e2-005369f73b) (client_ip: 192.168.1.30)
May 7 13:04:59 swift-proxy01 proxy-server: ERROR with Account server 10.10.2.32:6002/sdb1 re: Trying to HEAD /v1/AUTH_18c35fb509b84966aa7e5734eebdf8fb: Host unreachable (txn: tx3faaa6270e414611895e2-005369f73b) (client_ip: 192.168.1.30)
May 7 13:04:59 swift-proxy01 proxy-server: Handoff requested (2) (txn: tx3faaa6270e414611895e2-005369f73b) (client_ip: 192.168.1.30)
May 7 13:04:59 swift-proxy01 proxy-server: ERROR with Account server 10.10.2.33:6002/sdb1 re: Trying to HEAD /v1/AUTH_18c35fb509b84966aa7e5734eebdf8fb: Connection refused (txn: tx3faaa6270e414611895e2-005369f73b) (client_ip : 192.168.1.30)
May 7 13:04:59 swift-proxy01 proxy-server: Account HEAD returning 503 for [] (txn: tx3faaa6270e414611895e2-005369f73b) (client_ip: 192.168.1.30)
The storage nodes are pingable, iptables was shuted down. Ring contents:
[root@swift-proxy01 swift]# swift-ring-builder account.builder
account.builder, build version 5
262144 partitions, 3.000000 replicas, 1 regions, 5 zones, 5 devices, 0.00 balance
The minimum number of hours before a partition can be reassigned is 1
Devices: id region zone ip address port replication ip replication port name weight partitions balance meta
0 1 1 10.10.2.31 6002 10.10.2.31 6002 sdb1 100.00 157287 0.00
1 1 2 10.10.2.32 6002 10.10.2.32 6002 sdb1 100.00 157286 -0.00
2 1 3 10.10.2.33 6002 10.10.2.33 6002 sdb1 100.00 157286 -0.00
3 1 4 10.10.2.34 6002 10.10.2.34 6002 sdb1 100.00 157286 -0.00
4 1 5 10.10.2.35 6002 10.10.2.35 6002 sdb1 100.00 157287 0.00
[root@swift-proxy01 swift]#
[root@swift-proxy01 swift]# swift-ring-builder container.builder
container.builder, build version 5
262144 partitions, 3.000000 replicas, 1 regions, 5 zones, 5 devices, 0.00 balance
The minimum number of hours before a partition can be reassigned is 1
Devices: id region zone ip address port replication ip replication port name weight partitions balance meta
0 1 1 10 ...
Could it be the same problem as stated here? They resolved setting "mount_check = false" for account, container and object servers.
Here the config reference
I have added option "mount_check = false" on all nodes and on proxy server in account, container and object config files, then I restarted all services, but there is no any effect. Still have the problem.
Are swift-account services all up and running on storage nodes? Guide and troubleshooting swift guide
Yes, all services are started. But i think i found something. I've tried a couple of commands:
Then I've tried to change bind_ip in account-server.conf on first storage node from 127.0.0.1 to 10.10.2.31. And it looks like it worked:
What about bind_ip option in account, container and object config files on storage nodes and proxy server nodes? Where exactly I ...(more)
I was experiencing the same problem and your explanation saved me. Were you able to learn anymore as to why this needs to be done?