Ask Your Question
2

swift Account HEAD 503 Internal Server Error

asked 2014-05-07 04:15:10 -0500

Sapos gravatar image

Hallo!

Can't verify swift stat:

  [root@swift-proxy01 swift]# swift -V 2.0 -A http://192.168.1.29:5000/v2.0 -U admin:admin -K 12345As stat
    Account HEAD failed: http://192.168.1.30/v1/AUTH_18c35fb509b84966aa7e5734eebdf8fb 503 Internal Server Error

In proxy log I see:

May  7 13:04:55 swift-proxy01 proxy-server: Account HEAD returning 503 for [] (txn: tx5fedde22118b469b8e753-005369f737) (client_ip: 192.168.1.30)
May  7 13:04:59 swift-proxy01 proxy-server: ERROR with Account server 10.10.2.35:6002/sdb1 re: Trying to HEAD /v1/AUTH_18c35fb509b84966aa7e5734eebdf8fb: Connection refused (txn: tx3faaa6270e414611895e2-005369f73b) (client_ip             : 192.168.1.30)
May  7 13:04:59 swift-proxy01 proxy-server: ERROR with Account server 10.10.2.34:6002/sdb1 re: Trying to HEAD /v1/AUTH_18c35fb509b84966aa7e5734eebdf8fb: Connection refused (txn: tx3faaa6270e414611895e2-005369f73b) (client_ip             : 192.168.1.30)
May  7 13:04:59 swift-proxy01 proxy-server: ERROR with Account server 10.10.2.31:6002/sdb1 re: Trying to HEAD /v1/AUTH_18c35fb509b84966aa7e5734eebdf8fb: Connection refused (txn: tx3faaa6270e414611895e2-005369f73b) (client_ip             : 192.168.1.30)
May  7 13:04:59 swift-proxy01 proxy-server: Handoff requested (1) (txn: tx3faaa6270e414611895e2-005369f73b) (client_ip: 192.168.1.30)
May  7 13:04:59 swift-proxy01 proxy-server: ERROR with Account server 10.10.2.32:6002/sdb1 re: Trying to HEAD /v1/AUTH_18c35fb509b84966aa7e5734eebdf8fb: Host unreachable (txn: tx3faaa6270e414611895e2-005369f73b) (client_ip:              192.168.1.30)
May  7 13:04:59 swift-proxy01 proxy-server: Handoff requested (2) (txn: tx3faaa6270e414611895e2-005369f73b) (client_ip: 192.168.1.30)
May  7 13:04:59 swift-proxy01 proxy-server: ERROR with Account server 10.10.2.33:6002/sdb1 re: Trying to HEAD /v1/AUTH_18c35fb509b84966aa7e5734eebdf8fb: Connection refused (txn: tx3faaa6270e414611895e2-005369f73b) (client_ip             : 192.168.1.30)
May  7 13:04:59 swift-proxy01 proxy-server: Account HEAD returning 503 for [] (txn: tx3faaa6270e414611895e2-005369f73b) (client_ip: 192.168.1.30)

The storage nodes are pingable, iptables was shuted down. Ring contents:

[root@swift-proxy01 swift]# swift-ring-builder account.builder
account.builder, build version 5
262144 partitions, 3.000000 replicas, 1 regions, 5 zones, 5 devices, 0.00 balance
The minimum number of hours before a partition can be reassigned is 1
Devices:    id  region  zone      ip address  port  replication ip  replication port      name weight partitions balance meta
             0       1     1      10.10.2.31  6002      10.10.2.31              6002      sdb1 100.00     157287    0.00
             1       1     2      10.10.2.32  6002      10.10.2.32              6002      sdb1 100.00     157286   -0.00
             2       1     3      10.10.2.33  6002      10.10.2.33              6002      sdb1 100.00     157286   -0.00
             3       1     4      10.10.2.34  6002      10.10.2.34              6002      sdb1 100.00     157286   -0.00
             4       1     5      10.10.2.35  6002      10.10.2.35              6002      sdb1 100.00     157287    0.00
[root@swift-proxy01 swift]#
[root@swift-proxy01 swift]# swift-ring-builder container.builder
container.builder, build version 5
262144 partitions, 3.000000 replicas, 1 regions, 5 zones, 5 devices, 0.00 balance
The minimum number of hours before a partition can be reassigned is 1
Devices:    id  region  zone      ip address  port  replication ip  replication port      name weight partitions balance meta
             0       1     1      10 ...
(more)
edit retag flag offensive close merge delete

Comments

Could it be the same problem as stated here? They resolved setting "mount_check = false" for account, container and object servers.

Here the config reference

Antonio G. gravatar imageAntonio G. ( 2014-05-07 04:36:49 -0500 )edit

I have added option "mount_check = false" on all nodes and on proxy server in account, container and object config files, then I restarted all services, but there is no any effect. Still have the problem.

Sapos gravatar imageSapos ( 2014-05-07 06:53:49 -0500 )edit

Are swift-account services all up and running on storage nodes? Guide and troubleshooting swift guide

Antonio G. gravatar imageAntonio G. ( 2014-05-07 07:09:17 -0500 )edit

Yes, all services are started. But i think i found something. I've tried a couple of commands:

 curl http://10.10.2.31:6002
curl: (7) couldn't connect to host
[root@swift-proxy01 swift]# telnet 10.10.2.31 6002
Trying 10.10.2.31...
telnet: connect to address 10.10.2.31: Connection refused

Then I've tried to change bind_ip in account-server.conf on first storage node from 127.0.0.1 to 10.10.2.31. And it looks like it worked:

[root@swift-proxy01 swift]# swift -V 2.0 -A http://192.168.1.29:5000/v2.0 -U admin:admin -K 12345As stat
       Account: AUTH_18c35fb509b84966aa7e5734eebdf8fb
    Containers: 0
       Objects: 0
         Bytes: 0
  Content-Type: text/plain; charset=utf-8
   X-Timestamp: 1399464488.50562
    X-Trans-Id: txf9dde858cb62485ca042e-00536a2228
X-Put-Timestamp: 1399464488.50562

What about bind_ip option in account, container and object config files on storage nodes and proxy server nodes? Where exactly I ...(more)

Sapos gravatar imageSapos ( 2014-05-07 07:19:59 -0500 )edit

I was experiencing the same problem and your explanation saved me. Were you able to learn anymore as to why this needs to be done?

loopback127 gravatar imageloopback127 ( 2014-06-26 01:24:36 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
0

answered 2015-06-09 08:05:46 -0500

AntonioA gravatar image

I suggest you to check the selinux and firewalld service on object storage node. First test could be to try to disable both service. I have resolved the "Internal Server Error/Connection Refused" adding a rule to accept the port 6000/6001/6002.

Regards, Antonio

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2014-05-07 04:15:10 -0500

Seen: 5,977 times

Last updated: Jun 09 '15