Perform a GET and get http/ 1.1 503 Internal Server Error

asked 2013-07-10 18:01:43 -0500

jlu gravatar image

Hello,

I just setup swift env in my lab for testing. When i perform <curl -v="" -h="" 'x-auth-token:="" auth_tkadaffa35a3b4458f96a95c4377c3e496'="" http:="" 127.0.0.1:8080="" v1="" auth_admin=""> and I am getting http/1.1 503 Internal Server Error. Can someone please help me with this?

[root@cleverloadgen11 swift]# curl -v -H 'X-Auth-Token: AUTH_tkadaffa35a3b4458f96a95c4377c3e496' http://127.0.0.1:8080/v1/AUTH_admin * About to connect() to 127.0.0.1 port 8080 (#0) * Trying 127.0.0.1... connected * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)

GET /v1/AUTH_admin HTTP/1.1 User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2 Host: 127.0.0.1:8080 Accept: / X-Auth-Token: AUTH_tkadaffa35a3b4458f96a95c4377c3e496

< HTTP/1.1 503 Internal Server Error < Content-Type: text/html; charset=UTF-8 < Content-Length: 0 < Date: Wed, 10 Jul 2013 17:59:03 GMT < * Connection #0 to host 127.0.0.1 left intact * Closing connection #0 [root@cleverloadgen11 swift]#

Thank, Jimmy

edit retag flag offensive close merge delete

5 answers

Sort by ยป oldest newest most voted
0

answered 2013-07-11 03:28:53 -0500

There may be some config error during deployment, but bugs

edit flag offensive delete link more
0

answered 2013-07-11 06:44:10 -0500

jlu gravatar image

I rebuilt the rings to the correct mount points and restart all services. When perform <curl -v="" -h="" 'x-auth-token:="" auth_tk95a0d54e054e40a28f2386bba26c76bd'="" http:="" 10.218.36.74:8080="" v1="" auth_admin=""> and I am getting "Bad request with 400" error. The 503 error is no longer there. I am confused..:(

Thank you much!!

-Jimmy

<log from="" 1="" of="" the="" storage="" node="">

Jul 10 23:28:54 cleverloadgen16 object-expirer Unhandled exception: #012Traceback (most recent call last):#012 File "/usr/lib/python2.6/site-packages/swift/obj/expirer.py", line 91, in run_once#012 self.swift.get_account_info(self.expiring_objects_account)#012 File "/usr/lib/python2.6/site-packages/swift/common/internal_client.py", line 336, in get_account_info#012 resp = self.make_request('HEAD', path, {}, acceptable_statuses)#012 File "/usr/lib/python2.6/site-packages/swift/common/internal_client.py", line 162, in make_request#012 _('Unexpected response: %s' % (resp.status,)), resp)#012UnexpectedResponse: (UnexpectedResponse(...), 'Unexpected response: 400 Bad Request') (txn: tx57c71b791e794a31bcda7b3d462f9f55)

Jul 10 23:37:36 cleverloadgen15 account-replicator Exception dumping recon cache: #012Traceback (most recent call last):#012 File "/usr/lib/python2.6/site-packages/swift/common/utils.py", line 1284, in dump_recon_cache#012 with lock_file(cache_file, lock_timeout, unlink=False) as cf:#012 File "/usr/lib64/python2.6/contextlib.py", line 16, in __enter__#012 return self.gen.next()#012 File "/usr/lib/python2.6/site-packages/swift/common/utils.py", line 852, in lock_file#012 fd = os.open(filename, flags)#012OSError: [Errno 2] No such file or directory: '/var/cache/swift/account.recon' Jul 10 23:37:36 cleverloadgen15 account-replicator no_change:0 ts_repl:0 diff:0 rsync:0 diff_capped:0 hashmatch:0 empty:0 Jul 10 23:37:38 cleverloadgen15 object-server 10.218.36.76 - - [11/Jul/2013:06:37:38 +0000] "HEAD /sdb/231937/.expiring_objects" 400 43 "-" "tx2c7ac77aa8884fe0afc673851e616d41" "Swift Object Expirer" 0.0002 Jul 10 23:37:42 cleverloadgen15 object-server 10.218.36.76 - - [11/Jul/2013:06:37:42 +0000] "HEAD /sdb/231937/.expiring_objects" 400 43 "-" "tx6f12758f0342476eb9dfab996f9d278e" "Swift Object Expirer" 0.0002 Jul 10 23:37:51 cleverloadgen15 object-server 10.218.36.78 - - [11/Jul/2013:06:37:51 +0000] "HEAD /sdb/231937/.expiring_objects" 400 43 "-" "txee1462eb0ef44460bdb91c946fb3d4d7" "Swift Object Expirer" 0.0002

edit flag offensive delete link more
0

answered 2013-07-11 18:02:15 -0500

torgomatic gravatar image

The answer is in the stack trace you posted.

Quoting: "OSError: [Errno 2] No such file or directory: '/var/cache/swift/account.recon'"

Please make sure /var/cache/swift exists and is writable by the user that the Swift processes run as (probably "swift", but YMMV).

edit flag offensive delete link more
0

answered 2013-07-11 20:51:19 -0500

jlu gravatar image

Hi Samuel,

I had already created the file and changed the ownership to swift:swift and I don't see that error in the log.

This is my observation with the 400 bad request.

I have 5 storage nodes and 1 proxy node:

When I perform <curl -v="" -h="" "x-auth-token:="" auth_tk95a0d54e054e40a28f2386bba26c76bd"="" http:="" 10.218.36.74:8080="" v1="" auth_admin="">, I get a "400 Bad Request" in return.

What I noticed, on node3,5,6, i see these errors

node3: Jul 11 13:44:15 cleverloadgen13 object-server 10.218.36.74 - - [11/Jul/2013:20:44:15 +0000] "GET /sdb/260881/AUTH_admin" 400 36 "-" "tx49b006fb16554e818d29e96ecaef4c53" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" 0.0002

node5: Jul 11 13:44:15 cleverloadgen15 object-server 10.218.36.74 - - [11/Jul/2013:20:44:15 +0000] "GET /sdb/260881/AUTH_admin" 400 36 "-" "tx49b006fb16554e818d29e96ecaef4c53" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" 0.0002

node6: Jul 11 13:44:15 cleverloadgen16 object-server 10.218.36.74 - - [11/Jul/2013:20:44:15 +0000] "GET /sdb/260881/AUTH_admin" 400 36 "-" "tx49b006fb16554e818d29e96ecaef4c53" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" 0.0002

and node2 and node4 - i do not see such errors.

Thanks and appreciated for the help!!

-Jimmy

edit flag offensive delete link more
0

answered 2013-07-18 20:55:58 -0500

jlu gravatar image

This issue is resolved. It was due to port mismatch when building the ring and in the rsyncd.conf for *-server.conf.

Thanks, Jimmy

edit flag offensive delete link more

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2013-07-10 18:01:43 -0500

Seen: 22,509 times

Last updated: Jul 18 '13