SWIFT storage http error using JOSS API

asked 2014-05-13 14:50:28 -0500

raja gravatar image

updated 2014-05-13 15:21:05 -0500

We have a default Swift set up with a proxy and a total of 3 replica nodes.All parameters/set up is default(as far as I have checked proxy-server.conf and object-server.conf).We hit the proxy using JOSS API to do all the CRUD operations on file objects stored on the swift side. Auth method is basic while connecting to Swift. Though it doesn't matter what the code but still it is as simple as :

// where jossAccount is already bind to this class Container container = jossAccount.getContainer("container_name"); StoredObject object = container.getObject("name of the local object"); File uploadFile=new File("path to local object");
object.uploadObject(uploadFile);

There is a job/process(a small application on tomcat 8) which runs which asyncronously(though the concurrency would be quite low as 2-5 or in some cases linear) inserting/uploading files to the swift storage.After a while this process starts running I start to see all sorts of Http errors (503,404,400 etc) with the result that a lot of files fail to get uploaded.As a general observation I have seen this happening after the CONTAINER crosses 60-70 k objects/files. I have tried to put a re-trial logic around the above code if it fails which succeeded in achieving a higher through put but still a lot fails/kind of uncertain. I tried to check out the syslog on the proxy side but the only doubtful extract that I can find is "Trying to get final status of PUT to /v1/AUTH_system/528324b6aew/528356c8asdsds: Timeout (10s)" which is also something I tried to search for but nothing concrete I could find.Just in case : The Swift proxy is sitting on ubuntu 12.04 with WebOb version of 1.1.1

NOTE: The https errors are quite random but has nothing to do with the actual use case because I have checked for all the relevant details whenever such error has occurred.I am using pseudo folders inside containers to store objects

edit retag flag offensive close merge delete