PUT cost a lot of time
When I put a large number of 4K files, the performance is terrible.
My testing configuration: [bench] concurrency = 200 object_size = 4096 num_objects = 200000 num_containers = 200
Now I trace the code of PUT operation and find some problems. I seem looks like that three step will cost a lot of time in PUT operation.
in function "ObjectController::PUT":
for chunk in iter(lambda: reader(self.network_chunk_size), ''): upload_size += len(chunk) if time.time() > upload_expiration: self.logger.increment('PUT.timeouts') return HTTPRequestTimeout(request=request) etag.update(chunk) while chunk: written = os.write(fd, chunk) chunk = chunk[written:] sleep()
per lambda:reader will cost 600ms when I put 4K files. And also eventlet.sleep() will cost 400ms. At last, fsync() and async_update() will const 400ms.
Is there anyone who had faced this problem? How to fix this issue?
