What causes Swift performance to degrade when handling PUT requests?

asked 2013-07-16 02:26:49 -0600

kalrey gravatar image

updated 2013-07-16 18:39:43 -0600

smaffulli gravatar image

Hi,guys. I'm facing the problem of performance degradation of swift. I saw some information about swift while handling PUT request. The speed of PUT can reach to 1000/s. But it just reach to 200/s in my cluster of swift. I don't know how to figure out the problem I faced.

There are some information about my testing cluster of swift:

Test environment:
Ubuntu 12.04
1 proxy-node : 128GB-ram / CPU 16core / 1Gb NIC*1
5 Storage-nodes : each for 128GB-ram / CPU 16core / 2TB*4 / 1Gb NIC*1.

The ratio of storge-servers(account, container, object server) and disk is 1 : 1. At one storage node, it have four instance of account server, container server and object server.

My testing configuration:

concurrency = 200
object_size = 4096
num_objects = 200000
num_containers = 200
swift-bench 2013-07-10 18:15:50,339 INFO 100095 PUTS [0 failures], 138.4/s
swift-bench 2013-07-10 18:16:05,370 INFO 102283 PUTS [0 failures], 138.5/s
swift-bench 2013-07-10 18:16:20,379 INFO 104117 PUTS [0 failures], 138.2/s
swift-bench 2013-07-10 18:16:35,388 INFO 106291 PUTS [0 failures], 138.3/s
swift-bench 2013-07-10 18:16:50,418 INFO 108033 PUTS [0 failures], 137.9/s

Please give some suggestion what I can do to figure out the problem.

edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted

answered 2013-07-18 16:04:51 -0600

clayg gravatar image

What is your GET request rate on this setup?

There's a request amplification from the proxy to each of the 3 replica servers. But it seems like you should be able to get better.

I'd suggest tweaking the threadpool and worker settings.


edit flag offensive delete link more

answered 2013-07-26 01:42:23 -0600

wmeler gravatar image

updated 2013-07-26 01:43:37 -0600

You have 4*5 = 20 SATA disks, each capable of 100 IO/s, so your cluster can do no more than 2000 IO/s. One swift PUT probably have to authenticate, write object to disk, modify container and probably more(check/update quotas?), and all modifications have to be done on 3 replicas... So 2000/3=666, and 666/ your result 138 is something about 5 IO per PUT on single replica - maybe it is not so good, but I'm sure you can't go below 2 IO per PUT on swift.

edit flag offensive delete link more

Get to know Ask OpenStack

Resources for moderators

Question Tools


Asked: 2013-07-16 02:26:49 -0600

Seen: 505 times

Last updated: Jul 26 '13