Ask Your Question
0

The proxy node in swift

asked 2013-08-05 04:31:47 -0500

root gravatar image

updated 2013-08-07 11:05:10 -0500

smaffulli gravatar image

Hi developer, I have went through all the swift 1.8.1 code . I find the proxy node take to much heavy pressures. For example, assume there are 3 replicas for each partition, when upload an object, the client will first upload the object to proxy node , and then the proxy node will transfer the object to the three device due to 3 replicas. It means, the proxy node will upload the object to three device one by one. I think it will lead to heavy traffic for the proxy node. I think the more reasonable approach is the proxy node successfully upload the object to one device. And then the device synchronize the object to the left 2 device.

edit retag flag offensive close merge delete

Comments

I suggest you to ask this question to the English version of the site on https://ask.openstack.org/en/questions. There is a dropdown menu to switch language on the top right of this page.

smaffulli gravatar imagesmaffulli ( 2013-08-06 18:26:44 -0500 )edit

Thank you very much

root gravatar imageroot ( 2013-08-07 21:07:50 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
1

answered 2013-08-08 00:49:16 -0500

clayg gravatar image

Most deployments have more throughput between the proxy and object tier than between their proxy/load-balancing core's and any one client; so the amplification on PUT is not a limiting factor during upload.

Regardless, the consequences of loosing an object between a "successful" upload and an asynchronous offload is too dangerous. The proxy will even go so far as to "hand off" to secondary nodes if one of the primary locations is not responsive in order to ensure full durability immediately upon response to the client.

I should clarify if it wasn't already obvious that the proxy writes all three copies simultaneously - not "one by one". If any single connection drops out mid-stream the remaining two can continue to service the request successfully.

Other storage systems with different goals have taken the approach you've described - DRDB for example has an asynchronous mirroring mode. Recently Swift added support for write-affinity specifically for handling the case of geographically distributed clusters - but you may be able to leverage it to achieve a similar effect if you have use case that requires it:

http://docs.openstack.org/developer/swift/admin_guide.html#write-affinity-and-write-affinity-node-count

edit flag offensive delete link more

Comments

Thanks for your reply. I misunderstood before. I considered that the eventlet library executed the greenthread serially not parallelly. So I thought the proxy would upload the object to object server one by one. Thank you very much.

root gravatar imageroot ( 2013-08-10 06:00:17 -0500 )edit

Get to know Ask OpenStack

Resources for moderators

Question Tools

Stats

Asked: 2013-08-05 04:31:47 -0500

Seen: 220 times

Last updated: Aug 08 '13