Ask Your Question
0

About swift architecture

asked 2010-11-22 05:54:49 -0600

hz02ruc gravatar image

the Multi-server swift installation documentation says proxy is in an external network, while the storage nodes are not accessible from outside the cluster.

The question is : when uploading objects to the cluster, how are the real data transferred? the data routine is user(client) -> proxy node -> storage node ? (regardless of request itself, just data here)

If so, when there's a lot of requests for reading/writing, the proxy(network) should be a bottleneck, although we can add other proxy nodes.

edit retag flag offensive close merge delete

7 answers

Sort by ยป oldest newest most voted
0

answered 2010-11-23 02:43:30 -0600

hz02ruc gravatar image

Thanks John Dickinson, that solved my question.

edit flag offensive delete link more
0

answered 2010-11-23 02:08:06 -0600

notmyname gravatar image

Swift requires that 2 storage nodes successfully write the object before returning success (in the default config where there are 3 replicas). The third copy will be attempted, but if it fails, replication will handle creating it on the third object server. The second copy is essential, because a single hardware failure immediately after a successful write to a single object server may cause the data to be permanently lost.

To increase user performance (reduce write latency), the three writes to storage nodes happen concurrently. This means that the internal network ports on the proxy servers have an effective throughput of one-third their listed rate. In Rackspace's case, this means that if a user were able to get a sustained 3Gbps to the proxy servers (the load balancers, actually), this would be their network cap. In reality, since each object write is written to one spindle, the actual limiting factor is hard drive spindle speed.

If you were attempting to maximize large write speeds to swift (and cost is no factor), first get really fast disk IO, then fast networks to the storage nodes, then fast public access to the proxy servers (ensuring that the pubic speed is at least 3x the storage network). Finally, start increasing processors and memory on the proxy and storage nodes. For Rackspace, these optimizations would do very little for the average use case, and provide almost no benefit even for the exceptional use case. As always, your use case could be different, so your milage may vary.

edit flag offensive delete link more
0

answered 2010-11-23 01:47:44 -0600

hz02ruc gravatar image

Hi John,

I thought each external request would write data to only one storage node, then this storage node rsync the data to another two storage nodes before. Some distributed file systems handle the situation in this way. Could you please tell me is there any benefit for the concurrent write(from proxy to 3 storage node)?

edit flag offensive delete link more
0

answered 2010-11-22 15:31:41 -0600

cthier gravatar image

Hi Salvatore,

While the requests do stream through the proxy, the proxies are not really a bottleneck, as they scale horizontally like the rest of the system. Some of your questions about using more than one proxy node can be found with this question: https://answers.launchpad.net/swift/+question/134585 (https://answers.launchpad.net/swift/+...) .

The built in auth for swift is meant for development only, and not for a production system.

For the ring, we recommend administrating the ring on one server and then distributing the ring files out to all of the nodes (proxy and storage).

Since this question has come up more than once now, I'm going to create a bug report to add more clarification on how to add more proxies.

edit flag offensive delete link more
0

answered 2010-11-22 14:09:33 -0600

spiccolo gravatar image

Hi John, when you have more than one proxy and you add a storage node, you have to add it on each proxy with these commands

swift-ring-builder account.builder add z<zone>-<storage_local_net_ip>:6002/<device> 100 swift-ring-builder container.builder add z<zone>-<storage_local_net_ip_1>:6001/<device> 100 swift-ring-builder object.builder add z<zone>-<storage_local_net_ip_1>:6000/<device> 100

is it right?

And how do I manage account.ring.gz, container.ring.gz, and object.ring.gz? Which one I've to copy on storage node?

edit flag offensive delete link more
0

answered 2010-11-22 13:20:54 -0600

notmyname gravatar image

You are correct that the data flow is through the proxy nodes. At Rackspace, we have the proxy nodes connected to the public network via 10G connections and the storage nodes (internal network) connected via 1G. We have many more storage nodes than proxy nodes, and we aren't network limited by the proxies. Note that each external request to the proxy servers will turn in to 3 requests on the storage node network (three replicated writes that attempt to happen concurrently).

The way to configure two or more proxies is to have one external VIP that load balances across all of the proxy nodes.

The data flow is (client) -> (load balancer) -> proxy -> (3x storage node).

edit flag offensive delete link more
0

answered 2010-11-22 08:59:57 -0600

spiccolo gravatar image

And there is another question: how to configure corretly two or more proxy-node? Remember that you can have only one auth-node ... and this is anothe bottleneck when there are hardware failure ...

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

2 followers

Stats

Asked: 2010-11-22 05:54:49 -0600

Seen: 88 times

Last updated: Nov 23 '10