Dedicated replication network for swift

asked 2013-09-11 12:52:44 -0500

ebyenjoys gravatar image

updated 2013-09-11 12:57:37 -0500

helloo, I did read about Dedicated replication network from http://docs.openstack.org/developer/swift/replication_network.html

can anyone explian me how it works and what is the configuration for replicaion network needed?

edit retag flag offensive close merge delete

Comments

I would like to know about this as well. :)

serverascode gravatar imageserverascode ( 2013-09-16 15:06:20 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
2

answered 2013-09-17 01:59:10 -0500

clayg gravatar image

If you want to separate your replication traffic from your backend request traffic you'll need an isolated network (different ip) on each storage node.

I think http://docs.openstack.org/developer/swift/replication_network.html does a really good job of laying out all of the steps needed to simulate a separate replication network for testing, and enough information for deployers with Swift experience to apply the concepts to their environment. I think it'd be nice to see http://docs.openstack.org/developer/swift/howto_installmultinode.html updated; possibly with additional information on the dedicated replication network feature - but it's not always useful and even more rarely required - but that may change as more people experiment with geographically distributed clusters.

OVERVIEW

Generally moving replication traffic to a separate network requires moving two parts:

  • The data transfer channel (rsync)
  • The management channel (the HTTP REPLICATE method)

You'll also need to update the device info in your ring builder files.

RSYNC

The configuration of the named rsync endpoints (rsync calls them "modules") is not managed automatically by Swift, but it expects storage nodes to have named rsync endpoints in form rsync://<replication_ip>/<server-type>

[object]
max connections = 2
path = /srv/node/
read only = false
lock file = /var/lock/object.lock

or rsync://<replication_ip>/<server-type><replication_port>

[object6010]
max connections = 2
path = /srv/1/node/
read only = false
lock file = /var/lock/object6010.lock

... if you're running in vm_test_mode.

The replication "port" is my favorite part (rsync always runs on the same port), but it is used by the management channel.

HTTP REPLICATE

The HTTP REPLICATE verb on the management channel does some network (especially on the account/container servers) - and you can separate that out to the dedicated replication network by having a separate process listening for REPLICATE requests on replication network. The swift process management tool swift-init can make this easier on you by launching one server for each config.

So if you currently have /etc/swift/object-server.conf you'll want to create /etc/swift/objet-server/request.conf and /etc/swift/object-server/replication.conf so that when you (or your rc scripts) call swift-init object-server start you get one instance started from request.conf and configured for requests, and another started from replication.conf for replication management commands.

You'll need to set the bind_ip and bind_port options in the [DEFAULT] section. You'll also want to set replication_server = true in the [app:object-server] section of replication.conf and after you're sure the replication network is working to false in request.conf.

RINGS

It's easiest to add replication parameters when adding devices to the rings:

swift-ring-builder test.builder add --region 1 --zone 1 --ip 192.168.1.100 --port 6010 --replication-ip 192.168.2.100 --replication-port 8010 --device sda --weight 100

... but you can also update existing devices with set_info:

$ swift-ring-builder test.builder set_info 192.168.1.100 R192.168.3.100
Device d0r1z1-192.168.1.100:6010R192.168.2.100:8010/sda_"" is now d0r1z1-192.168 ...
(more)
edit flag offensive delete link more

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2013-09-11 12:52:44 -0500

Seen: 1,377 times

Last updated: Sep 17 '13