What are suffix directories in a partition?

asked 2013-05-02 09:52:42 -0600

I was going through a documentation of swift which mentioned the object replication process as under

". We now use a modification of this scheme in which a hash of the contents for each suffix directory is saved to a per-partition hashes file. The hash for a suffix directory is invalidated when the contents of that suffix directory are modified. The object replication process reads in these hash files, calculating any invalidated hashes. It then transmits the hashes to each remote server that should hold the partition, and only suffix directories with differing hashes on the remote server are rsynced. After pushing files to the remote server, the replication process notifies it to recalculate hashes for the rsynced suffix directories."

I am not too sure what does a suffix directory mean. If a suffix directory is a directory pair containing the accounts as the parent and the container as the sub directory within the partition directory such that for each partition the directory looks as under

+partition +account +container

then according to the logic sighted above, lets say, one of the objects suffers a bit rot, then does that mean the entire container will be replicated to lets say a handoff node? So does that mean the smallest unit of replication is a container within a partition. If not then please let me know what are directory suffixes and on what basis are they created?

edit retag flag offensive close merge delete

4 answers

Sort by » oldest newest most voted

answered 2013-05-03 19:06:49 -0600

Thanks clay.... That answered all my questions. I have some more querries on the handoff and I guess I will ask them seperately. Thanks again for the quick and prompt reply.

edit flag offensive delete link more

answered 2013-05-02 15:37:12 -0600

clay-gerrard gravatar image

That section isn't referring to the url structure; it's the filesystem layout on the storage node. Specifically under the 'objects' directory on the device.


. ./124900 ./124900/.lock ./124900/cc7 ./124900/cc7/79f90fbc6670a9af8ba360d8a130dcc7 ./124900/cc7/79f90fbc6670a9af8ba360d8a130dcc7/1367381436.64835.data ./8415 ./8415/.lock ./8415/21a ./8415/21a/0837da88da62b7be9e164cc0f400721a ./8415/21a/0837da88da62b7be9e164cc0f400721a/1367388052.84699.data ./36094 ./36094/.lock ./36094/abd ./36094/abd/233f8dbe2ee9f224472839034b75eabd ./36094/abd/233f8dbe2ee9f224472839034b75eabd/1367388052.83830.data

Looking at this you can see that in my ring the hash '233f8dbe2ee9f224472839034b75eabd' landed in the partition '36094'. (the hash is derived from 'account/container/objectname') In a more loaded out system that hash would share the partition with many other objects. And multiple objects will be grouped together under their "suffix" (literally the last few charcaters of the hash - 'abd').

Maybe consider again the the section on replication with this directory layout in mind and let me know if you have any questions...


edit flag offensive delete link more

answered 2013-05-03 05:19:32 -0600

Hey Thanks a lot Clay for your answer. So now considering your answer in mind, the suffix directory is /abd/ in this case. Now considering a bit rot in the 'XXXXXXeabd' object in this case, does that mean the object replicator will rsync the /abd/ directory. In other words if one of the objects suffer a bit rot in the abd directory, the entire abd directory will be rsynced?

Secondly, why is there two entries for the same hash- one with the hash value and the second with the hash value as a directory containing hash.data

edit flag offensive delete link more

answered 2013-05-03 16:50:22 -0600

clay-gerrard gravatar image

Yeah I think you've got it.

No the bit rot won't get replicated because a) bit rot won't invalidate the suffix in the per-partition pickle b) the auditor will but not until it quarantines the file.

Theres not really "two" entries, the output of find is just showing files and directories. The hash is the directory, the timestamp.data is part of conflict resolution.

edit flag offensive delete link more

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower


Asked: 2013-05-02 09:52:42 -0600

Seen: 156 times

Last updated: May 03 '13