Ask Your Question
0

Sqlite and .ts maintenance with deletes

asked 2013-01-04 09:34:54 -0600

andrefcruz gravatar image

I have a container that receives frequent PUTs/DELETEs. Over time I've noticed that the fs has a lot of empty .ts and the SQLite file has grown a lot.

The .ts nodes are not being reclaimed after 1 week, as they should, and it seems to me that data is never deleted from the SQLite file. How can I deal with these 2 issues?

Thanks.

edit retag flag offensive close merge delete

8 answers

Sort by ยป oldest newest most voted
0

answered 2013-01-15 18:01:21 -0600

andrefcruz gravatar image

How can I force activity on a certain suffix dir? Is it the suffix of the filename? Do I just have to upload an object to the same dir with a filename with the same suffix?

Thanks!

edit flag offensive delete link more
0

answered 2013-01-04 21:58:08 -0600

cthier gravatar image

Have you checked to make sure the replicators are running? They are what is responsible for reclimation.

edit flag offensive delete link more
0

answered 2013-01-04 22:37:10 -0600

andrefcruz gravatar image

They are running. Also, I've noticed that although the SQLite DB is large, there are no deleted objects older than a week, so data seems to be reclaimed there.

However, regarding the .ts nodes, their number is always increasing and I see nodes 2 months old which is the age of the system. Clearly they are not being reclaimed. Is there any scenario when this is supposed to happen?

Can I/should I delete them manually from the file system?

Thanks for the help.

edit flag offensive delete link more
0

answered 2013-01-08 16:20:12 -0600

cthier gravatar image

How active is the cluster? It looks like the reclamation happens when the suffix hashes are re-hashed (which usually only happens when a change, like a new object PUT, happens in that suffix. If nothing has changed in that suffix dir, then it is likely it hasn't had a chance to crawl, and thus reclaim the .ts files.

edit flag offensive delete link more
0

answered 2013-01-10 23:32:45 -0600

andrefcruz gravatar image

I'd say the cluster is pretty active. We get 400k uploads per day and the average file size is 2MB.

As for the "suffix dirs", I read the http://docs.openstack.org/developer/swift/overview_replication.html (http://docs.openstack.org/developer/s...) documentation but could not find out for certain what they mean. However, I have a lot of uploads/deletes for a specific container, and it is here that the problem has been worse since I already have much more tombstones than regular files. Since this container/dir is changed so frequently, shouldn't it be crawled and .ts files cleaned? Don't these happen in the same "suffix"?

Thanks for the help.

edit flag offensive delete link more
0

answered 2013-01-15 20:09:13 -0600

torgomatic gravatar image

The suffix is the last 3 nybbles of the hash of the object name (that's MD5(object-name + per-cluster-suffix)). It's there so that there's not a ton of subdirectories in the partition directory. Think of a URL like "http://archive.ubuntu.com/ubuntu/pool/main/c/cairo/cairo_1.6.0.orig.tar.gz"; the "/c/cairo/" part is just there to spread the files around a bit. Same thing.

So, if you wanted to force activity on a certain suffix directory, you'd need to come up with an object name such that the last 3 nybbles of hash(objectname) are the suffix and the first P bits are the same as the partition number, where P is your cluster's part_power setting. That's computationally feasible (find at most 44 bits of collision), but it's not trivial.

edit flag offensive delete link more
0

answered 2013-01-15 20:21:25 -0600

cthier gravatar image

As a little bit more drastic measure, you could delete the hashes.pkl file in the partition directory. This will cause the replicator to crawl all of the suffix dirs in that partition, which will take a bit of time, but also remove the .ts files.

edit flag offensive delete link more
0

answered 2013-01-15 21:51:08 -0600

andrefcruz gravatar image

Well, I suppose this is the best I can do. But it is rather painful doing this on multiple partitions....

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2013-01-04 09:34:54 -0600

Seen: 61 times

Last updated: Jan 15 '13