Ask Your Question

More quarantined files and in loop

asked 2019-04-14 20:58:48 -0500

My cluster swift have too many files going to quarantine and always, have apper new files in quarantined all time. Some time ago, one node has added in the cluster with different hash, but this went adjusted. The message of quarantined continue, and the swift cotinue moves data to quarantined and replicating the data 'supposedly' healthed. My disk is full. How i can you fix this?

edit retag flag offensive close merge delete


Similar problem: Somebody damaged their Swift config and paid for it with mass-quarantines.

The recommendation in that case was re-uploading objects rather than trying to repair the damage.

Bernd Bausch gravatar imageBernd Bausch ( 2019-04-14 22:23:48 -0500 )edit

Thanks, how i can free space im my device?

Antonio Novaes gravatar imageAntonio Novaes ( 2019-04-15 07:41:35 -0500 )edit

It seems quarantined objects are under /srv/node/d1/quarantined, at least in the Kilo version, so perhaps you should start there. Or maybe this is not the right solution; it just thought it was similar enough.

Bernd Bausch gravatar imageBernd Bausch ( 2019-04-15 07:48:27 -0500 )edit

how can to do analyze/troubleshooting?

Antonio Novaes gravatar imageAntonio Novaes ( 2019-04-15 11:46:15 -0500 )edit

Was solved. Host 3 was removed from the cluster and after some time the quarantine messages were gone. So we made a link to the beginning of the problem that started with the input from this host, even adjusting the settings did not work. Removing from the cluster, the problem has stopped. Thanks

Antonio Novaes gravatar imageAntonio Novaes ( 2019-04-24 08:23:22 -0500 )edit

1 answer

Sort by » oldest newest most voted

answered 2019-04-15 07:40:39 -0500

updated 2019-04-15 08:55:03 -0500

I'm suspicious of XFS:

On system failures, the XFS file system can sometimes truncate files it’s trying to write and produce zero-byte files. The object-auditor will catch these problems but in the case of a system crash it would be advisable to run an extra, less rate limited sweep to check for these specific files. (source the info: (

[root@xpto-object-storage ~]# file /srv/node/sde/quarantined/objects/86611a8eee5c285a433294f8f57086af-5619c2917eca4631b60840f148030522/

/srv/node/sde/quarantined/objects/86611a8eee5c285a433294f8f57086af-5619c2917eca4631b60840f148030522/ empty

But, in some times, the file is ok. In the test it was possible to compare a quarantine file with a file available via s3. This comparison was successful and the files have the same hash md5 and you can also view the contents of the two (PDF).

Important that many messages, if no all, log that hast the file is diferente the hash de folder/location.

edit flag offensive delete link more

Get to know Ask OpenStack

Resources for moderators

Question Tools


Asked: 2019-04-14 20:58:48 -0500

Seen: 99 times

Last updated: Apr 15 '19