Openstack Swift strange behaiour on copying back the Data Dirs

asked 2017-05-10 02:49:41 -0500

mgarg gravatar image

So, I am seeing some very strange behavior with Openstack Swift. If I start all the swift processes, and say do an operation (eg. upload a file), then if I kill all my Swift processes, copy all the data directories to some other temp location, delete the content of all the data directories, copy all the temp directories back to data dir and start my swift processes again, I am not able to perform a GET for the object. (I can access the container info but swift download fails).

I have spend hours debugging this very strange behavior. Anyone else has experienced this problem ?

Steps for reproducing the bug: (Assuming 4 Data Dirs say in /mnt/sdb1/ dir)

=> swift-init all start => swift upload some_container some_file
=> (Can be followed by any no of swift operations)
=> swift-init all kill
=> cp -R /mnt/sdb1/1 /mnt/sdb1/1_temp (copy from original to a temp dir)
=> cp -R /mnt/sdb1/2 /mnt/sdb1/2_temp (copy from original to a temp dir)
=> cp -R /mnt/sdb1/3 /mnt/sdb1/3_temp (copy from original to a temp dir)
=> cp -R /mnt/sdb1/4 /mnt/sdb1/4_temp (copy from original to a temp dir)
=> rm -rf /mnt/sdb1/1/* (delete all the contents of original data dir)
=> rm -rf /mnt/sdb1/2/* (delete all the contents of original data dir)
=> rm -rf /mnt/sdb1/3/* (delete all the contents of original data dir)
=> rm -rf /mnt/sdb1/4/* (delete all the contents of original data dir)
=> cp -R /mnt/sdb1/1_temp /mnt/sdb1/1 (copy to original from the temp dir)
=> cp -R /mnt/sdb1/2_temp /mnt/sdb1/2 (copy to original from the temp dir)
=> cp -R /mnt/sdb1/3_temp /mnt/sdb1/3 (copy to original from the temp dir)
=> cp -R /mnt/sdb1/4_temp /mnt/sdb1/4 (copy to original from the temp dir)

=> swift-init all start (start swift again)
=> swift list (this succeeds)
=> swift list some_container (this succeeds)
=> swift download some_container some_file (this fails)

This is very strange considering the fact that I just moved the content.

Any pointers ?

edit retag flag offensive close merge delete

Comments

Swift objects (and perhaps containers) are stored as files and extended attributes of those files. My guess is that you only copy the files, not the extended attributes. Perhaps cp --preserve=xattr is sufficient.

Bernd Bausch gravatar imageBernd Bausch ( 2017-05-10 03:28:39 -0500 )edit