Ask Your Question
0

Linux reports incorrect file size in object store. [closed]

asked 2014-03-20 09:33:00 -0500

Jimbo gravatar image

Using OpenStack Swift (Havana) on ARM platform running Fedora 19, noticing that Linux is reporting incorrect file size with the command "df -hP" and "du -sh" after uploading a file.

Mount options for object store:

$ mount |grep sda

/dev/sda on /data type xfs (rw,noatime,nodiratime,seclabel,attr2,nobarrier,inode64,noquota)

Create new file: 50M

$ dd if=/dev/zero of=/data/50M bs=10M count=5

State of the object store:

$ df -hP /data

Filesystem Size Used Avail Use% Mounted on /dev/sda 7.5G 83M 7.5G 1% /data

$ du -sh /data/50M

50M /data/50M

Upload the file to object store:

$ swift -v -V 2.0 -A http://127.0.0.1:25250/v2.0 -U admin:admin -K password upload Test /data/50M

State of the object store after upload:

$ df -hP /data

Filesystem Size Used Avail Use% Mounted on

/dev/sda 7.5G 1.2G 6.4G 16% /data

Show file size in object store:

$ find /srv/1/node/store/objects/ -name *.data |xargs du -sh

1.1G /srv/1/node/store/objects/473/cb1/7661c56b746713c79df51ca09926ccb1/1395323452.56928.data

$ find /srv/1/node/store/objects/ -name *.data |xargs ls -llAh

50M /srv/1/node/store/objects/473/cb1/7661c56b746713c79df51ca09926ccb1/1395323452.56928.data

Why would these commands (above) show two different sizes?

FYI - I tried the same experiment on a x86 platform and file size reported correctly (50M) by both commands.

Any help would be much appreciated.

Thanks, J

edit retag flag offensive reopen merge delete

Closed for the following reason the question is answered, right answer was accepted by SamYaple
close date 2014-03-22 19:22:08.559795

2 answers

Sort by ยป oldest newest most voted
1

answered 2014-03-22 09:41:22 -0500

Jimbo gravatar image

After further investigation, we determined that we needed to configure Swift to disable fallocate() calls. Apparently, this was causing fragmentation of the files as they were written to the object store. 'ls' and 'du' were reporting file sizes correctly because they were fragmented. We put "disable_fallocate = true" into the default section of the config files for account-server, container-server and object-server and the files now report their correct size.

Thanks to everyone for looking at this and considering a solution and/or responding. I think we're all set for now.

J

edit flag offensive delete link more

Comments

Thanks for reporting back! Glad it is working for you.

SamYaple gravatar imageSamYaple ( 2014-03-22 19:22:47 -0500 )edit
0

answered 2014-03-21 09:01:22 -0500

SamYaple gravatar image

Can you try this experiment on a random file on an XFS partition outside of swift? This sounds like a bug in the ls command or possibly in XFS.

What are the options you specify for? More specifically, why the double 'l' (read: ell) in the command ls -llAh?

edit flag offensive delete link more

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2014-03-20 09:33:00 -0500

Seen: 428 times

Last updated: Mar 22 '14