where is the code about objects mapping?

When I upload a object "ppt.pdf" to the container "container_test1" of swift SAIO, I find one of the path of the object is "/mnt/sdb1/1/node/sdb1/objects/497/382/7c5f513fe9483ed4d2646c059a79d382/1383891714.41626.data". I want to hnow where is the code to map a object from swift to the disk?

edit retag close merge delete

Sort by » oldest newest most voted

Thanks Kun Huang, that solved my question.

more

It's easy, and you don't need change source codes. Just try "swift-get-codes", following its help message and you could find physical address of an object, a container or an account.

more

notice this:

Use your own device location of servers: such as "export DEVICE=/srv/node"

If you installed your cluster by SAIO document, I think you should export your own environment variable DEVICE by your own config files.

more

Thanks, Kun Huang. I used both swift installed in 4 DELL servers and Swift SAIO for testing. I can upload and download. but when I used "swift-get-nodes", the Partition returned is not a folder in the storage node.

my DEVICE is set "/srv/node". There are many folders in “/srv/node/sdb1/objects/”, but there is not a folder named "61178". What's wrong?

There are some output of swift in 4 DELL servers, not of SAIO.

103748 126837 149607 173774 199160 224454 245960 35322 59406 81540 103759 126936 150199 173928 199274 224706 246115 35640 59883 81645 104006 127348 150388 174134 199316 224827 246328 35849 60625 82238 104023 127379 150421 174314 199607 22492 246504 35853 61046 8236 104036 12739 150443 174445 199656 224949 247236 3593 61104 82599 104155 127440 150555 174603 199993 22504 247474 36058 61285 829 104330 127876 150775 174775 200217 225222 248137 36533 61321 82982 104503 1279 151115 174919 200408 225251 249085 36593 6133 8299 10460 127971 152044 174941 200664 225523 249204 36968 61354 83013 104800 12821 152185 174971 200776 225548 249277 3722 61411 83035 104976 128482 152222 175017 201129 225639 249300 37808 61812 83806 104999 128577 152835 175140 201141 22576 24942 38130 62058 83874 105016 128778 153060 175340 201340 225873 249644 38514 62190 83956 105082 128972 153126 175360 201619 225888 249910 38592 62233 84005 105114 129266 153247 175482 201692 226246 249933 38613 62235 84290 105560 129368 153340 175781 201952 226464 250062 38616 62547 84448 105587 129420 153406 175810 202096 226583 2502 38885 62577 84471 105712 129613 153427 176219 202921 226675 250392 3896 62667 8464 105918 129644 153446 176406 203144 226927 250484 38968 62739 84936

swift@cloud1:~/openstack$swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing list cont1 boto-2.6.0.tar.gz swift@cloud1:~/openstack$ cd swift@cloud1:~$swift-get-nodes -a /etc/swift/object.ring.gz test cont1 boto-2.6.0.tar.gz Account test Container cont1 Object boto-2.6.0.tar.gz Partition 61178 Hash 3bbe955693bec810b7cfa75b9e8eb1af Server:Port Device 192.168.1.4:6010 sdb1 Server:Port Device 192.168.1.3:6010 sdb1 Server:Port Device 192.168.1.5:6010 sdb1 curl -I -XHEAD "http://192.168.1.4:6010/sdb1/61178/test/cont1/boto-2.6.0.tar.gz" curl -I -XHEAD "http://192.168.1.3:6010/sdb1/61178/test/cont1/boto-2.6.0.tar.gz" curl -I -XHEAD "http://192.168.1.5:6010/sdb1/61178/test/cont1/boto-2.6.0.tar.gz" Use your own device location of servers: such as "export DEVICE=/srv/node" ssh 192.168.1.4 "ls -lah${DEVICE:-/srv/node}/sdb1/objects/61178/1af/3bbe955693bec810b7cfa75b9e8eb1af/" ssh 192.168.1.3 "ls -lah ${DEVICE:-/srv/node}/sdb1/objects/61178/1af/3bbe955693bec810b7cfa75b9e8eb1af/" ssh 192.168.1.5 "ls -lah${DEVICE:-/srv/node}/sdb1/objects/61178/1af/3bbe955693bec810b7cfa75b9e8eb1af/"

swift@cloud1:~$ssh 192.168.1.3 "ls -lah${DEVICE:-/srv/node}/sdb1/objects/61178/1af/3bbe955693bec810b7cfa75b9e8eb1af/" swift@192.168.1.3's password: ls: 初始化月份字符串出错 ls: 无法访问/srv/node/sdb1/objects/61178/1af ...

more

The strategies and background is subtle, but you could find the codes in: L416~L427

https://github.com/openstack/swift/blob/master/swift/obj/diskfile.py#L416 (https://github.com/openstack/swift/bl...)

It's now whole, but an enough hint for you

or swiftstack‘s book http://swiftstack.com/book/

more

Thanks, Kun Huang. When I input "account:user, container, object", I want to get the address of one of the replicas in swift. Such as when I input "test:tester, container_test1, ppt.pdf", I hope to get "/mnt/sdb1/1/node/sdb1/objects/497/382/7c5f513fe9483ed4d2646c059a79d382/1383891714.41626.data". If there is a method without modifying the source code of swift, could you tell me how can I do that? It is perfect to obtain the address without modifying the source code. If it is essential to modify the source code to get the address, could you give me some advices?

more

Thanks, Kun Huang. As an Administrator, I want to have an interface to obtain the physical address of an object. Perhaps I need many objects addresses on time.

I want to achieve such a function that objects not being accessed for a long long time move from swift to other places. So I need to obtain the physical address of an object easy and fast. And it is better not disturbing the running swift. It is perfect to obtain the address without modifying the source code.

I think that your above advices can help me. Could you explain it with more details?

more

See the last part of your output, and you can find that is ssh command:

ssh

"<command>"

That is used to run <command> in target address. And what is output of ls? That is what you need now.

So try paste&copy and run that ssh command: )

more

I think you don't need worried about changes in source codes, because reasonable functions may have been alreadt implemented in community.

swift-get-nodes [-a] <ring.gz> <account> [<container>] [

more

Thanks, , Kun Huang. You are so kind. But when I have a try, I meet this swift@qwn-G31M-S2C:~\$ swift-get-nodes -a /etc/swift/account.ring.gz test Traceback (most recent call last): File "/usr/local/bin/swift-get-nodes", line 7, in <module> execfile(__file__) File "/home/swift/swift/bin/swift-get-nodes", line 22, in <module> from swift.common.utils import hash_path ImportError: cannot import name hash_path

I installed swift SAIO, and its version is swift 1.6.0.1

I find this is a bug. And Swift 1.10.0 has resolved it.

more