Ask Your Question
0

where is the code about objects mapping?

asked 2013-11-11 02:01:21 -0500

qi-weining gravatar image

When I upload a object "ppt.pdf" to the container "container_test1" of swift SAIO, I find one of the path of the object is "/mnt/sdb1/1/node/sdb1/objects/497/382/7c5f513fe9483ed4d2646c059a79d382/1383891714.41626.data". I want to hnow where is the code to map a object from swift to the disk?

edit retag flag offensive close merge delete

16 answers

Sort by » oldest newest most voted
0

answered 2013-11-13 11:07:45 -0500

qi-weining gravatar image

Thanks Kun Huang, that solved my question.

edit flag offensive delete link more
0

answered 2013-11-12 10:17:22 -0500

It's easy, and you don't need change source codes. Just try "swift-get-codes", following its help message and you could find physical address of an object, a container or an account.

edit flag offensive delete link more
0

answered 2013-11-13 02:44:55 -0500

notice this:

Use your own device location of servers: such as "export DEVICE=/srv/node"

If you installed your cluster by SAIO document, I think you should export your own environment variable DEVICE by your own config files.

edit flag offensive delete link more
0

answered 2013-11-13 03:26:10 -0500

qi-weining gravatar image

Thanks, Kun Huang. I used both swift installed in 4 DELL servers and Swift SAIO for testing. I can upload and download. but when I used "swift-get-nodes", the Partition returned is not a folder in the storage node.

my DEVICE is set "/srv/node". There are many folders in “/srv/node/sdb1/objects/”, but there is not a folder named "61178". What's wrong?

There are some output of swift in 4 DELL servers, not of SAIO.

103748 126837 149607 173774 199160 224454 245960 35322 59406 81540 103759 126936 150199 173928 199274 224706 246115 35640 59883 81645 104006 127348 150388 174134 199316 224827 246328 35849 60625 82238 104023 127379 150421 174314 199607 22492 246504 35853 61046 8236 104036 12739 150443 174445 199656 224949 247236 3593 61104 82599 104155 127440 150555 174603 199993 22504 247474 36058 61285 829 104330 127876 150775 174775 200217 225222 248137 36533 61321 82982 104503 1279 151115 174919 200408 225251 249085 36593 6133 8299 10460 127971 152044 174941 200664 225523 249204 36968 61354 83013 104800 12821 152185 174971 200776 225548 249277 3722 61411 83035 104976 128482 152222 175017 201129 225639 249300 37808 61812 83806 104999 128577 152835 175140 201141 22576 24942 38130 62058 83874 105016 128778 153060 175340 201340 225873 249644 38514 62190 83956 105082 128972 153126 175360 201619 225888 249910 38592 62233 84005 105114 129266 153247 175482 201692 226246 249933 38613 62235 84290 105560 129368 153340 175781 201952 226464 250062 38616 62547 84448 105587 129420 153406 175810 202096 226583 2502 38885 62577 84471 105712 129613 153427 176219 202921 226675 250392 3896 62667 8464 105918 129644 153446 176406 203144 226927 250484 38968 62739 84936

swift@cloud1:~/openstack$ swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing list cont1 boto-2.6.0.tar.gz swift@cloud1:~/openstack$ cd swift@cloud1:~$ swift-get-nodes -a /etc/swift/object.ring.gz test cont1 boto-2.6.0.tar.gz

Account test Container cont1 Object boto-2.6.0.tar.gz

Partition 61178 Hash 3bbe955693bec810b7cfa75b9e8eb1af

Server:Port Device 192.168.1.4:6010 sdb1 Server:Port Device 192.168.1.3:6010 sdb1 Server:Port Device 192.168.1.5:6010 sdb1

curl -I -XHEAD "http://192.168.1.4:6010/sdb1/61178/test/cont1/boto-2.6.0.tar.gz" curl -I -XHEAD "http://192.168.1.3:6010/sdb1/61178/test/cont1/boto-2.6.0.tar.gz" curl -I -XHEAD "http://192.168.1.5:6010/sdb1/61178/test/cont1/boto-2.6.0.tar.gz"

Use your own device location of servers: such as "export DEVICE=/srv/node" ssh 192.168.1.4 "ls -lah ${DEVICE:-/srv/node}/sdb1/objects/61178/1af/3bbe955693bec810b7cfa75b9e8eb1af/" ssh 192.168.1.3 "ls -lah ${DEVICE:-/srv/node}/sdb1/objects/61178/1af/3bbe955693bec810b7cfa75b9e8eb1af/" ssh 192.168.1.5 "ls -lah ${DEVICE:-/srv/node}/sdb1/objects/61178/1af/3bbe955693bec810b7cfa75b9e8eb1af/"

swift@cloud1:~$ ssh 192.168.1.3 "ls -lah ${DEVICE:-/srv/node}/sdb1/objects/61178/1af/3bbe955693bec810b7cfa75b9e8eb1af/" swift@192.168.1.3's password: ls: 初始化月份字符串出错 ls: 无法访问/srv/node/sdb1/objects/61178/1af ... (more)

edit flag offensive delete link more
0

answered 2013-11-12 03:19:06 -0500

The strategies and background is subtle, but you could find the codes in: L416~L427

https://github.com/openstack/swift/blob/master/swift/obj/diskfile.py#L416 (https://github.com/openstack/swift/bl...)

It's now whole, but an enough hint for you

If you're interested in the theory about this, try to read this blog: http://julien.danjou.info/blog/2012/openstack-swift-consistency-analysis (http://julien.danjou.info/blog/2012/o...)

or swiftstack‘s book http://swiftstack.com/book/

edit flag offensive delete link more
0

answered 2013-11-12 10:08:05 -0500

qi-weining gravatar image

Thanks, Kun Huang. When I input "account:user, container, object", I want to get the address of one of the replicas in swift. Such as when I input "test:tester, container_test1, ppt.pdf", I hope to get "/mnt/sdb1/1/node/sdb1/objects/497/382/7c5f513fe9483ed4d2646c059a79d382/1383891714.41626.data". If there is a method without modifying the source code of swift, could you tell me how can I do that? It is perfect to obtain the address without modifying the source code. If it is essential to modify the source code to get the address, could you give me some advices?

edit flag offensive delete link more
0

answered 2013-11-12 11:28:39 -0500

qi-weining gravatar image

Thanks, Kun Huang. As an Administrator, I want to have an interface to obtain the physical address of an object. Perhaps I need many objects addresses on time.

I want to achieve such a function that objects not being accessed for a long long time move from swift to other places. So I need to obtain the physical address of an object easy and fast. And it is better not disturbing the running swift. It is perfect to obtain the address without modifying the source code.

I think that your above advices can help me. Could you explain it with more details?

edit flag offensive delete link more
0

answered 2013-11-13 01:35:08 -0500

See the last part of your output, and you can find that is ssh command:

ssh

"<command>"

That is used to run <command> in target address. And what is output of ls? That is what you need now.

So try paste&copy and run that ssh command: )

edit flag offensive delete link more
0

answered 2013-11-12 13:01:58 -0500

I think you don't need worried about changes in source codes, because reasonable functions may have been alreadt implemented in community.

In your case, swift-get-nodes could help you:

swift-get-nodes [-a] <ring.gz> <account> [<container>] [] could output real/physical address of an account/container/object, without disturbing running swift cluster.

edit flag offensive delete link more
0

answered 2013-11-12 13:37:29 -0500

qi-weining gravatar image

Thanks, , Kun Huang. You are so kind. But when I have a try, I meet this swift@qwn-G31M-S2C:~$ swift-get-nodes -a /etc/swift/account.ring.gz test Traceback (most recent call last): File "/usr/local/bin/swift-get-nodes", line 7, in <module> execfile(__file__) File "/home/swift/swift/bin/swift-get-nodes", line 22, in <module> from swift.common.utils import hash_path ImportError: cannot import name hash_path

I installed swift SAIO, and its version is swift 1.6.0.1

I find this is a bug. And Swift 1.10.0 has resolved it.

Can I upgrade my swift SAIO instead of re install it?

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2013-11-11 02:01:21 -0500

Seen: 47 times

Last updated: Nov 13 '13