Ask Your Question
0

where is the code about objects mapping?

asked 2013-11-11 02:01:21 -0500

qi-weining gravatar image

When I upload a object "ppt.pdf" to the container "container_test1" of swift SAIO, I find one of the path of the object is "/mnt/sdb1/1/node/sdb1/objects/497/382/7c5f513fe9483ed4d2646c059a79d382/1383891714.41626.data". I want to hnow where is the code to map a object from swift to the disk?

edit retag flag offensive close merge delete

16 answers

Sort by » oldest newest most voted
0

answered 2013-11-12 03:19:06 -0500

The strategies and background is subtle, but you could find the codes in: L416~L427

https://github.com/openstack/swift/blob/master/swift/obj/diskfile.py#L416 (https://github.com/openstack/swift/bl...)

It's now whole, but an enough hint for you

If you're interested in the theory about this, try to read this blog: http://julien.danjou.info/blog/2012/openstack-swift-consistency-analysis (http://julien.danjou.info/blog/2012/o...)

or swiftstack‘s book http://swiftstack.com/book/

edit flag offensive delete link more
0

answered 2013-11-12 10:08:05 -0500

qi-weining gravatar image

Thanks, Kun Huang. When I input "account:user, container, object", I want to get the address of one of the replicas in swift. Such as when I input "test:tester, container_test1, ppt.pdf", I hope to get "/mnt/sdb1/1/node/sdb1/objects/497/382/7c5f513fe9483ed4d2646c059a79d382/1383891714.41626.data". If there is a method without modifying the source code of swift, could you tell me how can I do that? It is perfect to obtain the address without modifying the source code. If it is essential to modify the source code to get the address, could you give me some advices?

edit flag offensive delete link more
0

answered 2013-11-12 10:17:22 -0500

It's easy, and you don't need change source codes. Just try "swift-get-codes", following its help message and you could find physical address of an object, a container or an account.

edit flag offensive delete link more
0

answered 2013-11-12 11:28:39 -0500

qi-weining gravatar image

Thanks, Kun Huang. As an Administrator, I want to have an interface to obtain the physical address of an object. Perhaps I need many objects addresses on time.

I want to achieve such a function that objects not being accessed for a long long time move from swift to other places. So I need to obtain the physical address of an object easy and fast. And it is better not disturbing the running swift. It is perfect to obtain the address without modifying the source code.

I think that your above advices can help me. Could you explain it with more details?

edit flag offensive delete link more
0

answered 2013-11-12 13:01:58 -0500

I think you don't need worried about changes in source codes, because reasonable functions may have been alreadt implemented in community.

In your case, swift-get-nodes could help you:

swift-get-nodes [-a] <ring.gz> <account> [<container>] [] could output real/physical address of an account/container/object, without disturbing running swift cluster.

edit flag offensive delete link more
0

answered 2013-11-12 13:37:29 -0500

qi-weining gravatar image

Thanks, , Kun Huang. You are so kind. But when I have a try, I meet this swift@qwn-G31M-S2C:~$ swift-get-nodes -a /etc/swift/account.ring.gz test Traceback (most recent call last): File "/usr/local/bin/swift-get-nodes", line 7, in <module> execfile(__file__) File "/home/swift/swift/bin/swift-get-nodes", line 22, in <module> from swift.common.utils import hash_path ImportError: cannot import name hash_path

I installed swift SAIO, and its version is swift 1.6.0.1

I find this is a bug. And Swift 1.10.0 has resolved it.

Can I upgrade my swift SAIO instead of re install it?

edit flag offensive delete link more
0

answered 2013-11-12 14:19:05 -0500

If this is the first time you try swift, please use the newest codes. That error seems a very old bug.

edit flag offensive delete link more
0

answered 2013-11-13 00:59:57 -0500

qi-weining gravatar image

Thanks, , Kun Huang. You are so kind. Your advices almost resolve my problems. But there is one thing left. I want to get the object's full path, including the last "**.data". "swift-get-nodes" method tells me the path, not including the object's *.data file name.

Such as when I used "swift-get-nodes", I got "/mnt/sdb1/1/node/sdb1/objects/497/382/7c5f513fe9483ed4d2646c059a79d382/". But I also need the last "1383891714.41626.data". I hope I can get "/mnt/sdb1/1/node/sdb1/objects/497/382/7c5f513fe9483ed4d2646c059a79d382/1383891714.41626.data"

Thank you very much.

swift@cloud1:~$ swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing list test1 cert.key proxy-server.conf swift@cloud1:~$ swift-get-nodes -a /etc/swift/object.ring.gz test test1 cert.key

Account test Container test1 Object cert.key

Partition 34233 Hash 216e50dc72887cf0c7982b0edd5b63b6

Server:Port Device 192.168.1.3:6010 sdb1 Server:Port Device 192.168.1.5:6010 sdb1 Server:Port Device 192.168.1.4:6010 sdb1

curl -I -XHEAD "http://192.168.1.3:6010/sdb1/34233/test/test1/cert.key" curl -I -XHEAD "http://192.168.1.5:6010/sdb1/34233/test/test1/cert.key" curl -I -XHEAD "http://192.168.1.4:6010/sdb1/34233/test/test1/cert.key"

Use your own device location of servers: such as "export DEVICE=/srv/node" ssh 192.168.1.3 "ls -lah ${DEVICE:-/srv/node}/sdb1/objects/34233/3b6/216e50dc72887cf0c7982b0edd5b63b6/" ssh 192.168.1.5 "ls -lah ${DEVICE:-/srv/node}/sdb1/objects/34233/3b6/216e50dc72887cf0c7982b0edd5b63b6/" ssh 192.168.1.4 "ls -lah ${DEVICE:-/srv/node}/sdb1/objects/34233/3b6/216e50dc72887cf0c7982b0edd5b63b6/" swift@cloud1:~$

edit flag offensive delete link more
0

answered 2013-11-13 01:35:08 -0500

See the last part of your output, and you can find that is ssh command:

ssh

"<command>"

That is used to run <command> in target address. And what is output of ls? That is what you need now.

So try paste&copy and run that ssh command: )

edit flag offensive delete link more
0

answered 2013-11-13 01:59:40 -0500

qi-weining gravatar image

I meet another strange problem.

I use ssh to connect 192.168.1.3. And I can find the path "/srv/node/sdb1/objects". But I can not find the path /srv/node/sdb1/objects/34233. There is no such a path named 34233.

It is really confused me.

swift@cloud1:~$ ssh 192.168.1.3 "ls -lah ${DEVICE:-/srv/node}/sdb1/objects/34233/3b6/216e50dc72887cf0c7982b0edd5b63b6/"

swift@192.168.1.3's password: ls: 初始化月份字符串出错 ls: 无法访问/srv/node/sdb1/objects/34233/3b6/216e50dc72887cf0c7982b0edd5b63b6/: 没有那个文件或目录

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2013-11-11 02:01:21 -0500

Seen: 111 times

Last updated: Nov 13 '13