Ask Your Question

jcburke's profile - activity

2017-03-25 21:10:11 -0600 received badge  Popular Question (source)
2013-11-04 20:37:02 -0600 received badge  Famous Question (source)
2013-10-20 05:15:12 -0600 received badge  Taxonomist
2013-09-05 18:38:13 -0600 asked a question Transaction id in header?

This page: https://blueprints.launchpad.net/swift/+spec/transaction-id-headers (https://blueprints.launchpad.net/swif...) would seem to suggest that I should get the transaction ID as a header in each response I get from swift, but I am not seeing them. Is there some way to turn this behavior on? It would be really useful for debugging

2013-08-01 23:51:43 -0600 answered a question Getting 'not found' from some POST requests

Here are some logs: First the file being uploaded, then an example of a transaction where one of the three POSTs failed (so the client probably didn't get a 404) and an example where all 3 failed (so the client must have gotten a 404).

Aug 1 16:32:00 swift100 object-server 172.17.10.72 - - [01/Aug/2013:23:32:00 +0000] "PUT /d1/883816/AUTH_software/buildcache_test/test" 201 - "-" "txc34cfe1ffc024885bfa9cd71b90470d2" "-" 0.1009 Aug 1 16:32:00 swift103 container-server 172.17.10.71 - - [01/Aug/2013:23:32:00 +0000] "PUT /d2/467790/AUTH_software/buildcache_test/test" 201 - "txc34cfe1ffc024885bfa9cd71b90470d2" "-" "-" 0.0006 Aug 1 16:32:00 swift104 container-server 172.17.10.76 - - [01/Aug/2013:23:32:00 +0000] "PUT /d1/467790/AUTH_software/buildcache_test/test" 201 - "txc34cfe1ffc024885bfa9cd71b90470d2" "-" "-" 0.0005 Aug 1 16:32:00 swift104 object-server 172.17.10.72 - - [01/Aug/2013:23:32:00 +0000] "PUT /d3/883816/AUTH_software/buildcache_test/test" 201 - "-" "txc34cfe1ffc024885bfa9cd71b90470d2" "-" 0.0760 Aug 1 16:32:00 swift105 container-server 172.17.10.75 - - [01/Aug/2013:23:32:00 +0000] "PUT /d2/467790/AUTH_software/buildcache_test/test" 201 - "txc34cfe1ffc024885bfa9cd71b90470d2" "-" "-" 0.0005 Aug 1 16:32:00 swift105 object-server 172.17.10.72 - - [01/Aug/2013:23:32:00 +0000] "PUT /d3/883816/AUTH_software/buildcache_test/test" 201 - "-" "txc34cfe1ffc024885bfa9cd71b90470d2" "-" 0.0757

Aug 1 16:32:05 swift101 object-server 172.17.10.73 - - [01/Aug/2013:23:32:05 +0000] "POST /d1/883816/AUTH_software/buildcache_test/test" 404 - "-" "tx98457d933aa14602a37456ca5b013d87" "-" 0.0003 Aug 1 16:32:05 swift102 proxy-server ERROR with Object server 172.17.10.71:6000/d1 re: Trying to POST /AUTH_software/buildcache_test/test: ConnectionTimeout (0.5s) (txn: tx98457d933aa14602a37456ca5b013d87) (client_ip: 172.17.23.35) Aug 1 16:32:05 swift102 proxy-server ERROR with Object server 172.17.10.73:6000/d2 re: Trying to POST /AUTH_software/buildcache_test/test: ConnectionTimeout (0.5s) (txn: tx98457d933aa14602a37456ca5b013d87) (client_ip: 172.17.23.35) Aug 1 16:32:04 swift104 object-server 172.17.10.73 - - [01/Aug/2013:23:32:04 +0000] "POST /d3/883816/AUTH_software/buildcache_test/test" 202 - "-" "tx98457d933aa14602a37456ca5b013d87" "-" 0.0024 Aug 1 16:32:04 swift105 object-server 172.17.10.73 - - [01/Aug/2013:23:32:04 +0000] "POST /d3/883816/AUTH_software/buildcache_test/test" 202 - "-" "tx98457d933aa14602a37456ca5b013d87" "-" 0.0025

Aug 1 16:32:12 swift101 object-server 172.17.10.73 - - [01/Aug/2013:23:32:12 +0000] "POST /d1/883816/AUTH_software/buildcache_test/test" 404 - "-" "tx3f5e36fd69a54690b5050dc0643f520b" "-" 0.0002 Aug 1 16:32:11 swift102 proxy-server ERROR with Object server 172.17.10.75:6000/d3 re: Trying to POST /AUTH_software/buildcache_test/test: ConnectionTimeout (0.5s) (txn: tx3f5e36fd69a54690b5050dc0643f520b) (client_ip: 172.17.23.35) Aug 1 16:32:12 swift102 proxy-server ERROR with Object server 172.17.10.76:6000/d3 re: Trying to POST /AUTH_software/buildcache_test/test: ConnectionTimeout (0.5s) (txn: tx3f5e36fd69a54690b5050dc0643f520b) (client_ip: 172.17.23.35) Aug 1 16:32:12 swift102 proxy-server ERROR with Object server 172.17.10.71:6000/d1 re: Trying to POST /AUTH_software/buildcache_test/test: ConnectionTimeout (0.5s) (txn: tx3f5e36fd69a54690b5050dc0643f520b) (client_ip: 172.17.23.35) Aug 1 16:32 ... (more)

2013-08-01 22:36:59 -0600 answered a question Getting 'not found' from some POST requests

To answer your first question, yes, we do have object_post_as_copy set to false (I asked a question about it a little while back: https://answers.launchpad.net/swift/+question/232498 (https://answers.launchpad.net/swift/+...) ). As I said in that question, we do a lot of POSTs, so turning post_as_copy back on isn't really a viable option. I'm hoping that isn't the source of this issue; unfortunately we didn't have much logging in place back when we changed that option and I'm not sure if this problem was happening before we made the change. As for your second question, I'm not seeing transaction IDs in the headers returned to me by Swift. I can see these headers in the swift logs, but it would be great if I could directly tie them to the requests that failed rather than guessing based on what the log looks like. Am I doing something wrong that is causing me not to get these headers? I have been looking for them by talking to swift using curl in verbose mode.

2013-08-01 21:11:37 -0600 asked a question Getting 'not found' from some POST requests

For my use of Swift I need to check whether a file exists and update it's expiration time at the same time (or as close together as possible, doesn't need to be totally perfect), so I was planning on doing it by sending the POST request to update the expiration time and then checking whether the file exists by seeing if I got back a 202 or a 404. However, I have noticed with my cluster that I get quite a few false negatives: I have a test that uploads a file and then updates its expiration 1000 times, and on average about 3 of those attempts will return a 404. For comparison, when I run the same test and GET the file 1000 times instead of POSTing to it, I don't see any 404s (through 5 runs of the test). Is this to be expected? I know 3/1000 isn't high, but it's still enough to make me wary of relying on it since such mistakes could be rather costly in my case and there are a lot of files in play. Any idea what could be causing this? Also, I'm not sure if this is relevant, but for some reason these 404s are not evenly distributed amongst swift hosts: 2 of our 6 swift servers return about 60% of them, and one of the hosts only returns 4% of them. However, all of these hosts have the exact same configurations (we are running all of the swift services on each one) and from what I can tell receive roughly the same amount of traffic. Thanks in advance, any help will be greatly appreciated

2013-07-26 01:45:02 -0600 received badge  Notable Question (source)
2013-07-20 00:33:46 -0600 received badge  Popular Question (source)
2013-07-15 21:31:23 -0600 answered a question Slow response to expiration time update

Also, thanks for responding so quickly! You guys are clearly doing a great job staying on top of this forum

2013-07-15 21:29:35 -0600 answered a question Slow response to expiration time update

Thanks David Goetz, that solved my question.

2013-07-15 20:34:59 -0600 answered a question Slow response to expiration time update

Do you think you could explain a little more why this is the default, and what the consequences are of setting it to 'false'? It seems strange to copy the entire object every time its metadata is updated by default, and container-to-container sync seems like a rather obscure feature to be worth such a performance-costly default setting. Is there some other functionality that would break if I change this setting? Also, can it be set on a per-container basis, or does it have to apply to the whole cluster?

2013-07-15 20:19:49 -0600 answered a question Slow response to expiration time update

Thanks David Goetz, that solved my question.

2013-07-15 19:06:18 -0600 asked a question Slow response to expiration time update

Hi, I am using Swift as a cache for files that get up into the 10s of MB, and after fetching one I update its X-Delete-After time (in order to do approximate LRU by having unused stuff expire). I've noticed that for big files, this operation takes longer, which doesn't make sense since I am not sending any data, just headers, and I'm only getting back a small message, and on the server side this is (supposedly) handled asynchronously anyway. I don't have many files on swift since I am basically testing it out right now, so I don't think that is the problem. For example, updating the expiration time for a ~50MB file took min 2.21s, avg 7.5s, and max 28s (!). Here is the command I called 100 times to test this: curl -silent -k -X POST -H 'X-Auth-Token: [my auth token]' -H 'X-Delete-After: 604800' [my storage url]/[container]/[filename]

Any help would be greatly appreciated, taking 7 seconds on average to do this is really not acceptable for our use case (also, for the record, ping times to the servers in question are sub-ms and they have gigabit connections, so I don't think its a network problem).

2013-07-12 19:02:57 -0600 asked a question Swift: slow response to POST for large files

Hi, I am using Swift as a cache for files that get up into the 10s of MB, and after fetching one I update its X-Delete-After time (in order to do approximate LRU by having unused stuff expire). I've noticed that for big files, this operation takes longer, which doesn't make sense since I am not sending any data, just headers, and I'm only getting back a small message, and on the server side this is (supposedly) handled asynchronously anyway. I don't have many files on swift since I am basically testing it out right now, so I don't think that is the problem. For example, updating the expiration time for a ~50MB file took min 2.21s, avg 7.5s, and max 28s (!). Here is the command I called 100 times to test this: curl -silent -k -X POST -H 'X-Auth-Token: [my auth token]' -H 'X-Delete-After: 604800' [my storage url]/[container]/[filename]

Any help would be greatly appreciated, taking 7 seconds on average to do this is really not acceptable for our use case (also, for the record, ping times to the servers in question are sub-ms and they have gigabit connections, so I don't think its a network problem).