Ask Your Question
0

Most reliable way to test uploading a 5GB object.

asked 2010-10-22 17:30:38 -0500

twkiel gravatar image

Avoiding timeouts, what is the best method to send up a 5GB file (testing limits)?

edit retag flag offensive close merge delete

8 answers

Sort by ยป oldest newest most voted
0

answered 2010-11-05 15:46:02 -0500

dmitry-pha gravatar image

epochasset: Did you figure out the proper rate limiting settings?

edit flag offensive delete link more
0

answered 2010-10-26 18:37:26 -0500

twkiel gravatar image

I take this back, Amazon has a service for S3 that lets you send in portable storage for them to manually load. Sorry for the confusion, and thanks for your help.

edit flag offensive delete link more
0

answered 2010-10-25 18:09:05 -0500

gholt gravatar image

Ah gotcha, yeah there's just the one way right now. What are the optional ways with S3? We could investigate such features for Swift.

edit flag offensive delete link more
0

answered 2010-10-25 17:43:25 -0500

twkiel gravatar image

On the local network, with Swift installed with default settings, I occassionally get timeouts on files several GB in size. I assume this is due to the rate limiting settings. I was thinking as S3 does an alternate mechanism for uploading large files, possibly theres more than one way with Swift. If there is only the one way, I can just look into the rate limiting settings/network.

edit flag offensive delete link more
0

answered 2010-10-25 06:05:23 -0500

gholt gravatar image

Upload it on a network local to the cluster? I'm not sure I entirely understand the question though.

edit flag offensive delete link more
0

answered 2010-11-05 17:20:27 -0500

twkiel gravatar image

The example config appears to have ratelimiting off. But I also think the ratelimiting is by request, so a single operation is likely unaffected. At that point I would wonder what storage you are using, what size object times out in write, and how long it takes before failure.

edit flag offensive delete link more
0

answered 2010-11-05 17:08:27 -0500

dmitry-pha gravatar image

I'm uploading to the localhost and getting this error (only 1 node in this zone, maybe that's a problem). What setting should I change?

edit flag offensive delete link more
0

answered 2010-11-05 17:04:04 -0500

twkiel gravatar image

Internal tests seemed to have enough bandwidth to where it wasnt a problem. When I ran tests from a remote office I believe their usage caused it.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2010-10-22 17:30:38 -0500

Seen: 95 times

Last updated: Nov 05 '10