Ask Your Question
0

Performance Issue with Large Files

asked 2013-03-05 16:09:17 -0500

suppldp2001 gravatar image

I have installed Swift integrated with Keystone and when trying to retrieve large files (almost 3GB) it is taking 4 minutes approximately. I know that we should consider another aspects such as network bandwidth, etc. but I believe the problem comes from the fact that I use a single partition for storage (I followed "Swift - All In One" instructions) with a 30 GB disk containing 262144 partitions. My question is, can this generate a perfomance issue when trying to get objects from Swift? Is it not that a large number of partitions considering the disk size I used? I can provide more details if needed. Thanks in advance.

edit retag flag offensive close merge delete

4 answers

Sort by ยป oldest newest most voted
0

answered 2013-03-07 12:58:10 -0500

suppldp2001 gravatar image

What I basically would like to know is what is an acceptable data transfer bandwidth for Swift for files about 3 GB in size?

edit flag offensive delete link more
0

answered 2013-03-28 15:45:19 -0500

cthier gravatar image

I don't really have experience with glance sorry.

edit flag offensive delete link more
0

answered 2013-03-07 17:04:57 -0500

cthier gravatar image

For large files like that, swift will be limited by either your network throughput or your available disk io (whichever is smaller). A general rule of thumb is a 1G network and a single SATA drive can support on the order of magnitude of 100MB/s. This will be further limited if there is any other network traffic, or network shaping taking place, or if there is other disk io being done at the same time.

As a side note, if you are running everything on a single drive, then it is also likely that you have a lot of processes going on as well (like replication, etc.) that can eat up disk io if not properly tuned. You might try testing with the other services turned off to see if you need to tune those a bit further.


Chuck

edit flag offensive delete link more
0

answered 2013-03-28 15:34:01 -0500

suppldp2001 gravatar image

Thank you Chuck. I will take into consideration your advice.

I am using Swift as a storage backend for Glance. And every time I run a "glance delete" the image file is not deleted from Swift. Is it a common behaviour? Is this correct or the image file should also be deleted from Swift?

Thanks in advance.-

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2013-03-05 16:09:17 -0500

Seen: 53 times

Last updated: Mar 28 '13