Ask Your Question
0

Performance Issue with Large Files

asked 2013-03-05 16:09:17 -0600

suppldp2001 gravatar image

I have installed Swift integrated with Keystone and when trying to retrieve large files (almost 3GB) it is taking 4 minutes approximately. I know that we should consider another aspects such as network bandwidth, etc. but I believe the problem comes from the fact that I use a single partition for storage (I followed "Swift - All In One" instructions) with a 30 GB disk containing 262144 partitions. My question is, can this generate a perfomance issue when trying to get objects from Swift? Is it not that a large number of partitions considering the disk size I used? I can provide more details if needed. Thanks in advance.

edit retag flag offensive close merge delete
add a comment

4 answers

Sort by » oldest newest most voted
0

answered 2013-03-07 12:58:10 -0600

suppldp2001 gravatar image

What I basically would like to know is what is an acceptable data transfer bandwidth for Swift for files about 3 GB in size?

edit flag offensive delete link more
add a comment
0

answered 2013-03-28 15:45:19 -0600

cthier gravatar image

I don't really have experience with glance sorry.

edit flag offensive delete link more
add a comment
0

answered 2013-03-07 17:04:57 -0600

cthier gravatar image

For large files like that, swift will be limited by either your network throughput or your available disk io (whichever is smaller). A general rule of thumb is a 1G network and a single SATA drive can support on the order of magnitude of 100MB/s. This will be further limited if there is any other network traffic, or network shaping taking place, or if there is other disk io being done at the same time.

As a side note, if you are running everything on a single drive, then it is also likely that you have a lot of processes going on as well (like replication, etc.) that can eat up disk io if not properly tuned. You might try testing with the other services turned off to see if you need to tune those a bit further.

Chuck

edit flag offensive delete link more
add a comment
0

answered 2013-03-28 15:34:01 -0600

suppldp2001 gravatar image

Thank you Chuck. I will take into consideration your advice.

I am using Swift as a storage backend for Glance. And every time I run a "glance delete" the image file is not deleted from Swift. Is it a common behaviour? Is this correct or the image file should also be deleted from Swift?

Thanks in advance.-

edit flag offensive delete link more
add a comment

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

subscribe to rss feed

Stats

Asked: 2013-03-05 16:09:17 -0600

Seen: 73 times

Last updated: Mar 28 '13

Related questions

503 error running multiple-server

Auth server recovery

Openstack API wrappers

Regarding Swift API

Is OpenStack support CDN（Content Delivery Network),like Amazon?

What are suffix directories in a partition?

Rebalancing IO Overhead

Can't access to a container

Problem with folders/directories while using S3 middletier

how to check replicator work complete