Ask Your Question
0

swift performance for small objects

asked 2013-11-06 10:59:24 -0500

anonymous user

Anonymous

Is Swift appropriate for storing 500 million files each one between 20K to 50K for an application with very high write and read rates? Assuming we use multiple containers with each one holding a subset of the files, where are the performance bottlenecks for this use case?

edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted
1

answered 2013-11-07 20:01:32 -0500

mblandini gravatar image

updated 2013-11-11 04:50:27 -0500

koolhead17 gravatar image

Mercado Libre shared their case study at the OpenStack summit this week, video available @ http://youtu.be/EKsO2eiAdA8.

As mentioned in their presentation, they have an average image size of 20K, and >1B of them today working very well in their high-write scenarios.

edit flag offensive delete link more
0

answered 2013-11-06 13:41:01 -0500

torgomatic gravatar image

The filesystems on your data disks will start being the limiting factor pretty quickly. The best way to mitigate that is to try to keep inodes and directories in memory as much as possible, which you can do by setting the kernel's vfs_cache_pressure sysctl to 1, and also having plenty of RAM in the object-server boxes.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

2 followers

Stats

Asked: 2013-11-06 10:59:24 -0500

Seen: 446 times

Last updated: Nov 11 '13