Swift object-server disk write hotspots

asked 2015-07-10 20:08:14 -0600

James Swift gravatar image

Hey all,

We have a 3-node OpenStack Swift environment configured such that all Swift services are running on each of the three nodes, A, B, and C. Keystone (v2) is configured on node A and all traffic to date is being directed through the proxy-server of node A.

The Object Rings are configured such that 9 XFS file systems are used from each of the three nodes as follows:

swift-ring-builder object.builder
object.builder, build version 33
262144 partitions, 1.000000 replicas, 1 regions, 1 zones, 27 devices, 0.02 balance, 0.00 dispersion
The minimum number of hours before a partition can be reassigned is 1
The overload factor is 0.00% (0.000000)
Devices:    id  region  zone      ip address  port  replication ip  replication port      name weight partitions balance meta
             1       1     1     <IP removed>  6000     <IP removed>              6000     data0  10.00       9710    0.01
             2       1     1     <IP removed>  6000     <IP removed>              6000     data0  10.00       9710    0.01
             3       1     1     <IP removed>  6000     <IP removed>              6000     data1  10.00       9710    0.01
             4       1     1     <IP removed>  6000     <IP removed>              6000     data2  10.00       9710    0.01
             5       1     1     <IP removed>  6000     <IP removed>              6000     data3  10.00       9710    0.01
             6       1     1     <IP removed>  6000     <IP removed>              6000     data4  10.00       9710    0.01
             7       1     1     <IP removed>  6000     <IP removed>              6000     data5  10.00       9710    0.01
             8       1     1     <IP removed>  6000     <IP removed>              6000     data6  10.00       9710    0.01
             9       1     1     <IP removed>  6000     <IP removed>              6000     data7  10.00       9710    0.01
            10       1     1     <IP removed>  6000     <IP removed>              6000     data8  10.00       9710    0.01
            11       1     1     <IP removed>  6000     <IP removed>              6000     data1  10.00       9710    0.01
            12       1     1     <IP removed>  6000     <IP removed>              6000     data2  10.00       9710    0.01
            13       1     1     <IP removed>  6000     <IP removed>              6000     data3  10.00       9710    0.01
            14       1     1     <IP removed>  6000     <IP removed>              6000     data4  10.00       9710    0.01
            15       1     1     <IP removed>  6000     <IP removed>              6000     data5  10.00       9710    0.01
            16       1     1     <IP removed>  6000     <IP removed>              6000     data6  10.00       9710    0.01
            17       1     1     <IP removed>  6000     <IP removed>              6000     data7  10.00       9710    0.01
            18       1     1     <IP removed>  6000     <IP removed>              6000     data8  10.00       9710    0.01
            19       1     1     <IP removed>  6000     <IP removed>              6000     data0  10.00       9707   -0.02
            20       1     1     <IP removed>  6000     <IP removed>              6000     data1  10.00       9707   -0.02
            21       1     1     <IP removed>  6000     <IP removed>              6000     data2  10.00       9707   -0.02
            22       1     1     <IP removed>  6000     <IP removed>              6000     data3  10.00       9708   -0.01
            23       1     1     <IP removed>  6000     <IP removed>              6000     data4  10.00       9707   -0.02
            24       1     1     <IP removed>  6000     <IP removed>              6000     data5  10.00       9707   -0.02
            25       1     1     <IP removed>  6000     <IP removed>              6000 ...
(more)
edit retag flag offensive close merge delete