Seeking advice on vm "auto-cpu-pinning "

asked 2014-05-12 22:25:24 -0500

gaoshilin-zte gravatar image
We want to develop running vms pinned to physical CPUs based on openstack blueprint of auto-cpu-pinning.(https://blueprints.launchpad.net/nova/+spec/auto-cpu-pinning)

Now Describing of our initial thoughts, and hope anyone give some suggestions.

Precondition: 1) Now only consider KVM; 2) The VCPU number of a vm should be less then the intersection number of total physical CPU and vcpu_pin_set configured by nova.conf, once VCPU number more than the intersection number,all the vms of the host will be unpinned, and considered to use CPU super allocation; 3) Every VCPU should be pined to one physical CPU. 4) we will provides multiple pinning strategies finally,and which strategy will be used is configured by the administrator.

vcpupin strategies: Now only consider one strategy first.

Strategy 1(unnamed): 1) First NUMA info should be known from command “lscpu”,for example: NUMA node(s): 2 NUMA node0 CPU(s): 0-3,8-11 NUMA node1 CPU(s): 4-7,12-15 2) Then sort the CPUs by Numa Node list, let us call it CPU Queue for convenience.

    head                       CPU Queue                         tail
    -----------------------------------------------------------------
    | 0 | 1 | 2 | 3 | 8 | 9 | 10| 11| 4 | 5 | 6 | 7 | 12| 13| 14| 15| 
    -----------------------------------------------------------------

3) when first vm(instance_1) come, I push it into the head of the Queue, means the four vm’s VCPU will be pinned to physical CPU 0-3 one by one. then second vm, third vm…, In order of instance_id. If there is no enough CPU in the Queue,and you continue to deploy a vm,we think that CPU super allocation is confiured,then all the vms are unpinned,so the strategy itself doesn’t affect vm’s deployment.

    -----------------------------------------------------------------
    |  instance_1   | inst_2|inst_3 |  instance_4   |  instance_5   |
    -----------------------------------------------------------------
    | 0 | 1 | 2 | 3 | 8 | 9 | 10| 11| 4 | 5 | 6 | 7 | 12| 13| 14| 15| 
    -----------------------------------------------------------------

4) If some vms are shut down or canceled, the physical CPU pinned by them will be released from CPU Queue,but other vms will not be moved, so the CPU fragments will produce in the Queue.

    -----------------       ---------               -----------------
    |  instance_1   |       |inst_3 |               |  instance_5   |
    -----------------------------------------------------------------
    | 0 | 1 | 2 | 3 | 8 | 9 | 10| 11| 4 | 5 | 6 | 7 | 12| 13| 14| 15| 
    -----------------------------------------------------------------

5) Follow the above, if instance_4 is started again, it will be thrown into the original position.

    -----------------       ------------------------------------------
    |  instance_1   |       |inst_3 |  instance_4   |  instance_5   |
    -----------------------------------------------------------------
    | 0 | 1 | 2 | 3 | 8 | 9 | 10| 11| 4 | 5 | 6 | 7 | 12| 13| 14| 15| 
    -----------------------------------------------------------------

If instance_6 is deployed first, all the vms in CPU Queue must be aligned to the direction of the head of CPU Queue, because the tail of the Queue does not have enough CPU. And the CPU fragments will disappear after it in the Queue.

    -------------------------------------------------
    |  instance_1   | inst_3|  instance_5   | inst_6|
    -----------------------------------------------------------------
    | 0 | 1 | 2 | 3 | 8 | 9 | 10| 11| 4 | 5 | 6 | 7 | 12| 13| 14| 15| 
    -----------------------------------------------------------------

After deploying the instance_6,if we want to start the instance_4 again, instance_5 and instance_6 must be moved to the direction of the tail of CPU Queue, because there is no CPU between instance_3 and instance_5. If there is no enough CPU to pin instance_4, all the ... (more)

edit retag flag offensive close merge delete